That Theory of Change (ToC) has rapidly become a mainstream approach to evaluation raises several fundamental questions: are we uncritically adopting the ToC approach? In what contexts are ToCs suitable? And what are the challenges and opportunities in engaging with ToCs?
ToC has been described as a way of thinking, a process, and a product. The approach can begin by depicting the causal pathway linking the outcomes of a program to its activities. In a corresponding narrative, justifications and assumptions behind the linkages are provided. Done collaboratively, a ToC can help stakeholders predict and test change, helping to explain how and why a program works and thus serving as a useful tool for evaluation.
Reading ToC guidelines and reports, observing it used by evaluators, and applying it myself in international development contexts, I became curious about the use(fulness) of ToCs. In particular, I became interested in the limitations of ToCs, which I found were neglected in conversations, guidelines, and reports. I reviewed the literature to better understand some of the critiques of ToCs. In doing so, I identified seven concerns, some of which were found weakly justified while others pose valid challenges to ToC research and practice:
1. Confusion between ToCs and other evaluation approaches. It is unclear precisely what a ToC is and how it differs from other evaluation approaches such as logic models, logical frameworks, and impact pathways.
Limited evidence. There is extensive literature clarifying the difference between ToCs and other evaluation approaches. Considering the reflective practice competency of evaluation practice, evaluators should be knowledgeable about the differences and similarities between ToC and other theory-driven approaches to evaluation.
2. Conceptual vagueness. That a ToC is not a theory as is commonly understood in the social sciences creates confusion.
Partially supported evidence. The precise meaning of ToC is contested, but attempts have been made to define concepts and approaches. Rather than striving for a unified definition of ToC, evaluators should be aware of the diverse ToC meanings, purposes, and outcomes that exist, and be explicit about the conceptualizations they use.
3. Under-developed ToC. ToC presentations are typically vague, generic, and simplistic, particularly at the onset.
Partially supported evidence. It is not uncommon for ToCs to miss key steps such as outlining the assumptions, considering the context, or describing the nature of causal connections. Nevertheless, ToCs are meant to be dynamic, revisited, and revised as new knowledge is acquired.
4. Under-contributing theoretical knowledge. ToCs often under-contribute to the cumulation of theoretical knowledge.
Supported evidence. ToCs advancing theory beyond the program remains rare. Yet, that ‘theories’ are made explicit and therefore testable in ToCs enables ToCs to contribute to the development of theory for informing program strategies across different scales and settings.
5. Uncertainty in stakeholder engagement. While it is typically recommended that a wide range of stakeholders are invited to participate in ToC development, it is often unclear how, to what extent, and which stakeholders might be engaged.
Supported evidence. Explanations of and reflections on stakeholder engagement processes remain rare. Stakeholder engagement perhaps depends on the purposes of the ToC but more reflection experiences of stakeholder engagement are important for supporting this claim.
6. Neglecting context. Many ToCs are problematic in that they specify mechanisms linking activities to outcomes but pay little attention to how these mechanisms function across time and space.
Partially supported evidence. Context has been neglected in ToCs, but attempts have been made at integration. Evaluators should clarify how the problem and/or solution under consideration is created and sustained in context.
7. Overlooking complexity. Despite its emphasis on complexity, the presentation of ToCs tends to oversimplify real-world challenges.
Supported evidence. ToCs applying systems thinking and complexity-aware approaches remain rare. Complexity might be captured through creative visualizations capturing different pathways, feedback loops, program components, and relationships.
I found that ToC concerns focused mostly on how ToC has been operationalized rather than fundamental shortcomings of ToC per se. This finding does not dismiss the need for ToCs; instead, it questions how far ToC processes go in supporting reflection and learning. To support ToCs conceptually, clarity on what exactly is a ToC and how it differs from other evaluation approaches and social science theories are needed. At the methodological level, integrating context and complexity throughout the ToC process contributes not only to programming-specific knowledge but also to broader theoretical knowledge. To the extent that many evaluators hope to contribute to theory-driven social change, critical and reflexive development of theory-driven evaluation would represent a valuable step forward.
What concerns do you have about ToC? Do any of the above concerns resonate with you? How could ToCs be adapted in light of concerns?
Reference
Lam S. (2020). Toward learning from change pathways: reviewing Theory of Change and its discontents. Canadian Journal of Program Evaluation. 35(2): 188-203.
Photo by Daniel Ari Friedman on Foter.com / CC BY-NC