BookPDF Available

Purposeful Program Theory: Effective Use of Theories of Change and Logic Models

Authors:
  • BetterEvaluation

Abstract and Figures

Between good intentions and great results lies a program theory—not just a list of tasks but a vision of what needs to happen, and how. Now widely used in government and not-for-profit organizations, program theory provides a coherent picture of how change occurs and how to improve performance. Purposeful Program Theory shows how to develop, represent, and use program theory thoughtfully and strategically to suit your particular situation, drawing on the fifty-year history of program theory and the authors' experiences over more than twenty-five years. "From needs assessment to intervention design, from implementation to outcomes evaluation, from policy formulation to policy execution and evaluation, program theory is paramount. But until now no book has examined these multiple uses of program theory in a comprehensive, understandable, and integrated way. This promises to be a breakthrough book, valuable to practitioners, program designers, evaluators, policy analysts, funders, and scholars who care about understanding why an intervention works or doesn't work." —Michael Quinn Patton, author, Utilization-Focused Evaluation "Finally, the definitive guide to evaluation using program theory! Far from the narrow 'one true way' approaches to program theory, this book provides numerous practical options for applying program theory to fulfill different purposes and constraints, and guides the reader through the sound critical thinking required to select from among the options. The tour de force of the history and use of program theory is a truly global view, with examples from around the world and across the full range of content domains. A must-have for any serious evaluator." —E. Jane Davidson, PhD, Real Evaluation Ltd. Preview of 61 pages available here https://books.google.com.au/books/about/Purposeful_Program_Theory.html?id=zBoGbWnVsFsC&printsec=frontcover&source=kp_read_button&redir_esc=y#v=onepage&q&f=false
Content may be subject to copyright.
PART 1
Key Ideas in
Program Theory
COPYRIGHTED MATERIAL
3
1
The Essence of
Program Theory
A N APPLE A DAY KEEPS the doctor away—or does it? Thinking about
how we would fi nd out if this is true and how we might use those
ndings shows the value of program theory. In this chapter, we set out the
key ideas in program theory and show how program theory can be used to
learn from success, failure, and mixed results to improve planning, manage-
ment, evaluation, and evidence-based policy.
EVALUATION WITHOUT PROGRAM THEORY
Let us imagine that we have implemented a program based on the broad
policy objective of an apple a day in order to keep the doctor away. This pro-
gram, which we dubbed An Apple a Day, involves distributing seven apples a
week to each participant. A representation of this program without program
theory would simply show the program followed by the intended outcome
of improved health (Figure 1.1).
Purposeful Program Theory
4
Figure 1.1 An Evaluation of An Apple a Day Without Program Theory
This is what is often referred to as a black box evaluation: one that
describes an evaluation that analyzes what goes in and what comes out with-
out information about how things are processed in between.
PROGRAM OUTCOMES
Improved
health
ORIGINS OF “BLACK BOX”
Different sources have been suggested for the term black box. The current
Wikipedia entry for black box traces the term, when used for fl ight data recorders,
to World War II Royal Air Force terminology, when prototypes of new electronic
devices were installed in airplanes in metal boxes, painted black to avoid refl ections
and therefore referred to as black boxes.
Former electronics buff turned evaluator Bob Briggs, on the American
Evaluation Association’s discussion list EVALTALK (Briggs, 1998), reminisced how
electronics manufacturers would often cover components with opaque material to
prevent consumers from “opening the black box” to see how it worked (and assem-
bling their own version more cheaply). The parallel with evidence-based practices
is useful: program theory aims to help policymakers and practitioners “open up the
box” of successful programs to understand how it works rather than having to buy
the whole package and plug it in.
However, as the evaluator and author Michael Quinn Patton (1998) pointed out
in the same EVALTALK thread, the term can be seen as inappropriate: “Most uses
of ’black box’ or ’black box design’ carry a negative connotation. The association of
’black’ with negativity is what can be experienced as offensive, or at least insensi-
tive” (Patton, 1998). He suggested using instead terms such as empty box, magic
box, or mystery box designs to describe evaluations without program theor y.
The Essence of Program Theory 5
It can be diffi cult to interpret results from an evaluation that has no
program theory. For an intervention that involves a discrete product for indi-
viduals, an experimental or quasi-experimental design might be appropriate
for the evaluation. We will assume that people have been assigned to either a
treatment group, who received the program, or to a control group, who went
onto a waiting list to receive the program later if the evaluation shows it is
effective. “Keeping the doctor away” has been operationalized as “maintain-
ing or achieving good physical health.” Data collection has been carefully
designed to avoid measurement failure of outcome variables, with adequate
sample size, appropriate measures of health, and systems in place to avoid
accidental or deliberate data corruption.
Despite careful evaluation, it can be impossible to interpret evaluation
results correctly in the absence of program theory. If the program failed to
achieve signifi cant differences in health outcomes between the groups (apple
versus no apple), it might seem that the policy does not work—but it might
also be that it has not been implemented properly. Maybe the apples were
delivered but not eaten, or maybe they were too small, or too unripe, or too
overripe to work as expected. Although the evaluation might include some
measures of the quality and extent of implementation, it can be hard to
know what aspects should be included unless there is a program theory.
An evaluation using program theory would identify how we understand
this program works and what intermediate outcomes need to be achieved for
the program to work. This allows us to distinguish between implementation
failure (not done right) and theory failure (done right but still did not work).
Without program theory, it is impossible to know if we have measured the
right aspects of implementation quality and quantity.
If the results showed that the program seemed to have succeeded, as the
treatment group had signifi cantly better outcomes than the no-treatment
group, we might also have trouble using these results more broadly. If we
do not know what elements of the policy are important, we can only copy it
exactly for fear of missing something essential. It does not provide any guid-
ance for adapting the policy for other settings.
Finally, if we had mixed results, where the policy worked on only some
sites or for some people, we might not even notice them if we were looking
Purposeful Program Theory
6
only at the average effect. If we did see differential effects in different contexts
(for example, for men compared to women, or in urban areas rather than
rural areas), an evaluation without program theory leaves us in the position
of having to do simple pattern matching (for example, using the policy for
the groups or sites where it has been shown to work) but with little ability to
generalize to other contexts.
EVALUATION WITH PROGRAM THEORY
If we used a program theory approach, we would try to understand the
causal processes that occur between delivering apples and improved health.
We might start by unpacking the box to show the important intermediate
outcome that people actually eat the apples. The logic model diagrams in
Figure 1.2 show this: one in the form of a pipeline model and one as an out-
comes chain. The pipeline logic model represents the program in terms of
inputs, processes, outputs, and outcomes. The outcomes chain model shows
a series of results at different stages along a causal chain.
Although these look like many logic models that are used regularly in eval-
uation, they are not much of a theory; rather, they are more like a two-step
Pipeline model version
INPUTS
Apples
People in
poor health
PROCESSES
Apples
delivered
OUTPUT
Apples
eaten
OUTCOMES
Improved
health
Outcomes chain version
Apples
eaten
Improved
health
Figure 1.2 Simple Pipeline and Outcomes Chain Logic Models
The Essence of Program Theory 7
process, as Mark Lipsey and John Pollard (1989) called it, that identifi es an
intermediate variable without really explaining how it works. These models
make it clear that eating the apples is understood to be part of the causal
chain (rather than some other variable, such as social interaction with the
apple deliverer or physical exercise from playing with the apples). But they
do not explain how delivering apples leads to people eating apples or how
eating apples improves health.
A plausible explanation would be that delivering apples increases the
availability of fresh fruit, which leads to the apples being eaten, which
increases the amount of vitamin C in the diet, which improves the physi-
cal health of participants. This is only one possible explanation, of course.
Figure 1.3 shows this explanation as both a pipeline logic model and an
outcomes chain.
The diagrams in Figure 1.3 represent a program theory that articulates
the causal mechanisms involved in producing the two changes (changed
behavior and changed health status). The fi rst change relates to participants’
willingness to act in the way the program intended and the second to the
impacts of their actions. For many programs, it can be helpful to articulate
both types of changes in the program theory.
Pipeline model version
Outcomes chain version
INPUTS
Apples
People in
poor health
PROCESSES
Apples
delivered
OUTPUT
Improved
access to
fresh fruit
Apples eaten
SHORT-TERM
OUTCOMES
Improved
level of
vitamin C
OUTCOMES
Improved
health
Improved
access to
fresh fruit
Apples
eaten
Improved
level of
vitamin C
Improved
health
Figure 1.3 A Logic Model Showing a Simple Program Theory for An Apple a Day
Based on Improved Vitamin Intake
Purposeful Program Theory
8
Learning from Failure
An evaluation based on this program theory would collect data about
changes in access to fresh fruit, apple eating behavior, and nutritional status,
as well as overall health. If the intended outcomes have not been achieved,
we could work through the causal chain to see where it has broken down. If
the apples were not even delivered, there is obvious implementation failure;
if they were delivered but not eaten, then our theory of how to engage peo-
ple in changing their behavior seems not to work. Similarly, if the expected
health benefi ts had not been achieved, we would start by seeing if it was
because the apples had not been eaten. If the apples had been delivered and
had been eaten but without producing health improvements, then we have
a problem with the theory of change that underpins the program. Based on
these results, one option would be to reject the theory and look at other ways
of improving health. Another would be to look at dosage: maybe vitamin C
levels increased, but not enough to make a difference.
Learning from Partial Success
Developing a program theory also helps clarify differential effects, learning
from those participants for whom the program was effective. The simple
program theory is based on the assumption that the apples are both neces-
sary and suffi cient—that is, the apples will lead to good health in all circum-
stances and without contributions from other factors. Developing a more
complicated logic model would focus on the differential effects we might
expect for different types of participants, and we would collect and analyze
data to examine these. Disaggregating the data would investigate whether the
theory works in some contexts but not in others.
This review might show that the program works only for certain types
of participants—for example, those who are affected by diseases related to
inadequate nutrition. For people affected by infectious diseases, apples by
themselves might not be enough to improve health. Based on these results, we
might target the program to people most likely to benefi t: those with nutrition-
related diseases. Given the importance of the interaction between the interven-
tion and the characteristics of clients, it would be helpful to revise the theory of
change and its logic model to show this complicated causal path.
The Essence of Program Theory 9
If the program works for some groups but not for others or at some
sites but not others, it is important to try to understand why by identifying
possible explanations and then checking these out empirically. For example,
if the program worked for men but not for women, it might be because
of differences in labor force patterns which affected access to fresh fruit or to
differences in nutritional needs related to pregnancy. Finding exceptions
to the pattern (the men who did not improve and the women who did)
would provide more evidence to test these emerging program theories.
Learning from Success
Program theory has another benefi t when an evaluation fi nds that something
works: it helps in adapting the intervention to new situations. To be useful
for evidence-based policy and practice, a program theory evaluation needs to
identify the causal mechanism by which it works and determine whether this
is different for different people and in different implementation contexts.
To explore this use, imagine that the evaluation has found that the pro-
gram theory works: people are healthier when they eat an apple a day. Now
the job is to implement a new program based on this evidence. In this
case, the goal is not to understand failure but to understand success. Apples
might produce these effects through quite different theories of change, which
would lead us to quite different intervention theories and different program
activities to suit the context. We would immediately have many questions
about the statement. Does it work for everyone? Does it have to be a particu-
lar variety of apple (Granny Smiths? crab apples?), or does it apply to all vari-
eties? What if apples are not available? Can we substitute other fruit, or apple
juice, or vegetables? Would red onions work as well as red apples? An evalua-
tion without program theory would reveal only that it works, with no guid-
ance for how to translate the fi ndings to a particular situation. Without this
guidance, we can only blindly copy everything. With this guidance, we can
understand how we might adapt it and still achieve the intended results.
We previously sketched out a program theory with a theory of change of
providing a good source of vitamins in diets that are otherwise defi cient. To
test this out if we were implementing it would require data about people’s
nutritional status through either direct measures or relevant indicators so we
Purposeful Program Theory
10
could see if there was any change and also to identify the people we would
expect to get the most benefi t from the program. We would want to check
that they actually ate the apples. And we would want to rule out alternative
explanations by fi nding out if there had been other changes in their diets
that might have contributed to changes in their nutrition. If this is the case,
then other types of fruit are likely to be equally effective. In a country where
apples are hard to obtain or expensive, distribution or subsidization of local
fruit is likely to be an effective program, at least for people at risk of nutri-
tional defi ciency, if it is implemented well.
But maybe this is not how it works at all. Maybe it is not about the fl esh
or juice of the apples but their skin. The skin of apples contains a plant-based
chemical called quercetin. Some research studies have suggested quercetin
may help to prevent cancer, heart disease, and infl ammation of the prostate.
An evaluation would look at the intake of quercetin from various sources
and outcomes in terms of these specifi c diseases, focusing on outcomes for
people at risk of these diseases. If apples were not available, another source
of quercetin could be used. Red onions, a rich source of quercetin, might
be an effective substitute—an adaptation of the program that would not be
immediately obvious if we were thinking only about fruit.
Another possible explanation focuses on apples as a substitute for high-
calorie, low-nutrition snacks. Perhaps apples improve health by helping to
reduce obesity as people stop eating potato chips and doughnuts and choose
apples instead. An evaluation of this possibility would look at what people
were eating in addition to apples and whether there had been a decline in
their consumption of junk food. It also might measure short-term outcomes
such as body mass index (BMI) and percentage fat, which have been linked to
subsequent longer-term health outcomes. The evaluation would have to take
into account criticisms that have been made of BMI as an indicator and pre-
dictor of health. Making other low-calorie snacks such as carrots and celery
readily available might be equally effective. Figure 1.4 shows how these three
different change theories might plausibly explain why the policy works.
Other possible explanations, involving different theories of change,
would lead to different critical features in implementation that should be
ensured. For example, if health improvements came about through increased
The Essence of Program Theory 11
fiber consumption, eating the whole apple, not just drinking the juice,
would be important. Once the plausible theories have been identifi ed, they
can be used to guide data collection and analysis of an evaluation. They can
also be used to synthesize data from previous evaluations and research (we
discuss this in Chapter Four).
Success in terms of achieving intended results might not mean success
in terms of the theory. Another possible pattern of results is that the health
outcomes have been achieved but not the intermediate results of changes
in vitamin C. This would suggest that something other than the interven-
tion had caused the health improvements or that a quite different theory of
change was operating that did not involve vitamin C. Results like this would
indicate theory failure despite success in terms of results.
INPUTS
Apples
People in
poor health
INTERMEDIATE
RESULT
Apples
eaten
INTERMEDIATE
RESULT
Improved
nutritional
status
FINAL
RESULT
Improved
health
INPUTS
Apples
(could use
oranges)
People with
vitamin C
deficiencies
INTERMEDIATE
RESULT
Apples
eaten
INTERMEDIATE
RESULT
Adequate levels
of vitamin C
FINAL
RESULT
Reduced
incidence
of scurvy
INPUTS
Apples
(could use
red onions)
People at risk
of cancer,
heart disease
INTERMEDIATE
RESULT
Apples
eaten
INTERMEDIATE
RESULT
Increased levels
of quercetin
FINAL
RESULT
Reduced
incidence
of heart
disease,
prostate
inflammation
INPUTS
Apples
(could use
carrot sticks)
Obese and
overweight
people
INTERMEDIATE
RESULT
Apples
eaten
INTERMEDIATE
RESULT
Decreased
consumption of
junk food
FINAL
RESULT
Reduced
incidence
of obesity
and
associated
conditions
Figure 1.4 Logic Models Showing Different Possible Causal Mechanisms Involved
in Eating an Apple a Day
Purposeful Program Theory
12
Learning from “An Apple a Day
Speculating on different possible causal mechanisms enables us to develop an
evaluation that will collect and analyze data to be able to understand to what
extent, for whom, and why an intervention does or does not work. (Chapter
Fourteen describes how to use program theory to guide evaluation design.)
Although a single evaluation is limited in its scope, program theory makes it
easier to combine evidence from a number of studies. Table 1.1 summarizes
how an evaluation informed by program theory can distinguish among dif-
ferent types of success and failure.
The apple a day example shows the importance of developing program
theory that identifi es the causal mechanism that is understood to be involved
in producing the intended outcomes. This can help to produce more useful
evaluations and better evidence for policy.
Table 1.1 Using Program Theory to Interpret Evaluation Findings
Apples
Delivered
Apples
Eaten
Vitamin
C Levels
Raised
Health
Outcomes
Improved Interpretation
✗✗✗ ✗Implementation failure
✓✗✗ ✗Engagement or adherence failure
(fi rst causal link)
✓✓✗ ✗Theory failure (early causal link)
✓✓✓ ✗Theory failure (later causal link)
✓✓✓ ✓Consistent with theory
✓✓/✗✓/Partial theory failure (works in some
contexts)
✓✓✗ ✓Theory failure (different causal path)
The Essence of Program Theory 13
SUMMARY
This chapter has used a hypothetical example to explore how articulating a
program theory—an explicit statement of how change will occur and how
an intervention will produce these causal processes—can make evaluations
more useful. Throughout the rest of the book, we use examples from actual
evaluations to show how to develop, represent, and use program theory
for evaluation and other purposes.
EXERCISES
1. If a social marketing campaign was used instead of direct delivery of
apples for the Apple a Day program, what would implementation
failure look like? What would theory failure look like? What would
partial theory failure look like, where it works only in particular con-
texts?
2. Consider a policy that aims to increase student performance by
increasing teachers’ salaries. What might be some alternative causal
mechanisms that would produce the intended outcomes?
... At the start of PERFORM2Scale, the development of the ToC assisted in creating a common vision for the programme. We triangulated consortium members' experience-based knowledge 4 with research evidence during ToC development [15,28]. Following Stein and Valter's broad categories of the purpose of ToCs, the PERFORM-2Scale ToC articulated expected processes and outcomes that were reviewed over time (monitoring and evaluation purpose) and served as a thinking tool, helping the consortium to clarify and develop the theory behind the programme (learning purpose) [29]. ...
Article
Full-text available
Background Since 2017, PERFORM2Scale, a research consortium with partners from seven countries in Africa and Europe, has steered the implementation and scale-up of a district-level health management strengthening intervention in Ghana, Malawi and Uganda. This article presents PERFORM2Scale’s theory of change (ToC) and reflections upon and adaptations of the ToC over time. The article aims to contribute to understanding the benefits and challenges of using a ToC-based approach for monitoring and evaluating the scale-up of health system strengthening interventions, because there is limited documentation of this in the literature. Methods The consortium held annual ToC reflections that entailed multiple participatory methods, including individual scoring exercises, country and consortium-wide group discussions and visualizations. The reflections were captured in detailed annual reports, on which this article is based. Results The PERFORM2Scale ToC describes how the management strengthening intervention, which targets district health management teams, was expected to improve health workforce performance and service delivery at scale, and which assumptions were instrumental to track over time. The annual ToC reflections proved valuable in gaining a nuanced understanding of how change did (and did not) happen. This helped in strategizing on actions to further steer the scale-up the intervention. It also led to adaptations of the ToC over time. Based on the annual reflections, these actions and adaptations related to: assessing the scalability of the intervention, documentation and dissemination of evidence about the effects of the intervention, understanding power relationships between key stakeholders, the importance of developing and monitoring a scale-up strategy and identification of opportunities to integrate (parts of) the intervention into existing structures and strategies. Conclusions PERFORM2Scale’s experience provides lessons for using ToCs to monitor and evaluate the scale-up of health system strengthening interventions. ToCs can help in establishing a common vision on intervention scale-up. ToC-based approaches should include a variety of stakeholders and require their continued commitment to reflection and learning on intervention implementation and scale-up. ToC-based approaches can help in adapting interventions as well as scale-up processes to be in tune with contextual changes and stakeholders involved, to potentially increase chances for successful scale-up.
... Developing a TOC should be the first step in impact evaluation (Qiu et al., 2018). A TOC identifies the logical and ordered causal links moving from the intervention to intended impacts and typically includes a visual illustration (Funnell & Rogers, 2011). These links, visual or otherwise, should be viewed as the hypotheses to be tested in the impact evaluation. ...
Book
This open access book presents a nuanced and accessible synthesis of the relationship between land tenure security and sustainable development. Contributing authors have collectively worked for decades on land tenure as connected with conservation and development across all major regions of the globe. The first section of this volume is intended as a standalone primer on land tenure security and its connections with sustainable development. The book then explores key thematic challenges that interact directly with land tenure security, followed by a section on strategies for addressing tenure insecurity. The book concludes with a section on new frontiers in research, policy, and action. An invaluable reference for researchers in the field and for practitioners looking for a comprehensive overview of this important topic.
... Further possibilities for methodology improvements could also draw upon methodologies widely used in public policy environments based on the ideas around "Theory of Change". A theory of change is a rigorous, yet participatory process incorporating goals, conditions and interventions that bring some desired change arranged in a causal framework [20]. Use of domain specific constructs within such an integrated approach could also help address the abstraction gap. ...
Conference Paper
Full-text available
The last five years has seen an exponential increase in interest around the notion of a digital twin. Multiple systematic reviews have established a base set of findings which are now broadly taken as assumptions in the field. The findings have outlined frameworks, integration patterns, distinctions between product and process oriented digital twins, simulation and the leveraging of artificial intelligence to support prediction and optimisation. As the use cases of digital twins have evolved from a predominance in manufacturing and into the sociotechnical domain to support the social world, a gap emerges between the social requirements and the technical machinery of digital twins. This gap is significant and worthy of exploration as it presents important challenges for digital twin research including use of sociotechnical design methods, problems arising from a so-called abstraction gap and various epistemological concerns. This paper proposes an analytical route to ameliorating the sociotechnical gap which is discussed within a future notion of an Environment Digital Twin.
... Intervention Mapping uses logic models to describe how the various methods and strategies in an intervention influence health. Logic models typically describe causal relations between determinants and outcomes, usually graphically, and they can be extremely useful for understanding an EBI [18,19]. The purpose of this paper is to present the EBI Mapping process. ...
Article
Full-text available
Background Despite the development of numerous evidence-based interventions (EBIs), many go unused in practice. Hesitations to use existing EBIs may be due to a lack of understanding about EBI components and what it would take to adapt it or implement it as designed. To improve the use of EBIs, program planners need to understand their goals, core components, and mechanisms of action. This paper presents EBI Mapping, a systematic approach based on Intervention Mapping, that can be used to understand and clearly describe EBIs, and help planners put them into practice. Methods We describe EBI Mapping tasks and provide an example of the process. EBI Mapping uses principles from Intervention Mapping, a systematic framework for planning multilevel health promotion interventions. EBI Mapping applies the Intervention Mapping steps retrospectively to help planners understand an existing EBI (rather than plan a new one). We explain each EBI Mapping task and demonstrate the process using the VERB Summer Scorecard (VSS), a multi-level community-based intervention to improve youth physical activity. Results EBI Mapping tasks are: 1) document EBI materials and activities, and their audiences, 2) identify the EBI goals, content, and mechanisms of action, 3) identify the theoretical change methods and practical applications of those methods, 4) describe design features and delivery channels, and 5) describe the implementers and their tasks, implementation strategies, and needed resources. By applying the EBI Mapping tasks, we created a logic model for the VSS intervention. The VSS logic model specifies the links between behavior change methods, practical applications, and determinants for both the at-risk population and environmental change agents. The logic model also links the respective determinants to the desired outcomes including the health behavior and environmental conditions to improve the health outcome in the at-risk population. Conclusions EBI Mapping helps program planners understand the components and logic of an EBI. This information is important for selecting, adapting, and scaling-up EBIs. Accelerating and improving the use of existing EBIs can reduce the research-to-practice gap and improve population health.
... TCA frameworks generally call for solution designers to make their premises and assumptions clear (Funnell and Rogers, 2011;Vogel, 2012). For the TCA model proposed here, these assumptions include three core analytic variables: opportunity, motivation, and justice. ...
Article
Full-text available
While a wide consensus acknowledges that participation is critical for the successful implementation of change that improves the livelihoods of people and communities around the world, justly securing that participation from stakeholders (at both the design and implementation stages) remains a demanding problem. This paper proposes a heuristic model for increasing participation that not only helps to investigate instances of nonparticipation but also opens up alternative intervention strategies and pathways for designers and implementers to consider toward more justly increasing participation and overcoming nonparticipation. Applied to a successful case of participation in Gurúè District, Mozambique—where an 89% solution adoption of an improved postharvest seed storage method was measured two years after initial training—this paper demonstrates the key importance of designing opportunities and motivations for participation into any solutions or innovations but especially justice as a factor for successful realization of theory of change efforts (all the more so in developing nation contexts). Applied to a second case study, this paper also explores participation despite little to no motivation to do so. Aiming to afford designers and implementers of theory of change interventions a tool for more successfully and exactly matching innovation goals with innovation outcomes, the paper also addresses broader implications for the model within theory of change approaches generally.
... In other words, outcomes are not improved by interventions themselves but through the mechanistic responses that are triggered by those interventions. Therefore, the primary focus of inquiry centres on the "program theory"-a hypothesised description of what is supposed to be carried out in the implementation of programs and how and why that is expected to work [29,30]. A realist study begins with the development of an initial program theory, which is then tested usually with both quantitative and qualitative data: quantitative often focussing on contextual factors and outcomes, and qualitative on the mechanisms and social processes involved. ...
Article
Full-text available
Implementation science in healthcare aims to understand how to get evidence into practice. Once this is achieved in one setting, it becomes increasingly difficult to replicate elsewhere. The problem is often attributed to differences in context that influence how and whether implementation strategies work. We argue that realist research paradigms provide a useful framework to express the effect of contextual factors within implementation strategy causal processes. Realist studies are theory-driven evaluations that focus on understanding how and why interventions work under different circumstances. They consider the interaction between contextual circumstances, theoretical mechanisms of change and the outcomes they produce, to arrive at explanations of conditional causality (i.e., what tends to work, for whom, under what circumstances). This Commentary provides example applications using preliminary findings from a large realist implementation study of system-wide value-based healthcare initiatives in New South Wales, Australia. If applied judiciously, realist implementation studies may represent a sound approach to help optimise delivery of the right care in the right setting and at the right time.
Article
Full-text available
Background Improving livestock health is considered critical to address poverty, malnutrition and food insecurity in low- and middle-income countries (LMICs). Modifications of livestock management practices is also increasingly recognized as an important strategy to mitigate global threats such as climate change and novel disease emergence. Smallholders are, however, under various constraints which prohibit them from altering health practices for livestock and little is known about how the adoption of these practices may be promoted. The proposed scoping review aims to systematically map evidence around “what practices are (not) adopted by smallholders under what circumstances, how and why?.” Method and analysis We conducted initial scoping searches to broadly define types of animal health practices relevant for smallholders in LMICs and formulated search terms. A scoping review protocol was designed and registered. A systematic literature search will be conducted using electronic databases including CAB Abstract, Scopus, MEDLINE, EMBASE, and Web of Science Core Collection. Gray literature will be searched from AGRIS and Standards for Supporting Agricultural Livelihoods in Emergencies. Articles in English, pertaining to the animal health practices considered highly relevant will be considered eligible for inclusion. Articles will be screened at two stages by two independent reviewers; screening of titles, abstracts, and keywords, followed by full-article screening. The first reviewer will review 100% of the articles at both stages. The second reviewer will review a random sample of 20% of the articles at both stages. Any disagreements will be resolved using inputs from the third reviewer. A thematic analysis will be conducted to catalog contexts and mechanisms for adoption and discussed under a realist framework. Discussion Understanding of the mechanisms underlying the adoption of animal health practices by livestock smallholders in LMICs is crucial for successful implementation of interventions including those which are based on a One Health approach. This review will identify the extent of this knowledge across disciplines and inform future research priorities for the design of effective and feasible interventions which can contribute toward Sustainable Development Goal 2. Registration This protocol is registered within the Open Science Framework ( https://doi.org/10.17605/OSF.IO/FUQAX ).
Chapter
The main aim of this chapter is to give concrete reasons why systematic mixed-methods research (SMMR) is so important. SMMR is defined as using both quantitative and qualitative methods in one project. Some forms of mixed-methods research include a ‘reflexive’ stage, that is they involve self-reflection both at the personal and social level. This chapter sets out key terminology and draws connections between open systems, a depth ontology, and structuralism. Overall this creates a helpful platform for mixed-methods research. One innovation here is ‘open retroduction’, which means asking questions that require new evidence. Research that involves stakeholder voices is also encouraged. The terms ‘open systems’ and ‘retroduction’ have been introduced to affirm that a schism does not exist between quantitative and qualitative research methods. Lastly this chapter comments on the typical regression models that are widely used in the social sciences. Planning a regression model can be done partly by using a structuralist-institutionalist framing of variables. This chapter helps define the four main kinds of variables that one may include as independent variables in social science research. Regression of this kind brings mechanisms and events into the frame along with structural and institutional factors. Reflexivity and open retroduction are then two discrete ways to link the regression methods with the other methods in a mixed- or multiple-methods project.
Chapter
The impact of land tenure interventions on sustainable development outcomes is affected by political, social, economic, and environmental factors, and as a result, multiple types of evidence are needed to advance our understanding. This chapter discusses the use of counterfactual impact evaluation to identify causal relationships between tenure security and sustainable development outcomes. Rigorous evidence that tenure security leads to better outcomes for nature and people is thin and mixed. Using a theory of change as a conceptual model can help inform hypothesis testing and promote rigorous study design. Careful attention to data collection and use of experimental and quasi-experimental impact evaluation methods can advance understanding of causal connections between tenure security interventions and development outcomes.
ResearchGate has not been able to resolve any references for this publication.