No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Who Owns the Theory of Change?
Abstract
Policy evaluations in the UK have increasingly espoused a ‘Theories of Change’ (ToC) approach, drawing on North American experiences. Features of ToC include the expectation that all affected stakeholders will be involved in developing, agreeing, monitoring and evaluating a relevant theory for the proposed intervention, and the assumption that widespread stakeholder involvement will extend ownership of the intervention to achieve ‘total ownership’ and also improve attribution. Drawing on the experiences of three English evaluations (Health Action Zones, New Deal for Communities and Local Strategic Partnerships), this article examines the possibilities and limitations associated with the achievement of ‘total ownership’. Analysis reveals some important differences between the English and North American contexts and leads to the development of alternative models of ownership including ‘elite’,‘evaluator’,‘principal’ or ‘community’ ownership. The article concludes that if these models are more realistic than ‘total ownership’ in the English policy environment then this has implications for the appropriate application of ToC.
... This paper responds to this gap by adopting more holistic approaches to evaluate tourism interventions, and more specifically around the implementation of ToC to evaluate complex tourism interventions. As a participative approach to evaluation which empowers project implementers, ToC can contribute towards the achievement of positive, lasting social change (Bertella et al., 2021;Getz, 2019;Sullivan & Stewart, 2006;Warnoltz et al., 2022). ...
... This allows participants to reflect and share their thoughts with honesty (Vogel, 2012). Where this does not happen, it can be tempting for the researcher to take a more active role, significantly undermining the stakeholders' ownership of the ToC, and loosing much of the initial purpose and the opportunity to create a meaningful co-learning experience (Maini et al., 2018;Sullivan & Stewart, 2006). ...
... Scholars have discussed how the lack of engagement with project partners and consequent lack of learning from evaluations is a root cause for the failure to achieve lasting project legacies (Bertella et al., 2021;Niavis et al., 2022), particularly in Eu regional development projects (Stoffelen, 2018;Vanclay, 2015). Our findings suggest that evaluation can represent a valuable opportunity for learning if we transition from evaluating for to evaluating with project partners (Northcote et al., 2008;Sullivan & Stewart, 2006). Creating buy-in was often challenging, however, once partners were able to understand the benefits a ToC could bring to their evaluation, engagement occurred naturally. ...
... While the initial theory of change was used to guide the design and implementation of the Project in its pilot phase, evaluation of the Project's feasibility led to further modification of the theory of change. Stakeholders' involvement in the development of the theory of change also created a sense of ownership and buy-ins from relevant stakeholders [53]. Ideally, all stakeholders should have ownership over the theory of change [53]. ...
... Stakeholders' involvement in the development of the theory of change also created a sense of ownership and buy-ins from relevant stakeholders [53]. Ideally, all stakeholders should have ownership over the theory of change [53]. However, due to the complexity of the Project, which cuts across various care settings from hospital to home, this is often difficult to execute in practice. ...
Background
Set against the backdrop of a rapidly ageing population and growing emphasis on the importance of ageing-in-place, family members often assume the role of a caregiver. Navigating through a complex healthcare system while simultaneously juggling the daily care needs of their care-recipients, caregivers often become worn out by the intense caregiver stress and burden, neglecting their own well-being. This translates to adverse health and economic outcomes such as prolonged hospital stays and increased nursing home placement of care-recipients. Seeking to better support caregivers, we developed a theory of change to guide the design and implementation of a caregiver support programme – Project Carer Matters.
Methods
We applied theory of change methodology to explain how the Project’s interventions were hypothesised to lead to their identified short-to-long term goals, drawing on a causal analysis based on available evidence. The theory of change was developed with insights garnered from previous research studies conducted on caregiver stress, stakeholder engagement sessions and multiple dialogues with clinical experts and hospital leaders.
Results
Our final theory of change is the result of the evaluation of the Project in its pilot phase. It is populated with the resources, activities and short-to-long term outcomes that can be attributed or linked to the Project. Multiple meetings and discussion with stakeholders over the pilot prompted frequent practice of the Plan, Do, Study, Act model to refine the ongoing implementation process and the theory of change itself.
Conclusions
A theory of change is essential in guiding the design, implementation and evaluation of a complex health care intervention such as Project Carer Matters. The development of the theory of change is a journey and not a resultant product. This journey has also led us to learn that 1) a theory of change needs to be dynamic and ever evolving with time and context, 2) the perspectives of relevant stakeholders need to be included in this process to ensure the feasibility and sustainability of the project in the long run and 3) frequent stakeholder engagements are essential in enabling the implementation team to fine-tune the Project in an effective manner.
Trial registration
ClinicalTrials.gov, NCT05205135, registered on 24/01/2022.
... We define wilder rangelands and outline the mechanisms through which they may help improve human well-being and ameliorate the climate and biodiversity crises. Finally, we explain the implementation of wilder rangelands using a theory of change (ToC) approach outlined by Sullivan and Stewart (2006). ...
... We followed a ToC approach for developing and implementing wilder rangelands, as it is an efficient tool for designing, monitoring, and evaluating complex, multifaceted and long-term initiatives (Sullivan and Stewart 2006). This approach explores causal linkages between desired changes expected from certain interventions at different result levels (activities, outputs, outcomes, and impact), the actors, and factors influencing those changes (Fig. 3). ...
Rangelands face threats from climate and land-use change, including inappropriate climate change mitigation initiatives such as tree planting in grassy ecosystems. The marginalization and impoverishment of rangeland communities and their indigenous knowledge systems, and the loss of biodiversity and ecosystem services, are additional major challenges. To address these issues, we propose the wilder rangelands integrated framework, co-developed by South African and European scientists from diverse disciplines, as an opportunity to address the climate, livelihood, and biodiversity challenges in the world’s rangelands. More specifically, we present a Theory of Change to guide the design, monitoring, and evaluation of wilder rangelands. Through this, we aim to promote rangeland restoration, where local communities collaborate with regional and international actors to co-create new rangeland use models that simultaneously mitigate the impacts of climate change, restore biodiversity, and improve both ecosystem functioning and livelihoods.
... Connell and Kubisch define the theory-of-change approach as "a systematic and cumulative study of the links between activities, outcomes, and contexts of the initiative" (1998, p. 2). This approach was designed to evaluate and accommodate the multilevel, multi-dimensional impacts of comprehensive interventions in which the task of linking actions to outcomes is extremely complex, at a time when existing evaluation approaches were considered inadequate or inappropriate (Sullivan and Stewart, 2006). This evaluation model is part of the theoretical approaches to evaluation and is based on the idea that evaluators must help to identify the theory of action implicit in an intervention in order to define what should happen if the theory is correct (Sullivan and Stewart, 2006). ...
... This approach was designed to evaluate and accommodate the multilevel, multi-dimensional impacts of comprehensive interventions in which the task of linking actions to outcomes is extremely complex, at a time when existing evaluation approaches were considered inadequate or inappropriate (Sullivan and Stewart, 2006). This evaluation model is part of the theoretical approaches to evaluation and is based on the idea that evaluators must help to identify the theory of action implicit in an intervention in order to define what should happen if the theory is correct (Sullivan and Stewart, 2006). Part of this task is to identify the indicators of short-, medium-and long-term change that will let the evaluators determine what elements they need to form an evaluative judgment. ...
The purpose of this report is to present the background, methodology, and findings of a systematic review of methodologies of evaluative studies of violent extremism prevention programs internationally, published in English, French, and Spanish through December 2019.
From a total of 18,886 documents identified during the initial stage of this search, 219 studies included in 211 publications were selected.
This report is part of the first phase of the PREV-IMPACT Canada project which, supported by the Community Resiliency Fund of the Canadian Centre for Community Engagement and Violence Prevention and Public Safety Canada, aims to develop and implement Canadian models of program evaluation in the area of violent extremism prevention and, ultimately, to build the capacity of key stakeholders in Canada.
... Au-delà de ces apprentissages, les évaluations sont rarement pensées au moment de la conception, voire de la mise en oeuvre, du programme et apparaissent souvent tardivement comme un processus extérieur (pour ne pas dire un corps étranger) et déconnecté de l'initiative de prévention. Enfin, dans la plupart des programmes, l'absence de théories du changement constitue une difficulté additionnelle pour les évaluations (Bellasio et al., 2018 ;Chowdhury Fink et al., 2013 ;Williams et Kleinman, 2014 (Sullivan et Stewart, 2006). Ce modèle d'évaluation fait partie des approches théoriques de l'évaluation et repose sur l'idée que l'évaluateur ou l'évaluatrice doit contribuer à faire émerger une théorie d'action implicite inhérente à une intervention afin de définir ce qui doit se passer si la théorie est correcte (Sullivan et Stewart, 2006 (Mastroe et Szmania, 2016). ...
... Enfin, dans la plupart des programmes, l'absence de théories du changement constitue une difficulté additionnelle pour les évaluations (Bellasio et al., 2018 ;Chowdhury Fink et al., 2013 ;Williams et Kleinman, 2014 (Sullivan et Stewart, 2006). Ce modèle d'évaluation fait partie des approches théoriques de l'évaluation et repose sur l'idée que l'évaluateur ou l'évaluatrice doit contribuer à faire émerger une théorie d'action implicite inhérente à une intervention afin de définir ce qui doit se passer si la théorie est correcte (Sullivan et Stewart, 2006 (Mastroe et Szmania, 2016). ...
Ce rapport vise à présenter le contexte, la méthodologie et les résultats d’une revue systématique portant sur les méthodologies des études évaluatives de programmes de prévention de l’extrémisme violent à l’échelle internationale, publiées en anglais, français et espagnol jusqu’en décembre 2019. Sur un total de 18 886 documents identifiés pendant l’étape initiale de cette recherche, 219 études incluses dans 211 publications ont été retenues.
Ce rapport fait partie de la première étape du projet PREV-IMPACT Canada qui, soutenu par les Fonds pour la résilience communautaire du Centre canadien d’engagement communautaire et de prévention de la violence et Sécurité publique Canada, vise le développement et l’implantation de modèles canadiens d’évaluation des programmes en matière de prévention de l’extrémisme violent et, ultimement, le renforcement des capacités des personnes intervenantes clés au Canada.
... It is clear from the literature that it is advisable to develop a ToC and conduct subsequent reflections with a wide variety of stakeholders to extend ownership of the intervention, its implementation and evaluation [15,17,32]. While our consortium consisted of researchers who also had a role in intervention implementation and scaleup, key stakeholders (user organizations) and end-users (DHMTs) in each of the three countries were not involved in the ToC process, and no country-specific ToCs were developed. ...
... Reflections on the scale-up of the MSI and subsequent actions would probably have been different -more context-specific and maybe more successful -if these people had been involved. In our case, we can speak about 'evaluator ownership' of the ToC [32], mainly because of insufficient time and financial resources to include others in this process. Key stakeholders and endusers in the three countries were only implicitly involved in reflecting on parts of the theory of change when participating in the initial context analysis (which informed ToC development) and outcome and process evaluation interviews or group discussions (which informed annual ToC reflections within the consortium). ...
Background
Since 2017, PERFORM2Scale, a research consortium with partners from seven countries in Africa and Europe, has steered the implementation and scale-up of a district-level health management strengthening intervention in Ghana, Malawi and Uganda. This article presents PERFORM2Scale’s theory of change (ToC) and reflections upon and adaptations of the ToC over time. The article aims to contribute to understanding the benefits and challenges of using a ToC-based approach for monitoring and evaluating the scale-up of health system strengthening interventions, because there is limited documentation of this in the literature.
Methods
The consortium held annual ToC reflections that entailed multiple participatory methods, including individual scoring exercises, country and consortium-wide group discussions and visualizations. The reflections were captured in detailed annual reports, on which this article is based.
Results
The PERFORM2Scale ToC describes how the management strengthening intervention, which targets district health management teams, was expected to improve health workforce performance and service delivery at scale, and which assumptions were instrumental to track over time. The annual ToC reflections proved valuable in gaining a nuanced understanding of how change did (and did not) happen. This helped in strategizing on actions to further steer the scale-up the intervention. It also led to adaptations of the ToC over time. Based on the annual reflections, these actions and adaptations related to: assessing the scalability of the intervention, documentation and dissemination of evidence about the effects of the intervention, understanding power relationships between key stakeholders, the importance of developing and monitoring a scale-up strategy and identification of opportunities to integrate (parts of) the intervention into existing structures and strategies.
Conclusions
PERFORM2Scale’s experience provides lessons for using ToCs to monitor and evaluate the scale-up of health system strengthening interventions. ToCs can help in establishing a common vision on intervention scale-up. ToC-based approaches should include a variety of stakeholders and require their continued commitment to reflection and learning on intervention implementation and scale-up. ToC-based approaches can help in adapting interventions as well as scale-up processes to be in tune with contextual changes and stakeholders involved, to potentially increase chances for successful scale-up.
... By involving the different level stakeholders, ToCs encourage local organisations to take ownership of, and the responsibility for, the course of the interventions (Sullivan and Stewart, 2006). This requires sensitivity to perennial power imbalances within project structures, which in reality is rarely achieved (Ferguson et al., 2010). ...
... Instead, it is thought to reside in the weight of evidence, produced by other members of the community or by the researcher community, independently sifted through external review' (Clegg, 2007, p. 417). This singular logic mindset and expert supremacy, however, is precisely the problem that the theories of change were meant to address by engaging with multiple-level stakeholders (Sullivan and Stewart, 2006). Once an intervention is considered 'successful'-often with the help of an RCT-its theory of change is taken out of its original context and implemented in another through scaling up; losing both its place specificity and its participatory credentials. ...
Theories of change (ToCs) are development tools that describe how a certain set up of conditions and actions leads to producing specific results. They are used to design, monitor, evaluate, and scale interventions and are increasingly required by both donor and research agencies. In this paper, we explore the links between theories of change and actual social scientific theories about change, which have not been explored in academic literature. We make three essential contributions. First, we explain how the rising interest in evidence-based policy fuelled the evolution of development management tools, including Logit Models, Log Frames and theories of change. Second, we argue that the narrow understanding of ‘research evidence’ resulted in scientific contributions being limited to conducting experimental evaluations (RCTs) meant to test and validate the projects’ theories of change. This we criticise as both methodologically unsound and epistemologically limiting. Third, we introduce alternative sources of ‘evidence’ that credit the social science theories and methods and engage meaningfully with local stakeholders (target communities and implementing staff).
... By involving the different level stakeholders, ToCs encourage local organisations to take ownership of, and the responsibility for, the course of the interventions (Sullivan and Stewart, 2006). This requires sensitivity to perennial power imbalances within project structures, which in reality is rarely achieved (Ferguson et al., 2010). ...
... Instead, it is thought to reside in the weight of evidence, produced by other members of the community or by the researcher community, independently sifted through external review' (Clegg, 2007, p. 417). This singular logic mindset and expert supremacy, however, is precisely the problem that the theories of change were meant to address by engaging with multiple-level stakeholders (Sullivan and Stewart, 2006). Once an intervention is considered 'successful'-often with the help of an RCT-its theory of change is taken out of its original context and implemented in another through scaling up; losing both its place specificity and its participatory credentials. ...
This chapter discusses the nature and role of evidence in advocacy by civil society organizations (CSOs) in the context of international development. It focus on CSOs’ efforts at the direct influencing of policymakers. The chapter show the important and diverse roles of evidence in CSO advocacy, while problematizing the supposed objective quality of evidence and its usage. It identifies the ways in which evidence creation and usage is shaped in relational dynamics between CSOs and the policymakers they target, and between CSOs that are differently positioned in power relations. Charting the political dynamics involved in the relations and interactions involved, the chapter calls for a recognition of the implications of these for inclusion, ownership, representation and legitimacy, and presents ways forward drawing on these insights.
... By involving the different level stakeholders, ToCs encourage local organisations to take ownership of, and the responsibility for, the course of the interventions (Sullivan and Stewart, 2006). This requires sensitivity to perennial power imbalances within project structures, which in reality is rarely achieved (Ferguson et al., 2010). ...
... Instead, it is thought to reside in the weight of evidence, produced by other members of the community or by the researcher community, independently sifted through external review' (Clegg, 2007, p. 417). This singular logic mindset and expert supremacy, however, is precisely the problem that the theories of change were meant to address by engaging with multiple-level stakeholders (Sullivan and Stewart, 2006). Once an intervention is considered 'successful'-often with the help of an RCT-its theory of change is taken out of its original context and implemented in another through scaling up; losing both its place specificity and its participatory credentials. ...
This chapter advances a perspective that recognizes diverse epistemic resources that engage with gender in international development. Rather than assuming that knowledge is exclusively produced through a decontextualized academic viewpoint, the chapter highlights the entanglement of knowledge with the material conditions in which it is produced. Building on insights from feminist epistemology, the chapter examines the situated knowledge about the notion of ‘gender equality’ in the CGIAR according to three different sites of knowledge production: institutional documents, gender specialists, and men and women from rural communities. The chapter identifies potential opportunities of embracing a transdisciplinary perspective on gender mainstreaming by expanding the treatment of gender equality into one that acknowledges its contested and context-specific nature. The chapter concludes by highlighting pathways for gender transformation in agricultural research for development work.
... By involving the different level stakeholders, ToCs encourage local organisations to take ownership of, and the responsibility for, the course of the interventions (Sullivan and Stewart, 2006). This requires sensitivity to perennial power imbalances within project structures, which in reality is rarely achieved (Ferguson et al., 2010). ...
... Instead, it is thought to reside in the weight of evidence, produced by other members of the community or by the researcher community, independently sifted through external review' (Clegg, 2007, p. 417). This singular logic mindset and expert supremacy, however, is precisely the problem that the theories of change were meant to address by engaging with multiple-level stakeholders (Sullivan and Stewart, 2006). Once an intervention is considered 'successful'-often with the help of an RCT-its theory of change is taken out of its original context and implemented in another through scaling up; losing both its place specificity and its participatory credentials. ...
Introduction Development projects are based on explicit or implicit assumptions of how and why a certain intervention will work. Over the past three decades, these assumptions have captured the attention of academics and practitioners alike, sparking interest in the so-called theories of change (Brown, 2020; Prinsen and Nijhof, 2015). In the most common sense, theories of change (ToCs) are explanations of how groups of stakeholders expect to reach a commonly understood long-term goal (Brest,
... It can be seen as the foundation of an evaluation, as it shows where the evaluation should focus on and how it can produce relevant information (Es and Guijt 2015). A ToC provides an overview of all relevant different impact goals and shows a visualization how it is expected that the activities consequently will lead to these goals (Sullivan and Stewart 2006). A good ToC includes all expected and direct benefits and assumptions, but also tries to capture the unexpected and indirect, positive and negative consequences and assumptions.Desk research, literature reviews and interviews or dialogues with stakeholders can help to gain understanding of all these expectations and related assumptions. ...
... These assumptions are different hypotheses that can be validated (Ebrahim 2019). After developing the ToC, it is time to validate the underlying logic or theory by means of a dialogue with relevant stakeholders, such as experts or researchers (Sullivan and Stewart 2006). They can help cultural organizations to sharpen the ToC in order to create a ToC that is trustworthy and realistic. ...
Cultural participation is associated with many benefits. Therefore, it is seen as an instrument to achieve social change and development. However, clear empirical proof for these claims is limited. There is a need for new frameworks and clear evaluation methods. This paper presents a conceptual framework that enables cultural organizations to understand the purposes of impact measurement. It also shows the steps cultural organizations can take to measure and manage their social impact. Consequently, it enables them to measure the impact on their beneficiaries while the lessons learned can help them to improve and manage their performance.
... Theory of Change (ToC) approaches were first developed in the US to evaluate complex community initia tives but are increasingly being used in the UK to evaluate policy initiatives (Sullivan and Stewart 2006) To guide the evaluation of the programme the team developed a ToC based on an academic literature review, the Dasgupta Review (Dasgupta 2021) and feedback from the delivery team and programme funder. ...
We are in a time of rapid change on multiple levels. Change can be seen as positive by one group and negative by another. As a result, different perspectives on any given change can draw completely different conclusions. In these proceedings we want to address different approaches to change from all kinds of perspectives within the realm of citizen science and participatory research. We discuss both active, transformative change, and the observation of change monitored by citizen science in all kinds of disciplines. We highlight the potential of citizen science to be a change maker in research and society, and as a tool to manage the change happening around us. The proceedings "Change - The transformative power of citizen science" showcase a selection of topics that have been presented and discussed at the ECSA/ACSC 2024 double conference in Vienna and highlight the transformative power, citizen science can have.
... Theory of Change (ToC) approaches were first developed in the US to evaluate complex community initia tives but are increasingly being used in the UK to evaluate policy initiatives (Sullivan and Stewart 2006) To guide the evaluation of the programme the team developed a ToC based on an academic literature review, the Dasgupta Review (Dasgupta 2021) and feedback from the delivery team and programme funder. ...
The National Education Nature Park aims to involve every nursery, school, and college in England in enhancing the biodiversity on their site, whilst supporting young people’s wellbeing, pro-environmental behaviours, and green skills. Young people gather environmental data using citizen science research, and then through collaboration and collective decision-making, they design and implement their own nature recovery actions. But will this participation in community and citizen science lead to behaviour change and environmental action, and how can we build participants’ sense of agency to take environmental action through our programme? Here, we present our Theory of Change for the Nature Park and the design features of the programme that connect participation in citizen science with achieving two crucial types of change - environmental change in the form of biodiversity gain, and the behaviour change that underpins it.
... This theory examines the underlying causes of exploitation, forced labour, and vulnerabilities among migrant workers and identifies the roles of relevant stakeholders in addressing these issues (Women, U.N. 2017). The TOC was developed through extensive consultations with representatives from civil society, business, and government, as it recognises that no single actor can bring about change on their own (Sullivan & Stewart, 2006). ...
The study aimed to investigate the experiences and exploitation of Ugandan migrant domestic workers who returned from Saudi Arabia. The study followed a constructivist paradigm, an interpretive framework that focuses on individuals' understanding of the world and their creation of unique meanings. A qualitative approach was utilised, specifically employing a phenomenological case study design. Data was gathered through unstructured interviews with twenty-eight (28) migrant domestic workers. Thematic analysis was employed to analyse the data. The findings regarding the lived experiences and exploitation of returnee Ugandan migrant domestic workers were visualised through concept maps created using NVivo version 10.0. The study discovered that the returnee workers' experiences were primarily related to their interactions with various individuals in their employers' households, including dealing with abusive employers, inconsiderate children and relatives, and challenging working conditions. Other experiences involved working in an unfriendly environment, carrying out difficult tasks, facing heavy workloads, experiencing occupational health difficulties, enduring physical abuse, inadequate rest, food deprivation, sexual advances, and false allegations. Consequently, providing legal protection for domestic workers in Saudi Arabia is crucial. Although progress has been made in addressing legal gaps, particularly regarding weekly rest days, more work remains. This includes implementing or expanding minimum wages and aligning working hour restrictions with those applicable to other workers. It is therefore recommended that Uganda and Saudi Arabia revise labour export laws to ensure that the rights of migrant domestic workers are respected.
... Stakeholders' involvement in the development of the theory of change also created a sense of ownership and buy-ins from relevant stakeholders [41]. Ideally, all stakeholders should have ownership over the theory of change [42]. However, due to the complexity of the Project, this is often di cult to execute in practice. ...
Background: Set against the backdrop of a rapidly ageing population and growing emphasis on the importance of ageing-in-place, family members often assume the role of a caregiver. Navigating through a complex healthcare system while simultaneously juggling the daily care needs of their care-recipients, caregivers often become worn out by the intense caregiver stress and burden, neglecting their own well-being. This translates to adverse health and economic outcomes such as prolonged hospital stays and increased nursing home placement of care-recipients. Seeking to better support caregivers, we developed a theory of change to guide the design and implementation of a caregiver support programme – Project Carer Matters.
Methods: We applied theory of change methodology to explain how the Project’s interventions were hypothesised to lead to their identified short-to-long term goals, drawing on a causal analysis based on available evidence. The theory of change was developed with insights garnered from previous research studies conducted on caregiver stress, stakeholder engagement sessions and multiple dialogues with clinical experts and hospital leaders.
Results: Our final theory of change is the result of the evaluation of the Project in its pilot phase. It is populated with the resources, activities and short-to-long term outcomes that can be attributed or linked to the Project. Multiple meetings and discussion with stakeholders over the pilot prompted frequent practice of the Plan, Do, Study, Act model to refine the ongoing implementation process and the theory of change itself.
Conclusions: A theory of change is essential in guiding the design, implementation and evaluation of a complex health care intervention such as Project Carer Matters. The development of the theory of change is a journey and not a resultant product. This journey has also led us to learn that 1) a theory of change needs to be dynamic and ever evolving with time and context, 2) the perspectives of relevant stakeholders need to be included in this process to ensure the feasibility and sustainability of the project in the long run and 3) frequent stakeholder engagements are essential in enabling the implementation team to fine-tune the Project in an effective manner.
Trial Registration: ClinicalTrials.gov, NCT05205135, registered on 24/01/2022.
... Despite its widespread use to explain reforms, the theory of educational change faces conceptual criticisms, such as the fact that it does not prescribe speci c evaluative methods or how theories of change are articulated (Blamey & Mackenzie, 2007;Sullivan and Stewart, 2006). The theory of change is more theoretical than practical in practice (Mason & Barnes, 2007). ...
The most recent policy on the abolition of the student allowance has generated mixed reactions from students and stakeholders. The study examined the effects of the removal of allowance for trainees in colleges of education on access to colleges of education. The study adopted the mixed methods approach utilising sequential explanatory design to collect both quantitative and qualitative. The investigation in which a quantitative phase of data collection and analysis was followed by a qualitative phase. Descriptive statistics were employed in the analysis. The abolition of the allowance for trainees has led to an increase in enrolment and improved infrastructure in the colleges. With regard to the standard of education, the lack of financial assistance for trainee teachers has led to a low level of attendance and learning outcomes as some trainees miss classes to engage in economic activities to support themselves financial whiles they are in the teacher colleges of education. It is also clear from the finding that the removal of the teacher trainee allowance puts additional costs on parents and trainees The reforms have increased access to colleges of education as compared to the quota system. The Government of Ghana should consider abolishing the quota system for student participation in colleges of education. This is because the quota system component of the teacher education allowance scheme is a rather inhibiting factor. Government should also consider decentralizing financing and encouraging distance learning models as an alternative.
... The theory of change can be developed independently through the integration of stakeholders at the initial stage. Involving the relevant decision-makers during the early phase of development will foster a mutual understanding of the processes required for the desired social change (Sullivan and Stewart, 2006). An example of a well-structured theory of change plan is USSE2's target to support 10,000 orphans within five years. ...
In the emerging field of academic research on sustainability, studies on social sustainability within social entrepreneurship are largely underexplored due to the widespread misconception of understanding the distinction between these two concepts. Sustainability in social entrepreneurship is defined as the process of developing sustainable solutions for social, economic, or environmental problems that are not being addressed with the ability to endure over time by continually improving on operational efficiencies. Social entrepreneurial sustainability should cover the three sustainability dimensions, which are social sustainability, economic sustainability, and environmental sustainability. This study aims to explore the dimension of social sustainability within social entrepreneurship specifically. The study seeks to understand how social entrepreneurs achieve social sustainability. This study employs an exploratory qualitative approach based on case studies of social enterprises operating in the United States. The findings reveal various themes such as value creation, establishing smart partnerships and social innovation that social entrepreneurs undertake to be socially sustainable. This study looks to provide significant empirical findings that will be invaluable to promote further research in this nascent field of social sustainability within social entrepreneurship that will benefit academicians and practitioners worldwide.
... By involving the relevant decisionmakers during the early phase of development, it will foster a mutual understanding of the processes required for the desired social change. In addition, areas of responsibility may be identified, assigned, and delegated to improve program development (Sullivan & Stewart, 2006). Even though their initial reports indicate that theory of change framework is gaining importance in the program design stage by utilising resources in a more organised manner, there are still setbacks encountered in employing the theory of change framework. ...
In the emerging field of academic research on social entrepreneurship, studies on social entrepreneurial motivation and sustainability are largely underexplored due to the complexity of understanding the connection between these two constructs. This study aimed to provide theoretical, practical, and empirical evidence of how social entrepreneurs are motivated to operate their social ventures structured in an entrepreneurial and business-like manner to achieve sustainability. This study employed an exploratory qualitative approach based on case studies of social enterprises operating in Malaysia and the United States. This study explored the motivational factors related to critical dimensions such as social, economic, behaviour, and governance that influenced the CEOs and founders of social enterprises operating their unique entities to attain sustainability for their ventures. The theory of change and the logic model are business processes that were investigated and extended into the social impact measurement using the social return on investment or the balanced scorecard approach. The empirical findings indicate motivational factors that drove the studied social entrepreneurs to pivot, forge new partnerships, and create social and technical innovation to ensure continued sustainability for their social enterprises. The cross-comparison study revealed similarities and differences between these two countries. The development of a new concept termed ‘mission agility’ is another pivotal contribution that will benefit academicians worldwide. Hence, this study provided pioneering work in the theoretical development of linking motivation to sustainability in social entrepreneurship.
Practitioners and policymakers will be able to utilise the conceptual framework established by this study to navigate through the current economic landscape that will enable social enterprises to be sustainable in facing the post-pandemic era. This study delivered invaluable theoretical, practical, and empirical contributions to social entrepreneurship and sustainability.
... To deal with the attribution problem, eminent scholars in public policy and strategy have suggested adopting cause-and-effect models (Ansoff, 1965;Chandler, 1962;March & Simon, 1985;. Such a recommendation that has led several authors to develop causation approaches and models to link policy program goals to policy outcomes (Barnabè et al., 2013;Eden & Ackermann, 2004;Kaplan & Norton, 1992;Rosemberg & Posner, 1979;Saaty, 1990;Sullivan & Stewart, 2006). ...
This chapter will show how DPM may provide the actors in a public value-driven performance regime with the methodological framework to implement policy learning for assessing community outcomes, illustrated by different examples. Before presenting the empirical work, the following sections will discuss the main methodological challenges associated with outcome-based performance assessment to set the field for showing how DPM may implement policy learning in public value-driven performance regimes.
... To deal with the attribution problem, eminent scholars in public policy and strategy have suggested adopting cause-and-effect models (Ansoff, 1965;Chandler, 1962;March & Simon, 1985;. Such a recommendation that has led several authors to develop causation approaches and models to link policy program goals to policy outcomes (Barnabè et al., 2013;Eden & Ackermann, 2004;Kaplan & Norton, 1992;Rosemberg & Posner, 1979;Saaty, 1990;Sullivan & Stewart, 2006). ...
... It is also recommended for complex interventions (Rogers, 2008). The main weaknesses of ToC are related to the oversimplification of reality, the omission of externalities, the consumption of time and resources, and the debate over who owns ToC (Sullivan & Stewart, 2006) or subjectivity or context-specific analysis. Moreover, 'it should be noted that the application of ToCs are based on a deeper assumption of development and therefore do not replace the critical understanding of development perspectives' (Zheng et al., 2018, p. 6). ...
Information and Communication Technologies (ICT) are considered a cross-cutting tool that contributes to meeting the global challenges set out in the Sustainable Development Goals (SDGs). However, in many countries, there is still a significant connectivity gap between cities and rural areas. Using a case study approach and the Digital-for-development paradigm proposed by Heeks, this paper explores an innovative strategy for addressing the rural connectivity gap and examines its impact on the SDGs. The model under analysis is the Rural Mobile Infrastructure Operator (RMIO). The specific case analyzed is the first company operating under an RMIO figure and offering services in underserved rural areas of Peru. The results show that the RMIO strategy primarily contributes to some specific targets of SDGs 3, 9, and 17. Key stakeholders can use the methodology and results of this study to develop strategies to address the connectivity divide and promote the achievement of the SDGs in rural contexts. .
... Il y a plusieurs types d'évaluation sur la théorie. Nous avons choisi la méthode de la théorie du changement (Théorie of change -ToC) telle que décrite par De Silva (96,98,99). En effet, la ToC décrit les composantes interventionnelles, les résultats ...
Les Objets connectés et les applications santé (Ocas) sont de plus en plus présents dans le quotidien pour influencer les comportements de santé dont un déterminant principal est la nutrition (alimentation et l’activité physique). Bien que la connaissance des pratiques liées aux Ocas se précise, leurs conditions d’efficacité sur les modifications de comportements restent peu explorées. Dans ce travail il s’agit d’identifier les composantes et les mécanismes efficaces à mobiliser dans la réalisation d'un Ocas afin d’accompagner le changement de comportement en prenant en compte les Inégalités Sociales de Santé (ISS) et les conditions de viabilité nécessaires à leur utilisation. Ces éléments permettront aux professionnels qui les conçoivent et les conseillent mais aussi aux utilisateurs d’avoir un outil d’analyse de leur efficacité. Pour cela nous avons mis en place une démarche structurée participative de construction de consensus entre praticiens et utilisateurs en se basant sur les références suivantes : les recommandations du Medical Research Council (MRC), la théorie du changement et l’approches des changements de comportement de S. Michie avec les « Behavior Change Techniques » (BCTs) et la « Behavior Change Wheel ».Nous avons ainsi élaboré un cadre d’analyse comprenant : - Une théorie d’intervention formalisée sous forme d’une grille d’analyse des Ocas nutrition : la « Grille de changement de comportement ». - Une liste de critères d’analyse de la prise en compte des ISS dans les Ocas sous forme d’une grille d’analyse : la « Grille de critères ISS ». - Une liste de critères d’analyse de la viabilité sous forme de grille d’analyse : la « Grille de critères viabilité ».Le modèle créé avec les trois grilles est complémentaire des approches du type Health Technology assessment. Il est adaptable en fonction des objectifs et des moyens disponibles. Il peut être utilisé pour concevoir des Ocas, analyser des Ocas existantes ou encore, en l’adaptant sous forme d’une grille simplifiée, pour guider les utilisateurs à choisir un Ocas correspondant à leur besoin. Il reste à en tester la praticité au sein de différents usages.
... It can be achieved by using a wide array of data collection and analysis methods. Subsequently, areas of responsibilities may be identified, assigned, and delegated to improve program development (Sullivan & Stewart 2006). ...
This study examined the relationship between social entrepreneurship and sustainability among owner-managed hotels in Rivers State, Nigeria. Cross sectional research design was adopted. Our respondents were owner-managers constituting the population of the study. From the field survey, we retrieved and analyzed seventy-four copies of questionnaire from the participants; Spearman's rank order correlation coefficient statistical tool was used to determine the relationship existing between the variables while the p-value obtained was used to test hypotheses developed for the study. The findings demonstrated a favorable significant association between the social entrepreneurship dimension of social innovation and the sustainability indicators of economic and environmental sustainability. The researchers came to the conclusion that social entrepreneurship has a strong link to long-term sustainability. This gave rise to our recommendations: that social entrepreneurs to develop their capabilities in resource bricolage, the art of maximizing the outcomes derived from the available resources, in order to achieve their goals over the long term.
... Finally, this endeavor requires (1) an adaptive development mode that supports reflective practice and organizational learning (Ling 2012), inclusion, and sense-making with multiple competing explanations of why and how change happens and (2) an understanding of the project's Theory of Change (Sullivan and Stewart 2006) to identify the key dependencies upon systems and subsystems which lie outside the formal structures of the intervention. ...
This article reports on the experiences and results of the European Union-funded Horizon 2020 project TCBL which has been successful in creating a European network of Textile and Clothing Business Labs aimed at the sustainable transformation of one of the most problematic industries in both social and environmental terms. The approach followed by the project was based on the diffused creation of value by and for all stakeholders, including consumers. This, in turn, implies a systemic transformation of business models, brought about by all players in the sector engaging in the experimentation of new processes and transaction patterns. In this way, all stakeholders were able to reap the benefits of innovation, and the lever of competitive advantage shifted from price to knowledge, collaboration, and shared values. In the meantime, the COVID-19 pandemic has had a devastating effect on the business models of the luxury and fast-fashion brands for which TCBL has aimed to offer an alternative path, also loosely in line with the provisions of the European Green Deal and the United Nations 2030 Agenda. Given the results attained, a two-pronged strategy for the constitution of a sustainable post-project TCBL ecosystem is now being implemented.
... Designing a unified theory for the whole organization can be an enormous, if not impossible, task and may not be the best option, especially if the ToC is to be implemented in different contexts and levels. A further consideration is ownership of the process, "total ownership" being the ideal in which all stakeholders are engaged in developing and evaluating the ToC (Sullivan and Stewart 2006). ...
Transparency is now a core principle in environmental and resource governance. Responding to calls for a clearer identification of pathways from transparency to effective change, this article identifies three “Theories of Change” for governance-by-disclosure and applies them to the Extractive Industries Transparency Initiative (EITI). Among the best known global transparency initiatives, the EITI has used an inclusive multistakeholder governance model and elaborate compliance standards, disclosing trillions of dollars in natural resource revenues. Yet, after two decades, the EITI is still largely without an explicit and proven theory. This study finds that a Theory of Change for the EITI is possible, valuable, and even necessary as the EITI risks becoming obsolete in some participating countries. The proposed Theories of Change provide valuable templates for environmental and resource governance, yet such models need to reflect national contexts, needs, challenges, and objectives to ensure fit and effective implementation, including measures enforcing accountability.
... It can be achieved by using a wide array of data collection and analysis methods. Subsequently, areas of responsibilities may be identified, assigned, and delegated to improve program development (Sullivan and Stewart 2006) Even though their initial reports indicate that theory of change framework is gaining importance in the program design stage, there are still setbacks encountered in employing the theory of change framework. They cited that the evaluation process remains widely regarded as an afterthought activity. ...
In the emerging field of academic research on social entrepreneurship, studies linking social entrepreneurship to sustainability is gaining interest due to the importance of connecting these two constructs. The purpose of this study is to propose an updated conceptual framework that links social entrepreneurship to sustainability. A literature review of social entrepreneurship journals was conducted, resulting in identifying four key dimensions in social entrepreneurship which are social, economic, behaviour and governance. These four dimensions have been categorised to have an effect on social entrepreneurship and sustainability. Additionally, the theory of change and logic model are business processes that have been identified, extending into the measurement of social impact by using either the social return on impact or balanced scorecard approach to complete the conceptual framework. Hence, this study is envisioned to provide original work in the theoretical development of linking social entrepreneurship to sustainability that will benefit academicians and practitioners alike.
... Given this complexity, we contructed the aims of the PRiSE programme through a theory of change (TOC; Sullivan and Stewart, 2006). This approach is recommended by several organisations and has been applied to other STEM outreach/engagement programmes (e.g. . ...
Physics in schools is distinctly different from, and struggles to capture the excitement of, university research-level work. Initiatives where students engage in independent research linked to cutting-edge physics within their school over several months might help mitigate this, potentially facilitating the uptake of science in higher education. However, how such initiatives are best supported remains unclear and understudied. This paper evaluates a provision framework, `Physics Research in School Environments' (PRiSE), using survey data from participating 14-18 year-old students and their teachers to understand their experience of the programme. The results show that PRiSE appears to provide much more positive experiences than typical university outreach initiatives due to the nature of the opportunities afforded over several months, which schools would not be able to provide without external input. The intensive support offered is deemed necessary, with all elements appearing equally important. Based on additional feedback from independent researchers and engagement professionals, we also suggest the framework could be adopted at other institutions and applied to their own areas of scientific research, something which has already started to occur.
... 34 A clearly elucidated TOC not only improves the chances that an intervention affects deforestation 35 by helping ''identify assumptions, enabling factors and stumbling blocks,'' 34 but also builds policy legitimacy by engaging stakeholders in a discussion on how to best achieve a given end. 35, 36 Here, we draw upon theoretical and empirical evidence to map the TOC underlying each of the identified policy options to facilitate a discussion of the potential impact of the identified policies on reducing imported deforestation. ...
Despite the importance of tropical forest conservation in achieving global sustainability goals and the key role of forest-risk commodity trade in driving deforestation, consumer country policy options for reducing imported deforestation have received limited scholarly attention. Drawing on gray literature and a European Commission public consultation, we identify 86 policy options for the European Union to address deforestation. We assess the political feasibility and map the “theory of change” (TOC)—the causal chain through which the policies address deforestation—for each of these policy options, identifying a trade-off between feasibility and potential impacts: information-based and cooperative policies, which dominate our sample, typically exhibit high feasibility, but mostly lack convincing TOCs, while more stringent regulatory and market-based policy options generally have lower feasibility. We propose three principles for overcoming the feasibility-impact dilemma: (1) build policies on proven TOCs, (2) use policy mixes, and (3) work with key stakeholders, supply chains, and regions.
... One way to overcome this temptation is to use a Theory of Change approach. This was initially developed to evaluate complex initiatives (Sullivan and Stewart, 2006) and support long-term behaviour change. A key benefit of using a Theory of Change is that it starts with the end goal, and then requires the identification of intermediate outcomes through a process of backwards mapping that will, over time, lead to that goal. ...
A major focus in the science, technology, engineering and mathematics (STEM) public engagement sector concerns engaging with young people, typically through schools. The aims of these interventions are often to positively affect students’ aspirations towards continuing STEM education and ultimately into STEM-related careers. Most school engagement activities take the form of short one-off interventions that, while able to achieve positive outcomes, are limited in the extent to which they can have lasting impacts on aspirations. In this paper, we discuss various different emerging programmes of repeated interventions with young people, assessing what impacts can realistically be expected. Short series of interventions appear also to suffer some limitations in the types of impacts achievable. However, deeper programmes that interact with both young people and those who influence them over significant periods of time (months to years) seem to be more effective in influencing aspirations. We discuss how developing a theory of change and considering young people’s wider learning ecologies are required in enabling lasting impacts in a range of areas. Finally, we raise several sector-wide challenges to implementing and evaluating these emerging approaches.
... ToC is distinct from other approaches as it not only focuses on how to create social change by empowering individuals, but it also makes explicit the causal pathways through which change may happen [14,15]. The output of ToC workshops, the ToC map, is a theory-supported hypothetical visual pathway which demonstrates how a public health intervention can bring about specific long-term change through a logical sequence of intermediate outcomes [16][17][18]. This paper describes the process and outcomes of using the ToC approach to involve stakeholders to identify gaps in the implementation of the SMHP, and how these gaps could be addressed by developing contextually relevant interventions and harnessing multi-sectoral partnerships to deliver the SMHP at-scale in the public schools of rural Rawalpindi, Pakistan. ...
Background
Ninety percent of children with mental health problems live in low or middle-income countries (LMICs). School-based programs offer opportunities for early identification and intervention, however implementation requires cross-sector collaboration to assure sustainable delivery of quality training, ongoing supervision, and outcomes monitoring at scale. In Pakistan, 35% of school-aged children are reported to have emotional and behavioral problems. As in many other LMICs, the government agencies who must work together to mount school-based programs have limited resources and a limited history of collaboration. The “Theory of Change” (ToC) process offers a way for new partners to efficiently develop mutual goals and long-term prospects for sustainable collaboration.
Objective
Develop a model for scale-up of school based mental health services in public schools of Pakistan.
Methods
We used ToC workshops to develop an empirically supported, ‘hypothesized pathway’ for the implementation of WHO’s School Mental Health Program in the public schools of rural Pakistan. Three workshops included 90 stakeholders such as policy makers from education and health departments, mental health specialists, researchers, head teachers, teachers and other community stakeholders including non-governmental organizations.
Results
The ToC process linked implementers, organizations, providers and consumers of school mental health services to develop common goals and relate them (improved child socioemotional wellbeing, grades and participation in activities) to interventions (training, monitoring and supervision of teachers; collaboration with parents, teachers and primary health care facilities and schools). Key testable assumptions developed in the process included buy-in from health care providers, education officials and professionals, community-based organizations and families. For example, teachers needed skills for managing children’s problems, but their motivation might come from seeking improved school performance and working conditions. Poverty, stigma and lack of child mental health literacy among teachers, administration, and parents were identified as key hypothesized barriers. Children and their families were identified as key stakeholders to make such a program successful.
Discussion
ToC workshops assisted in team building and served as a stakeholders’ engagement tool. They helped to develop and support testable hypotheses about the structures, collaborations, and knowledge most important to scaling-up school based mental health services in Pakistan.
... This exclusion is commonly mirrored within conservation contexts (Armitage et al., 2020). Therefore, the perceived "sense-of-ownership" of a ToC, and its development process, has implications for how the ToC is received by actors and the intervention's success (Koleros & Mayne, 2019;Sullivan & Stewart, 2006;Van Tulder & Keen, 2018). ...
Contemporary conservation must address social well‐being while still protecting biodiversity. Accordingly, the objective of the Convention on Biological Diversity's recent Zero Draft Post‐2020 Global Biodiversity Framework is to sustainably meet the needs of people while reducing biodiversity loss. However, frequent “failures” in achieving this social‐ecological balance necessitates more holistic, systematic, and adaptive post‐2020 conservation interventions. The Theory of Change (ToC) approach provides a useful and flexible tool to support this endeavor. However, debate persists over its usefulness, and “best” manner of use. This paper explores the elements of, and proposes a framework for developing robust conservation ToC pathways. The framework emphasizes the importance of producing a shared vision of desired results and actions, and associated causal assumptions, among actors. Furthermore, evaluation is considered key to informing required ongoing adaptation to better achieve desired results. The paper also critically explores the challenges associated with ToC, and makes recommendations for its improved use in post‐2020 conservation. In particular, we aim to inform the implementation and mainstreaming of the Post‐2020 Global Biodiversity Framework, especially at a national‐ and local‐level. The framework and discussion should be relevant to a broad range of conservation actors at various scales that must address linked social and ecological objectives.
... Archer et al., Under Review, and references therein). 145 Given this complexity, we detail the aims of the PRiSE programme through a theory of change (Sullivan and Stewart, 2006). ...
We introduce a scalable framework for protracted research-based engagement with schools called Physics Research in School Environments (PRiSE) which has transformed cutting-edge space science, astronomy, and particle physics into accessible 6-month independent research projects for schools. The programme's theory of change presents how PRiSE aims to impact on a diverse range of 14–18 year-old students, supporting and enhancing their physics aspirations, as well as influencing teachers' practice and their school environments to potentially enable wider impacts. We explore the considerations made in developing the programme to help enact these theorised changes, in particular detailing the structure, support, and resources offered by active researchers as part of PRiSE. Through feedback from participating students and teachers, we assess the provision within this framework. This illustrates that the model appears to provide highly positive experiences that are otherwise not accessible to schools and that the extraordinary level of support offered is deemed necessary with all elements appearing equally important. Researchers and public engagement professionals seem receptive to the PRiSE framework of schools engagement and it has started to spread to other institutions.
... A TOC explains how and why a given policy intervention is expected to achieve change, detailing "who will do what differently and why" . A clearly elucidated TOC not only improves the chances that an intervention will be effective in reducing deforestation (Garcia et al., 2020), by helping "identify assumptions, enabling factors and stumbling blocks" but builds policy legitimacy by engaging stakeholders in a discussion on how to best achieve a given end (Garcia et al., 2020;Sullivan and Stewart, 2006). In addition, a clear TOC is crucial for designing impact evaluations of forest conservation policies (Baylis et al., 2016;Garcia et al., 2020) by identifying confounders and hypotheses. ...
... A TOC explains how and why a given policy intervention is expected to achieve change, detailing "who will do what differently and why" . A clearly elucidated TOC not only improves the chances that an intervention will be effective in reducing deforestation (Garcia et al., 2020), by helping "identify assumptions, enabling factors and stumbling blocks" but builds policy legitimacy by engaging stakeholders in a discussion on how to best achieve a given end (Garcia et al., 2020;Sullivan and Stewart, 2006). In addition, a clear TOC is crucial for designing impact evaluations of forest conservation policies (Baylis et al., 2016;Garcia et al., 2020) by identifying confounders and hypotheses. ...
Consumer countries play an important role in driving tropical deforestation through imports of forest risk commodities (FRCs), but their role in reducing deforestation has received limited scholarly attention. Drawing on suggestions from grey literature and a European Commission (EC) public consultation, we identify 1,141 policy proposals for the EU and other consumer countries to address tropical deforestation, which we summarize into 86 unique policy options. Two-third of these are informational and supportive policies, while regulatory and especially market-based options are less-often proposed. We assess the political feasibility and map the 'theory of change' (TOC) through which the policies address tropical deforestation. Several feasible options exist, though most of these rely on less coercive-informational and cooperative-policy instruments. Most policy proposals lack an explicit and proven TOC, reducing their potential impact on reducing deforestation. Due diligence and multi-stakeholder fora stand out as policies that are feasible, yet rest on convincing TOCs. To increase the feasibility and impact of consumer-country deforestation policy options, we propose three principles: i) Build policies on a clear theory of change, ii) apply policy packaging and sequencing, and iii) work with stakeholders in key supply-chains and regions, broadening scope over time.
... The 'theory of change' originated from the Aspen round table (Connell & Kubisch,1999;Anderson, 2005) to evaluate the impact of policy (Sullivan & Stewart, 2006 (Taplin et al., 2013, pp.2). An explicit theory of change can be beneficial as it makes the intention of practices, for example, a programme direction, available for scrutiny (Connell & Kubisch, 1999;Funnell & Rogers, 2011). ...
Making value judgements on one’s academic work is a fundamental skill, particularly for those within the higher education sector. Evaluative practices such as reflection in and on action (Schön, 1987; Cowan, 2006) and surveys on teaching effectiveness (Jones, Gaffney-Rhys and Jones, 1987; Praslova, 2010) are often proposed as a means of gathering evidence. The aim of this paper is to ask what theories of change (ToC) (Connell and Kubisch, 1999), selfevaluative practices, and barriers to effective implementation could be identified and what level of freedom and autonomy lecturers experience in evaluating their own work. This small study used a RUFDATA evaluative tool (Saunders, 2000) with sixteen full-time lecturers and took place in an Institute of Technology (IoT) in Ireland. Five theories of change were identified (1) transformative focused, (2) profession focused, (3) discipline focused, (4) workplace focused and (5) module focused, along with six self-evaluation practice themes and nine barriers to effective practice. Utilising Bamber’s (2011b) discretion framework, participants were found to have autonomy in choosing what and how to evaluate their practice. The practical knowledge (Habermas, 1971) presented here offers an alternative approach to how academic work is ‘valued’, judged and understood from the lecturer’s perspective. This paper proposes the view of self-evaluation, not as a set of standalone practices, but as part of a sequential process arising from a lecturer’s purposeful academic intentions. The theories of change are used to judge the value of their work within the setting. Therefore, the proposed categories can offer a discourse to justify specific self-evaluative practices for quality assurance and enhancement.
... It also takes into account key recommendations of global experts who endorse that high-quality EmOC should be universally available and accessible, that all women should deliver their infants in the presence of a professional, skilled birth attendant, and that these key services should be integrated into health systems (192). reported that the participation of stakeholders can vary substantially in practice (242). ...
Primary health care (PHC) is usually the first point of contact people have with a country's health system. The aim of PHC is to provide comprehensive, accessible, community-based care that meets the health needs of individuals throughout their lifetime. Humanitarian emergencies (HEs) are characterised by an inability of affected populations to cope with an event using their own resources. In HEs, international actors often provide humanitarian assistance to affected populations. The majority of health services are delivered at the PHC level. In fact, approximately 90% of the activity of the largest humanitarian medical actor, Medecins sans Frontieres (MSF), is conducted at the PHC level each year. Despite the significance of PHC activity in HEs, there are currently no established guidelines for the humanitarian community on service implementation, particularly related to the context, national health system capacity and the expectations of affected communities. The overarching aim of this thesis is to provide empirical evidence to describe the PHC system of a HE from the perspective of international actors. To achieve this aim, I addressed three key research questions: 1. What is currently known about how PHC services are delivered in HEs by international actors? 2. How do key concepts of PHC apply in a HE? and 3. What does a health systems approach look like with respect to PHC delivery in a HE? To answer the first question, I undertook a scoping narrative literature review of peer-reviewed literature from 1978 to 2016 and grey literature from 2013 to 2018. I included primary reports of PHC interventions delivered in the acute phase of a HE by international actors, and analysed these interventions against an existing PHC framework. I found that the PHC system collapses during a HE, that international actors delivered PHC services according to their own capacity, setting their own aims and objectives, and that little consideration was given to community empowerment in service delivery. I used field visits to MSF projects in northern Nigeria and Lebanon as case studies to answer the second research question. In northern Nigeria, a visit and realist analysis of a MSF maternal health care project highlighted the importance of understanding the 'context' of an intervention, particularly the role of PHC in comprehensively addressing maternal and neonatal mortality and morbidity. In the Bekaa valley region of Lebanon, I used routinely collected patient data on non-communicable diseases (NCDs) from four MSF PHC clinics to investigate the concepts of geographic accessibility, availability of facilities, adjustment to population health needs and continuity of care for patients seeking NCD care. I found that access to care was dependent on context, that there was a relationship between continuity of care and access to a clinic, and that humanitarian access plays a key role in these settings. The final research question was answered using principles of complex adaptive systems theory and the findings from the case studies. I developed a conceptual framework to explain the dynamic relationship between the national health system of a country, the system created by international actors and that of individuals and communities affected by a HE. The findings presented in this thesis have important implications for practice and further research. International health actors working in HEs need to better understand the context in which PHC services are delivered to provide effective and relevant health care. The principles of PHC are relevant in HEs, however they need to be adapted. If we are to achieve the goals of the Declaration of Astana to 'leave no one behind', we must place greater emphasis on understanding the inter-dependent relationships between the national health system, international actors and communities themselves.
... One way to overcome this temptation is to use a Theory of Change approach. This was initially developed to evaluate complex initiatives (Sullivan and Stewart, 2006) and support long-term behaviour change. A key benefit of using a Theory of Change is that it starts with the end goal, and then requires the identification of intermediate outcomes through a process of backwards mapping that will, over time, lead to that goal. ...
A major focus in the STEM public engagement sector concerns engaging with young people, typically through schools. The aims of these interventions are often to positively affect students' aspirations towards continuing STEM education and ultimately into STEM-related careers. However, most schools engagement remains in the form of short one-off interventions, which obviously have limited scope for real lasting impact. In this paper we discuss various different programmes of repeated interventions with young people, assessing what impacts can realistically be expected. Short series of interventions, rather than one-offs, appear to also suffer some limitations in the types of impacts achievable. However, deeper programmes that interact with both young people and those that influence them seem to be more effective. We discuss how developing a Theory of Change and considering young people's wider learning ecologies are required in enabling such lasting impacts. Finally, we raise several sector-wide challenges to implementing and evaluating these emerging approaches.
... We produced an alternative, robust TOC for the Better-Birth intervention (Fig. 2) using arrows to designate hypothesized causal relationships, shading to identify the desired unit of analysis for each variable, and superscripts to identify which variables were measured in each Better-Birth development phase. This retrospective TOC was developed primarily by program evaluators and implementors, but prospectively engaging community members and frontline healthcare providers can provide additional insight into local context and enhance community buy-in [41,42]. Many variables identified in this TOC as playing important roles in the BetterBirth Program, such as birth attendant ability, were not measured during the development phases. ...
Background:
Despite extensive efforts to develop and refine intervention packages, complex interventions often fail to produce the desired health impacts in full-scale evaluations. A recent example of this phenomenon is BetterBirth, a complex intervention designed to implement the World Health Organization's Safe Childbirth Checklist and improve maternal and neonatal health. Using data from the BetterBirth Program and its associated trial as a case study, we identified lessons to assist in the development and evaluation of future complex interventions.
Methods:
BetterBirth was refined across three sequential development phases prior to being tested in a matched-pair, cluster randomized trial in Uttar Pradesh, India. We reviewed published and internal materials from all three development phases to identify barriers hindering the identification of an optimal intervention package and identified corresponding lessons learned. For each lesson, we describe its importance and provide an example motivated by the BetterBirth Program's development to illustrate how it could be applied to future studies.
Results:
We identified three lessons: (1) develop a robust theory of change (TOC); (2) define optimization outcomes, which are used to assess the effectiveness of the intervention across development phases, and corresponding criteria for success, which determine whether the intervention has been sufficiently optimized to warrant full-scale evaluation; and (3) create and capture variation in the implementation intensity of components. When applying these lessons to the BetterBirth intervention, we demonstrate how a TOC could have promoted more complete data collection. We propose an optimization outcome and related criteria for success and illustrate how they could have resulted in additional development phases prior to the full-scale trial. Finally, we show how variation in components' implementation intensities could have been used to identify effective intervention components.
Conclusion:
These lessons learned can be applied during both early and advanced stages of complex intervention development and evaluation. By using examples from a real-world study to demonstrate the relevance of these lessons and illustrating how they can be applied in practice, we hope to encourage future researchers to collect and analyze data in a way that promotes more effective complex intervention development and evaluation.
Trial registration:
ClinicalTrials.gov, NCT02148952; registered on May 29, 2014.
... ToC approaches were initially developed in the US as a way of evaluating complex community initiatives (Sullivan and Stewart 2006) but have been used in the UK as a way to evaluate policy initiatives, such as Full Service Extended Schools (Dyson and Todd 2009). ...
There is concern about the low numbers and diversity of young people choosing careers and study subjects in science, technology, engineering and maths (STEM) at university and beyond. Many interventions aimed at addressing this issue have focused on young people aged 14+ years old. However, these interventions have resulted in little improvement in the numbers and diversity of young people progressing into STEM careers. The aim of this study is to ask “What are the affordances of a Theory of Change (ToC) for increasing the diversity and number of young people choosing a career in STEM post-18?” An innovative ToC is introduced which provides the theoretical underpinnings and context for the complex mix of interventions necessary to lead to a significant change in the number and diversity of those choosing STEM careers. Case studies of interventions developed using the ToC are presented. This approach, and associated ToC, is widely applicable across STEM, education and public engagement fields.
... The above can be represented in a ToC map or narrative for testing. The theory-building process [26] is one that is cumulative and iterative, combining scientific data from the literature (empirical and theoretical) with knowledge from experts (researchers and practitioners) and from patients or users [27]. ...
Background:
Connected health devices and applications (referred to hereafter as "SDApps" - Smart devices and applications) are being portrayed as a new way for prevention, with the promise of accessibility, effectiveness and personalization. Many effectiveness evaluations (experimental designs) with strong internal validity exist. While effectiveness does appear to vary, the mechanisms used by these devices have not yet been thoroughly investigated. This article seeks to unpack this black box, and describes the process of elaboration of an intervention theory for healthy eating and physical activity SDApps. It includes a set of requirements relative to their impact on social health inequalities.
Methods:
To build this theory, we drew on theory-driven approaches and in particular on the theory of change (ToC) method. To this end, we developed a cumulative and iterative process combining scientific data from the literature with knowledge from experts (researchers and practitioners) and from patients or users. It was a 3-step process, as follows: 1 - identifying the evidence base; 2 - developing the theory through design intervention and creating realistic expectations, including in our case specific work on social health inequalities (SHIs); 3 - modeling process and outcome.
Results:
We produced an evidence-based theory according to the ToC model, based on scientific evidence and knowledge from experts and users. It sets out a causal pathway leveraging 11 key mechanisms - theoretical domains - with which 50 behavior change techniques can be used towards 3 ultimate goals: Capacity, Opportunity, Motivation - Behavior (COM-B). Furthermore, the theory specifically integrates requirements relative to the impact on SHIs.
Conclusions:
This theory is an aid to SDAapp design and evaluation and it can be used to consider the question of the possible impact of SDApps on the increase in inequalities. Firstly, it enables developers to adopt a more overarching and thorough approach to supporting behavior change, and secondly it encourages comprehensive and contributive evaluations of existing SDApps. Lastly, it allows health inequalities to be fully considered.
... Theory of change is a participatory technique for the evaluation of traditionally ''hard to measure'' community interventions (Center for Theory of Change 2017), which has become popular in international development, among NGOs, philanthropies, and corporate social responsibility programs. The framework enjoyed a particularly strong uptake in the UK's public sector (Blamey and Mackenzie 2007;Sullivan and Stewart 2006) and it is from here that it entered into Global Mental Health. The Department for International Development (DFID) became a strong advocate of Theory of Change around 2010 (Vogel 2012) and recommended its use in all of their sponsored interventions. ...
Global Mental Health has developed interventions that strive to work across great difference—variously conceptualized as cultural, socio-economic, geographic, or pertaining to the characteristics of health systems. This article discusses how the evaluation framework Theory of Change (ToC) facilitates the production of ‘global’ knowledge across such differences. Drawing on 14 months of multi-sited fieldwork among Global Mental Health actors in Europe, North America and South Africa, it traces the differential use of ToC in GMH interventions. While much critical scholarship of Global Health metrics holds that techniques of quantification rely on universals that necessarily betray the “real world”, ToC unsettles these critiques. It comes into view as an epistemic and relational device that produces ‘contingent universals’—concepts that are true and measurable until they stop working in the field, or until the parameters of ‘what works’ shift to a new iteration. As such, Theory of Change produces actionable—rather than true—knowledge attuned to open-ended change, both desirable (impact) and unforeseen (adaptation). Its effects, however, are ambiguous. ToC presents us with a horizoning technique that enables what I call “virtuous failure” within the evidence-based paradigm. It may equally harbor the potential to disrupt distinctions such as bricolage (tinkering) and design (planning) and their respective politics, as it may tie neatly into audit cultures, depending on its use. The article analyzes the novel stakes of reflexive evaluation techniques and calls on anthropology and critical Global Health for renewed empirical engagement.
Despite numerous initiatives made by various organisations to increase employment opportunities for individuals with autism spectrum disorder (ASD), they remain unemployed. The unemployment rate for individuals with ASD remains high. Therefore, this study seeks to understand the employment challenges for individuals with ASD and develop innovative solutions to address them. This study selected six founders of social enterprises in Klang Valley, Malaysia, that target individuals with ASD as participants. The study uses a face-to-face interview as a means of data collection. By implementing the thematic analysis, the study will uncover key themes and patterns that can inform future initiatives and policies to promote inclusion and diversity in the Malaysian workforce. These findings will contribute to a more comprehensive understanding of the challenges faced by marginalised groups and the strategies that can be implemented to address them effectively.
Development banks, non-governmental organizations, and international institutions regularly implement projects in low-income countries to assist them in their development process. These projects, often called “programs,” are of different natures and target different sectors in the economy, as well as different aspects of society or different demographic groups. This chapter discusses econometric methods based on panel data to evaluate the impact of such programs. Those who participate or benefit from these programs are called the beneficiaries. Often, the evaluation consists of comparing the change in a target outcome (e.g., income) between the beneficiaries and a control group, often called the counterfactual. To do so, both groups should be comparable; therefore, careful attention should be paid when choosing both the beneficiaries and the counterfactual, while avoiding selection bias. This chapter further discusses the sampling strategy for selecting both groups, including the calculation of the sample size, propensity score matching methods to define the final evaluation sample, and ways to handle potential selection bias in panel data using difference-in-difference techniques. The chapter also shows implementation of these techniques in Stata software using a case study and data from the “Training for Rural Economic Empowerment (TREE)” program implemented in Zimbabwe in 2010 by the International Labour Organization.
Background:
Adult patients admitted to intensive care units in the terminal phase experience high symptom burden, increased costs, and diminished quality of dying. There is limited literature on palliative care engagement in ICU, especially in lower-middle-income countries. This study explores a strategy to enhance palliative care engagement in ICU through a stakeholder participatory approach.
Methods:
Theory of Change approach was used to develop a hypothetical causal pathway for palliative care integration into ICUs in India. Four facilitated workshops and fifteen research team meetings were conducted virtually over three months. Thirteen stakeholders were purposively chosen, and three facilitators conducted the workshops. Data included workshop discussion transcripts, online chat box comments, and team meeting minutes. These were collected, analysed and represented as theory of change map.
Results:
The desired impact of palliative care integration was good death. Potential long-term outcomes identified were fewer deaths in ICUs, discharge against medical advice, and inappropriate admissions; increased referrals to palliative care; and improved patient and family satisfaction. Twelve preconditions were identified, and eleven key interventions were developed. Five overarching assumptions related to contextual factors influencing the outcomes of interventions.
Conclusion:
Theory of change framework facilitated the identification of proposed mechanisms and interventions underpinning palliative care integration in ICUs.
This chapter examines the role of expertise in public policy collaboration. It explores how expertise influences and is influenced by collaboration, and the ways in which expertise and collaboration interact productively and unproductively. It examines the challenges and opportunities collaboration presents to expert identities and performances, the role of actors and agency in enabling and constraining expertise in collaboration, and the implications for public policy in different places and spaces. Finally, it focuses on the specific expertise required to collaborate well and the particular role played by ‘boundary spanners’ in public policy collaboration.
In the last years with the affirmation of the Theory of Change (ToC—Connel and Kunisch in New approaches to evaluating community initiatives 2:1–16, Connell and Kubisch, New Approaches to Evaluating Community Initiatives 2:1–16, 1998), a lot of tools for the evaluation of projects having social relevance have been developed. The need to measure their impact has led to leave the use of traditional qualitative measurement instruments joints for quantitative tools, which calculate the effectiveness of the resources employed and the generated impact by social projects (Anderson in The community builder's approach to Theory of Change, Anderson, A.A., (2006). The community builder's approach to Theory of Change.). Among them, recently it is paid attention on the Social Return on Investment (SROI—Lingane and Olsen in California management review 46:116–135, Lingane and Olsen, California Management Review 46:116–135, 2004), a model to account for created value. It includes not only the return of investments but also the benefits for the broader public in the social, economic and environmental spheres. The research aim is to verify and calculate SROI as a valid tool to measure the impact generated by social projects in transitional countries, especially those of health international cooperation. Indeed, the economic quantification of the SROI rate appears as an important element to improve reporting and accountability both to the lender, the gesture and the users. The case study has been identified as appropriate methodology for conducting this research (Yin 2003; Eisenhardt in Academy of Management Review 14:532–550, Eisenhardt, Academy of Management Review 14:532–550, 1989). The project ‘Introducing Health Information System (IHIS): a modern approach to transparency and accountability in the Albanian public health’ conducted by the Tuscany Region resulted relevant for the involvement of different actors and its action in a transition country as Albania. This project was realized from 2014 to 2016 using financing from IADSA (Italian Albanian Debt for Development Swap Agreement) to support the Albanian government in the NHS technological and accountant development. Now it’s possible to adequately observe the generated impact 5 years after the conclusion of IHIS project. The necessary data for the calculation of SROI was collected by desk analysis of project's documents and reports and the conduction of interviews and focus groups to a specimen of involved stakeholders. Results show the good capability of SROI to quantitatively evaluate the impact produced by a health international cooperation project, also if complex and articulated as that studied. In particular, the SROI rate of this project is 9.4, an optimal rate (for 1.00€ invested, the project generated an economic impact of 9.40€). Therefore, it is demonstrated the optimal use of the allocated resources obtaining a strong effective impact in the Albanian health care system and on the network of involved stakeholders. Another important aspect is the reinforcement of results dissemination in the society, for the simplicity and easy comprehension of the SROI rate and the internal restitution promoted by SROI analysis. So SROI proves as a good instrument both to measure and to share the economic value of generated impact.KeywordsSocial projectsImpactEvaluation toolsInternational cooperationSROIHealthJEL ClassificationH43H51I15
Investments flowing into blue economy projects are estimated to be much lesser than the requirements, for achieving the targets set out in the UN Sustainable Development Goals. Blue economy projects are typically financed through conventional means of public and development finance. However, the nature and characteristics of blue economy projects transcend the need to extend beyond the conventional financing options of multilateral/bilateral aid. The objective of this article is to assess if the existing blue economy initiatives are adequate to the sectoral investment needs and to develop contours of a framework that could accelerate the blue economy investments. The research finds that the current initiatives such as blue bonds are relatively small and accelerating investments requires access to additional financing instruments and a transformative change in participating stakeholders. Using a Theory of Change approach, contours of a framework that pools in low-cost funds from a diverse set of investors to be deployed for either public sector promoted large impact projects or individual blue economy projects through market-based instruments are suggested. The findings contribute to the ongoing debate on how to improve the financial capability of various blue economy stakeholders and enable them to configure more sustainable financing mechanisms.
In realist evaluation, where researchers aim to make program theories explicit, they can encounter competing explanations as to how programs work. Managing explanatory tensions from different sources of evidence in multi-stakeholder projects can challenge external evaluators, especially when access to pertinent data, like client records, is mediated by program stakeholders. In this article, we consider two central questions: how can program stakeholder motives shape a realist evaluation project; and how might realist evaluators respond to stakeholders' belief-motive explanations, including those about program effectiveness, based on factors such as supererogatory commitment or trying together in good faith? Drawing on our realist evaluation of a service reform initiative involving multiple agencies, we describe stakeholder motives at key phases, highlighting a need for tactics and skills that help to manage explanatory tensions. In conclusion, the relevance of stakeholders' belief-motive explanations ('we believe the program works') in realist evaluation is clarified and discussed.
In the UK a great deal of attention is currently focused on the potential of the `theories of change' approach to evaluating complex public policy interventions. However, there is still relatively little empirical material describing its application. This article discusses the use of `theories of change' in the national evaluation of English Health Action Zones (HAZs). It locates `theories of change' within the wider context of evaluation approaches and assesses its strengths and weaknesses as an evaluation framework.
The article then focuses on a key aspect of complex public policy interventions — cross-sector collaboration. Drawing on data about cross-sector partnerships and community involvement from the English HAZ evaluation, the article explores the contribution of `theories of change' towards examining the building of collaborative capacity in HAZs.
The article also describes the `co-research' approach being employed within the national HAZ evaluation. It discusses how this approach can complement the use of `theories of change', contribute to managing change within organizations and communities and facilitate more effective use of evaluation within a local health context.
When New Labour came to power in the UK in 1997 it brought with it a strong commitment to reducing inequality and social exclusion. One strand of its strategy involved a focus on area-based initiatives to reduce the effects of persistent disadvantage. Health Action Zones (HAZs) were the first example of this type of intervention, and their focus on community-based initiatives to tackle the wider social determinants of health inequalities excited great interest both nationally and internationally. This article draws on findings from the national evaluation of the initiative. It provides an overview of the HAZ experience, and explores why many of the great expectations associated with HAZs at their launch failed to materialise. It suggests that, despite their relatively limited impact, it is best to consider that they made a good start in difficult circumstances rather than that they failed. As a result there are some important lessons to be learned about the role of complex community-based interventions in tackling seemingly intractable social problems for policy-makers, practitioners and evaluators. Social programs are complex undertakings. They are an amalgam of dreams and personalities, rooms and theories, paper clips and organisational structure, clients and activities, budgets and photocopies, and great intentions.
Powerful, rich, and well educated people tend to live longer and healthier lives than their less advantaged counterparts. These socioeconomic inequalities in health have been observed in a range of societies—developed, developing, market led, welfare state, and communist. Their expression, however, may vary according to how the particular society is stratified—for example, by income or wealth in the United States, by social class in the United Kingdom, or by education in Europe. They occur across a wide range of causes of death and types of illness, have been observed since accurate statistics were first available, and seem to have been increasing.1
Several governments have recently proposed strategies to reduce socioeconomic inequalities in health.2–5 An issue rendering strategy development in this field difficult is that, although a lot of information is available about the magnitude and causes of socioeconomic inequalities in health, rather less information is available about the effectiveness of policies in reducing them.6 The recent Cross-Cutting review in England noted that intervention research is scanty compared with the much larger body of observational evidence that describes inequalities.5 This is shown by the fact that the review contains six boxed lists, containing between them 50 examples …
What counts is what works – but how can we actually tell what works? And what can we do with such knowledge to influence policy and practice? As all parts of the public sector embrace ‘evidence’ as a means of providing more effective and efficient public services, this book provides a contribution to such debates. The authors consider the role of evidence in specific public policy areas (healthcare, education, criminal justice, social care, welfare, housing, transport, and urban renewal), using experts in each field to explore the creation, dissemination, and use of evidence within each. They consider in particular: How is research evidence of service effectiveness created? How does such evidence shape policy and influence service delivery? What efforts are being made to encourage greater utilisation of evidence in policy and practice? The rich cross-sectoral accounts of the many and diverse activities in each sector provide an insight into the ebb and flow of evidence as guidance to policy and practice. The book develops perceptive analyses of outstanding problems, and raises challenging agendas for service development and future research. The authors conclude with the all-important question of the implementation of evidence-based practice.
This article examines the New Labour agenda for the governance of localities, focusing on three central themes: community leadership, improving the management of public services and building social capital. Potential areas of policy tension and conflict are identified between these themes. The different forms of Community Governance are outlined and their coverage of the three themes described. This helps to elaborate and explain the tensions that exist within the New Labour agenda. The article then specifies the key challenges facing elected local government if modernisation and democratisation is to be delivered through a framework of Community Governance.
This paper describes how we strengthened the theory of change approach to evaluating a complex social initiative by integrating it with a quasi-experimental, comparison group design. We also demonstrate the plausibility of selecting a credible comparison group through the use of cluster analysis, and describe our work in validating that analysis with additional measures. The integrated evaluation design relies on two points of comparison: (1) program theory to program experience; and (2) program cities to comparison cities. We describe how we are using this integrated design to evaluate the Robert Wood Johnson Foundation’s Urban Health Initiative, an effort that aims to improve health and safety outcomes for children and youth in five distressed urban areas through a process of citywide, multi-sector planning and changed public and private systems. We also discuss how the use of two research frameworks and multiple methods can enrich our ability to test underlying assumptions and evaluate overall program effects. Using this integrated approach has provided evidence that the earliest phases of this initiative are unfolding as the theory would predict, and that the comparison cities are not undergoing a similar experience to those in UHI. Despite many remaining limitations, this integrated evaluation can provide greater confidence in assessing whether future changes in health and safety outcomes may have resulted from the Urban Health Initiative (UHI).
Urban policy in England can be characterised on the one hand as a fitful succession of discontinuous initiatives, the latest being a phase of competition and place marketing, but on the other hand as a continuity of centralisation, fragmentation and organisational proliferation. Policy integration and interdepartmental collaboration at central and local government levels has been weak. Proposals for a new Cabinet Committee, Ministers appointed with responsibility for individual cities, an integrated Regeneration Budget, and decentralised regional administration offer the potential for a reversal of centralist trends. At the same time, however, they embody a managerial, competitive and corporatist policy culture which runs counter to the spirit of a new localism in regeneration policy.
Central-local relations have been of particular interest since the Labour government came to power in 1997. Both academics and practitioners have pointed to tensions within the Labour government's reform agenda—between a ‘top-down’ and ‘bottom-up’ approach; between a drive for national standards and the encouragement of local learning and innovation; and between strengthening executive leadership and enhancing public participation. It is argued that while Labour's modernisation strategy has clear elements of a top-down approach (legislation, inspectorates, white papers, etc) there is also a significant bottom-up dimension (a variety of zones, experiments and pilots, albeit with different degrees of freedom). This article utilises a multi-level governance framework of analysis and argues that, while much of the research using such frameworks has hitherto focused on the EU, recent developments in governance at neighbourhood, local authority, sub-regional and regional levels facilitate its application within a nation state. The central thesis is that, while there is extensive interaction between actors at sub-national level, this should not be seen as a proxy for policy influence. The local political arena is characterised less by multi-level governance than by multi-level dialogue. Sub-national actors participate but they are rarely major players in shaping policy outcomes: the plurality which characterises sub-central governance does not reflect a pluralist power structure.
Since the Blair government was elected in Britian in May 1997, there has been an explosion of area-based initiatives aimed at regenerating deprived communities and reducing poverty and social exclusion. All are partnerships that place a high value on collaboration, in contrast to the previous government's faith in competition and quasi-markets. At the same time, these new experiments in 'joined-up government' are expected to put the interests of consumers and communities before the innate conservatism of providers of services. In addition, many of the new area-based initiatives are charged with the responsibility for 'constantly reviewing, evaluating and, where necessary, adapting policies to ensure they achieve the maximum effect.
Please address correspondence to: Prof. Helen Sullivan, Research Director
- H E L E N S U L L I Va N Research
- Cities At
- Centre
H E L E N S U L L I VA N is research director at the Cities Research Centre, University of the West of England, Bristol, UK. Please address correspondence to: Prof. Helen Sullivan, Research Director, Urban Governance, Cities Research Centre, Faculty of the Built Environment, University of the West of England, Coldharbour Lane, Bristol BS16 1QY, UK. [email: helen.sullivan@uwe.ac.uk]
Cross-Cutting Issues in Public Policy and Public Services
- S Richards
- M Barnes
- A Coulson
- L Gaster
- B Leach
- H Sullivan
Richards, S., M. Barnes, A. Coulson, L. Gaster, B. Leach and H. Sullivan (1999) Cross-Cutting Issues in Public Policy and Public Services. London: Department of the Environ-ment, Transport and the Regions.
Collaboration and Co-ordination in Area Based Regeneration Initiatives
- M Stewart
- G Gillanders
- S Goss
- L Grimshaw
Stewart, M., G. Gillanders, S. Goss, L. Grimshaw, L. et al. (2002) Collaboration and Co-ordination in Area Based Regeneration Initiatives. London: DETR.
Cross-Cutting Issues in Local Government
- M Stewart
- S Goss
- G Gillanders
- R Clarke
- J Rowe
- H Shaftoe
Stewart, M., S. Goss, G. Gillanders, R. Clarke, J. Rowe and H. Shaftoe (1999) Cross-Cutting Issues in Local Government. London: DETR.
Finding out What Works: Building Knowledge about Complex Community Based Initiatives
- A Coote
- J Allen
- D Woodhead
Coote, A., J. Allen and D. Woodhead (2004) Finding out What Works: Building Knowledge about Complex Community Based Initiatives. London: King's Fund.
Applying a Theory of Change Approach to Two National, Multisite Comprehensive Community Initiatives: Practitioner Reflections
- S Herbert
- A Anderson
Herbert, S. and A. Anderson (1998) 'Applying a Theory of Change Approach to Two National, Multisite Comprehensive Community Initiatives: Practitioner Reflections', in K. Fulbright-Anderson, A. C. Kubisch and J. P. Connell (eds) New Approaches to Evalu-ating Community Initiatives, vol. 2, Theory, Measurement and Analysis. Washington, DC: Aspen Institute.
Assessing the Impact of Urban Policy A New Commitment to Neighbourhood Renewal: National Strategy Action Plan
- B Robson
- M Parkinson
- F Robinson
Robson, B., M. Parkinson, F. Robinson and associates (1994) Assessing the Impact of Urban Policy. London: HMSO. Social Exclusion Unit (2001) A New Commitment to Neighbourhood Renewal: National Strategy Action Plan. London: Stationery Office.
Voices from the Field II: Reflections on Comprehensive Community Change
- A C Kubisch
- P Auspos
- P Brown
- R Chaskin
- K Fulbright-Anderson
- R Hamilton
Kubisch, A. C., P. Auspos, P. Brown, R. Chaskin, K. Fulbright-Anderson and R. Hamilton (2002) Voices from the Field II: Reflections on Comprehensive Community Change. Washington, DC: Aspen Institute.
An Evaluation of Urban Policy Urban Policy Evaluation: Challenge and Change
- M Bradford
- B Robson
Bradford, M. and B. Robson (1995) 'An Evaluation of Urban Policy', in R. Hambleton and H. Thomas (eds) Urban Policy Evaluation: Challenge and Change, pp. 37–54.
What Works? Evidence Based Policy and Practice in Public Services Sullivan and Stewart: Who Owns the Theory of Change? Department ofHealth Action Zones: Invitation to Bid', Executive Letter
- H T O Davies
- S M Nutley
- P C Smith
Davies, H. T. O., S. M. Nutley and P. C. Smith (2000) What Works? Evidence Based Policy and Practice in Public Services. Bristol: Policy Press. Sullivan and Stewart: Who Owns the Theory of Change? Department of Health (1997) 'Health Action Zones: Invitation to Bid', Executive Letter, 65. London: DoH.