Content uploaded by Denny John
Author content
All content in this area was uploaded by Denny John on Jun 21, 2020
Content may be subject to copyright.
A preview of the PDF is not available
Does incorporating participation and accountability improve development outcomes? Meta-analysis and framework synthesis
Abstract and Figures
Does engaging citizens in the planning, management and oversight of public services –
such as health care, social protection or physical infrastructure – that are critical to
enabling large-scale development of populations impact the quality of and access to
services and citizens’ quality of life? This systematic review examined high quality
evidence from 35 citizen engagement programmes in low- and middle-income
countries that promote the engagement of citizens in service delivery through four
routes: participation (participatory priority setting); inclusion of marginalized groups;
transparency (information on rights and public service performance); and/or citizen
efforts to ensure public service accountability (citizen feedback and monitoring);
collectively, PITA mechanisms.
Figures - uploaded by Denny John
Author content
All figure content in this area was uploaded by Denny John
Content may be subject to copyright.
... These approaches are often framed in terms of the concept of social accountability. Popularised in the early part of the last decade following the publication of the influential 2004 World Development Report (World Bank 2004), social accountability has been the subject of significant policy diffusion globally and has grown to be a key part of externally-funded development programmes (e-Pact 2016; Waddington et al. 2019). Joshi (2014) points out that various formations of social accountability appeal to different kinds of outcomes; from the more instrumental immediate fixes to service provision that come from better understanding service user needs, to more transformative changes in how citizens and the state relate to one another in terms of rights and responsibilities. ...
... Within civil society they might involve supporting the growth of civic organisations and collectives that act to represent particular interests or constituencies, advocate for change, and build reformist movements and momentum. Positive outcomes of attempts at these kinds of support to civil society have been reported in independent reviews of DFID programmes (ICAI 2013a), in recent systematic reviews (Waddington et al. 2019), and internal reviews of international NGO programmes on civic involvement in governance in conflict-affected contexts (Amakom et al. 2018;Fooks 2013: 6). ...
... The macro-evaluation suggested that within the programmes reviewed, achieving higher-level policy changes through such approaches required the integration of citizens' experiences within the formal institutionalised spaces of the service delivery and governance system, although this was not sufficient in the examples analysed to generate those changes in isolation. A recent systematic review of evaluation evidence from a range of citizen engagement modalities, including those associated with social accountability approaches, drew a similar conclusion that localised improvements in services and relationships were common but these were not necessarily linked to evidenced improvements in what they termed 'wellbeing' outcomes (Waddington et al. 2019). Fox (2015) highlights that efforts to mobilise citizens or institutions to raise their voice and make louder demands for public accountability often fail to activate institutionalised or formal mechanisms that can effectively sanction decision makerswhat he calls 'voice without teeth'. ...
... These approaches are often framed in terms of the concept of social accountability. Popularised in the early part of the last decade following the publication of the influential 2004 World Development Report (World Bank 2004), social accountability has been the subject of significant policy diffusion globally and has grown to be a key part of externally-funded development programmes (e-Pact 2016; Waddington et al. 2019). Joshi (2014) points out that various formations of social accountability appeal to different kinds of outcomes; from the more instrumental immediate fixes to service provision that come from better understanding service user needs, to more transformative changes in how citizens and the state relate to one another in terms of rights and responsibilities. ...
... Within civil society they might involve supporting the growth of civic organisations and collectives that act to represent particular interests or constituencies, advocate for change, and build reformist movements and momentum. Positive outcomes of attempts at these kinds of support to civil society have been reported in independent reviews of DFID programmes (ICAI 2013a), in recent systematic reviews (Waddington et al. 2019), and internal reviews of international NGO programmes on civic involvement in governance in conflict-affected contexts (Amakom et al. 2018;Fooks 2013: 6). ...
... The macro-evaluation suggested that within the programmes reviewed, achieving higher-level policy changes through such approaches required the integration of citizens' experiences within the formal institutionalised spaces of the service delivery and governance system, although this was not sufficient in the examples analysed to generate those changes in isolation. A recent systematic review of evaluation evidence from a range of citizen engagement modalities, including those associated with social accountability approaches, drew a similar conclusion that localised improvements in services and relationships were common but these were not necessarily linked to evidenced improvements in what they termed 'wellbeing' outcomes (Waddington et al. 2019). Fox (2015) highlights that efforts to mobilise citizens or institutions to raise their voice and make louder demands for public accountability often fail to activate institutionalised or formal mechanisms that can effectively sanction decision makerswhat he calls 'voice without teeth'. ...
Development donors invest significantly in governance reform, including in contexts
characterised by conflict and fragility. However, there is relatively little comparative study of their change strategies, and little understanding of what works and why. This paper explores the strategies of six recent DFID-funded programmes in Mozambique, Myanmar, and Pakistan with empowerment and accountability aims. Document review and field interviews are used to analyse the application of multi-scalar or multi-level change strategies, since such approaches are hypothesised to potentially generate more leverage for public accountability reforms. Analysis suggests that these strategies can strengthen citizen ability to navigate governance systems to resolve problems and claim accountability, and can bolster pro-accountability coalitions’ internal solidarity and external legitimacy. Multi-level strategies also appear associated with establishing more significant pressure for reform than exclusively local or national approaches. Yet conventional project reporting focuses on counting activities and outputs rather than analysing the dynamic, interactive processes at work in these strategies, and few evaluations are publicly accessible. To fully understand what kinds of action strengthen citizen demands for accountability requires a more transparent and rigorous approach to learning from donor-led governance interventions.
... Interestingly, a review of provider responsiveness to social accountability initiatives found health provider receptivity to citizen demands is mediated by the extent to which the social accountability initiatives provide personal and professional support to providers [11]. In addition, a review of evidence on interventions promoting external participation and accountability in low-and middle-income countries suggests that collaborative approaches to service provider engagement in transparency and accountability processes appear more effective than confrontational approaches and that many such interventions experience challenges stemming from lack of positive engagement with supply-side actors [12]. The collaborative approach to social accountability is line with the CQI philosophy that quality is a product of linked chain. ...
... In the health sector, there is promising evidence that collaborative social accountability approaches-like the CSC and the similar Citizen Voice and Action (CVA) approach-can improve relationships and spaces for negotiation between the community and health providers, and improve community empowerment, provider responsiveness, and ultimately enhance the availability, access, quality and uptake of services [12,13,14,15,16,17,18,19,20]. ...
... This is important not only because frontline health workers play a critical role in the functioning of these social accountability approaches, but also because these approaches may lead to changes in service delivery and health worker behaviors that can be best (and is some cases only) ascertained from health worker self-reports. While several studies examining health sector collaborative social accountability approaches have used community and client-reported qualitative and quantitative data as well as process data (e.g., Score Card data) to understand the impact of these approaches, there has been less published on the impact of these approaches using health worker-reported service provision-related data [12,13,14,15,16,17,18,19,20]. To our knowledge, a quantitative survey of health workers to better understand a collaborative social accountability approach's impact on their self-reported responsibilities and service provision as well as governance-related outcomes has not been carried out. ...
Background
Social accountability approaches are increasingly being employed in low-resource settings to improve government services. In line with the continuous quality improvement (CQI) philosophy that quality is the product of a linked chain, collaborative social accountability approaches like the Community Score Card (CSC) aim to empower clients and frontline service providers to transform their own lives and hold public officials to account for state obligations. Despite being a critical focus of collaborative social accountability approaches, to our knowledge, a quantitative survey of health workers to understand the impact of these approaches on their self-reported responsibilities and service provision has not been conducted. To fill this gap, we carried out a quantitative survey with health workers to assess the CSC’s impact on health worker-reported service responsibilities and provision and complement women’s self-reports.
Methods
We evaluated the effect of the CSC on reproductive health-related outcomes using a cluster-randomized design in Ntcheu district, Malawi. We matched 10 pairs of health facilities and surrounding catchment communities; one from each pair was randomly assigned to the intervention and control arms. The intervention communities and health workers each completed 3–4 cycles of the CSC process by endline. We then surveyed all health workers in the 20 intervention and comparison sites at endline (n = 412) to estimate the intervention’s impact.
Results
Significantly (p < .05) more health workers in the CSC intervention areas compared to control areas reported responsibility for antenatal care, comprehensive antenatal care counseling, recording of the number of pregnant and postpartum women seen each month, and the average age of their last family planning client was younger. In addition, marginally significantly (p < .10) more health workers in treatment versus control areas report visiting women at their home at least once during their pregnancy. However, health worker-reported responsibility for HIV testing was significantly lower in intervention areas than in control.
Conclusions
The CSC aims to empower health workers to collaborate with the community and rest of the health system to identify and overcome the diverse and context-specific range of performance barriers they face. In doing so, it aims to support them to demand and ensure quality care for themselves from the health system so they can, in turn, deliver quality services to clients. Our results contribute to the evidence that the CSC may hold promise at improving service provision. While there is increasing evidence that collaborative social accountability approaches like the CSC are effective means to improving reproductive health-related service provision and outcomes in low-resource settings, additional research is needed.
... In addition to allies within health systems or donors, respondents here also cited the importance of engaging a broad coalition of civil society in CLM efforts, especially in order to build the reach of CLM efforts in a resource efficient way. The literature suggests that beyond this particular benefit, social accountability efforts that consist of or engage with a broad variety of community and civil society organizations may help to grow the social capital of the effort and help to increase bargaining power with duty bearers [18]. Interestingly, the need for CLM to representative of broad swaths of community also emerged in this study as it relates to approaches to data collection. ...
... In this study, participants noted that they were at time challenged by gaps in skills related to both data collection and advocacy. The inclusion of mechanisms that aimed to build capacity for collective action has been noted as an important facilitator of accountability and transparency work that engages community [18]. Respondents in this study made clear that this kind of capacity building is often under-funded within their projects. ...
Achieving the global HIV, tuberculosis, and malaria targets will require innovative strategies to deliver high quality and person-centered health services. Community-led monitoring (CLM) is a rapidly proliferating health systems strengthening intervention for improving healthcare services and documenting human rights violations, through social empowerment and political accountability. Driven in part by increasing financial support from donors, a growing number of countries are implementing CLM programs. This study aimed to identify early challenges and lessons learned from CLM implementation, with the aim of informing and improving the implementation of CLM programs and ultimately achieving greater impact on the delivery of services. Twenty-five CLM implementors representing 21 countries participated in an interview. Early generation of buy-in from diverse stakeholders was noted as critical for CLM success. Leveraging existing networks of service users and community organizations to implement CLM also helped to maximize program reach and resources. Uncertainty around CLM’s purpose and roles among CLM stakeholders resulted in challenges to community leadership and ownership of programs. Respondents also described challenges with underfunded programs, especially advocacy components, and inflexible donor funding mechanisms. Critical capacity gaps remain around advocacy and electronic data collection and use. With the rapid expansion of CLM, this study serves as an important first step in characterizing challenges and successes in the CLM landscape. Successful implementation of CLM requires prioritizing community ownership and leadership, donor commitment to sustainable and reliable funding, and strengthened support of programs across the data collection and advocacy lifecycle.
... The latest report published by the International Initiative for Impact Assessment indicates that incorporating local participation and accountability often improves developmental outcomes as it stimulates active citizen engagement in service delivery. Moreover, the application of the local understanding of the observed changes enhances community interest in ecosystem monitoring (Waddington et al., 2019). ...
... CBM cases at the international level indicate that CBM is beneficial to resource users, as it helps them participate directly in field data collection to support the conservation and management of natural resources (Fernandez-Gimenez et al., 2008;Gérin-Lajoie et al., 2018;Weston & Conrad, 2015;Van Rijsoort & Jinfeng, 2005). CBM enhances the accountability and transparency of research projects through partnership, such that local participants are able to see the changes happening in their landscapes (Waddington et al., 2019). It also fosters community pride and enhances social values and efficacy as is evidenced by IGES programs across many parts of the world. ...
Community-based monitoring (CBM) is a widely used form of scientific data collection in which local community members directly participate in ecosystem research and also help the process by sharing traditional ecological knowledge and local understanding of land and resources. This paper presents a review of the challenges and opportunities associated with CBM projects in Canada and internationally. While Canadian cases are the primary focus, international examples are drawn on to provide additional context. Based on our review of 121 documents and publications, we found that CBM helps fill science research gaps by providing access to continuous data sets on the ecosystems being studied. CBM also increases the credibility of the data among users, as the community itself takes part in the environmental monitoring process. CBM supports cross-cultural learning and the co-production of knowledge by using traditional ecological knowledge with science, thereby enabling researchers, scientists and community members to learn from one another. Our review suggests that although there are multiple successes, CBM faces several challenges that constrain its progress including funding shortages, lack of support for local stewardship, and inadequate training for local users in the operation of equipment and data collection methods. Data sharing and rights on the use of data are also constraining to the long-term success of CBM programs.
... Las organizaciones públicas son responsables ante varios foros diferentes que aplican diferentes conjuntos de criterios, lo que hace que la rendición de cuentas sea múltiple, compleja, potencialmente inconsistente y ambigua (Ozga, 2020). Las relaciones de rendición de cuentas presuponen tanto que el actor que debe rendir cuentas jugará un papel activo en proporcionar información sobre su comportamiento y ajustarlo, como que el foro que exige que alguien rinda cuentas buscar activamente información, discutir asuntos de rendición de cuentas y utilizar los instrumentos con los que cuenta para corregir el comportamiento del actor (Waddington et al., 2019). La literatura sobre rendición de cuentas también describe diferentes clasificaciones para la misma. ...
La participación de los GAD´S los cuales se encuentran conformados por varios funcionarios aportan abiertamente hacia la construcción de políticas públicas, a partir del conjunto de recursos de la sociedad, para solventar diversas adversidades en concordancia con el Estado como parte de un sistema democrático. En tanto, la rendición de cuentas es una relación entre un actor y un foro, en la que el primero explica y justifica la conducta y, el segundo, emite juicios como mecanismo de control social. El objetivo de esta investigación es proponer un modelo de participación del GAD que permita mejorar la rendición de cuentas del cantón Riobamba. La metodología empleada fue básica-proyectiva, diseño no experimental, transversal y niveles de conocimiento, descriptivo, explicativo, predictivo y prospectivo, el instrumento empleado estuvo validado por 5 doctores en el área de Gestión Pública y Gobernabilidad, así también la población estuvo compuesta por 861 sujetos donde se aplicó un muestreo no probabilístico por conveniencia obteniendo 170 funcionarios públicos. Los hallazgos denotaron que el 56,5% de los funcionarios no están conformes en la manera que se ha venido manejando la figura de la rendición de cuentas en el municipio, siendo el porcentaje más bajo 2,4% y el más alto 41,2%. Las conclusiones destacan la importancia de impulsar el proceso de rendición de cuentas contribuyendo al desarrollo sostenible basado en los resultados en las necesidades de la sociedad.
... However, when relationships are poor, public forums that degenerate into name-andshame sessions may make matters worse. Facilitated, collaborative meetings that jointly engage citizens and service providers in monitoring are often more effective than confrontational meetings (Waddington et al. 2019). ...
Ineffective monitoring and weak accountability mechanisms have greatly affected the realization of the benefits of decentralization in Uganda. To address this, the government initiated community advocacy forums or citizen barazas. The objective was to enhance public involvement in holding the government accountable for service delivery, especially, in relation to the resources spent. The authors of this report conducted a cluster randomized control trial to understand the impact of these barazas on public service delivery and to inform policymakers about the effectiveness of this initiative.
... The policy framework should be aligned to the needs of the community to ensure successful completion and also the clarity regarding the responsibility, accountability and sharing of benefits needs to be specifically mentioned. The effective community participation needs to be assessed by the variables like access and equity, transparency, accountability (Waddington et al. 2019) and regulations. Here access and equity refers to the fair distribution and allocation of resources, work or benefits among members of community; transparency and accountability is essential to make community members answerable to their action and behavior as well as responsive toward accomplishment of goals; regulations means the existence of laws, rules and policies to regulate community members. ...
Unanticipated, unpredictable social and environmental changes adversely affect communities living in tropical constrained environment. This conceptual paper investigates the adaptations and resilience of communities earning their livelihood through the tourism sector. This chapter is built through extensive literature reviews and proposes a conceptual framework focusing on community development, sustainable development and tourism. The proposed strategic framework model highlights the need for engagement of resident community in the process of tourism development in constrained environments. It suggests that the community participation will be helpful in effective policy planning, decision-making, implementation, monitoring and evaluating policies. The significant contribution of this research is in the continuous engagement of community members in the planning of tourism activities, through the strategic framework. The insightful information on implications of community participation will be helpful for researchers, policy makers and especially the tourism industry.
Background
Across the globe, gender disparities still exist with regard to equitable access to resources, participation in decision‐making processes, and gender and sexual‐based violence. This is particularly true in fragile and conflict‐affected settings, where women and girls are affected by both fragility and conflict in unique ways. While women have been acknowledged as key actors in peace processes and post‐conflict reconstruction (e.g., through the United Nations Security Council Resolution 1325 and the Women, Peace and Security Agenda) evidence on the effectiveness of gender‐specific and gender‐transformative interventions to improve women's empowerment in fragile and conflict‐affected states and situations (FCAS) remains understudied.
Objectives
The purpose of this review was to synthesize the body of evidence around gender‐specific and gender‐transformative interventions aimed at improving women's empowerment in fragile and conflict‐affected settings with high levels of gender inequality. We also aimed to identify barriers and facilitators that could affect the effectiveness of these interventions and to provide implications for policy, practice and research designs within the field of transitional aid.
Methods
We searched for and screened over 100,000 experimental and quasi‐experimental studies focused on FCAS at the individual and community levels. We used standard methodological procedures outlined by the Campbell Collaboration for the data collection and analysis, including quantitative and qualitative analyses, and completed the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) methodology to assess the certainty around each body of evidence.
Results
We identified 104 impact evaluations (75% randomised controlled trials) assessing the effects of 14 different types of interventions in FCAS. About 28% of included studies were assessed as having a high risk of bias (45% among quasi‐experimental designs). Interventions supporting women's empowerment and gender equality in FCAS produced positive effects on the outcomes related to the primary focus of the intervention. There are no significant negative effects of any included interventions. However, we observe smaller effects on behavioural outcomes further along the causal chain of empowerment. Qualitative syntheses indicated that gender norms and practices are potential barriers to intervention effectiveness, while working with local powers and institutions can facilitate the uptake and legitimacy of interventions.
Conclusions
We observe gaps of rigorous evidence in certain regions (notably MENA and Latin America) and in interventions specifically targeting women as actors of peacebuilding. Gender norms and practices are important elements to consider in programme design and implementation to maximise potential benefits: focusing on empowerment only might not be enough in the absence of targeting the restrictive gender norms and practices that may undermine intervention effectiveness. Lastly, programme designers and implementation should consider explicitly targeting specific empowerment outcomes, promoting social capital and exchange, and tailoring the intervention components to the desired empowerment‐related outcomes.
Pollution is the leading cause of disease, disability and premature death around the world. But its impacts are not evenly distributed. Poor and marginalized communities often bear the brunt of the environmental, health, and socioeconomic impacts, especially in middle- and low-income countries. Far removed from powerful decision-makers, these communities face multiple barriers in their efforts to fight pollution. Many are unaware of their rights to access pollution information or participate in policymaking decisions, and often lack the advocacy skills needed to form strategic partnerships and effectively engage government officials. Access to information, public participation and justice are environmental rights that not only are fundamental to good environmental governance, but also offer an important tool for improving the development, implementation and enforcement of pollution laws. Strategically applying these rights can enable civil society and local communities to evaluate the environmental and social justice aspects of pollution, demand better compliance with laws and help build a pollution accountability movement. Developed in partnership with more than a dozen civil society organizations across six countries, this toolkit offers practical guidance on how to use environmental rights to fight air, water and solid waste pollution. Designed to support civil society, local community activities and those concerned about pollution, the toolkit enables everyone, everywhere to conduct policy research, collect and use pollution information in decision-making processes, form coalitions and develop advocacy campaigns. Its eight modules feature important concepts, research indicators, worksheets and templates, which can be modified to fit a country’s context and downloaded to complete as a workbook. From Thailand to Morocco to Jamaica, civil society organizations and local communities have already used this toolkit to strengthen their right to a clean, healthy and safe environment. In Indonesia, for example, the Ministry of Environment and Forestry passed a new List of Public Information regulation that required over 111 documents, maps, and reports to be made proactively available. And in Morocco, there is growing civil society engagement with government officials over a newly passed Right to Information law. Communities around the world can follow in their footsteps and use this toolkit to protect lives, livelihoods and the environment from pollution.
This Campbell systematic review examines the effectiveness, efficiency and implementation of cash transfers in humanitarian settings. The review summarises evidence from five studies of effects, 10 studies of efficiency and 108 studies of barriers and facilitators to implementation of cash‐based humanitarian assistance.
Studies assessing effectiveness of cash‐based approaches were experimental and quasiexperimental studies. Studies analyzing efficiency were experimental, quasi‐experimental or observational studies with a cost analysis or economic evaluation component. Studies examining barriers and facilitators included these study types as well as other qualitative and mixed methods studies.
Unconditional cash transfers and vouchers may improve household food security among conflict‐affected populations and maintain household food security among food insecure and drought‐affected populations. Unconditional cash transfers led to greater improvements in dietary diversity and quality than food transfers, but food transfers are more successful in increasing per capita caloric intake than unconditional cash transfers and vouchers. Unconditional cash transfers may be more effective than vouchers in increasing household savings, and equally effective in increasing household asset ownership. Mobile transfers may be a more successful asset protection mechanism than physical cash transfers. Cash transfers can be an efficient strategy for providing humanitarian assistance. Unconditional cash transfer programmes have a lower cost per beneficiary than vouchers which, in turn, have a lower cost per beneficiary than in‐kind food distribution. Cash transfer programs can also benefit the local economy. Voucher programmes generated up to 1 equivalent provided to beneficiaries and unconditional cash transfer programmes generated more than 1 provided to beneficiaries. Intervention design and implementation play a greater role in determining effectiveness and efficiency of cash‐based approaches than the emergency context or humanitarian sector. Factors which influence implementation include resources available and technical capacity of implementing agencies, resilience of crisis‐affected populations, beneficiary selection methods, use of new technologies, and setting‐specific security issues, none of which are necessarily unique to cash‐based interventions.
Plain language summary
Cash‐based humanitarian assistance approaches can increase food security and are more cost effective than in‐kind food transfers
Both cash‐based approaches and in‐kind food assistance can be effective means of increasing household food security for people who live in areas of conflict.
The review in brief
Cash‐based approaches have become an increasingly common strategy for the provision of humanitarian assistance. Both cash‐based approaches and in‐kind food assistance can be effective means of increasing household food security among conflict‐affected populations and maintaining household food security among food insecure and drought‐affected populations. Cash transfers are more cost effective than vouchers which are more cost effective than in‐kind food assistance.
What is this review about?
This review assesses the effects of cash‐based approaches on individual and household outcomes in humanitarian emergencies. It also assesses the efficiency of different cash‐based approaches and identifies factors that hinder and facilitate programme implementation.
What is the aim of this review?
This Campbell systematic review examines the effectiveness, efficiency and implementation of cash transfers in humanitarian settings. The review summarises evidence from five studies of effects, 10 studies of efficiency and 108 studies of barriers and facilitators to implementation of cash‐based humanitarian assistance.
What are the main findings of this review?
What studies are included?
Studies assessing effectiveness of cash‐based approaches were experimental and quasi‐experimental studies. Studies analyzing efficiency were experimental, quasi‐experimental or observational studies with a cost analysis or economic evaluation component. Studies examining barriers and facilitators included these study types as well as other qualitative and mixed methods studies.
Unconditional cash transfers and vouchers may improve household food security among conflict‐affected populations and maintain household food security among food insecure and drought‐affected populations. Unconditional cash transfers led to greater improvements in dietary diversity and quality than food transfers, but food transfers are more successful in increasing per capita caloric intake than unconditional cash transfers and vouchers. Unconditional cash transfers may be more effective than vouchers in increasing household savings, and equally effective in increasing household asset ownership. Mobile transfers may be a more successful asset protection mechanism than physical cash transfers.
Cash transfers can be an efficient strategy for providing humanitarian assistance. Unconditional cash transfer programmes have a lower cost per beneficiary than vouchers which, in turn, have a lower cost per beneficiary than in‐kind food distribution. Cash transfer programs can also benefit the local economy. Voucher programmes generated up to 1 equivalent provided to beneficiaries and unconditional cash transfer programmes generated more than 1 provided to beneficiaries.
Intervention design and implementation play a greater role in determining effectiveness and efficiency of cash‐based approaches than the emergency context or humanitarian sector. Factors which influence implementation include resources available and technical capacity of implementing agencies, resilience of crisis‐affected populations, beneficiary selection methods, use of new technologies, and setting‐specific security issues, none of which are necessarily unique to cash‐based interventions.
What do the findings of this review mean?
Unconditional cash transfers and vouchers can be effective and efficient ways to provide humanitarian assistance.
Each assistance modality has different advantages and disadvantages that should be considered in the design of future interventions. However, no definitive conclusions on the effectiveness of cash transfer or voucher programmes could be drawn that are universally applicable for humanitarian policy.
Further development of the evidence base, with more rigorous evaluations comparing the effectiveness of different cash‐based approaches and transfer modalities, as well as approaches to comparing costs and benefits of cash‐transfer and voucher programmes, is needed to further strengthen the evidence base.
How up‐to‐date is this review?
The review authors searched for studies published up to November 2014. This Campbell systematic review was published in December 2017.
Executive summary
BACKGROUND
Humanitarian actors have a responsibility to ensure that assistance is provided in a way that minimizes risks and maximizes benefits to people affected by crisis. However, there are many challenges in evaluating ‘what works’ in addressing the needs of crisis‐affected populations, and translating research evidence into practice in complex environments with limited resources.
Humanitarian assistance has traditionally been provided in the form of in‐kind goods and services: temporary shelters, food and non‐food items, water and medical care. However, as the nature of humanitarian crises has shifted over the last few decades, cash‐based approaches have become an increasingly common strategy for the provision of humanitarian assistance and are widely considered an appropriate, and sometimes preferable, substitute for in‐kind assistance when conditions permit.
Increasing use of cash‐based approaches has been accompanied by efforts to evaluate cash‐based interventions and develop recommendations for implementation in a range of settings. Systematic reviews of evidence in humanitarian settings are,however,relatively rare, and, to the best of our knowledge, this is the first systematic review of the effects of cash‐based approaches in emergencies to date.
OBJECTIVES
The primary objective of this review wasto assess and synthesize existing evidence on the effects of cash‐based approaches on individual and household outcomes in humanitarian emergencies. The secondary objective was to assess the efficiency of different cash‐based approaches and identify factors that hinder and facilitate programme implementation.
REVIEW METHODS
We followed standard methodological procedures for review of experimental and quasi‐experimental studies to assess the effects of unconditional cash transfer, conditional cash transfer and voucher programmes for crisis‐affected populations. We also adapted these procedures to review economic studies assessing the efficiency of cash‐based approaches and observational, qualitative and mixed method studies assessing the factors that facilitate or hinder the implementation of cash‐based approaches in different settings.
We conducted comprehensive searches of published and unpublished literature in November 2014. Two independent research assistants screened all identified studies to determine eligibility for inclusion in the review. We then extracted data from all included studies using a standardized coding tool and critically appraised the studies using existing tools appropriate for the different study designs.
Due to the heterogeneity of the comparisons and outcomes reported in the included studies, we were not able to synthesize the studies using meta‐analysis. Instead, we have presentedthe results in tables and synthesised the findings narratively. We used narrative and thematic synthesis to address the secondary objective. We conducted these analyses in parallel, and have reported on each separately in subsequent chapters of this review.
REVIEW FINDINGS
Out of 4,094 studies identified in the initial search, a total of 113 publications (108 unique studies) were included in this systematic review. Only nine studies were found in peer‐reviewed publications. Overall, we have considered the body of evidence reviewed to have been of low quality due to methodological limitations. While the evidence reviewed offers some insights, the paucity of rigorous research on cash‐based approaches limits the strength of the conclusions. This is not uncommon amongtopics related to humanitarian assistance. The following table summarizes the types of studies reviewed in each section of this report:
Review Topic Studies Reviewed Study Characteristics
Effects of cash‐based approaches on:
• • Individual and household‐level economic outcomes
• • Sector‐specific humanitarian outcomes
• • Cross‐cutting humanitarian outcomes
5
Settings: DR Congo, Ecuador, Niger, Lebanon, YemenInterventions: Unconditional cash transfers, vouchersStudy designs: Randomized control trials (factorial and stratified cluster designs), regression discontinuity
Efficiency of cash‐based approaches in achieving humanitarian objectives
10
Settings: DR Congo, Ecuador, Niger, Lebanon, YemenInterventions: Unconditional cash transfers, vouchersStudy designs: Cost, cost‐efficiency, cost‐effectiveness, cost‐benefit, market impact
Factors facilitating and hindering realization of cash programme activities and the achievement of humanitarian objectives in different contexts
108
Settings: Afghanistan, Bangladesh, Belize, Bosnia and Herzegovina, Burundi, Chile, DR Congo, Ecuador, Ethiopia, Haiti, India, Indonesia, Kenya, Japan, Jordan, Lebanon, Lesotho, Niger, Malawi, Mozambique, Occupied Palestinian Territory, Pakistan, the Philippines, Somalia, South Sudan, Sri Lanka, Sudan, Swaziland, Turkey, Uganda, Vietnam, Yemen, Zambia, ZimbabweInterventions: Unconditional cash transfers, vouchers, conditional cash transfers (cash for work)Study designs: descriptive (quantitative), qualitative (narrative or thematic analysis), mixed methods
Main findings:
Effectiveness of cash‐based approaches (chapter 5)
Five studies assessed the effects of cash‐based approaches, four of which assessed effects on household level food security outcomes. Unconditional cash transfers and vouchers may improve household food security among conflict‐affected populations and maintain household food security within the context of food insecurity crises and drought. Studies found that unconditional cash transfers led to greater improvements in dietary diversity and quality than food transfers. Food transfers were found to be more successful in increasing per capita caloric intake than unconditional cash transfers and vouchers.
Few studies measure changes in household economic indicators, other sectoral outcomes and cross‐cutting outcomes. Unconditional cash transfers may be more effective than vouchers in increasing household savings, and equally effective in increasing household assets. Mobile transfers may be a more successful asset protection mechanism than physical cash transfers.
Efficiency of cash‐based approaches (chapter 6)
Ten studies assessed the efficiency of cash based approaches. Cash transfers and vouchers may be more cost‐efficient than in‐kind food distribution. Studies found that unconditional cash transfer programmes have a lower cost per beneficiary than comparison interventions (either vouchers, in‐kind food distribution or both); and vouchers have a lower cost per beneficiary than in‐kind food distribution. In‐kind food distribution has substantially higher administrative costs per dollar value provided to a beneficiary than unconditional cash transfers.
Cash‐based approaches may have positive economic multiplier effects. Voucher programmes generated up to 1 equivalent provided to beneficiaries and unconditional cash transfer programmes generated more than 1 provided to beneficiaries.
Factors facilitating and hindering implementation of cash‐based approaches (chapter 7)
Evidence suggests that intervention design and implementation play a greater role in determining effectiveness and efficiency of cash‐based approaches than the emergency context or humanitarian sector.
Specific factors shown to influence implementation include resources available and technical capacity of implementing agencies, resilience of crisis‐affected populations, beneficiary selection methods, use of new technologies, and setting‐specific security issues, none of which are necessarily unique to cash‐based interventions.
CONCLUSIONS AND RECOMMENDATIONS
Despite the widespread use and increasing number of evaluations of cash‐based humanitarian assistance, there is a paucity of rigorous evidence about how best to address the needs of crisis‐affected populations. This is not surprising, as studies meeting the methodological criteria for inclusion in most systematic reviews are relatively rare in emergency settings.
Findings suggests that both cash‐based approaches and in‐kind food assistance can be effective means of increasing household food security among conflict‐affected populations and maintaining household food security among food insecure and drought‐affected populations; each assistance modality has different advantages and disadvantages that should be considered in the design of future interventions. However, no definitive conclusions on the effectiveness of cash transfer or voucher programmes could be drawn that are universally applicable for humanitarian policy. Further development of the evidence base, with more rigorous evaluations comparing the effectiveness of different cash‐based approaches (or combinations of approaches) and transfer modalities, as well as standardized approaches to documenting and comparing both costs and benefits of cash‐transfer and voucher programmes, is needed to further strengthen the evidence base in this area.
This Campbell systematic review assesses the effectiveness of community monitoring interventions in reducing corruption. The review summarises findings from 15 studies, of which seven are from Asia, six from Africa and two from Latin America.
Community monitoring interventions can reduce corruption. They also improve use of health services, but no significant effect is found on school enrolments or dropouts. There is no improvement in health service waiting times, but there is an improvement in weight for age, though not child mortality. There are beneficial effects on education outcomes as measured by test scores.
Community monitoring interventions appear to be more effective in improving outcomes when they promote direct contact between citizens and providers or politicians, and when they include tools for citizens to monitor the performance of providers and politicians. In all cases, findings are based on a small number of studies. There is heterogeneity in the findings with respect to health and education. Hence it is difficult to provide any strong, overall conclusions about intervention effectiveness.
PLAIN LANGUAGE SUMMARY
COMMUNITY MONITORING INTERVENTIONS CAN REDUCE CORRUPTION AND MAY IMPROVE SERVICES
Community monitoring interventions (CMIs) can reduce corruption. In some cases, but not all, there are positive effects on health and education outcomes. Further research is needed to understand contexts and designs for effective interventions.
WHAT DID THE REVIEW STUDY?
Corruption and inefficient allocation of resources in service delivery are widespread in low‐ and middle‐income countries. Community monitoring interventions (CMIs) are intended to address this problem. The community is given the opportunity to participate in monitoring service delivery: observing and assessing providers' performance to provide feedback to providers and politicians.
This review assesses the evidence on the effects of community monitoring interventions on corruption and access and quality of service delivery outcomes. The review also considers the mechanism through which CMIs effect a change in corruption and service delivery outcomes, and possible moderating factors such as geographic region, income level or length of exposure to interventions.
WHAT STUDIES ARE INCLUDED?
To assess the effect on corruption included studies had to have either an experimental or a quasi‐experimental design. Qualitative studies were included to assess mechanisms and moderators.
The review assesses 15 studies of 23 different programmes' intervention effects. The studies were conducted in Africa (6), Asia (7) and Latin America (2). Most studies focused on programmes in the education sector (9), followed by health (3), infrastructure (2) and employment promotion (1).
What is the aim of this review?
This Campbell systematic review assesses the effectiveness of community monitoring interventions in reducing corruption. The review summarises findings from 15 studies, of which seven are from Asia, six from Africa and two from Latin America.
WHAT ARE THE MAIN RESULTS OF THIS REVIEW?
Community monitoring interventions can reduce corruption. They also improve use of health services, but no significant effect is found on school enrolments or dropouts. There is no improvement in health service waiting times, but there is an improvement in weight for age, though not child mortality. There are beneficial effects on education outcomes as measured by test scores.
Community monitoring interventions appear to be more effective in improving outcomes when they promote direct contact between citizens and providers or politicians, and when they include tools for citizens to monitor the performance of providers and politicians.
In all cases, findings are based on a small number of studies. There is heterogeneity in the findings with respect to health and education. Hence it is difficult to provide any strong, overall conclusions about intervention effectiveness..
WHAT DO THE FINDINGS OF THIS REVIEW MEAN?
The evidence identifies CMIs as promising. That is, there is evidence that they are effective. But the evidence base is thin, the interventions do no work in all contexts, and some approaches appear more promising than others.
Future studies should assess the effectiveness of different types of community monitoring interventions in different contexts, sectors and time frames to identify when and how such programmes may be most effective in improving outcomes. There is a need for adequate information and tools to assist citizens in the process of monitoring. Research about these mechanisms and their moderation of the effectiveness of CMIs should be a priority for further research in the area.
HOW UP‐TO‐DATE IS THIS REVIEW?
The review authors searched for studies published until November 2013. This Campbell systematic review was published in November 2016.
Executive summary/Abstract
1.1 Background
In many low‐ and middle‐income countries (L&MICs) corruption and mismanagement of resources are prevalent in the public sector. Community monitoring interventions (CMIs) aim to address such issues and have become common in recent years. Such programmes seek to involve communities in the monitoring of public service providers to increase their accountability to users. However, their effectiveness in reducing corruption and improving access and quality of services remain unclear.
1.2 Objectives
This review aims to assess and synthesise the evidence on the effects of CMI interventions on access to and quality of service delivery and corruption outcomes in L&MICs. More specifically, the review aims to answer three main questions:
• 1) What are the effects of CMIs on access to and quality of service delivery and corruption outcome measures in L&MICs relative to no formal community monitoring or CMIs with less community representation?
• 2) What are the mechanisms through which CMIs effect a change in service delivery and corruption outcomes?
• 3) Do factors such as geographic region, income level or length of exposure to interventions moderate final or intermediate outcomes?
1.3 Search Methods
We searched for relevant studies across a broad range of online databases, websites and knowledge repositories, which allowed the identification of both peer reviewed and grey literature. Keywords for searching were translated into Spanish, French, and Portuguese and relevant non‐English language literature was included. We also conducted reference snowballing and contacted experts and practitioners to identify additional studies. We used Endnote software to manage citations, abstracts, and documents. First stage results were screened against the inclusion criteria by two independent reviewers, with additional supervision by a third.
1.4 Selection Criteria
We included studies of CMI in countries that were classified as L&MICs according to the World Bank definition at the time the intervention being studied was carried out. We included quantitative studies with either experimental or quasi‐experimental design to address question 1. In addition, both quantitative and qualitative studies were eligible for inclusion to address questions 2 and 3.
1.5 Data Collection and Analysis
Two reviewers independently coded and extracted data on study details, design and relevant results from the included studies. Studies were critically appraised for potential bias using a predefined set of criteria. To prepare the data for meta‐analysis we calculated standardised mean differences and 95 per cent confidence intervals (CI) for continuous outcome variables and risk ratios and risk differences and 95% CI for dichotomous outcome variables. We then synthesised results using statistical meta‐analysis. Where possible we also extracted data on intermediate outcomes such as citizen participation and public officials and service providers' responsiveness.
1.6 Results
Our search strategy returned 109,017 references. Of these 36,955 were eliminated as duplicates and a further 71,283 were excluded at the title screening stage. The remaining 787 papers were included for abstract screening and 181 studies were included for full text screening. Fifteen studies met the inclusion criteria for addressing question 1. Of these, ten used randomised assignment and five used quasi‐experimental methodologies. An additional six sibling papers were also included to address questions 2 and 3. Included studies were conducted in Africa (6), Asia (7) and Latin America (2). The 15 studies included for quantitative analysis evaluated the effects of 23 different CMIs in the areas of Information Campaigns (10), Scorecards (3), Social Audits (5), and combined Information campaigns and Scorecards (2). Most studies focused on interventions in the education sector (9), followed by health (3), infrastructure (2) and employment promotion (1).
Corruption outcomes
Included studies on the effects of CMI on corruption outcomes were implemented in infrastructure, education and employment assistance programmes. The overall effect of CMI as measured by forensic economic estimates in two studies suggest a reduction in corruption (SMD=0.15, 95% CI [0.01, 0.29).
Three studies (comprising four interventions) measured perception of corruption as an outcome measure. A meta‐analysis of two of these studies showed evidence for a reduction in the perception of corruption among the intervention group (risk difference (RD) 0.08, 95% CI [0.02, 0.13]). Another study, which was not included in the meta‐analysis due to a lack of comparability in outcome, suggests an increase in perceptions of corruption in the intervention group (SMD ‐0.23, 95% CI [‐0.38, ‐0.07]).
Access to services
A number of different outcome measures were included as proxies for access to service delivery. One study examined the effects of an information campaign and a combined information and scorecard campaign on health care utilisation. The information campaign showed no significant effect in the short term, but the information campaign and score card combined resulted in an increase in utilisation both in the short term (SMD 2.13, 95% CI [0.79, 3.47]) and the medium term (SMD 0.34, 95% CI [0.12, 0.55]).
The overall effects of two CMI interventions on immunisation outcomes suggest a positive effect in the short term (Risk Ratio (RR): 1.56, 95% CI [1.39, 1.73]). However, the medium term effect reported from one of these interventions is smaller and less precise (RR 1.04, 95% CI [‐0.52, 2.61]). Another study reporting on a range of measures of access to health services suggests an overall positive effect (RR 1.43, 95% CI [1.29, 1.58]).
Meta‐analysis of four studies which evaluated the effects of CMI on school enrolment showed an overall positive effect, but the estimate cross the line of no effect (SMD 0.09, 95% CI [‐0.03, 0.21]). The overall effect across on drop‐out across four studies is no different from zero (SMD 0.0, 95% CI [‐0.10, 0.10]).
Quality of services
For health related interventions child death and anthropometric outcomes were considered proxies for quality of service. A meta‐analysis of two studies which examined the short term effects of a score card and a combined score card and information campaign using child deaths as an outcome is not clear (RR 0.76 [0.42, 1.11]). For the score card and information campaign intervention data was available on the medium term effects and the estimate is similarly imprecise (RR 0.79, 95% CI [0.57, 1.08]). The average effect on weight for age, based on the same two studies, suggests an overall beneficial effect (RR 1.20, 95% CI [1.02, 1.38]). For the combined score card and information campaign intervention with data on medium term effects the results suggest the benefits were sustained (RR 1.29, 95% CI [1.01, 1.64]). The same two studies also looked at waiting times for services and the results suggest no difference in this outcome (RR 0.99, 95% CI [.80, 1.17]).
In education interventions test scores were used as a proxy outcome measure for quality of service. The overall effect across six studies was 0.16 (SMD, 95% CI [0.04, 0.29]).
The limited number of studies included in our review, and the limited number of included studies with information on intermediate outcomes in particular limited our ability to answer our second and third research questions regarding the mechanisms through which CMIs effect change and whether contextual factors such as geographic region, income level or length of exposure to interventions moderate final or intermediate outcomes.
Nonetheless, some exploratory evidence is provided in response to these questions, which may inform further research in the area. Some likely important moderators of the effect of CMI are having an accountability mechanism for ensuring citizen participation, availability of information and tools for citizens engaged in the monitoring process and pre‐existing beliefs regarding the responsiveness of providers to citizen's needs
1.7 Authors' conclusions
This review identified and analysed available evidence regarding the effects of CMIs on both access to and quality of service delivery and on corruption outcome measures in L&MICs. Overall, our findings were heterogeneous making it difficult to provide any strong, overall conclusions as to the effectiveness of CMIs.
However, the results suggest CMIs may have a positive effect on corruption measures and some service delivery measures.
We found the overall effect of CMIs on both forensic and perception based measures of corruption to be positive. In improving access to public sector services results were more variable. Effects on utilization of health services are not clear, but we observe an improvement in immunization rates. In the education sector, we did not find evidence of an effect on proxy access measures such as school enrollment and dropout.
We used child anthropometric measurements and deaths and waiting times for services as proxy measures for service quality in the health sector and test scores in the education sector. The evidence from two studies suggests improvements in weight for height, but no difference in child deaths or in waiting times for services. The results suggest an improvement of quality of services, as measured by improvements in test scores.
Despite limitations in our ability to synthesise evidence on the mechanisms which moderate the effects of CMIs, some important preliminary evidence was uncovered. Firstly, we identified a lack of accountability in ensuring the involvement of citizens in CMIs as an important potential bottleneck to effectiveness. Secondly, we identified the need for adequate information and tools to assist citizens in the process of monitoring. Further research on these mechanisms and their moderating effect on the effectiveness of CMIs should be a priority for further research in the area.
Market-based reforms have been promoted over the past decades to improve the performance of the power sector. This systematic review assesses the effect of market-based reforms in developing countries on intermediate outcomes like technical efficiency and the resulting impacts on electricity access. Using a pool of 70 well-designed qualitative and quantitative studies, the review synthesizes impacts of private sector involvement, privatisation, liberalisation, and regulation. This mixed-methods approach detects only few and mostly weak effect patterns for reform types sufficiently evaluated in the primary literature. The qualitative synthesis further distils factors that likely contribute to successful electricity sector reforms as tentative guidance for coherent policy delivery.
Decentralized Governance and Accountability - edited by Jonathan A. Rodden February 2019