ArticlePDF Available

The evidence barometer: increasing effectiveness of basic service interventions through the use of best available evidence

Authors:
  • Effective Basic Services (eBASE) Africa, Cameroon
  • Effective Basic Services (eBASE) Africa

Abstract and Figures

This abstract was submitted for the Evidence Leaders Africa 2020 conference and it represents an innovative work that was carried to facilitate the uptake of evidence by government and development agencies in running their programs.
Content may be subject to copyright.
The Evidence Barometer: Increasing Effectiveness of Basic Service
Interventions through the Use of Best Available Evidence.
Introduction
There has been a steady growing political focus on the quality of government programs, development
projects, and aid that is provided especially from development agencies. Despite efforts to improve
programs that support basic services, there is a lack of a sound scientific approach in measuring the level
of evidence used in development projects in public policies. Meanwhile, these programmes are the
theatres for which evidence from impact evaluations to evidence synthesis are meant to be used.
Objective
To develop and pilot an evidence brokering tool which facilitates measurement of the level of evidence
for policies and development projects and make recommendations for evidence use.
Methods
We designed an evidence brokering tool (Evidence barometer) adapted for use at the central level
(ministry) and decentralized level (districts) by policymakers and development agencies. The evidence
barometer is an automated tool that measures 10 evidence points including stakeholders’ engagement,
evidence search, and use, equity, peer review, addressing conflict of interest, the rigor of evaluation,
systems integration, resource use, ethics consideration, and report publication. Each category score
provides sets of recommendations to ensure best practices are used and how best practices can be
obtained. A maximum score of 27 is possible for each recommendation. Prior to 2017, the tool had 8
points, two points (equity and publishing reports were included in 2018) and scores were not the same
for all points e.g. stakeholder engagement can score a maximum of 4 points while peer-reviewing can
score a maximum of 2 points.
These recommendations are in simple policy and practice language.
We used the evidence barometer to evaluate project proposals in health, environment, disability, and
education, to measure how these projects planned to use evidence-based approaches and more
importantly make recommendations to include appropriate use of evidence in project implementation.
We also used the tool to evaluate projects implemented prior to 2015 (when the tool was designed) and
after recommendations from the tool were being applied.
Our initial contact was with the North West Regional delegation of public health in Cameroon during a
workshop where we did a presentation of the tool and how it could be useful in improving their projects.
We had the chance to discuss with other key stakeholders (government, development agencies, CSOs and
consumers) present who found the tool interesting during the workshop and we were invited for further
presentations at a national conference.
Results
We have developed a tool for evidence brokering called the evidence barometer for use by policymakers
and development agencies (fig 1). We evaluated 13 projects; government (04), development agencies (03)
and CSOs (06) since 2013. We compared scores before and after the development of the tool. Projects
gained 4.8 points in use of best practices with use of the evidence barometer (R: 5-16) by 2017 (fig 2).
Conclusion/Discussion
In his book ‘Effectiveness and Efficiency’ Archie Cochrane stated ‘It is surely a great criticism of our
profession that we have not organized a critical summary, by specialty or subspecialty, updated
periodically, of all relevant randomized controlled trials.’ This challenge has inspired huge databases of
synthesized evidence not only in healthcare but also in other basic services. However, the use of this
evidence remains low especially so for LMICs where best practices require interventions from
policymakers for implementation and often policymakers require some arm bending from development
partners to act.
This tool will be useful for policymakers and development agencies to say with surety that their projects
are evidence based or that they are implementing best practices or ‘what works’ as it was developed on
different evidence-based criteria. It also highlights key components required for evidence implementation
to be effective.
Perspectives
We intend to seek funding and organize capacity building workshops with key evaluators at the different
government agencies to improve their knowledge and use of the tool. We also intend to open room for
collaborations with organizations across Africa working in evaluation such as AFIDEP as this will be key in
pushing forward the use of the tool in other African countries. We also intend to continuously monitor
the use of this tool yearly and take up challenges to help improve the tool, and to continuously provide
consultations to these agencies when the need arises.
Discussions are underway with senate health committee on mainstreaming this tool. However, this
requires education, sensitization, and capacity building for senate and parliament members to better
understand evidence based decision making.
Challenges and lessons learned from applying this to government projects.
Challenges
1. Evidence Hesitancy: This is a concept we have documented during our work with government
agencies whereby there is the resistance to take up evidence-based recommendations to
improve on project development and implementation.
2. Lack of capacity: There is limited capacity to fully understand and implement evidence-based
recommendations.
3. Costs: Given that this was not funded, the cost of getting this government project was incurred
by the different authors.
Lessons Learned
1. It is not always a straight forward fix when working with government agencies especially at the
central level due to the bureaucratic bottlenecks at this level.
2. Issues around equity and conflict of interest are rarely addressed during the development and
implementation of government projects.
3. Funding is important to drive innovation.
Figure 1: Using the Evidence Barometer for Disability
Figure 2: Evidence Scores For Development Projects 2013 - 2019
0
0.5
1
1.5
2
2.5
3
Evidence Search & Use
Equity
Stakeholder Engagement
Addressing COI
Peer Review
Report published
Evaluation Rigour
Systems Integration
Resource Use
Ethics Considerations
Using the Evidence Barometer for Disability
Protocol Recommendations
2013 2013 2014 2015 2016 2016 2017 2017 2017 2017 2018 2019 2019
Score 512 11 14 12 16 17 17 16 16 24 26 24
0
5
10
15
20
25
30
Evidence Score
Evidence Scores For Development Projects 2013 - 2019
... La société civile et les centres de recherche en Afrique ont utilisé diverses approches pour encourager l'utilisation des données probantes de recherche dans les politiques et la pratique (Kamga &Okwen, 2020;Yuh et coll., 2020) et des interventions de suivi visant à promouvoir l'adoption des données probantes par les citoyens (Carneiro, 2020) Cependant, avant la pandémie du covid 19, la désinformation et de la mésinformation existaient déjà sur d'autres sujets (telles que l'autonomisation des femmes, l'éducation des filles, le plaidoyer contre le mariage précoce et la violence contre les femmes, la santé de la mère et de l'enfant, le changement climatique et la sécurité alimentaire), ce qui a eu un impact négatif important sur l'utilisation des résultats de la recherche dans la prise de décision par les décideurs politiques. Si l'impact des rumeurs sur le processus décisionnel de Covid 19 est ressenti à l'échelle mondiale, mettant en lumière l'impact considérable que la désinformation et la mésinformation peuvent avoir sur le processus décisionnel des politiques publiques, il convient de noter l'impact que ces dernières peuvent avoir sur d'autres secteurs vulnérables. ...
... Le baromètre des données probantes est un outil automatisé qui mesure 10 points de données probantes, notamment l'engagement des parties prenantes, la recherche et l'utilisation des preuves, l'équité, l'examen par les pairs, la gestion des conflits d'intérêts, la rigueur de l'évaluation, l'intégration des systèmes, l'utilisation des ressources, la prise en compte de l'éthique et la publication du rapport. Il s'agit d'un outil de courtage de données probantes adapté pour être utilisé au niveau central (ministère) et au niveau décentralisé (districts) par les décideurs politiques et les agences de développement (Kamga & Okwen, 2020 ...
... Le baromètre des données probantes est un outil automatisé qui mesure 10 points de données probantes, notamment l'engagement des parties prenantes, la recherche et l'utilisation des preuves, l'équité, l'examen par les pairs, la gestion des conflits d'intérêts, la rigueur de l'évaluation, l'intégration des systèmes, l'utilisation des ressources, la prise en compte de l'éthique et la publication du rapport. Il s'agit d'un outil de courtage de données probantes adapté pour être utilisé au niveau central (ministère) et au niveau décentralisé (districts) par les décideurs politiques et les agences de développement (Kamga & Okwen, 2020).  I have been presented the information about this study and its contents have been well understood  Or, they have read out to me the contents of the information sheet, and its contents are understood  I have been presented the goals and objectives of the study and they have been well understood  I have received answers to all questions I asked  The risks and benefits have been presented and explained to me  I have fully understood that I can accept or refuse to participate in the study  My consent does not clear the study investigators of their responsibilities, I conserve all my rights as guaranteed by the law. ...
ResearchGate has not been able to resolve any references for this publication.