Local Accountability

Abstract

The current shortage of health workers in many low-income countries poses a threat to the quality of health services. When the number of patients per health worker grows sufficiently high, there will be insufficient time to diagnose and treat all patients adequately. This paper tests the hypothesis that a high caseload reduces the level of effort per patient in the diagnostic process, using a new data set from rural Tanzania. Tanzania has a severe shortage of health workers, and previous research has pointed at high workload as a main reason for sub-standard clinical performance. We observed and evaluated the level of effort of 159 clinicians in 2,095 outpatient consultations at 126 health facilities with different levels of caseload per clinician. Surprisingly, we find no association between caseload and the level of effort per patient in the diagnostic process. In fact, clinicians appear to have ample amounts of idle time. We conclude that health workers are not overworked and that scaling up the number of health workers in this setting is unlikely to raise the quality of health services. A more promising measure for improved quality is to raise the level of formal clinical training among the clinicians, although training alone seems far from enough to raise quality to adequate levels.

Full text

Seminar Paper No. 749
LOCAL ACCOUNTABILITY
by
Martina Björkman, Ritva Reinikka and
Jakob Svensson
INSTITUTE FOR INTERNATIONAL ECONOMIC STUDIES
Stockholm University

Seminar Paper No. 749
Local Accountability
by
Martina Björkman, Ritva Reinikka and Jakob Svensson
Papers in the seminar series are published on the internet
in Adobe Acrobat (PDF) format.
Download from http://www.iies.su.se/
ISSN: 1653-610X
Seminar Papers are preliminary material circulated to
stimulate discussion and critical comment.
November 2006
Institute for International Economic Studies
Stockholm University
S-106 91 Stockholm
Sweden

Local Accountabilit y
Draft: Novem ber 2006
Martina Björkma n
∗
, Ritva R e in ik ka
o
, and Jak ob Sv ensson
#
Abstract: Identifying and implementing incentives that give rise to a strong relationship
of accountability between service providers and beneficiariesisviewedbymanyascritical
for improving service delivery. How to achieve this in practice and if it at all works,
however, remain open questions. Systematic evaluation of service delivery innovations
to increase accountability can show what works, what doesn’t and why, a first step to
scaling up success. This paper discusses one such attempt: a randomized evaluation of a
Citizen Report Card project at the community lev el in primary health care in Uganda.
The Citizen Report Card project collected quantitative information on the quality and
quan tity of health service provision from citizens and public health care providers. This
information were then assembled in "easy access" report cards that were disseminated,
together with practical information on ho w best to use this information, in community,
staff, and interface meetings b y local community organizations in order to enhance citizens’
ability to monitor the health care providers. The intervention improved both the quality
and quantity of health service provision in the treatment communities: One year into
the program, average utilization was 16 percent higher in the treatment communities; the
weight of infants higher, and the number of deaths among children under-five markedly
lower. Treatment communit i es became more extensively involved in monitoring pro viders
following the intervention, but we find no evidence of increased go vernment funding. These
results suggest that the improvements in the quality and quantity of health service delivery
resulted from increased effort by the health unit staff to serve the community.
This paper draws on Björkman and Svensson (2006). The project, Citizen Report
Cards, was a collaborative effort involving many people. Foremost, we are deeply indebted
to Frances Nsonzi for her contributions in all stages of the project. We also would like
to acknowledge the important contributions of Gibwa Kajubi, James Kanyesigye, Ivo
Njosa, Abel Ojoo, Omiat Omongin, Anthon y Wasswa, Carolyn Winter, and the field
and data staffs that we have worked with over a number of years. We wish to thank
the Ugandan Ministry of Health, Planning Division, the World Bank’s Country Office in
Uganda, and the Social Development Department of the World Bank for their cooperation.
Finally, we wish to thank the BNPP and Knowledge for Change Trust Funds and and the
Swedish International Development Agency, Department for Research Cooperation for
their generous funding for this research.
∗
IGIER - Bocconi University and CEPR. Email: martina.bjorkman@unibocconi.it.
o
World Bank. Email: rreinikka@worldbank.org
#
Institute for International Economic Studies, Stock holm Univ ersity, NHH, and
CEPR. Email: jakob.svensson@iies.su.se.
1

1 Introduction
It is becoming increasingly clear that imp rov em ents in health and educa-
tional outcomes in dev eloping countries require more than additional funds.
A wealth of anecdotal, and recently more systema tic evidence, show that pro-
vision of public services to poor people in developing countries is constrained
by weak incen tives of service providers - - sc hools and health clinics are not
open when they are suppose to; teac h ers and health w orkers are frequently
absent from sc h ools and clinics and when presen t spend a significant time
not serving the intend ed beneficiaries; equipment, even when functioning, is
not used; drugs are misused; and public funds are expropriated.
1
As argued in Chau dhury et al. (2006), this evidence is symptom atic of
failures in "street-level" institution s and go vernance, i.e. the relationships
of accountabilit y at the pro vid er lev el are dysfunctional. Howe ver, although
these failures are a direct hindran ce to economic and social dev elop m ent,
until recently, they ha ve received m uch less attention in the literature than
have weaknesses in macro institutions. In this paper, we try to partly close
this gap.
Two important relationships of accountably in service deliv er y are those
of provider-to-state (whic h in turn is partly driven by that of politicians-to-
citizens) and that of prov id er-to-th e citizen/client.
On the form er, the policymaker prov ides resources and delegates po wers
and responsibilit y for collective objectives to the service providers. En force-
ability and answerability (getting information about perform a nce) com e into
play wh en the policymaker also specifies the rewards (and possibly the penal-
ties) that depend on the service pro v ider’s actions and outputs. In practice,
enforceability and answ er ab ility is ach ieved through delegation, i.e. some-
one in the institutional hierarchy is assigned to monitor, con trol, and penal-
ize/rew a rd agen ts further do w n in the hierarc hy (i.e. the provider). The tacit
assum ption is that more and better enforcement of rules and regulations will
strengthen the pro viders’ incen tives to increase both the quan tity and qual-
it y of service provision. This is by far the most common approach to hold
providers accoun tab le. How ever, in many poor coun tries, the institutions as-
signed to monitor the providers are typically w eak and malfunctioning, and
1
For anecdotal and case study evidence, see World Development Report 2004 (World
Bank, 2004). Chaudhury et al. (2006) provide new and systematic evidence on the rates of
absen t eeism based on surveys in which enumerators made unannounced visits to primary
schools and health clinics in seven developing countries. Averaging across countries, 35
percent of health workers were found to be absent. Banerjee et al. (2004) and Dufloand
Hanna (2005) confirm these findings. On misappropriation of public funds and drugs, see
Reinikka and Svensson (2004) and McPake et al. (1999).
2

may themselves act under an incen tive system that pro v ides little incen tives
to effectively mon itor the pro vid ers.
2
As a result, the relationsh ip of accoun t-
ability of provider-to-state is in many developing countries is ineffective.
Partly in response to the failures of these traditional mechanisms of en-
forceability and answerabilit y, it has been argu ed that more effortmustbe
placed on strengthening beneficiary control, i.e. strengthen providers ac-
countability to citizen-clients.
3
In theory, beneficiary con trol, or communit y-
based monitorin g, have at least three advantages. First, it is lik e ly to be
cheaper for the beneficiaries to monitor the providers since they (at least as a
group) are better informed about the staff’s behavior than the external agent
assigned to supervise the pro v ider. Second, they may have means to punish
the pro vider that are not available to others, suc h as verb al complaints or
social opprobrium (Banerjee and Duflo, 2006). Third, to the exten t that the
service is valuable to them , they should ha ve strong incen tives to monitor and
reward or punish the pro vider - - incen tives that the external agent assigned
to supervise the pro vider may lack. Ho wever, there are also potentially large
problems associated with communit y-based monitoring. Assembling infor-
matio n about performan ce ("answerability") and acting on this informat ion
are subject to possibly large free-riding problems: the community w ould
lik e to ensure that the pro vider performs, but everyone would rather ha ve
someone else collecting information and monitoring performance. Second ,
beneficiary con tr ol is unlik ely to w ork if citizens do not ha ve a high demand
for the service or hav e access to easily available (and affordable) options (pri-
vate pro v iders). In that case, the expected relativ e return to monitoring the
public pro v ider will be lo w . T h ird, the community m u st also have some di-
rect or indirect w ay of sanctioning or rewarding the pro vider (or some higher
level arm of the state). Finally, an y project, and maybe communit y-b ased
interventions in particu lar, may be subject to captu re. For exa m p le, the elite
ma y corrupt the collection or dissemination of information or may prev en t
citizens from speaking out or putting pressure on the provider. Th us, in the
end, if and to which exten t providers accountabilit y to citizen-clients can be
strengthen and if so to what exten t such an institutional reform improves
outcom es is an empirical question.
Empirically, the c hallenges when establishing whether strengthening pro viders’
2
As an example, most anticorruption programs rely on legal and financial institutions
— judiciary, police, and financial auditors — to enforce and strengthen accountability in
the public sector. In many poor countries, though, these legal and financial institutions
are often corrupt themselves. Not surprisingly, there is scant evidence that devoting
additional resources to existing legal and financial government monitoring institutions
reduce corruption (Svensson, 2005).
3
See, for example, the 2004 World Development Report (World Bank 2003).
3

accountability to citizen-clients can improve outcom e are twofold. First, an
intervention has to be designed so that, if properly implemented, it enhances
citizens/clien ts ability to monitor and con trol the provider. Second, to evalu-
ate the impa ct of the in terven tion , one needs to establish a credible comp ari-
son group — a group of observational units (e.g. communities) whic h would, in
the absence of the intervention, have had outcomes similar to those exposed
to it.
Our approac h to deal with the first challen ge is to induce variation in two
important elements of the accoun t ab ility relationship: access to informa tion,
and participation and local organization capacit y. Imp rov ed access to infor-
mation about the beneficiaries’ (as a group) experiences and entitlements is
critical for citizens’ ability to mon itor service providers. Although people
know wheth er their o w n ch ild died or not, and whether the health workers
did an ything to help them, they t ypically do not hav e information on ag-
gregate outcomes, such as how man y c hildren in their community did not
survive bey on d the age of 5 or where citizens, on a verage, seek care. Provi-
sion of information on outcom es and performance also improves users abilit y
to ch allen ge abuses of the system, since reliable qua ntitativ e information is
more difficult for service pro viders to brush aside as anecdota l, partial, or
simply irrelevan t. Enhan ced participation and local organization capacit y is
in tend ed to m inim ize collective action problems and thus to get citizens to
act on the information being provided.
Citizen report cards, which is the community-based program evaluated
in Björkman and Sv en sson (2006), is one in terven tio n where these elements
take a cen tral focus.
4
A Citizen report card is a tool to collect feedback from
the users (and poten tial users) of public services (in this case primary health
care pro v iders) and disseminate this information back to the citizens/u sers so
they have reliable information about how their communit y at large views the
quality and efficacy of service delivery. It also pro vides the community with
an opportunit y to comp are service delivery in their community vis-à-vis other
communities, or across districts and municipalities in the coun try at large.
The citizen report card meth odology also emphasizes the activ e dissemination
4
The best known examples of citizen report cards is probably those developed by the
Public Affairs Centre in Bangalore, India (Paul, 2002). Citizens were asked to rate service
access and quality and to report on concerns about public services, general grievances,
and corruption. The information was summerized in report cards that were reported in
the press and in civic forums. Citizen report cards have spread beyond Bangalore to cities
in Ken ya, Mozambique, the Philippines, Ukraine, and Vietnam. They have been scaled
up in India to cover urban and rural services in 24 states. Overall, the citizen report
cards have stimulated considerable media and political attention, and there is, despite
any scientific evidence to back this up, general acknowledgment in policy circles of their
positive contribution to service improvements (see e.g. World Bank, 2003).
4

of informatio n in order to create a wareness and inv oke participation of the
comm unit y.
We rely on a random ized design to deal with the second challenge. By
rando m ly assigning com munities into a treatmen t group (i.e. communities
in which the Citizen report card project w ere implemented) and a control
group (i.e. communities in which the Citizen Report Ca rd project was not
implemented), we are relatively confident about the absence of confound-
ing factors. In addition, the intervention we evaluate was run on a large
scale - - approximately 5,000 households from 50 "communities" from nine
districtsinUgandahavebeensurveyedintworounds,andintotalthere
are approximately 110,000 households residing in the treatment and con trol
communities.
5
This increases our confidence in the external validity of the
results.
Björkman and Sv ensson (2006) show that the community-based moni-
toring interven t ion (Citizen report card project) increased the quality and
quantit y of primary health care provision and resulted in improved health
outcomes. One year into the program , utilization (for general outpatient
services) w as 16 percen t higher in the treatmen t facilities. We also find sig-
nifican t differences in deliv er ies at the treatment facilities, and in the use of
antenatal care and family planning. Treatment practises, as expressed both
in perception responses by households and in more quan titative indicators
(immunization of c h ildre n, waiting time, exam ination procedures) improv ed
significantly in the trea tm ent communities. We find a small but significant
difference in the weig ht of infants and a markedly low er n u mber of deaths
among children under-five in the treatmen t comm unities. No effect is found
on investments or financial or in-kind support (from the go vernment), sug-
gesting that the changes in the quality and quan tity of health care provision
are due to behavioral changes of the staff. Moreover, w e also find evidence
that the treatment clinics started sharing information about treatmen t prac-
tises, availabilit y of drugs, and service delivery in general, in response to the
in terven tion and that the treatment communities began to monitor the health
unit more extensively. This reinforces our confidence that the findings on the
quality and quantit y of health care prov ision resulted from increased efforts
by the health unit staff to serve the communit y in ligh t of better community
monitoring.
5
A "community" is operationalized as the households (and villages) residing in the
five-kilometer radius around the facility; see section 5.
5

2 Literature Review
There is a growing empirica l literature on the relationship bet ween informa -
tion dissemination (through the media) and accountability. Alm ost exclu-
sively this literature studies the relationships of accountability of politicians
to citizens and deal with one - - periodic elections out of several mechanism
through which citizens can make politicians and policym akers accountable.
For example, Strömberg (2003, 2004) considers how the press influences re-
distributive progra m s in a model of electoral policies, where the role of the
med ia is to raise vo ter a wareness, ther eby increasing the sensitivity of turnout
to favors granted. Besley and Burgess (2002) focus on the media’s role in
increasing political accountability, also in a model of electoral policies. Fer-
raz and Finan (2005), study the effects on the probabilit y of the incumbent
winnin g the election of m aking information about corruption in the local gov-
ernments public. Besley and Prat (2005) study the in terdependence bet ween
media and gov ernment accoun tabilit y, but focus on the rev erse relationship:
ho w the go v ernmen t can influence what information will be pro vided. Our
work differs in sev era l important dimension s. First, we focus on mec ha nism s
through which citizens can make providers, rather than politicians, account-
able. Th us, we do not study the design or allocation of public resources
across communities or programs, but rather ho w already existing resources
are utilized. Second, we use micro data from hou seho lds and health clinics
rather the disaggreg ated nationa l accounts dat a. Finally, we identify im -
pact using an experimental design, rather than exploiting non-experimen tal
data. The source of identification will th us come directly from a randomized
experimen t.
Reinik ka and Svensson (2005a) also study the relationsh ip between in-
formatio n, accountability, outcomes at the prov ider level. They exploit a
newspaper campaign aimed at reducing the capture of public funds by pro-
viding schools (parents) with information to monitor local officials’ handling
of a large education gran t program (capitation grant). They find that the
newspa per campaign was highly successful. Head teach ers in schools closer to
a newspaper outlet are more knowledgeable of the rules go verning the grant
program and the timing of releases of funds b y the cen tral governmen t. These
sc h ools also managed to claim a significantly larger part of their en titlem ent
after the newspaper camp aign had been initiated. Reinikka and Sv e nsso n
(2005b) and Björkman (2006) tak e these results as a starting poin t to ex-
plore the effects of increased "client pow er" on school outcom es. They show
that the reduction in capture had a positive effect on both enrollment and
studen t learning. The newspaper campaign in Uganda, ho w ever, may not be
easy to scale up in other sectors or for more complex government programs.
6

Specifically, the capitation gran t is a very simple entitlem ent project and a
relativ e ly small item in a vast government bu dg et. They also identify impact
using a non-experimental approach, rather than an experimen tal design as
w e do here.
Impro ving governance and public service delivery through community
participationisanapproachthathasgainedprominenceinrecentyears.
For exam ple, the 2004 World Developmen t Report is entirely devoted to
the concept of increasing poor citizens’ voice and participation in service
delivery in order to help them monitor and discipline providers. However ,
despite the enth usiasm for comm unity participation and monitoring, there
is little credible evidence on the impact of policy in terv entions aimed at
achieving these. On the one hand, most comprehensive comm unity based
monitoring initiativ es have not been rigorously evaluated. On the other hand,
the few studies relying on rigorous impact evaluation strategies have not
evaluated more comprehensiv e attempts to inform and involve the communit y
in monitoring public officials.
On the latter issue, Olk e n (2005) evaluates different w ays of m onitor ing
corruption in a road construction project in Indonesia. In one of the ex-
perim ents, in vitat ions were sen t out to village-level meetin gs where project
officials documented how they spent project funds for local road construction.
How ever, although the invitations increa sed the num ber of people particip at-
ing in the meetin gs, the meetings we re still dominated by members of the
village elite. Moreov e r, corruption is not easily observable and project offi-
cials m ay very well be able to hide it when reporting on ho w funds were used.
The data also reveal that corruption problems w ere seldom discussed in these
meetin gs.
6
Th us, it is unclear to what exten t non-elite communit y members
were really more informed about corruption in the project, or if they had any
means of influencing outcomes, in response to the in terven tion. Given these
constra ints, it is not surprising that Olken (2005) only finds minor effects of
the intervention.
Using a random ized design, Banerjee, Deaton and Duflo (2004) evaluate
a project in Rajasthan in India where a member of the community w as paid
to chec k once a week, on unannounced days, whether the auxiliary nurse-
midw ife assigned to the health center was present in the cen ter . Unlike
Olken’s study, getting reliable information is not a concern here. In fact,
6
The information problem is illustrated in the novel but burdensome way in which
Olken (2005) estimates the extent of corruption. Specifically, Olken (2005) assembled a
team of engineers and surveyors who dug samples in each road to estimate the quantity of
materials used and then, using price information from local supplies, estimated the extent
of "missing" expenditures. The corruption estimates were not reported in the village
meetings.
7

external monitors confirmed the absence rates documen ted by the community
members assigned to the project. The issue is rather how the informed
community member could use his or her information on absen teeism to in vok e
comm unit y participation. The interv ention had no impact on attendance.
Thus,havingoneinformedperson,evenifthisisdoneisastructuredand
regular way , may not have much impact.
Jimen ez and Sa wada (1999) examine ho w decen tralizin g educational re-
sponsibility to communities and sch ools affects studen t outcom es. They
study El Salvador’s Community-Managed Sc h ools Program, EDUC O , and
its effect on studen ts’ ac h ievement on standardized tests and attendan ce as
com par ed to students in tradition al schools. EDU C O sc h ools are m anag ed
autonom ou sly b y community education associations whose elected members
are parents of the students. The comm unit y education associations are re-
sponsible for hiring (and firing) teachers, closely monitorin g teachers’ perfor-
mance, and equipping and maintaining the sc hools. The results sho w that
enhanced comm unit y and paren tal in v olv ement in EDUC O sc hools improved
students’ language skills and reduced studen t absences. A key estimation
issue in this paper is endogenous program participation and although the
authors instrument for program participation by using the proportion of
ED U C O sc h ools in a m unicipa lity, it is not obvio us that they manag e to
obtain the causal treatmen t effect.
3 Communit y-based Monitoring
How is accountability ach ieved in the public sector? To begin to answer this
question, compare ho w accoun tab ility is ach ieved in many market contexts.
In the market, dissatisfied consumers can successfully use the exit option;
i.e., if the price is too high or the qualit y too low, the consumer can choose
not to buy the good or buy from another producer. If man y consumers act
in the sam e w ay, this will influence the producer’s profitability and, in the
end, its survival in the market. The exit mechanism, however, may not work
well in the pub lic sector. First of all, in som e cases there may be no easily
available alternative to the local pu blic pro v id er - - sa y a primar y health
clinic. More important, the link bet ween the public pro vider ’s performance
and its financial position (or its staff’s remuneration) is often weak or non-
existent. Typically, public money does not follow the patients and hiring,
salaries, and prom otions are determined largely by educational qualifications
8

and seniorit y.
7
In theory, gover nm ents ha ve tried to compensate the lac k of a well-
functioning exit mec hanism (or competition effect) b y increasing control and
supervision, i.e. to strengthen the relationship of accoun tably of the pro vider
to the state. The political agency literature demonstrates that when indi-
viduals and households are well informed and hav e mechanisms to sanction
politicians - - for exam ple the right to v o te them out of office - - politicians
have potentially strong incen tives to monito r and pressure public institutions
to do what individuals and households, whom they represent, want.
8
How-
ev er , the mounting evidence of failures in "street-level" institutions across
the dev eloping world suggest that this mec h anism is clearly not enough.
Why doesn’t the political system generate dema nd s for stronger supervi-
sion and con trol of pro v iders? There are at least three explanations for this.
First, supervision and control are not perform ed solely b y politicians them -
selv es but instead delegated to various institutions and agencies. Howe ver,
man y poor countries lack the trustworthy machinery/institutions (judges,
court personnel, police, auditors) needed to supervise and enforce rules.
Thus, politicians, even if they wanted , are restricted (at least in the short
run) in their attempts to supervise and control pro viders. Seco nd, while
well-functioning legal and financial systems can curtail ob vious cases of mis-
managemen t, they only partially constrain the discretionary po wers of public
sector managers and employ ees. The complexity of the tasks perform ed by a
t ypical public sector unit and its informational advantage relativ e to the users
of public services m ake it nearly impossible to design legal and accounting
measures to address all t ypes of misuse and thus to curtail less ob v ious cases
of mismanagement (suc h as shirking, budget prioritization in fa v or of staff,
political considerations). Third, campaignin g on crac k ing do w n poor perfor-
mance of health staff ma y not be a winning strategy as noted in Chaudhury
et al. (2006). The pro viders are an organized interest grou p, whereas clien ts,
particular in health, are diffuse. T ho se poor enough to use public clinics
may have less political power than the organized and middle class health
workers. In many countries, including Uganda, those who are moderately
well off use private clinics. This pattern may create a self-reinforcing cycle
of low qua lity, exit of the politically influen tial from the public sector, and
further deterioration of quality (Chaudhury et al. 2006, Hirschman 1970).
This effect is com pounded by the fact that people lack information on ser-
vice delivery outco m es. As stressed by Khema ni (2006), although people
7
This is the case in the health sector in Uganda but also the case more generally in
developing countries, as discussed in Chaudhury et al. (2006).
8
For references and textbook treatment of the literature, see Persson and Tabellini
(2000).
9

ha ve private information about health outcomes and whether the pro vider
did an yth ing to help them, they are unlik ely to hold their remotely located
politician accountable for this private experience. They typically do not have
inform ation on agg regate development outcomes, such as ho w many children
in their community did not surviv e beyo nd the age of 5, where citizens, on
a verage, seek care, and extent of imm u niza tion of children in the communit y,
that are more likely to be linked to the actions of their political represen-
tativ es. And ev en if they can guess that others in their neigh borhood are
suffering similar tragedies, people might be sceptical of using these estim ates
of actual dev elopment outcom es as an indicator of politicians’ performance,
focusing instead on simple actions they can directly observ e, suc h as an-
nouncement of a price subsidy, provision of jobs, or infrastructure projects.
Politicians will respond accordingly : they will focus attention to inefficient,
and som etim es ineffective, policies of targeted transfers, shifting effort and
resources a way from supervision and reform of the public sector (Keefer and
Khem ani, 2005). The public, in turn, realizes these incen tives and their ex-
pectations of what they can get from the pro vid ers that are mean t to serve
them are therefore low .
As men tion ed earlier, partly in response to the failures in "street-lev el" in-
stitutions, it has been argued that more effo rt must be placed on strengthen-
ing beneficiary con tr ol.
9
In theory, beneficiary control, or community-b ased
mon itoring, ha ve at least three advan ta ges. First, it is likely to be ch eaper
for the beneficiariestomonitortheproviderssincethey(atleastasagroup)
are better informed about the staff’s behavior than the external agent as-
signed to supervise the pro v ider. Second, they may have means to punish the
provider that are not available to others, such as v er ba l complaints or social
opprobrium (Banerjee and Duflo,2006). Third,totheextentthattheser-
vice is valuable to them, they should have strong incentives to monito r and
reward or punish the pro vider - - incen tives that the external agent assigned
to supervise the provider may lac k.
In practise, community-b ased monitoring can take a variety of forms.
10
How ev er, the different innovations share some comm on features. Specifi-
cally, a key ingredien t is better information about the beneficiaries’ (as a
group) experiences and entitlem ents. Access to suc h information is viewed
as critical for citizens’ ability to monitor service providers. It also improves
9
See, for example, the 2004 World Development Report (World Bank 2003).
10
Examples of this approach include participatory budgeting in Porto Allegre, Brazil;
citizen report cards in Bangalore, India; right to information on public works and public
hearings or jan sunwais in Rajasthan, India; public information campaign to reduce cap-
ture of school funds in Uganda; and community scorecards in Malawi (see Reinikka and
Svensson, 2004; World Bank, 2003; Paul, 2002; and Singh and Shah, 2002).
10

users abilit y to challenge abuses of the system, since reliable quantitativ e
inform a tion is more difficult for service prov iders to brush aside as anec-
dotal, partia l, or simp ly irrelevant. How ever, better information in itself
may not be enough. For beneficiary control to be effective, the expected
return to monitoring must be higher than to the alternativ es - to either do
nothin g (for examp le, self-treatme nt rather than visiting the dysfunctional
public health care pro vider) or use alternativ e pro viders (private for-profitor
not-for-profit health providers). The relative returns depend on a variet y of
factors. Forem ost, it depends on the communit y’s abilit y to take collective
action. Com munity-b ased monitoring is subject to possibly large free-riding
problems: the comm unit y would lik e to ensure that the staff do their job,
but every on e wo uld rather have som eone else to do the m onito ring . For
these reasons, many com munity -base d monito ring initiatives place a strong
empha sis on encouragin g active participation by communit y members as a
way to minim ize the collectiv e action problem. Second, if the beneficiaries
do not ha ve a high demand for the service, or if there are easily a vailable
options (private pro viders), the expected relative return to monitoring the
public provider will be lo wer. Finally, the relative retu rns also depend on cit-
izens’ abilit y to either directly or indirectly sanction or rew ard the provider
(or some higher lev el arm of the state).
Thus, there are both clear potential advanta ges and potentially large
problem s associated with communit y-b ased m onito ring. In the end, if and
to which extent com munity based m on itoring works is therefore an empirical
question. We turn to this next.
4 In stitutional setting
Ugan da , like many newly independent countries in Africa, had a function ing
health care system in the early 1960’s. Accessibilit y and affordability were
relatively extensive. The 1970 ’s and 198 0s sa w the collapse of Gov ern m ent
services as the country underwent political uphea val. Health indicators fell
dramatically during this period un til peace w as restored in the late 1980s.
Since then, the Government has been implemen ting major infrastructure
rehabilitation programs in the public health sector. Some health indicators
ha v e impro ved, while others have not. For example, the infant mortalit y
rate stagnated at 88 deaths per 1,000 liv e births during the latter half of the
1990s (Republic of Uganda 2002, Moeller 2002) and maternal mortalit y has
remained high and immunization rates stagnant since the late 1990’s. This
is despite a GDP growth rate of 64 percent and a 40-percent reduction in
11

consumption po verty in the 1990s (Appleton 2001)
As of 2001, public health services are free of charge. Anecdotal and survey
evidence (see below), ho wever, suggests that users still encounter varying
costs and that suc h costs defer man y, especially the poor, from accessing
services.
The health sector in Uganda is composed of four types of facilities: hos-
pitals, health cen ters, dispensaries (health center III), and aid posts or sub-
dispensaries. These facilities can be government, private for-profit, or pr ivate
not-for-profit operated and o wned. The focus of this impact evaluation is on
the dispensary (leve l III). Dispensaries are closest to the users and the low est
tier of the health system where a professional intera ction between users and
providers tak es place. Most dispensaries are rural (89 percent). According
to the government health sector strategic plan, the standard for dispensaries
includes preventive, promotiona l, outpatien t care, maternity, general ward,
and laboratory services (Republic of Uganda 2000). Dispensaries are m anned
by a clinical officer (who can be a medical doctor). In our sample of facil-
ities, on average, a dispensary wa s staffed b y a clinical officer, three n ur ses
(including midwives), and three n ursing aids or other assistants.
The health sector in Uganda is decentralized and supervision and con tro l
of the dispensaries are go verned at the district lev el. A n umber of actors are
responsible for the functioning of the dispensaries. The most important local
actor is the Health Unit Management Com m ittee (HU MC), which is the main
link between the communit y and the health facilit y. Each dispensary has an
HU M C which consists of members from both the health facility staff (the
in-charge) and non-political representativ es from the community (elected b y
the sub-coun t y local council). The HUMC should monitor drugs and finances
disbursed to the facility, as well as the day-to-day running of the health
facilit y (Republic of Uganda 2000). The HUM C can warn the health facility
staff on m atters of indiscipline, rudeness to patients and misappropriations of
funds b y recommending that the staff be transferred from the health facility.
How e ver, the HUMC has no authority to dismiss the health facility staff.
In cases of problem s at the health facilit y, the working pr actice is that the
c h airperson of HUMC raises the issue with the in-charge. If there is no
improv em ent, the matter should be referred to the Health Sub-district which,
if it fails, w ill refer the errand to the Director of District Health Services.
The Health Sub-district mon itors funds, drugs and service delivery at the
dispensary. Supervision meeting s b y the Health Sub-district are supposed
to appear quarterly but, in practise, monitoring is infrequent. The Health
Sub-d istrict, as well as the Director of D istrict Health Services, ha ve the
autho rity to reprimand, but not dismiss, health facility staff for indiscipline.
Cases of dismissal are reported to the Chief Adm inistrative Officer of the
12

District who will then report suc h cases to the District Service Comm ission ,
which is the appoin ting auth o rity for the district and has the authority to
suspend or dismiss staff.
Com munit y Based Organizations (CB Os) are also importan t actors in
the health service deliv er y system at the local lev el. CBO s involved in health
main ly focus on undertaking health education activities in anten atal care,
family planning, HIV/AIDS prevention, etc.
5 The Pr ogram: C it iz en R e port Ca rd
In response to perceiv ed continu ed wea k health care deliv ery at the primary
level, a pilot project (Citizen report cards) aimed at enh ancin g community
involvement and monitorin g in the delive ry of primary health care wa s initi-
ated in 2004. The project w as carried out b y staff from the World Bank and
Stockholm Univ ersity, in cooperation with a number of Ugandan practition-
ers, 18 community-based organizations, and the Uganda Ministry of Health,
Planning Division. The 50 project facilities (all in rural areas) w ere dra w n
from nine districts in U gand a (see Björkm an and Svensson, 2006, for details).
Defining the catchme nt area (or the “communit y ”) of each dispensary as the
households (and villages) residing in the five-kilometer radius around the
facilit y, approximately 110,000 households reside in the communities suppos-
edly served b y these 50 facilities.
The facilities w ere first stratified b y location (districts) and then by the
num ber of household s residing in the catchment areas. From each group, half
the units were randomly assigned to the treatment group and the remaining
25 units w ere assigned to the control group. H en ce, within eac h district,
there exist both treatmen t and contr ol units.
The Citizen report card project had four components: (a) collecting quan-
titative information from users (citizens) and service provid ers using micro
survey tech niques; (b) assembling this information in "easy access" report
cards; (c) disseminating the report cards to users and providers in suc h a w ay
as to create a wareness and invok e participa tion; (d) providing communities
with practical information on ho w to best use the inform ation to mon itor
and, in the end, impro v e the qualit y and quantit y of service pro vision. These
componen ts are discussed next.
Data collection was governed b y two objectiv es. First, data were required
to assem ble report cards on how the community at large views the qualit y and
efficacy of service delivery. We also wanted to contrast the citizens’ view with
that of the health unit staff. Second, data were required to rigorously evaluate
13

the imp act. To meet these objectiv es, two surveys w e re implemented: a
survey of health care providers and a survey of health care users. T hese
surveys were implemented both prior to the intervention (data from these
surveys formed the basis for the in terven tion) and one year after the project
had been initiated.
A quantitativ e service deliv er y survey (QSDS ) has been used to collect
data from the health service providers. The QSDS collected detailed quan-
titative data on performance and outcomes from the providers. In many
respects, a QS D S is similar to a standard firm-level survey. The k ey differ-
ence is that it explicitly recognizes that agents in the service delivery system
may have a strong incentive to misreport (or not report) key data. To this
end, the data are obtained directly from the records kep t by facilities for
their own need (i.e. daily patient registers, stock cards, etc.) rather than
from administrativ e records submitted to the local governmen t. The former,
often available in a highly disaggregate format, were considered to suffer the
least from any incentiv e problems in record-k eep ing .
The user/household survey collected quan titative, and some perception
based, data on both househ old s’ health outcomes and health facility perfor-
man ce. It included indices of performance param eters suc h as availabilit y,
access, reliability, quality and satisfaction. Data were collected on all dif-
feren t services provid ed b y the health facility, i.e. daily out-patient service,
family planning, imm unization, and antenatal care. To the exten t that it was
possible, household responses were supported by patien t records; i.e., patien t
exercise books and immunization cards. These records helped the household
recall details about its visits to the health facility and also minimized the
problem of misreporting. The post-interv ention household surv ey also in-
cluded a shorter module on health outcom es, including data on under-five
mortality, and all infants in the surveyed households were w eighed.
Astratified random sample of households within the catchm ent area of
the facility w ere surv eyed. In total, roughly 5,000 households hav e been
surveyed in eac h round. In a typica l com munity, households from six villages
were surv eyed.
11
Thedatafromthetwopre-interventionsurveyswereanalyzedanda
smaller subset of the findin gs w ere assem bled in report cards for the treat-
ment localities.
12
The data included in the report cards w ere iden tified as
key areas subject to improvem ent and include utilization, services, drugs
and user c h arges and comparison s vis-à-vis other health facilities in the dis-
11
See Björkman and Svensson (2006) for a detailed explanation of the design and im-
plementation of the surveys.
12
Thus, the design and size of the surveys were largely driven by the second objective —
to evaluate impact.
14

trict and the coun try at large. E ach treatment facilit y and its communit y
had a unique report card summarizing the key findings from the surveys in
a form a t accessible to the communities.
The report cards w ere translated into the main language spoken in the
comm unit y.
13
To support the illiterate community members, posters were
specifically designed and painted by a graphical artist so that otherw ise com-
plex information and concepts w ere easily understood. As the information
in the report card wa s largely statistical, the posters con vey ed the principal
ideas suc h as where people go to seek medical care, reasons for this beha vior
etc.
14
The informa tion in the report cards was disseminated to citizens and
pro viders using a "participatory rural appraisal approac h".
15
The informa-
tion dissemination process was facilitated b y staff from Com munity-b a sed
Orga nization s (C B O ). T hese facilitators we re perceiv ed to be a good conduit
through whic h the Citizen Report Card project could be delivered, since they
were in constan t in ter action with the com munities and had a manda te dra w n
from a long-term presence on the grou nd wo rking with the communit y. In
addition , they could easily make follo w-up visits and provid e support to the
communities.
16 ,17
The objective of the dissemination process w as threefold. First, to allo w
the communit y members themselves to analyze and draw conclusions from
the summary findin gs in the report cards. Second, to dev elop a shared view
on how to monitor the provider by discuss ing and deco mposin g the various
elements of accoun tability in the primary health sector (who is accountable
to whom; what is a particular actor accoun table for; how can these actors
account for their actions,andhowaretheseelementsreflectedinthereport
card findings). Third, to ensure that the process is not captured by the
13
In the end, the report cards were translated into six different languages: Ateso (Soroti),
Lusoga (Iganga), Lango (Apac), Luganda (Masaka, Wakiso, Mukono and Mpigi), Run-
yankore (Mbarara) and Lugbara (Arua).
14
See Björkman and Svensson (2006) for prototypes of these posters.
15
Participatory rural appraisal (PRA) is a label given to a gro wing family of participa-
tory approaches and methods with the common aim of enabling people to make their own
appraisal, analyses, and plans. PRA evolved from a set of informal techniques used by
development practitioners in rural areas to collect and analyze data (World Bank, 1996).
16
The CBO facilitators were trained for seven days in data interpretation and dissemi-
nation, utilisation of the participatory methodology, and conflict resolution and manage-
ment. In addition, a trained enumerator recorded the findings and the process of the CBO
facilitated intervention.
17
It should be noted that various CBOs (including some participating in the project)
alsooperateinthecontroldistricts. Thus,thepresence(andnumbers)ofCBOsinthe
project communities is similar across treatment and control groups.
15

eliteoranyotherspecific sub-group of the community. To this end, a vari-
ety of methods were used, including maps, diagrams, role-play, focus group
discussions and action planning.
18
The inform ation dissemination process was conducted in three separate
meetin gs: a community meeting; a staff meeting; and an in terface meeting.
The community meeting wa s a two-da y (afternoons) ev ent with approx-
imately 100 in vited participants draw n from the surveyed villages in the
catchment area of the health facility. The in v ited participants from eac h
village consisted of a selection of rep resentatives from different spectra of
society (i.e. yo un g, old, disabled, wo m en , moth ers, leaders). The facilitators
mob ilize d the villag e members by cooperating with Local (Village) Council
representativ es in the catch m ent area. Invited participants were asked to
spread the word about the meeting and, in the end, a large n umber of un-
invited participants from other villages who had found out about the even t
also attended the meeting. A typical village meeting was attended b y more
than 150 participan ts per day.
In the community meeting, the facilitators shared the information in the
report card with the communit y members using methods that aimed to solv e
the collectiv e action problem. These participatory methods enabled the com-
m unity mem bers to make their o wn appraisals and plans for action and mon-
itoring (see Björkman and Svensson (2006) for information on the specific
meth ods used in the informa tion disseminatio n process). In addition to dis-
seminating findings in the report card, the facilitators also presented infor -
matio n on patients’ righ ts and entitlements.
19
At the end of the meeting, the
community’s suggestions for improv em ents (and ho w to reac h them without
additional resources) were summarized in an action plan. The action plan
con ta ined information on health issues/services that had been identified as
the most important to address; how these issues could be addressed; and
ho w the comm unit y could monitor impro v emen ts (or lack thereof). After
this two-da y meetin g, participants from each village were giv e n posters and
copies of the report card to bring back to their villages and share with their
village members.
18
See Björkman and Svensson (2006) for a more detailed description of the various
methods.
19
Information on patien ts’ rights and entitlements was based on the Yello w Star program.
In 2000, the MoH developed a quality of care strategy called the Yellow Star Program with
the aim of improving and maintaining basic standards of care at government and NGO
health facilities. The rationale behind this strategy was the general concensus that the
quality of health services had been a major deterrent to service utilization. The Yellow
Star Program lists a set of basic standards of quality. The standards fall into six categories:
Infrastructure and Equipment; Management systems; Infection prevention: Information,
Education and Communication; Clinical skills; and Client services.
16

The health facilit y staff meeting w as a one-day (afternoon) meeting held
at the health facility with all health facilit y staff presen t. In this meeting,
the facilitators contrasted the information on service provision as reported b y
the pro vider with the findings from the household surv ey, i.e. the report card.
The meeting enabled the pro viders to review and analyze their performance,
and compare their performan ce with other health clinics in the district and
across the country.
Follo w ing the com munit y and the health facility meeting w as an interface
meetin g with participants (c ho sen at the community meeting) from villages
in the catchmen t area and all health facilit y staff. The objective of this
meeting was to agree on a strategy for improved health care pro vision, based
on the action plan developed in the comm unity meeting and the discussions
from the health facilit y meeting. During the in te rfa ce meeting, the commu-
nity and the health facility staff presen ted and discussed their suggestions
for impro vements. A role-pla y wa s used to disseminate the results from the
survey and in this pla y, the communit y and the staff took reverse roles. The
participants discussed their rights and entitlements and their roles and re-
sponsibilities as patients or medical staff. The outcome of this meeting was
a joint action plan describing ho w the staff and the comm unity collectively
can best improv e service deliv ery within the existing resource en velope. The
plan con tained and reflected the community ’s and the service provider’s con-
sensus on what needs to be done, how, when, and by whom. The joint action
plan iden tified how the communit y was to monitor the pro vider and a time
plan. Copies of the action plan w ere kept with the comm unity and the health
facilit y to support the monitorin g process that follo wed.
The Citizen Report Card process involv ed both follow-up and repeat en-
gagements with the aim of institutionalizing the process. To this end, the fa-
cilitators supported the communities with follow-up meetings. This w as done
as an integ rat ed part of the CBO’s ordinary work in the villages. Each com-
m unit y had appro ximately two follow-up meetings in the subsequen t months.
After a period of six mon t hs, the communities and health facilities w ere
revisited and a mid-term review was conducted. The mid-term review w as
a repeat engagement on a smaller scale which included a one-day commu-
nity meeting and a one-day in terface meeting and it aimed at tracking the
implem entation of the action plan, possibly drawing new areas for concern,
and coming up with a new set of recomm enda tion s for improvem ent. The
action plans made in the earlier interven tion w ere prin ted on posters and
they formed the ground for the discussions in the mid -term review. The
facilitators presented the inform atio n on the prin t ed action plans, followed
b y focus group discussions on the progress. During the interface meeting,
the health facility staff and the community members join tly discussed sug-
17

gestions on actions for improving or sustaining the progress of the previously
determin ed action plan. In cases where improvements had not been made,
new recommend ations w er e agreed upon and noted in the updated action
plan and in cases where improvem ents had been made, suggestions for sus-
tainability wer e recorded in the plan. The updated action plan w as kept w ith
the community and the health facilit y to assist in the continued work and
monitoring process.
6 Evaluation Design and pre-treatment char-
acteristics
Empirically, the c hallenge when establishing whether (and if so which) insti-
tutional arrange m ents can foster a stronger degree of accountability between
service providers and citizens, is to establish a credible comparison group — a
group of observational units (e.g. communities) whic h w ou ld, in the absence
of the in terven tio n, have had outcomes similar to those exposed to the inter-
ventio n. To achieve this, w e rely on a random ized design, i.e. facilities w ere
first stratified by location (districts) and then by the number of households
residing in the communities. From each group, half the units w ere randomly
assigned to the treatment group and the remaining 25 units w ere assigned
to the con trol group. Since treatment status was randomly assigned across
health units (and their catchm ent areas), program participation is not corre-
lated in expectation with either the observed or the unobserved health unit
or communit y c h aracteristics.
Prior to the interven tion, the treatment and the con trol groups were sim-
ilar in mo st ch aracter istics. Thus, the randomization appears to ha ve been
successful. As reported in Björkman and Svensson (2006), there are no sta-
tistically significa nt differences across the two groups in utilization (num-
ber of outpatient treated and deliv eries per month), use of differ ent service
providers (including drug shops) in case of illness, wa iting time, equipm ent
usage, gov ern m ent fund in g, citizens’ perceptions of staff behavior, catchment
area cha ract eristics (suc h as the n umber of villages and households in catc h-
ment area), distan ces from th e health facility to the nearest local council and
gover nment facilit y, supply of drugs, user-ch arg es, or health facility charac -
teristics (such as type of water source, availability of drinking water at the
facilit y, whether a separate m atern ity unit is a vailable, electricit y shortages).
18

7Results
Because of the randomized design of the project, causal effects can be deter-
mined by simply comparing means across treatmen t and control groups. Be-
low w e discuss the m ain findings reported in Björkman and Svensson (2006).
Björkm an and Svensson (2006) study four sets of outco m es: utilization (or
quantity of health care), treatment practises (as a measure of quality of
health care), health outcomes (a measure of both quantity and quality), and
price setting. They also study what observable initiativ es were tak en by the
comm unities to strengthen their role as monitors of the health pro vider.
Theinterventionhadlargeandsignificant effects on utilization. Björk-
man and Sv en sso n (2006) exploit two sour ces of information on utilization.
First, detailed informatio n on the n umber of out-patien ts, the number of
deliveries, the n umber of an ten atal care patien ts, and the nu mber of people
seeking family planning services w ere assembled by counting the n umber of
patients from health facility ’s daily patient records, maternity unit records,
the antenatal care register, and the family planning register. Second, deta iled
inform ation on each househ old member’s decision of where to seek care in
case of illness that required treatment were collected in the househo ld surv ey.
Based on facility level records, the differences in means across treatmen t
and con trol groups are positive and significant across all four services (out-
patien ts, deliveries, anten atal care patien ts, and people seeking family plan-
ning services). One year in to the program, utilization (for general outpatient
services) is 16 percent higher in the treatmen t facilities. The difference in the
num ber of deliveries at the facility (albeit starting from a low level) is ev en
larger (68 percen t). There are also positive and significant differences in the
num ber of patients seeking anten atal care (20 percent) and family planning
(63 percen t).
A simila r patter n is evident from the utilizatio n data collected from house-
holds. There is a positiv e and significa nt differen ce in the use of the project
facilit y between the treatm ent and con tro l facilities following the in terven-
tion. The increase, 15 percent higher in the treatment group as compared to
the con tr ol group, is similar to that found using facilit y records. The utiliza-
tion pattern deriv ed from the household data also reveals that households in
the treatment community reduced the n umber of visits to traditional healers
and the extent of self-treatmen t follow in g the interve ntion, while there are no
statistically significant differences across the two groups in the use of other
pro viders (not-for-profitandforprofit facilities). Th us, household s in the
treatm ent communities switc h ed from traditional healers and self-treatment
to the project facility in response to the interven t ion.
Björkman and Svensson (2006) use t w o complemen tary approac hes to
19

measuring the quality of care. First, they study beha vioral (and measurable)
changes at the facility level, such as immunization of child ren , waiting time,
examination procedur es, and household’s perceptions of the quality of care.
Second, they study changes in health outcomes (whic h is a function both of
the qualit y and quantity of care).
Househ olds in the treatment communities reported large improvem ents
in the quality of care. For example, a majority (54 %) of the households
surveye d report that the qualit y of services provided at the project facility
hasimprovedinthefirst year of the project, while m ost households in the
con tro l communities (53 %) perceive that the qualit y of services pro v ided
at the project facility has become worse (or at least not improved). Similar
c h anges are apparent in househ olds’ perceptions about the c hang e in staff
politeness during the first year of the project, cha ng e in availability of medical
staff, attention giv en to the patien t by the staff when visiting the project
facilit y, and whether the patien t felt he/she was free to express herself when
being examined.
The results using perception data are confirmed by looking at quanti-
tative indicators of service delivery. For example, most (50 percent) of the
patien ts in the treatment community reported that equipm ent (for instance
therm om eter or blood pressure equipment) w as used during the examination.
How ever, in the control communities a majorit y (59 percent) reported that no
equipment was used the last time the respondent (or the respondent’s c hild)
visited the project clinic. The difference in m ean outcomes are statistically
different.
20
Another indicator of quality is w aiting time, defined as the difference
betw een the time the user left the facility and the time the user arriv ed at
the facility minus the exam in atio n time. On average, the waiting time was
133 minutes in the control facilities and 117 in the treatm ent facilities. Ag ain
the difference is significan t.
Immunization coverage is another quality measure. The househ old surv ey
collected inform ation on how many times (doses) in total each child has been
20
There is no easily measured indicator that can be used to evaluate whether and how
patien ts in the project facilities receive better treatment. Naturally, the relevant treatment
is conditional on illness and the condition of the patient. However, since the project was
randomly allocated across communities, there is no reason to believe that the type of illness
and the condition of the patients should be systematically different across groups. It is
possible that, due to the intervention, patients with more severe illnesses seek care at the
project facilities in the treatment area and that this, in turn, can have a direct impact on
observed treatment practises. However, the evidence does not support this claim. We have
information on reported symptoms for which the patient seeks care (from the household
survey). There are no systematic differences in reported symptoms across treatment and
control communities. See Björkman and Svensson (2006) for details.
20

immunized with polio, DPT , BCG, and measles. To the exten t that this is
possible, these data w ere collected from immunization cards.
According to the Uganda National Expanded Program on Immunization
(UN E P I), eac h child in Uganda is suppose to be immunized against measles
(one dose at 9 months and two doses in case of an epidemic); DPT (three
doses at 6 weeks, 10 w eeks and 14 weeks); BCG (one dose at birth or during
first con tac t with health facilit y ); and polio (three doses, or four if delivery
takes place at the facility, at 6 weeks, 10 weeks, 14 weeks). To account
for these immunization requirements, we create dummy variables taking the
value one if child i of cohort (age) j had receiv ed the required dose(s) of
mea sles, DPT, BC G , and polio, respectively, and zero otherw ise. We then
compar e mean outcomes, i.e., the share of children for each age group (0-
12 months, 13-24 months, 25-36 months, 37-48 mon ths, and 49-60 mon ths)
that has received the required dose(s) of m easles, DPT , BCG , and polio,
respectively.
There are significant positiv e differences bet ween the treatment and the
control community for all four vaccines, although not for all cohorts. Approx-
imately 40 percent of children und er one y ea r ha ve receiv ed at least one dose
against measles. There is no significan t difference bet ween treatment and
con trol groups. For one-year old children (13-24 months), ho wev er, w e find
asignificant difference. In the contro l group, 83 percent of the c hildren ha ve
been immunized, while the corresponding n umber in the treatment group is
5.2 percentage points higher. A sma ller, but significant, differen ce also shows
up in the cohort of three year old ch ildren (37-48 mon ths). For polio we find
positive and significant differences in all bu t the oldest age group (regressions
6-9). The difference is largest for the you ng est cohort (4.7 percent points).
This corresponds to a 13 percen t increase in the treatmen t group compared
to the con trol group. For DPT, w e find a significant positiv e differen ce in
two out of fiv e cohorts and for BCG we find a positive and significant dif-
ference (7 percen tage poin ts) in the y oungest cohort. Th us, while w e do not
find significan t differen ces in all cohorts, immunization co verage increased
significantly follo w in g the inter vention.
The main objective of the community -ba sed monito ring project was to
improv e health outcomes in rural areas of Uganda where health indicators
have been stagnating. To ac hieve this objective, the project in tended to
enhan ce communities’ abilities to mon it or the public health care pro v ider,
thereby strengthening pro viders’ incen tiv es to increase both the qualit y and
the quantity of primary health care provision. As reported above, the project
was successful in raising both utilization and, to the exten t that this can be
measured, qualit y of services. Next, w e turn to health outcomes.
Data on t wo health outcom es were collected. First, w e collected infor-
21

matio n on wheth er the household had suffered from the death of a child
(under five y ears) in 2005 (i.e., the fir st yea r of the communit y monitoring
project). Second, all infants (i.e. all c hildren under 18 months of age) and
c h ildren (bet ween 18 and 36 months of age) in the surveyed households w ere
w eighed.
21
3.2 percen t of the surv eyed households in the treatment communit y had
suffered from the death of a c h ild in 2005. The corresponding nu mber in the
con trol comm unity is 4.9 percen t. The difference, 1.7 percen tage poin ts, is
statistically significant. With a total of approximately 55,000 households re-
siding in the treatment com munities, the treatment effect (0.017) corresponds
to 546 a verted under-fiv e deaths in the treatment group in 2005 follo wing the
in terven tion.
22
Given the sample size, we pool the data on the weight of infants and study
the differences in the average w eight of infan ts bet ween 1-18 months and 19-
36 months, respectiv e ly. The in terven tion resulted in an average increase in
weight of 0.13 kilograms for the group 1-18 mon ths. For the average child (age
nine months and weight 7.9 kilogram s), this represents a 1.6 percent increase
in weight between the treatment and the contr ol group one year into the
program. Albeit small, the difference is statistically significan t. There is no
significant difference in weight of c hild ren between 18-36 months of age.
As of 2001, public health services are free of charge. Howev er, the sur-
vey evidence indicates that patients still encounter varying costs, although
a large majority of patien ts do not pay (informal) user fees. In the pre-
treatm ent data, 7 percent of the households surv eyed reported having to pay
user charges for out patien t services; approximately 15 percent had to pay
for injections (when needed); and 67 percent paid for deliv ery.
23
The intervention had no significant effect on the share of households that
needed to pa y for drugs or delivery. Howev er, it had an impact on general
out patien t services as w ell as on injections
Althoug h the Citizen report card project was a structured in terven tion,
it left plenty of room for the com munities to choose if and ho w to react to
the information being dissem inated . The aim of the project was that the
report card informa tion would pro vid e a spark for community action but
also provide citizens with hard data throug h which the health facility could
21
See Björkman and Svensson (2006) for details.
22
Note, though, that since villages closer to the facility w ere oversampled, the sample
of treatment villages is not fully representative of the total population in the treatment
communities.
23
Average payment (for those that had to pay) was UGX 1,435 (USD 0.80) for out-
patien t service, UGX 370 (USD 0.21) for injections, and UGX 4,955 (USD 2.75) for deliv-
ery.
22

be evaluated/monitored. We turn now to some evidence on if new initiatives
and processes were initiated or strengthened as a result of the in terven tio n.
To avoid influen cing local initiatives, w e ch ose not to have enumerators
spending time in the field after the first round of meetings. Therefore, w e were
not able to document all actions tak en b y the com munities in response to the
in terven tion . Still, w e ha ve tw o sources of information on how processes in
the com munity chan ged follow ing the inter ven tion . First, the CBO s involv ed
in dissem ina ting the report card information submitted reports on what type
of c h anges they observed. The evidence from these reports suggests that the
project influenced the way in which the providers w er e being monitored. This
eviden ce is supported b y facilit y and household survey data as w e ll as data
assembled through a Local (village) council surv ey.
Accord ing to the CBO reports, the com munity-b ased mon itoring process
that followed the first set of meetings (comm unit y, facility and interface meet-
ings) was a join t effort mainly managed by the village local councils, HUMC
(Health Unit Managem ent Com m ittee) and the community members. In
the communities, the performance of the health facilit y was discussed during
village mee tings. The Local Council survey confirm s this claim. A typical
village had, on av era ge , six village meetings in 2005. In those meetings, 89
percent of the villages discussed issues concerning the project health facilit y.
The main subjects of discussion in the villages concern ed the action plan
(30 percen t of the villages) or parts of it suc h as behavior of the staff (49
percent of the villages), drug deliveries at the health facilit y (48 percent of
the villages), and that go vernment health services are supposed to be free of
c h arge (68 percent of the villages).
The CBO s report that concerns raised b y the village mem bers were car-
ried forward b y the local council to the healt h facility or the HUMC. Howeve r,
although the HUMC w as view ed as an entit y that should play an important
role in monitoring the provider, it w as in many cases viewed as being ineffec-
tive. As a result, misman aged HU MCs w ere re-elected, while others felt the
pressure from the com munity to act and follow up on the issues covered in the
action plan. Once more, these reports are confirmed in the surv ey data: more
than one third of the H U M C s in the treatmen t communities w ere reelected or
received new members followin g the initial in terven tio n. Further, the CBO s
report that the communit y also mo nitored the health facilit y staff during
health visits to the clinic, when they rew ar ded and questioned issues in the
action plan which had or had not been addressed. Tools suc h as suggestion
boxes (where community members could anon ymously leave suggestions for
c h ange or comment on the lack of change that wa s supposed to hav e taken
place), n u mbered w aitin g cards (to ensure a first-come-first serv e basis), and
duty roasters, w e re also put in place in sev eral treatment facilities.
23

Totheextentwecanmeasureit,theCBOreportsareconfirmed by
the survey dat a. O ne year into the project, treatment facilities are more
lik e ly to have suggestion boxes (no control facility had these, while 36 % of
the treatment facilities did) and numbered w aiting cards (only one control
facilit y had these, while 25 % of the treatment facilities did). There are also
differences bet ween the treatment and control facilities in the extent to which
inform ation is posted on free-services and patient’s righ t s and obligations.
24
Househ olds in the treatment communities are also better inform ed about
various aspects of service pro vision follow ing the in terven tion . For exam ple, a
significantly larger nu mber of households have receiv e d informa tion about the
importance of visiting a health clinic for media l treatment and the dangers of
self-treatment and for family planning. The treatment com munity is sligh tly
more lik ely (although most household s do not kno w this) to know when the
project facility receiv es drug deliv eries. There are also differen ces bet ween
the treatment and con trol group in the exten t to which the performan ce of
the staff at the project facilit y is discussed in Local Council meetings, and
whether , one y ear in to the project, community members know anyon e who
is a mem ber of the HUMC and have kno wledge of the HUM C’s roles and
responsibilities.
8Conclusion
In response to the failures of traditional mechanisms of enforceability and an -
swerabilit y (the relationship of accountably of pro vid er to state), it has been
argued that more effort must be placed on strengthening beneficiary con trol,
i.e. strengthen pro vide rs accoun tability to citizen-clien ts. Ho wever, factual
evidence in support of this strategy has, to date, been missing. In this paper,
w e hav e studied the effects of enhancing rural communities’ abilit y to hold
prima ry health care providers accountable. We find that both the quality and
the quantity of health service provisio n improv ed in the treatment com mu-
nities: One year in to the program, a verage utilization w a s 16 percen t higher
in the treatment communities; the w eight of infants higher, and the n umber
of deaths among c hildren under-five markedly lower. Treatmen t communi-
ties becam e more extensively involv ed in mon itoring the provid ers follo w ing
theinterventionandtheresultssuggestthatthehealthunitstaff responded
by exerting a higher effort into serving the comm unit y. By strengthening
the pro viders’ incentiv es to serve the poor, health provision and, in the end,
24
These data were collected through visual checks by the enumerators.
24

health outcomes can be significantly improved.
Although the Citizen report card project appears to be successful, it is
too early to use these findings as a basis for con tinued or increased support
and funding for various activities with the aim of strengthening beneficiary
con tro l. There are still a n u mber of outstanding issues. One important con-
cern is the extent to whic h the processes initiated b y the Citizen Report
Card project are sustained. Since the project is ongoing and scaled up to
involve an additional 25 health facilities, this process can be studied over
time. It is also possible that ev en better results can be achieved by combin-
ing bottom-up monitoring (comm unit y based monitoring) with a top-do wn
appro ach (supervision and possibly sanctions/rewards from someone in the
institutional hierarchy assigned to monitor and control the primary health
care prov iders). The evaluation of such a project is curren tly underway.
It is also importan t to subject the project to a cost-benefit analysis and
relate the cost-benefit outcomes to other possible interv entions. This would
require putting a value on the improvem ents w e ha ve documented. To pro -
vide a fla vor of such a cost-benefit analysis, consider the findings on averting
the death of a c hild under-five. The in ter ven tion resulted in 1.7 percent-
age poin ts few er child deaths during the first project year in the treatment
communities. To the exten t that this n u mber is represen tative of the total
treatment population, this w ould imply that appro ximately 500 under-five
deaths w ere a verted as a result of the in tervention. A bac k-of-the-env elope
calculation then suggests that the in terven tion, only judg ed on the cost per
death av erted, m ust be considered to be fairly cost-effective. The estimated
cost of av erting the death of a child under-five is $300 in the Citizen report
card project. This can be compared to the numbers reported by Filmer and
Pritchett (1999). They con tra st the cost of averting the death of a child
derived from increasing public expenditu res on health (regression estimates
range from $47,112 to $100,927), to more con ventional health in terv entions
basedoncost-effectiveness estimates of the minimum required cost to a vert
a death (ranges from $1,000 to $10,000 for diarrheal diseases, from $379 to
$1,610 for acute respiratory infection, $78 to $990 for malaria, and $836-
$3,967 for complication s of pregnancy).
25
25
These numbers should be viewed with extreme caution. For the cost-benefitestimates
of the Citizen report card project, it should be noted that the sample is, by construc-
tion, not fully representative of the population (since villages closer to the facility were
oversampled). Naturally, the 95 percent confidence interval would also include a much
smaller estimate of program impact than the 1.7 percentage points used here. Moreover,
since the largest cost item was the collection of data and these data were used partly in
the intervention and partly to evaluate impact, the cost is a rough estimate. Filmer and
Pritchett’s (1999) estimates of the cost of averting a child death derived from increasing
25

The Citizen report card project w as implemented in nine differ ent districts
of Uganda and reac hed approximately 55,000 households. Thus, in this di-
men sion, the project has already show n that it can be brought to scale. Still,
this project is a con trolled experiment in some dimension. Specifically, data
collection and data analyses w ere supervised by the evaluators. To the exten t
that these tasks w ere delegated to local actors in the various communities,
they could have been subject to capture. This is an issue on which our find-
ings do not shed an y light. What our findings strongly suggest, though, is
that experimentation and evaluation of new tools to enhance accountabilit y
should be an in tegr al part of the researc h agenda on impro vin g outcomes of
social services. This is an area where at presen t, research on what w ork s and
wha t does not w o rk is clearly lagging behind policy.
public expenditures on health are subject to a variety of estimation problems and the
health interventions based cost-effectiveness estimates of the minimum required cost to
av ert a death are, as noted by Filmer and Pritchett, at best suggestive.
26

References
Appleton , Simon (2001), "The Ric h A re Just Like Us, Only Rich er’: Poverty
Functions or Consumption Functions?", Journa l of A frican Econom ie s 10(4):
433-469.
Banerjee, Abhijit and Esther Duflo (2005), "Addressing Absence", Journal
of Ec onomic Persp ectives 20 (1): 117-132.
Banerjee, Abhijit, Angu s Deaton and Esther Duflo (2004), "Wealth, Health,
and Health Service Delivery in Rural Rajasthan", Americ an Economic Re-
view Papers and Proceedings 94(2): 326-330.
Banerjee, Abhijit, and Ruimin He (2003), “The World Bank of the Future”,
American Economic Review 93(2): 39-44.
Besley, Timothy, and Robin Burgess (2002), “The Political Economy of Gov-
ernment Responsiveness: Theory and Evidence From India”, The Quarterly
Journal of Economics 177(4):1415-51.
Besley,Timothy,andAndreaPrat(2005),“Handcuffs for the Grabbing
Hand ? Media Capt ure and Gov ern m ent Accountability ”, American Eco-
nomic R eview, forthcom in g.
Björkman , Martina (2006), "Does M oney Matter for Student P erform ance?
Evidence from a Gran t Program in Ug anda", Working P aper, IIES, Stoc k-
holm Univ e r sity.
Björkm an, Martina and Jak ob Sv ensson (2006), "P ow er to the P eople: Evi-
dence from a Randomized Experiment of a Citizen Report Card Project in
Uganda", Working P aper, IIES, Stoc kholm University.
Chaudhury, Nazm ul, Jeffrey Hamm er, Mic hael Kremer, Karthik Muralidha-
ran, and F. Halsey Rogers (2006), "M issing in Action: Teacher and Health
Worker Ab sence in Deve loping Countries", Journal of Economic Perspe ctives
20(1): 91-116.
Esther, Duflo and Rema Hanna (2005), "Monitoring Works: Getting Teach-
ers to Come to School", Working P aper, Department of Economics and
Poverty Action Lab, MIT.
Ferrez, Claudio and Frederico Finan (2005), "Exposing Corrupt Po liticians:
The Effect of Brazil’s Publicly Released Audits on Electoral Outcomes",
Working P a per, Departm ent of Ag ricultural and Resource Economics, Uni-
versity of California , Berk e ley.
Film er, Dea n and Lant Pritc h ett (1999) , "The Impact of Pub lic Spending on
Health: Does Money Matter?", Social Science and Medicine 49(10).
27

Jimenez, Emman uel and Yasuyuki Sawada (1999), "D o Community-Managed
Sc hools Work? An Evaluation of El Salvador’s EDUC O Program", The
WorldBankEconomicReview13(3): 415-441.
Keefer, Philip and Stutti Khem ani (2005), "Democracy, Public Expendi-
tures, and the Poor: Und er standin g Political Incentiv es for Pro v iding Pu blic
Services", World Bank Research Observ er , 20: 1-27 .
Khem ani, Stuti (2006), "Can Information Cam paigns Overcome Political
Obsta cles to Serving the Poor," mimeo, The World Bank.
Malena, Carmen, Reiner Forster and Janmejay Singh (2004), "Social Ac-
countability: An Introduction to the Concept and Emerging Practice", So-
cial Development Papers 76, Participation and Civic Eng agem ent Grou p,
The World Bank.
McPake , Barbara, Delius Asiimwe, Francis Mwesigye, Mathias Ofumbi, Lis-
beth Ortenblad, Pieter Streefland and Asaph Turinde (1999), “The Economic
Behavior of Health Work ers in Uganda: Implications for Quality and Accesi-
bilit y of Public Health Services,” Social Science and Medicine 49(7): 849-865.
Moeller, Lars Christian (2002), “Ugan da and the Millenn ium Development
Goals” , Human Dev elop m ent Netwo rk, World Bank, Washington D.C, Processed.
Olken, Ben (2005), " Monitorin g Corruptio n: Eviden ce from a Field Exper-
iment in Indonesia" , NBER Working Paper No.11753.
P ersson, Torsten and and Guido Tabellini (2000), Political economics: Ex-
plaining Ec onomic Policy, MIT Press, Cambridge, MA.
Reinikka, Ritva and Jakob Svensson (2004), “Local Capture: Evidence from
a Central Gov ernm e nt Transfer Progra m in Uga nda ”, The Quarterly Journal
of Ec onomics 119 (2): 679-705.
Reinikka, Ritva and Jakob Sven sson (2005a), "The Pow er of Information:
Evidence from a New spa per Campaign to Reduce Cap ture”, Working paper,
IIES, Stockh olm Universit y.
Reinikka, Ritva and Jakob Svensson (2005b), "Fighting Corrup tion to Im-
pro v e Schooling: Evidence from a Newspaper Campaign in Uganda”, Journal
of the European Econom ic Association 3 (2-3): 259-267.
Reinikka, Ritva and Jakob Svensson (2005c), "Working for God”, Working
paper, IIES, Stockholm Univ ersity.
Republic of Uganda (2000), “National Health P olicy and Health Sector Strate-
gic Plan 2000/01-2004/05”, Ministry of Health, Kampala
Republic of Uganda (2002), “Infant Mortality in Uganda 1995-2000: Why
the Non-Im prov em ent?”, Discussion Paper No. 6, Planning and Economic
Developmen t, Kampala.
28

Sam uel P aul (2002), Holdin g the State to Account: Citizen Monitor in g in
Action, Books for Change, Bangalore.
Singh, Janmeja y and Meera Shah (2002), “Communit y Score Cards in rural
Malawi”, World Bank, Washington, D.C. Processed
Ström berg, David (2003), “Mass Media and Public P olicy”, Eur op ean Ec o-
nomic R eview 45(4-6): 652-63.
Ström berg, Da vid (2004), “Radio’s Impact on Public Spending”, The Quar-
terly Journal of Economics 119(1): 189-221.
Sv ensson, Jakob (2005), “Eight Questions about Corruption”, Journal of
Economic Perspectives, 19 (3): 19-42.
World Bank (2003), Making Service Work for the Poor People, World Devel-
opment Report 2004, World Bank and Oxford Univ ersity Press.
World Bank (1996), "World Bank Participation Sourcebook", Environmental
Departme nt papers No. 19, Washington.
World Health Organization (2006), Wor ld Health Statistic s 2006, WHO Press,
Switzerland.
29

SEMINAR PAPER SERIES
The Series was initiated in 1971. For a complete list of Seminar Papers, please contact
the Institute.
2005
734.
Mats Persson, Torsten
Persson, and Lars E.O.
Svensson
Time Consistency of Fiscal and Monetary Policy:
A Solution
735.
Matthias Doepke and
Fabrizio Zilibotti
Patience Capital and the Demise of the Aristocracy
736.
Alessandra Bonfiglioli
How Does Financial Liberalization affect
Economic Growht?
737.
Alessandra Bonfiglioli
Equities and Inequality
738.
Virginia Queijo
How Important are Financial Frictions in the U.S.
and Euro Area?
739.
Assar Lindbeck
Sustainable Social Spending
740.
Vasco Cúrdia and
Daria Finocchiaro
An Estimated DSGE Model for Sweden with a
Monetary Regime Change
2006
741.
Mats Persson and
Claes-Henric Siven
The Becker Paradox and Type I vs. Type II Errors
in the Economics of Crime
742.
Assar Lindbeck and
Mats Persson
A Model of Income Insurance and Social norms
743.
Assar Lindbeck, Mårten
Palme and Mats Persson
Job Security and Work Absence: Evidence from a
Natural Experiment
744.
Martina Björkman
Income Shocks and Gender Gaps in Education:
Evidence from Uganda
745.
Anna Larsson
Monetary Regimes, Labour Mobility and
Equilibrium Employment
746
Harry Flam and Håkan
Nordström
Trade Volume Effects of the Euro: Aggregate and
Sector Estimates
747.
Mirco Tonin
The effects of the minimum wage in an economy
with tax evasion
748.
Stefano DellaVigna and
Ethan Kaplan
The Fox News Effect: Media Bias and Voting
749
Martina Björkman,
Ritva Reinikka and
Jakob Svensson
Local Accountability
ISSN: 1653-610X
Stockholm, 2006

Institute for International Economic Studies