Conference PaperPDF Available

Contracting-out Primary Health Care Services using Performance-Based Payments: An evaluation of the Honduras' Experience

Authors:

Abstract and Figures

Honduras has recently contracted-out the provision of primary health care services to decentralized providers in order to expand health care coverage in rural areas. This paper evaluates the impact of contracting-out on health outputs and outcomes related to access, quality, and equity in health care. The impact evaluation design is quasi-experimental, and uses the propensity score matching method to select a control group of municipalities with public provision statistically comparable to those that contracted-out services. Estimates from a fixed effect model indicate that contracting-out was effective to increase access to health care, but there was no evidence of a significant impact on quality of care. The positive effects on health care utilization were higher in municipalities with extremely-high poverty levels, indicating that contracting-out also improved equity in health care.
Content may be subject to copyright.
Contracting-out Primary Health Care Services using Performance-Based
Payments: An evaluation of the Honduras’ Experience
Mercedes Vellez
University of Rome II Tor Vergata
Center for the Study of State and Society (CEDES)
This version: January 2015
Abstract
Honduras has recently contracted-out the provision of primary health care services to decentralized
providers in order to expand health care coverage in rural areas. This paper evaluates the impact
of contracting-out on health outputs and outcomes related to access, quality, and equity in health
care. The impact evaluation design is quasi-experimental, and uses the propensity score matching
method to select a control group of municipalities with public provision statistically comparable
to those that contracted-out services.Estimates from a fixed effect model indicate that contracting-
out was effective to increase access to health care, but there was noevidence of a significant impact
on quality of care. The positive effects on health care utilization were higher in municipalities
with extremely-high poverty levels, indicating that contracting-out also improved equity in health
care.
Keywords: contracting-out, access, quality, equity, infant mortality, propensity score matching.
JEL classification: C23, H44, I18
1
1 Introduction
In the past two decades, an increasing number of developing countries have turned to contracting-
out health care services to non-state providers in order to improve the performance of their health
systems. Management or service delivery contracts between the public sector and non-state
providers typically define the services to be provided, and establish payments based on
performance through a system of rewards and sanctions. On the other hand, non-state providers
are given full autonomy in allocating financial and human resources either to manage existing
public health care services or to provide services where none exists. The growing interest in
separating finance from provision finds support on the rationale that providers make more efficient
budget allocations and exert more effort to increase the quantity and quality of services when
payments are linked to measurable targets. This contrasts to most public health facilities which are
generally financed through budgetary allocations based on historical budget, lack incentive
mechanisms and strong supervision, and have rigid management of human resources.
Systematic reviews of empirical literature have raised concerns about the performance
dimensions and methodologies used to assess the effectiveness of contracting-out health care
services. Access is the most common dimension addressed using indicators such as coverage rates,
availability, and quantity of services provided. However, other performance indicators such as
efficiency,quality, and equity, are relatively less known and yet deserve attention [Slack and
Savedoff (2001); and Loevinsohn (2008)]. Several authors argue that the available empirical
studies fall short of conducting a rigorous evaluation of the contracting-out approach because of
poor quality of data and methodological pitfalls that undermine the internal validity of their results.
Liu et al. (2008) stress that most evaluation studies suffer from inappropriate design in estimating
contracting-out impacts due to lack of baseline data, statistically equivalent control group, or
potential confounders effects. Lagarde and Palmer (2009) and Loevinsohn and Harding (2005)
also emphasize the existence of methodological weaknesses in evaluation studies and call for new
research using experimental or quasi-experimental analysis. More rigorous research, therefore, is
needed to add to the body of knowledge concerning the impact of this increasingly popular health
reform.Honduras is a successful example of expanding health care coverage in rural areas through
the separation of financing and provision. As part of a broad health reform, the Ministry of Health
(MOH) contracted-out the provision of primary health care services to the so called decentralized
providers including NGOs, communal associations, municipal commonwealths, and Mayor’s
offices. Decentralized providers were financed through a pay-for-performance scheme and were
granted full autonomy and flexibility to allocate their budget, purchase drugs and medical supplies,
hire and fire staff, and negotiate wages.
MOH (2009) and Garcia Prado and Lao Peña (2010) evaluate the performance of the
contracting-out model in Honduras comparing access, quality and efficiency indicators between
decentralized and public providers. Both studies conclude that contracting-out had a positive effect
on access, but evidence is inconclusive with regard to improvements in quality and overall
2
efficiency.
1
Focusing on very few health facilities, their findings have a fairly limited scope, being
pertinent to assess the implementation of the contracting-out model in local contexts but not to
establish national patterns or trends.
In this paper, I provide empirical evidence of the impact of contracting-out on health
outputs and outcomes related to access, quality, and equity in health care addressing some of the
methodological pitfalls identified in other evaluation studies of similar experiences in different
countries. In particular, I use a quasi-experimental approach, the propensity score matching (PSM)
technique, to select a control group of municipalities with public provision statistically comparable
to the contracting-out municipalities, and then apply a fixed effect model to estimate the
contracting-out effects for the period 2008-2010. This impact evaluation is the first attempt to
assess the effectiveness of the whole contracting-out experience in Honduras with a national scope.
The paper is organized as follows: section 2 reviews theoretical and empirical literature on
contracting-out public services, section 3 describes the contracting-out model in Honduras, section
4 presents the methodology used, section 5 summarize the data, section 6 presents the empirical
results, and section 7 concludes.
2 A review of the literature on contracting-out
Separating finance from provision in health care has several advantages. Contracting-out grants
non-state providers full autonomy and flexibility to manage financial and human resources
facilitating more efficient budget allocations. It contributes to avoid bureaucratic red tape, such as
bypassing public procurement systems, and reduce problems related with public health workforce
absenteeism. Contracting-out also enables authorities to focus on their intrinsic role of guiding the
health system, such as policy and planning, financing, regulation, and monitoring and evaluation
activities, giving special emphasis on the achievement of health care targets.
Management or service delivery contracts between the public sector and non-state
providers typically define the package of services, and establish payments based on performance
through a system of rewards and sanctions. Thus, non-state providers are expected to exert more
effort in providing health care services when payments are linked to measurable targets.
However, contract incompleteness may lead to an excessive focus on the quantity of
services provided without concomitant attention given to the quality of service [Loevinsohn
(2008)]. Quality of care can be challenging to measure to the extent that it depends on variables
that are unobservable, such as effort exerted by the non-state provider, or unverifiable by third
parties and thus cannot be enforced in court. For instance, it may be difficult to specify contract
1
Regarding quality, MOH (2009) finds that clinical protocols for the treatment of diarrhea and pneumonia were met
more successfully at contracting-out facilities, whereas Garcia Prado and Lao Peña (2010) show better drug
availability, equipment, and cleanliness. However, the studies obtain opposite results about waiting times. With regard
to efficiency, both studies suggest that contracting-out facilities exhibited higher labor productivity and lower wage
costs than public health facilities, but also higher unitary costs of drugs.
3
provisions regarding medical staff skills for delivering health care services. The combination of
poorly measurable quality targets and other easily measurable tasks, may lead an opportunistic
non-state provider to underperform on the former to cut cost and maximize budget surplus.
Previous literature on public versus private provision has raised concerns about
maintaining an appropriate quality level when the public sector contracts services out and quality
dimensions are non-contractible. Hart et al. (1997) develop an incomplete-contract model that
highlights the risk of quality degradation in the presence of strong cost-saving incentives for
private providers. They conclude that public provision would be preferable when cost reductions
have large deleterious effects on non-contractible quality and there is little scope for efficiency-
enhancing innovation. However, informal mechanisms such as reputation and brand names [Klein
and Leffler (1981); Banerjee and Duflo (2000); Dellarocas et al. (2006); Bar-Isaac and Tadelis
(2008], and contract renewal and long-term relationships [MacLeod (2007); Calzolari and
Spagnolo (2009); Iossa and Rey (2010)] may mitigate the risk of non-contractible quality
degradation depending upon the public sector’s flexibility and discretion in contracting-out
services.
Even if quality is contractible, it may be too costly to monitor and achieve through payment
deductions. As emphasized by Levin and Tadelis (2010), high transaction costs may also tip the
scale against private provision. Thus, public provision would prevail when costs borne by the
public sector for specifying and enforcing quality are high and exceeds gains expected from
enhanced efficiency of the private sector. In addition to the presence of significant opportunities
for cost reductions that negatively impact on quality and negligible efficiency-enhancing
innovation, the superiority of public over private provision has been also stressed in cases where
competition is weak, and consumer choice is ineffective [Shleifer (1998)].
Empirical research on the effectiveness of contracting-out
In the past two decades, many developing countries have introduced reforms aimed at improving
the performance of their health care systems through the separation of financing from provision of
health care. Table 1 reports a summary of contracting-out examples in developing countries. Most
contracting-out experiences had the primary goal of expanding health care coverage in rural areas
(Costa Rica, Guatemala, Honduras, Haiti). Others countries, instead, turned to contracting-out to
overcome a low demand for public health care services (Bolivia), or a high demand for unqualified
drug sellers and increased absenteeism rate in public facilities (Cambodia). Non-state providers
were mostly NGOs, although in some cases the government contracted-out health care to worker-
controlled health care cooperatives (Costa Rica), or even local governments, such as municipal
commonwealths and Mayor’s offices (Honduras). Contracting-out experiences were varied in
terms of the allocation of health facilities: some non-state providers were assigned to remote areas
where there was basically no provision of public health care services (Guatemala), others instead
came to provide services at health facilities previously managed by the public sector (Costa Rica,
Bolivia,) or both of the above (Honduras).
4
Contracting-out has mainly focused on the provision of primary care related to maternal
and child health, and secondary care such as hospital management (Bolivia). Most agreements
took the form of service delivery contracts under which non-state providers assumed the
responsibility for hiring new personnel, maintaining existing equipment and purchasing inputs.
Some countries adopted a mixed approach that grants less autonomy to non-state providers. For
example, Guatemala and Cambodia introduced a second modality of contracting-out (known as
mixed provider or contracting-in) in which NGOs provide health services through medical staff
appointed by the MOH. Similarly, in Honduras, when decentralized providers were assigned to a
health facility previously managed by the public sector, they used public servants personnel
appointed before decentralization, hiring the additional staff needed to fill up empty vacancies.
Regarding the payment mechanism, all contracting-out experiences included a pay-for-
performance component, which stipulated penalties for not achieving selected targets. In the
Rwanda case, NGOs were financed through a fixed payment for each targeted service delivered
combined with a quality of care index used as a multiplicative factor that increases or lowers
payments. In Honduras, instead, service provisionand quality targets werebundle into a final score
to determine the amount of withhold payments, thus raising concerns that decentralized providers
may outperform on easier targets and underperformed on more difficult ones.
There are several empirical studies assessing the effectiveness of contracting-out health
care services that use different types of research design. Only few studies use an experimental or
quasi-experimental design to evaluate the impact of contracting-out. A remarkably study is the one
by Bloom et al., (2006), who analyze the experience of Cambodia and exploit the pre-intervention
design that randomly assigned districts into contracting-out and control groups. Since not all
randomly selected districts actually end up contracting-out health care because of limited
competition, the authors use the initial randomization of treatment as an instrumental variable for
actual treatment to estimate the effect of treatment on the treated. Similarly, the original random
selection of contracting-out districts in Rwanda was later reassigned, leading Basinga et al. (2010)
to use a quasi-experimental approach and conduct a difference-in-difference analysis to estimate
the impactof the contracting-out. More recently,Cristia et al. (2011) use a difference-in-difference
specification exploiting a further expansion of the contracting-out experience in Guatemala.
Other studies use less rigorous methodologies that limit the internal validity of their
findings, e.g., simple descriptive statistics, post-intervention cross-section or time-series analysis
without controlling for unobserved heterogeneity and potential confounders. For instance, Gauri
et al. (2004) use a panel data to compare the performance of the cooperative clinics against the
universe of public clinics at the same level of complexity in Costa Rica controlling for potential
community level covariates. Yet the study does not take into account time-invariant unobserved
factors that could have affected both cooperatives and health outputs, thus increasing the risk of
bias. Eichler et al. (2007) evaluate the effectiveness of the shift in payment mechanism from cost
reimbursement to pay-for-performance for NGOs contracts in Haiti. Arguing that treatment and
control NGOs were not observationally equivalent because the former was selected on the basis of
its “readiness” to responsible management, the paper estimates a panel data model that controls
5
for unobserved differences among groups. Lavadenz et al. (2001) conduct a descriptive analysis
to assess the effectiveness of the contracting-out experience in Bolivia comparing the evolution of
a few performance indicators before and after the intervention, and confronting results with a
control municipality. Nevertheless, both studies disregard other factors affecting performance and
hence, observed differences between treatment and control groups may not be attributable only to
the implementation of contracting-out. Other empirical studies rely on retrospective cross-section
and descriptive analysis to evaluate the performance of contracting-out health care vis a vis public
provision. These are the cases of Guatemala [Danel and La Forgia (2005)] and Honduras [MOH
(2009) and Garcia Prado and Lao Peña (2010)].
Among the performance indicators used to evaluate contracting-out experiences, access is
the most common dimension addressed. Results indicate that contracting-out can be very effective
to improve access to health care, by increasing ambulatory visits, institutional births, and bed
occupancy rates in Bolivia [Lavadenz et al. (2001)], and the number of general visits in Costa Rica
[Gauri et al. (2004)]. In Guatemala, immunization rates, prenatal care coverage, and the likelihood
of having child growth-monitoring checkups were higher in contracting-out than in the public
provider catchment areas, with no discernible difference concerning family planning services
[Danel and La Forgia (2005); Cristia et al. (2011)]. Similarly, the Cambodian experience shows
that whilecontracting-out had no effect on family planning and institutional births, it had a positive
impact on vitamin A supplementation, prenatal care, skilled birth attendance, full immunization,
and use of public facilities [Bloom, et al, (2006)]. In Haiti, the introduction of the pay-for-
performance scheme led to an increase in immunization rates and institutional births, but not
necessarily in prenatal and postnatal care visits [Eichler et al. (2007)]. In Rwanda, contracting-out
increased the probability of having an institutional birth and receiving child visits, but no impact
was found on prenatal care and child immunization services. Thus, contracting-out had higher
impact on those targets with higher monetary incentives (such as institutional births), and more in
control of the provider i.e., less dependent on patient’s decisions, like immunizations [Basinga et
al. (2010)]. Finally, decentralized health facilities in Honduras delivered a higher number of
domiciliary, prenatal and postnatal visits, family planning services, and institutional births than
public health facilities [MOH (2009)]. Also, in terms of geographic and physical access, facilities
were located closer to referrals hospitals and had more functioning vehicles to transport patients
than their public counterparts [Garcia Prado and Lao Peña (2010)].
Efficiency and quality dimensions of contracting-out health care services are relatively less
known. Results suggest that contracting-out enhanced efficiency, by increasing labor productivity
in Guatemala and Honduras [Danel and La Forgia (2005); MOH (2009); Garcia Prado and Lao
Peña (2010)], and by reducing utilization of more expensive services (such as specialist visits, lab
exams, and drugs per visit) at cooperative clinics in Costa Rica [Gauri et al. (2004)].
Evidence of potential reductions in production costs for similar services is, however,
mixed. MOH (2009) reports lower cost per visit in contracting-out facilities in Honduras, whereas
in Costa Rica total expenditure per capita in the cooperative clinics was lower than in public clinics
[Gauri et al. (2004)]. However, Danel and La Forgia (2005) find opposite results in Guatemala.
6
Moreover, it remains unclear whether contracting-out lowers total health care expenditures,
including private out-of-pocket, public expenditures, and costs of contract management and
monitoring and evaluation. Bloom et al. (2006) find that contracting-out reduced out-of-pocket
expenditures, while it increased public health spending, thus concluding that overall health
expenditures most likely decreased or stayed constant.
There is a wide variation in the indicators used to address quality of care. Some studies
investigate onthe impact of contracting-out on the accomplishment of clinical protocols as a signal
of improved quality. In Guatemala, pregnant women and children in contracting-out catchment
areas were more likely to receive tetanus toxoid and iron supplementation during pregnancy, and
oral rehydration solution to treat diarrhea [Danel and La Forgia (2005)]. In Honduras,
accomplishment of clinical protocols for the treatment of diarrhea and pneumonia were also higher
in contracted-out facilities [MOH (2009)], whereas in Rwanda, contracting-out increased the
likelihood of receiving tetanus vaccines in prenatal visit [Basinga, et al. (2010)]. Other studies,
instead, focus on the quality of contracting-out health facilities. In Honduras, those facilities
exhibited more availability of drugs, better equipment, and higher cleanliness and share of trained
personnel than public facilities, with no advantage on average waiting times [MOH (2009); Garcia
Prado and Lao Peña (2010)].
Other studies opt to investigate patients’ perceived quality using user satisfaction surveys,
yet results on this indicator are mixed. Patients in contracting-out facilities were more satisfied
with the care received than those in public facilities in Bolivia and Guatemala [Lavadenz et al.
(2001) andDanel and La Forgia (2005)]. On the contrary, a poorer perception of quality was found
in contracting-out districts in Cambodia possibly because the high dispersion of health facilities
make patients to travel larger distances and wait longer to receive care compared to alternative
sources of care (such as unqualified drug sellers). Also patients may have a different perception
about what constitute an appropriate care, being the provision of vitamin injections and glucose
trips a natural indicator of quality of care [Bloom et al. (2006)].
Finally, the empirical evidence is scarce regarding the impact of contracting-out on equity
issues and health related outcomes. Contracting-out may improve equity in access to health care
if services are well targeted and benefit more poor households. MOH (2009) finds that most
families visiting the facilities had low income levels and poor life conditions, suggesting an
improvement in the equality of access. Regarding health outcomes, Bloom et al., (2006) finds that
contracting-out reduced the chance of reporting illness, and the incidence of diarrhea in children,
but had no effect on child survival. In this paper, I provide further evidence of the effect of
contracting-out on access to health care, but most importantly, I intent to fill the knowledge gap
regarding equity and health outcomes, the latter used as a signal of quality.
3 The decentralized model of primary health care provision in Honduras
3.1 Background
7
Honduras is the second poorest country in Central America Region with a population of 7.5 million
inhabitants and a GDP per capita of USD 3,509 in PPP terms. Around 60 percent of thepopulation
has incomes below the poverty threshold, and about half of people live in rural areas. Maternal
and child mortality rates have decreased steadily in the previous decade, but despite improvements
of national health indicators, there are still significant differences between urban and rural areas
[PAHO (2009)]. The vast majority of the population receives health care services at facilities from
the MOH’s network, less than 10 percent have access to the Honduran Social Security Institute
(IHSS), and the share of people covered by the private sector is negligible.
Since the early 1990s, the government has implemented health reforms oriented to reinforce the
regulatory and managerial capacities of the MOH, and to promote decentralization and social
participation. In 2004, theMOH’s reorganization includedthedelegation of functionsto the Health
Region Departments (HRD) (called “Region Sanitaria Departamental) which assumed the
responsibility for managing the health budget, ensuring access and quality of health care services
to their population. In order to expand health care service coverage in rural areas, the MOH adopted
the contracting-out approach of health care delivery financed through government funds from debt
relief and supported by multilateral and bilateral organizations (i.e., the World, Bank, the Inter-
American Development Bank, and USAID). Contrary to procurement best practices, contracts
were not awarded through a competitive bidding among potential providers, but instead, seeking
for community support, contracts were offered to either current municipal commonwealths, NGOs
already operating in the area, or communal associations legally formed to be able to contract with
the MOH. At the beginning, the decentralization focused on unserved populations in remote areas,
that is contracting-out health services at previously closed rural health facilities. But later on,
contracts were also awarded to replace the entire municipal network of publicly managed primary
care health facilities.
Starting in 2005, the MOH has progressively expanded the contracting-out model
throughout the territory. Table 2 presents the evolution of the decentralization in the period 2005-
2010. At the beginning only eight municipalities, spread across six departments, contracted-out
health care services to expand coverage to 60,133 inhabitants. Contracts were awarded to one
municipal commonwealth, three communal associations, and two NGOs, managing a total of 17
health facilities. The first expansion occurred two years later adding three municipalities and 21
health facilities, and benefiting a total population of 116,521 people. But it was in 2008 when the
major expansion of the contracting-out model took place, allocating the management of 171 health
facilities to a total of 21 decentralized providers. The share of contracting-out municipalities rose
dramatically from a 4 percent to 19 percent, and the benefited population reached to 475,000
people. In this year, the MOH also incorporated single Mayor’s offices as providers along with the
already existent municipal commonwealths, communal associations, and NGOs. At the end of
2010, there were a total of 25 decentralized providers that managed 203 health facilities distributed
8
across 56 municipalities providing health care services to over 700,000 people (about 9.5 percent
of Honduras’ population).
2
3.2 The contract
3
The management and service delivery contract (called “Convenio de Gestion”) between the MOH
and decentralized providers defines: i) the basic package of services (BPS) to be delivered; ii) the
beneficiary population and the network of health facilities to be managed; iii) the decentralized
providers’ managerial responsibilities; iv) the payment mechanism; v) the monitoring and
evaluation (M&E) scheme; and vi) the contract duration and early termination provisions.
Basic Package of Services
The BPS is delivered free of charge and includes activities related to health promotion (educative
meetings on health care prevention, healthy food, safety water, and disposals), health prevention
(family planning, immunizations, community integrated childhood care), health care (prenatal and
postnatal care, institutional birth, emergencies, and infant morbidity care), and health surveillance
(notification of vector transmitted diseases, maternal and infant mortality, and HIV cases).
Population and Network of Health Facilities
The contract specifies the beneficiary population in the catchment areas and the network of health
facilities that decentralized providers will be responsible for.Health care services may be provided
through domiciliary visits in the community or at the network of primary care health facilities
according to theirlevel of complexity. Thesmallest facilities are rural health centers (CESAR) run
by an auxiliary nurse providing services to around 3,000 people, followed by the health centers
with a primary care physician, a dentist, and an auxiliary nurse (CESAMO) that provide care to
3,000 to 6,000 habitants. CESAR and CESAMO opening hours are 8 hours a day, 5 days a week.
Then the maternal-infant clinics (CMI) provide birth, and obstetric and child emergency care
services 24 hours a day, 7 days a week. In almost all cases, health facilities’ ownership remains in
the MOH with the exception of a few CMIs that were built more recently with donation funds.
Provider’s Managerial Responsibilities
2
There is one municipality that contracted with two different decentralized providers: one of them was offered the
management of the CMI, whereas the other provider managed the health facilities. Thus, they may be considered as
two different municipalities with common characteristics.
3
This subsection is based on a report prepared to the Inter-American Bank [Vellez (2010)].
9
In order to ensure the provision of the BPS, decentralized providers are granted full autonomy and
flexibility to allocate their budget, purchase drugs and medical supplies
4
, hire and fire staff, and
negotiate wages of the contracted personnel. When health facilities’ physicians and nurses staff
includes public servants appointed before contracting-out, decentralized providers hire the
additional personnel needed to fill up the empty vacancies and comply with the minimum
standards required by the MOH. Moreover, they are required to employ health promotion staff to
pursue outreach activities in the community. Contracted personnel are generally hired on annual
basis for a monthly salary. Their wages are lower than their public servants analogues because
they lack additional benefits such as seniority and remote area compensations. As an incentive to
perform, the contract establishes that decentralized providers should reward their personnel
whenever they received the one percent bonus for good performance (see Payment Mechanism
Section below). Contract renewal does not work as a credible threat to encourage good
performance because labor supply often falls short inremote rural areas. Physicians and nurses are
not originally from these communities, but rather come from urban areas with high unemployment
looking for temporal job opportunities.
Decentralized providers usually have a higher availability of drugs since they avoid slow
public procurement practices and weak logistical systems by direct purchasing. They buy drugs
from the lower price supplier among three alternatives, and when credit channels fall short, they
are forced to purchase drugs from the lower second price supplier. As expected, decentralized
providers pay higher drug prices than the MOH because neither they can exploit the benefits from
economies of scale nor have the bargaining power to deal with pharmaceutical suppliers.
According to Vellez (2010), average overpricing in drug purchases was 175 percent for a group of
38 essential medicines, and moreover, price differences reached 400 percent for generic medicines
such as acetyl salicylic.
Payment Mechanism
The MOH finances decentralized providers through a pay-for-performance scheme based on a
prospective fixed-price payment per capita subject to rewards and penalties. Those who manage a
CMI receive, in addition, a production-based payment for each institutional birth which givesthem
incentives to increase the amount of services provided consistently with the MOH’s goals.
By allocating risk to providers, fixed-price payments generate incentives to reduce costs in
order to appropriate the budget surplus. As it was mentioned before, this may induce a degradation
of quality. Since fixed-price payments represent, on average, more than 86 percent of total
financing, they need to be accompanied by a good M&E system to reduce the risk of limiting the
quantity or quality of the services. Thus, the annual fixed-price payment per capita receives a one
percent bonus if the M&E final score exceeds a certain threshold of good performance, and it
suffers deductions when the final score falls short of the threshold. The bonus is expected to
4
Vaccines are provided by the MOH.
10
stimulate their personnel and to maintain the infrastructure and equipment at the health facilities,
but it was eventually suspended in 2010 due to MOH’s budgetary restrictions.
Decentralized providers are not allowed to charge user fees for the BPS with the exception
of ambulance transportation for non-pregnant women and children above 5 years old. This
additional funding along with other revenues from municipal contributions and donations,
constitute a solidarity fund enabling decentralized providers to improve and maintain the maternal-
infant accommodation. These structures provide temporary housing to pregnant women who are
prompt to deliver and live far away from the CMI to avoid an emergency transportation.
Monitoring and Evaluation
Decentralized providers are subject to a regular M&E system. A monitoring team composed by
HRD’s staff visits contracting-out health facilities to review clinic histories, check for drugs and
medical supplies availability, and control consultations, compliance of biosafety measures,
immunizations and domiciliary visits records, among others activities. Given the complexity of
the M&E process, it takes around three days to evaluate a single decentralized provider, entailing
considerable transportation, food, and accommodation expenditures for the monitoring team
because of the long distances between the HRD and the rural areas where health facilities are
located.On quarterly basis, HRD evaluates providers’ performance through management, health
promotion and prevention, quality of services, and CMI indicators, and at the end of the contractual
year, the MOH conducts an annual evaluation to assess decentralized providers’ overall
performance. Each performance indicator has an arbitrary value assigned intending to reflect their
relative importance in the monitoring tool. The final M&E score, ranging from 0 to 100, is defined
as a weighted average of the four components, and thus, bundles management aspects with health
prevention outputs and quality issues (See Table A.1 in the Appendix for a list of M&E indicators,
and Table A.2 for details on final score formulas, bonuses and deductions).
As reward and penalties are not directly linked to each target but rather to the final score,
there is a risk that decentralized providers divert efforts from targets difficult to achieve towards
easy tasks, with no effect on the overall performance. For instance, they may achieve a good final
score (above 85) by accomplishing targets for health facilities supplies (such as medicines or
biosafety measures) and disregarding those for health outputs (such as immunization coverage).
In fact, according to the official monitoring tool, a hypothetic decentralized provider failing to
achieve essential maternal and child health care outputs (such as prenatal care or child
immunizations targets) could obtain a final M&E score that exceeds the minimum value to avoid
payment deductions and receive the bonus.
Moreover, decentralized providers may underperform in quality targets and yet achieve a
good final score. For example, theymay not suffer any penaltyeven whenthe use of the partogram
is incorrect, there is not an immediate transfer of patients with fetal distress to hospitals, and the
management of prenatal care, deliveries, and obstetric and newborn complications was not
11
performed according to clinical protocols.
5
These examples highlight the importance of the M&E
design to generate appropriate incentives and achieve desirable health output and outcome targets.
Unfortunately, as emphasized by Vellez (2010), the MOH does not conduct a regular analysis of
M&E results to identify and address systematic failures of decentralized providers toreach specific
targets.Publicly managed health facilities, instead, are not subjected to this M&E mechanism, and
therefore, it is not possible to compare them with decentralized providers in terms of final scores.
Only data on health outputs and outcomes (such as coverage rates, serviceutilization, and mortality
rates) are available for both groups, and hence used for evaluating the contracting-out model in
this paper.
Contract Duration and Early Termination
The contract duration is 12-months, and it is renewed on an annual basis. The reasons that may
trigger an early termination of the contract include a default of contract provisions from either or
both parties, MOH payment delays for more than 45 days, decentralized providersbankruptcy,
unjustified service suspension for more than 5 consecutive days, two consecutives monitorings or
annual evaluation final scores below 70, probated negligence on the part of the decentralized
provider, mutual agreement among parties, force majeure events, and provider’s corruption
practices. The contract does not envisage compensations that each party may be entitled to receive
if the early termination occurs as suggested by best practices in contract design [Iossa et al. (2007a)
and (2007b)].
Short contract duration and early termination provisions may encourage decentralized
providers’ good performance because of the fear of not renewal or contract cancelation. But in the
Honduras case, neither of them represents a real threat because there is no enough competition to
replace poor performing providers, and therefore, contract cancelation may end up in service
disruption. The MOH’s strong preference for awarding contracts to municipal commonwealths
and communal associations rather than NGOs, based on the community perception that NGOs
involvement implies a privatization of services, also limits the potential international competition.
6
4 Identifying the impact of contracting-out health care services
This paper estimates the impact of contracting-out health care to decentralized providers on health
outputs and outcomes related to access, quality and equity. In particular, I will explore to what
5
These targets refer to quality indicators No 2 and 4, and CMI indicators No 7, 8, 10, 11, and 15 in Table A.1.
6
In the period 2005-2010, just one NGO (International AID) was substituted for a communal association (La Flecha)
to provide health care services in the municipality of Macuelizo.
12
extent contracting-out municipalities show better maternal and child care indicators such as visits
per capita, prenatal care, institutional birth, child immunizations, and infant mortality rates.
Theoretically, the causal impact of a treatment or intervention Ion an outcome of interest
Yin the population is the difference between the outcome Yin the presence of the treatment and
without it: =|= 1 |= 0 . Since the same individuals cannot be under treatment
and without treatment at the same time, the second term is not observed and represents the
counterfactual event, i.e., what would have been Yif individuals had not been exposed to
treatment? In our context, this translates to the question of what would have happened with health
outputs and outcomes in contracting-out municipalities if they had not decentralized health care
provision. So a key goal of an impact evaluation is to estimate the counterfactual by identifying a
valid comparison group, the so called “control group”, in which individuals are statistically
identical to those in the treatment group before the intervention took place. Thus, if treatment and
control groups are identical except that one group is treated and the other isnot, then any difference
in outcomes must be due to the treatment or intervention.
Different approaches could be used to generate statistically equivalent treatment and
control groups. Whereas a prospective impact evaluation randomly assigns individuals into
treatment and control groups before the intervention, a retrospective approach generates both
groups ex-post, i.e., after the intervention has been implemented [Gertler et al. (2011)]. Since
municipalities that contracted-out health services were not randomly assigned, but instead selected
according to explicit criteria, this impact evaluation is based on the second approach.
The first step of my identification strategy is to geographically restrict the sample of
municipalities with public health care provision to those departments that at least have one
contracting-out municipality.
7
The analysis focuses on 13 out of 18 departments that had
contracted-out health care provision by 2010.
8
Thus, there are a total of 251 municipalities from
which 56 had contracted-out health care. Next, I use a quasi-experimental method of sampling, the
PSM technique, to select a group of municipalities with public provision that is comparable to the
contracting-out municipalities. PSM allows the selection of a control group whose distribution of
pre-intervention characteristics is similar to that of the treatment group. But instead of matching
municipalities on every pre-intervention characteristic, PSM condenses all of them into a single
number, the propensity score, which is the probability of being treated (contracted-out) as a
function of pre-intervention covariates [Rosenbaum and Rubin (1983)]. Formally, propensity
scores are obtained from a logit model that uses municipalities’ cross-section data:
= 1|=exp( )
1 + exp( ) (1)
7
Cattaneo et al. (2009) uses a similar strategy to investigate the impact of a program to replace dirt floors with cement
floors on child health and adults happiness in Mexico.
8
The departments of Atlántida, Cortes, Gracias a Dios, Islas de la Bahia, and Ocotepeque are excluded from the
analysis.
13
where the binary indicator equals 1 if health care services have been contracted-out in the
municipality ibefore 2010, or 0 if services remained under public provision, and the vector
contains pre-intervention municipal socioeconomic characteristics that were used by the MOH to
select municipalities for implementing the contracting-out model.
Using contracting-out and control group municipalities, I then estimate a two-way fixed-
effect linear regression model that controls for unobserved time-invariant municipality
characteristics (that might affect both decentralization and the output variables)
9
, and time-varying
factors that are common to both groups of municipalities.
10
Thus, the model allows for correlation
between the unobserved effect and the explanatory variables, and consistently estimates the impact
of contracting-out, provided that decentralization varies over time and is uncorrelated with time-
varying unobserved heterogeneity affecting health outputs and outcomes [Wooldridge (2002)]. I
consider the following panel data model:
= + + + + + 2
where is the health output or outcome of municipality iin year t, is an indicator variable that
equals 1 if health care services are contracted-out in municipality iin year tand 0 otherwise, is
a fixed-effect unique to municipality i, is a time-effect common to all municipalities in period
t, is a vector of control variables, and the term is a municipality time-varying error assumed
to be independent identically distributed.
11
In order to rule out potential factors affecting health outputs and outcomes, the set of
control variables include municipal socioeconomic characteristics, such as annual income per
capita, literacy rate, and share of rural population, and availability of the incentive bonus at
municipal level. There is convincing evidence that households’ socioeconomic status accounts for
a large part of the variation in health care utilization at least at the household level. Higher income
and literacy rates are associated with higher service utilization since richer and more educated
individuals tend to seek morehealth care. Proximity to health services has also been stressed as an
important factor. Larger distance to health care facilities, typical from rural areas, may restrict
access and prevent individuals to get opportune medical care when needed. Rural populations are
particularly disadvantaged as they often lack reliable means of transportation [Loevinsohn et al.
(2006); Munthali (2007); Aquino et al. (2009); Kesterton et al. (2010)]. Last but not least,
9
The inclusion of municipal fixed-effects reduces the risk of a selection bias arising from unobserved municipal
characteristics. Despite PSM removes observed pre-intervention differences between contracting-out municipalities
and the control group, it assumes that there is no selection into treatment on the basis of unobservable characteristics.
This assumption would not hold if, for example, municipalities more prone to and committed with decentralization
exert more effort in providing health care services leading to higher health outputs and outcomes. Therefore, the
pooled OLS estimator of the impact of contracting-out would be biased and inconsistent.
10
Time effects capture trends related to rural developments, such as transportation and communication improvements,
that may lead to better access to health care services.
11
This implies that the decentralization of health care in one municipality does not affect health outputs and outcomes
in other municipality.
14
performance payments are expected to motivate good performance. I take advantage of the bonus
variation across groups and within municipalities (because it was not available every year) to
control for the effect of monetary incentives’ availability (not necessarily granted) on health
outputs and outcomes.
Equation (2) assumes that the impact of contracting-out is constant over time. But it may
well happen that the effect of contracting-out varies over time as decentralized providers acquire
more experience in managing health care services. This seems a plausible assumption since the
majority of contracting-out municipalities adopted decentralization in 2008, and therefore, it may
be expected, ceteris paribus, an improvement of health outputs and outcomes over time. Therefore,
I estimate another model specification:
= + + + + + + 3
where is an interaction term between the contracting-out indicator and year dummies, and
the rest of the variables are the same of equation (2).
5 Data
The empirical analysis uses two datasets.
12
The first one is a cross-section dataset on municipal
socioeconomic characteristics prior to the adoption of the contracting-out model gathered from the
2001 Census, United Nations Development Program (UNDP), and the MOH Statistics Division.
It is used to generate the control group of municipalities with public provision, and contains 251
municipalities from which 56 have contracted-out health care services.
Table 3 reports mean differences of municipal characteristics between contracting-out and
the full sample of municipalities with public provision in the selected departments. Most variables
are statistically significant suggesting that, on average, municipalities which eventually
contracted-out health services exhibited higher malnutrition levels, a larger proportion of rural
population, housing without water and sanitation, and households living in poverty with
unsatisfied basic needs, compared to those with public provision. Also, they had a higher municipal
development index and availability of CMIs, but a relatively less educated population (according
to the illiteracy rate, years of education, and the HDI).
The second dataset is a panel for the period 2008-2010 that contains information on health
outputs and outcomes at municipal level (e.g., visits per capita, prenatal care, institutional birth,
child immunizations, and infant mortality rates) used to estimate the impact of decentralization.
13
For contracting-out municipalities, data were obtained from the recording system of the Extension
12
See Table A.2 in the Appendix for detailed information on variables’ definitions and sources.
13
Except for those municipalities that contracted-out health services in 2009 and 2010, the data is post-intervention
because few decentralized providers existed before 2008 and no reliable data were collected at the onset of the
decentralization experience.
15
of Coverage and Financing of Health Services Unit (UECF) of the MOH, in turn reported by
decentralized providers on a monthly basis. For municipalities with publicly managed health
services, instead, comparable information was provided by the MOH Statistics Division.
Information on control variables (i.e., income per capita, literacy rate, proportion of rural
population, and incentive bonus) were gathered from UNDP, the MOH Statistics Division, and
service delivery contracts.
Health outputs and outcomesindicators are constructed by normalizing the number of cases
by the population of interest: the total municipal population is used when the entire network of
health facilities was contracted-out, whereas the population in the catchment areas is used when
only some health facilities were managed by decentralized providers. Figure 1 presents average
health outputs and outcomes over time for three groups of municipalities, i.e., those with
contracting-out health care,the full sample of municipalities with public provision, and the control
group (matched-sample of municipalities with public provision after applying thePSM technique).
As the control group is statistically equivalent to contracting-out municipalities, comparisons will
be made between these two groups.
Contracting-out municipalities exhibit significant improvements in most health outputs
over time compared to the control group. Part of this success may result from the emphasis placed
on health promotion and preventive care under the decentralized model. Unlike public health
facilities, decentralized providers employ trained health promoters who identify unvaccinated
children and pregnant women and explain the importance of preventive care in order to encourage
health care utilization in the community. The relatively poorer performance observed in 2008 may
be explained by weak managerial skills and lack of experience in providing health care services
[Vellez (2010)], since 22 out of 30 decentralized providers, for which health output data is
available, joined the contracting-out model in that year.
14
The number of visits per capita may be seen as an indicator of effectiveness since a
potential unmetdemand for health care prevails in populations with high levels of unsatisfied basic
needs (UBN). Visits per capita significantly differ between groups in which decentralized
providers, on average, outperformed their public counterparts by 16 percent in the period.
Regarding maternal health care outputs, adequate prenatal care is essential for the
identification and subsequent management of preventable conditions during pregnancy that may
endanger the mother and her child. Institutional delivery, on the other hand, is an important
indicator for reducing maternal and infant mortality since the combination of skilled attendance,
appropriate supplies, and hygienic settings lowers the risk of complications that may cause death
or illness to the mother and her baby [Campbell and Graham (2006)].
Results suggest that decentralized providers successfully attracted new pregnant women to
receive first-time prenatal services, and provided follow-up prenatal care and delivery services to
those already enlisted. Prenatal care coverage (i.e., the share of pregnant women that had at least
one prenatal care visit) in contracting-out municipalities increased 15 percentage points in the
14
In addition, as some new decentralized providers started in the second semester of 2008, output data had to be
annualized to allow comparability.
16
period, but differences between groups are not statistically significant. Also, prenatal care visits
per pregnancy in contracting-out municipalities increased sharply (around 31 percent accumulated
in the period), and were considerably higher than in municipalities with public provision. Pregnant
women in contracting-out municipalities had, on average, nearly 4 prenatal visits, while those in
the control group received 3 prenatal visits in 2010. As expected, payments for each institutional
birth were effective to encourage decentralized providers for increasing the quantity of deliveries
at health facilities. The share of pregnant women that gave birth at health facilities in contracting-
out municipalities rose dramatically from 57 percent in 2008 to 80 percent in 2010, and exceeded
those in the control group. Yet differencesbetween groups are not statistically significant probably
because the number of municipalities with CMI and institutional birth data is very small.
15
With regard to child health care outputs, contracting-out also led to a substantial increase
in immunizations coverage. The share of children under 1 year old that received the BCG vaccine
in contracting-out municipalities reached nearly 70 percent in 2010, and exceeded more than 34
percentage points that of the control group. Sabin and DTP/Hep B/Hib coverage exhibits similar
patterns. The proportion of children under 1 year old that completed the 3er dose increased from
72 percent in 2008 to more than 80 percent in 2009-2010 in contracting-out municipalities, while
it remainedrelativelyconstant around 81 percentinthe control group. Differences between groups,
however, are not statistically significant.
As far as health outcomes are concerned, infant mortality rates are gold standards for
measuring quality of care, and are included as targets in the M&E along with other quality
indicators. It is apparent that although contracting-out achieved better health outputs there was not
a concomitant improvement in health outcomes. Contracting-out municipalities exhibited a mild
6 percent decrease in infant mortality rates compared to a 27 percent decrease in the control group
in the period. The fact that leading causes of infant mortality in Honduras are associated with
perinatal conditions (such as low birth weight, asphyxia, and infections) suggests that despite the
significant increase of prenatal care and institutional delivery services, there are quality issues in
the provision of health care [PAHO (2009)].
6 Empirical Results
6.1 Balancing groups: contracting-out and public provision
Given that contracting-out municipalities were not randomly chosen, a direct comparison between
them and municipalities with public provision would not help to determine whether differences in
health outputs and outcomes across groups resulted from the decentralization itself or instead from
differences in municipal characteristics. As the two groups indeed were not equivalent in terms of
15
There are only 6, 15 and 21 contracting-out municipalities with institutional birth coverage data in 2008, 2009, and
2010, respectively.
17
observable pre-intervention socioeconomic characteristics (see Table 3), it is likely that they would
have also displayed different health outputs and outcomes even in the absence of decentralization.
In order to select an observationally equivalent group of municipalities with public
provision, I first estimate a logit model of the probability that a municipality with public provision
in 2005 shifted to contracting-out before 2010 conditional on pre-intervention characteristics.
Estimation results from equation (1), reported in Table 4(column 1), show that municipalities were
more likely to contract-out health care services if they had lower levels of development and
malnutrition, and higher availability of health facilities and CMIs. Few estimated coefficients are
statistically significant because of multicollinearity among explanatory variables. But since the
interest lies in the predictive power of the whole model rather than individual coefficient estimates,
multicollinearity is not a concern. Therefore, I use the saturated model that includes all municipal
characteristics to obtain the propensity scores.
Next, based on the predicted propensity scores, I match contracting-out municipalities and
those with public provision using the one-to-one nearest neighbor matching algorithm with
replacement. This algorithm allows one municipality with public provision to serve as the match
for more than one contracting-out municipality.
16
The resulting control group is composed by 36
municipalities with public health care provision.
17
Balancing tests evaluating the common support condition suggest that matching
municipalities are a valid control group to obtain an unbiased treatment effect. First, a comparison
of pre-intervention characteristics between contracting-out municipalities and the control group
show that, after matching, mean differences are eliminated [see Table 4 (columns 2-4)]. Second,
the Kolmogorov-Smirnov test does not reject the null hypothesis of equality of distributions in
propensity scores, indicating that contracting-out and control municipalities share similar pre-
intervention characteristics.Third, although not reported, estimated coefficients from a logit model
that uses only contracting-out and control municipalities were not statistically significant, thus
indicating that observable differences between the two groups are eliminated.
18
6.2 Impact of contracting-out health care
16
Matching without replacement, instead, may perform poorly when there is little overlap of the propensity scores or
when the control group is small because treated units are matched to observations that are not necessarily similar [see
Dehejia and Wahba (2002) for further details].
17
By Department, the control group of municipalities with publicly managed health facilities is: 1) Choluteca
(Marcovia, El Corpus); 2) Comayagua (San Jose del Potrero, Minas de Oro); 3) Copan (El Paraiso, Veracruz, Santa
Rosa de Copan, San Agustin, San Nicolas, Concepcion); 4) El Paraiso (Liure); 5) Francisco Morazán (Lepaterique,
San Miguelito); 6) Intibuca (Masaguara, Colomoncagua, Dolores, Jesus de Otoro, Camasca); 7) La Paz (Chinacla,
Santa Ana, La Paz, San Pedro de Tutule); 8) Lempira (La Campa, Santa Cruz, Talgua, San Francisco, Las Flores, La
Union, Erandique); 9) Olancho (Guata); 10) Santa Barbara (Quimistan, San Vicente Centenario); 11) Valle (Nacaome,
Aramecina); and 12) Yoro (Arenal, Olanchito). None municipality was selected from the department of Colon.
18
In addition, I tried alternative nested logit models and matching algorithms (the k-nearest neighbor and the caliper)
and obtained different control groups. The control group used in this paper, proved to be the most similar to
contracting-out municipalities based on the three balancing tests.
18
Access - Health Outputs
Table 5presents the impact of contracting-out health care services on different indicators of access
using contracting-out municipalities and the control group. For each indicator, columns 1 and 2
report regression results for equation (2) and (3), respectively.
Results show a significant increase of health care visits per capita: point estimates indicate
that, on average, contracting-out municipalities provided 0.22 more visits per inhabitant than
public providers, which represents a 19 percent increase with respect to the control group in 2008.
When analyzingthe impact by year (column 2),it turns out that most of the improvement happened
in 2009.
As far as maternal health care is concerned, the effects of contracting-out appear to be
evident in 2009-2010, one year after the intervention took place for most contracting-out
municipalities. Decentralized providers were more effective in delivering follow-up prenatal care
to already enlisted pregnant women than engaging new ones to receive first-time prenatal care or
delivery services. For instance, the impact of contracting-out on prenatal visits per pregnancy
ranges between 22 and 27 percent in 2009-2010: according to point estimates, pregnant women in
contracting-out municipalities had, on average, 0.58 to 0.73 more prenatal visits than those living
in the control municipalities. Decentralized providers also outperformed their public counterparts
in terms of prenatal care coverage although to a lesser extent: the proportion of pregnant women
that received first-time prenatal care was 16 percent higher (with a point estimate of 0.11) in
contracting-out municipalities than in the control group. Nevertheless, estimation results show no
significant impact of contracting-out on institutional birth coverage in any year. Although
decentralized providers exhibited remarkably higher rates than public providers, observations are
not enough for the coefficients to be statistically significant.
19
With regard to child health care indicators, contracting-out led to large improvements in
immunization coverage. Interestingly, the most remarkable case is the BCG vaccine where point
estimates show an increase of 0.39 in the share of children under 1 year old that received the
vaccine with respect to the control group, which represents a 128 percent impact (column 1).
Improvements are concentrated to a large extent in 2008 contrasting with the rest of health output
indicators (column 2). There is also evidence of a positive effect of contracting-out on Sabin and
DTP/Hep B/Hib vaccination coverage in 2009-2010, with point estimates around 0.17 and 0.20
for both vaccines (column 2). Thus, the share of vaccinated children in contracting-out
municipalities increased 20 to 24 percent with respect to the control group.
As noted before, decentralized providers exhibit worse health outputs compared to public
providers in 2008 probably due to their lack of experience in managing health services. As a result,
most output indicators show no impact of decentralization in 2008. Column (3) reports estimation
results for equation (2) using observations in the period 2009-2010 and including department fixed
19
A null result may not necessarily mean that there is no impact. Rather it may reflect a lack of statistical power to
detect an effect because that the sample size is insufficient.
19
effects instead of municipal fixed effects. According to this new specification, contracting-out had
an impact of 19 percent on visits per capita, 16 percent on prenatal coverage, 34 percent on prenatal
visits per pregnancy, 157 percent on BCG coverage, 12 percent on Sabin coverage, and 11 percent
on DTP/Hep B/Hib coverage. However, no impact was found for institutional birth coverage.
Furthermore, I investigate the relative performance of different types of decentralized
providers by including a contracting-out dummy for each type
20
[see column (4)]. In terms of
visits per capita, municipal commonwealths and NGOs performed better than public providers
delivering 23 percent and 21 percent more visits, respectively, but there is no significant difference
with respect to communal associations. Prenatal care coverage is 17 to 18 percent higher for
communal associations and municipal commonwealths than in public providers’ areas, but there
is no significant difference between the latter and NGOs. With regard to prenatal visits per
pregnancy, municipal commonwealths show a higher performance than communal associations
and NGOs, accounting for a 36, 35, and 30 percent increase with respect to public providers,
respectively. Municipal commonwealth also outperformedother providers on BCG coverage rates.
They exhibit a 171 percent increase in coverage compared to 143 percent and 137 percent for
NGOs and communal associations, respectively. For Sabin and DTP/Hep B/Hib vaccines, NGOs
takes the lead in coverage having a 22 percent and 20 percent increase with respect to public
providers.
For most health output indicators, I find evidence that contracting-out health care services
to decentralized providers had a positive impact on access to health care. This is particularly
relevant because the bulk of financing received by decentralized providers comes from capitation
payments that provide incentives to reduce costs and keep any budget surplus. Findings suggest
that decentralized providers were neither skimping on quantity of care nor discouraging new
patients in order to reduce costs. Nevertheless, there is little evidence of a positive impact of
monetary incentives on health outputs because the coefficients for the bonus dummy are seldom
statistically significant.
Quality - Health Outcome
Table 6 reports the impact of contracting-out on health outcomes. Infant mortality rates reflect the
overall quality and accessibility of primary care services for pregnant women and their infants.
Pregnant women who do not receive prenatal care have more chances of delivering a baby with
low birth weight, which is one of the main causes of infant deaths. As prenatal and early infant
care allow the identification and treatment of preventable conditions that may endanger the mother
or her baby, the regression models control for prenatal care coverage, prenatal visits, and BCG
coverage in addition to the municipal socioeconomic characteristics.
21
20
Mayor’s offices provide health care services in only 3 municipalities, and were grouped with municipal
commonwealths because they share similar characteristics such as proportion of rural population, education level, and
average household income.
21
Another appropriate control would be institutional birth coverage, but it is not included because of the small number
of available observations.
20
There is no evidence that contracting-out contributed to reduce infant mortality rates.
Estimates show no statistically significant differences between decentralized and public providers,
although it has been a relatively slight decline in the number of deaths over time. There is only one
significant coefficient corresponding to communal associations, suggesting that infant mortality is
5.5 higher than in public providerscatchment areas. As far as control variables are concerned, the
estimated coefficients of performance bonus, prenatal care coverage and BCG coverage, contrary
to the expected, are not statistically significant. Moreover, results indicate that municipalities that
provided more prenatal visits during pregnancy had also higher mortality rates.
These findings suggest that better health outputs did not necessarily translate into better
health outcomes. Thus, improvements in access may have been accompanied by a reduction in the
quality of services, as argued by the theoretical literature. As PAHO (2009) underscores, the fact
that perinatal conditions persist as the main causes of infant mortality albeit the increase prenatal
care services and institutional births, may be indicating that the provision of such services do not
have a sufficient level of quality. Moreover, the current M&E design associated with performance
payments do not generate appropriate incentives since it allows decentralized providers to avoid
payment deductions even when targets related to quality of services are not met.
Equity - UBN
I estimate the impact of contracting-out on equity of access in order to investigate whether poorer
populations benefited more from decentralization. Following Galiani et al. (2005), who study the
effect of water privatization on child mortality, I use the proportion of households with UBN to
define three levels of poverty: i) municipalities with UBN below 67 percent (medium); ii)
municipalities with UBN between 67 and 75 percent (high), and those with UBN above 75 percent
(extremely-high). There is not a category for low UBN as it ranges from 46 to 88 percent in the
municipalities of the sample.
22
Table 7 reports the estimated effects at different ranges of poverty at the municipality level
for the period 2009-2010. The reported coefficients correspond to the interaction terms between
the contracting-out dummy and the UBN indicator function. Results suggest that contracting-out
showed a progressive effect on increasing equity of access since it benefited poorest municipalities
to a greater extent. For most health outputs, municipalities with extremely-high poverty levels
exhibited a higher impact of contracting-out that those with high and medium levels of UBN.
Point estimates indicate impacts in the order of 14 to 28 percent on prenatal coverage, 30
to 50 percent on prenatal visits, 195 to 215 percent on BCG vaccine, and 37 to 43 percent on Sabin
and DTP/HepB/Hib coverage in municipalities with high and extremely-high poverty levels,
respectively. While for visits per capita contracting-out had a higher impact (25 percent) in
municipalities with high levels of UBN rather than extremely-high or medium poverty levels, no
significant impact was found for institutional births and infant mortality rates.
22
Categories’ cut-off values are arbitrary to ensure that similar number of observations lies in each category.
21
7 Conclusions
This paper contributesto the empirical literature of the effectiveness of contracting-out health care
services. Using a quasi-experimental design, the paper evaluated the impact of decentralization on
outputs and outcomes related to access, quality and equity in health care. I found evidence that
contracting-out was effective toincreaseaccess to health care in Honduras having a positive impact
on maternal and child care outputs. Visits per capita, prenatal care, and immunization coverage
were significantly higher incontracting-out than in municipalities with public provision of services
in 2009-2010. In 2008, instead, decentralized providers did not outperform public providers,
probably because of their lack of experience in managing health services and the natural learning
process that any reform entails. Estimation results show no significant effect of contracting-out on
institutional birth rates due to the small sample, although decentralized providers exhibit higher
coverage. Contracting-out also contributed to reduce inequities in access by progressively
increasing health care provision in municipalities with high and extremely-high poverty levels.
Nevertheless, the data do not support the existence of a positive effect of contracting-out
on health outcomes such as infant mortality rates. Whereas better health outputs in contracting-out
municipalities suggest that decentralized providers did not skimp on quantity of services or
discourage new patients in order to reduce costs, the lack of impact on infant mortality rates may
imply that improvements have been accompanied by a compensating reduction in the quality of
care provided. The combination of a M&E design that poorly aligns incentives for quality
enhancement and the persistence of preventable causes of infant mortality also suggests existence
of deficiencies in the quality of care.
The papers’ findings suggest that contracting-out the provision of health care services is an
effective policy to increase access for vulnerable populations, and thus supports the MOH’s
interest in expanding the model to other rural areas. But efforts should still be made to enhance the
quality of care and ensure that higher outputs ultimately contribute to improve health outcomes of
the population. The provision of maternal and child care services is necessary but not sufficient to
achieve better health outcomes, it is also crucial that service delivery is of a high quality. In this
regard, a strategy towards quality enhancing may include the analysis of M&E results to identify
systematic failures in achieving quality targets followed by a modification in the M&E design that
unbundles the quality component in order to align providesincentives and avoid the risk of
underperformance in quality targets.
Municipal commonwealths appeared to be a better alternative for contracting-out health
care than communal associations and NGOs, and thus the MOH should focus on these types of
providers when planningthe expansion of the model. Special focus should be placed on improving
the technical and managerial capabilities of new decentralized providers since the data show lower
health outputs in 2008 when most of municipalities started to contract-out the provision of health
care services.
22
A potential limitation of the analysis is that it focused on a relatively short period of time
to evaluate performance, and in some cases it substituted departments for municipal fixed effects
reducing the likelihood of unbiased treatment effects. A further extension of the analysis to 2011-
2012 including also additional municipalities that possibly contracted-out health care services
would certainly overcome this issue.
Acknowledgements
I express especial gratitude to Dr. Mirna Moreno of the MOH for providing me with valuable
information used in this research. I also thank Maria Georgina Diaz from the Department of
Statistics, the UECF Unit staff, the Regional Sanitary Departments and decentralized providers for
providing health outputs and outcomes data. I am grateful to my supervisor Prof. Giancarlo
Spagnolo for their enthusiastic guidance, and to Juan Pradelli for his continuous help and support.
Referencies
Aquino, R., Oliveira, N., and Barreto, M. (2009). “Impact of the Family Health Program on Infant
Mortality in Brazilian Municipalities”, American Journal of Public Health, 99(1):87-93.
Banerjee, A. and Duflo, E. (2000). “Reputation Effects and the Limits of Contracting: A Study of
the Indian Software Industry”, The Quarterly Journal of Economics, 115(3): 989-1017.
Bar-Isaac, H. and Tadelis, S. (2008). “Seller Reputation”, Foundations and Trends in
Microeconomics, 4(4): 273-351.
Basinga, P., Gertler, P., Binagwaho, A., Soucat, A., Sturdy, J., and Vermeersch, C. (2010). “Paying
Primary Health Care Centers for Performance in Rwanda”, Policy Research Working Paper No
5190, The World Bank, January 2010.
Bloom, E., King, E., Bhushan, I., Kremer, M., Clingingsmith, D., Loevinsohn, B., Hong, R., and
Schwartz, J.B. (2006). “Contracting for Health: Evidence from Cambodia”, Brookings Institution,
mimeo.
Calzolari, G. and Spagnolo, G. (2009). “Relational Contracts and Competitive Screening”, CEPR
Discussion Paper No DP7434.
Campbell, O. and Graham, W. (2006). “Strategies for reducing maternal mortality: Getting on with
what works”, Lancet, 368 (9543):1284-1299.
Cattaneo, M., Galiani, S., Gertler, P., Martinez, S., and Titiunik, R. (2009). “Housing, Health, and
Happiness”, American Economic Journal: Economic Policy, 1(1): 75-105.
23
Cristia, J., Evans, W., and Kim, B. (2011). “Does Contracting-out Primary Care Services Work?
The Case of Rural Guatemala”, Working Paper Series No IDB-WP-273, Inter-American
Development Bank, November 2011.
Danel, I. and La Forgia, G. (2005). “Contracting for Basic Health Care in Rural Guatemala-
Comparison of the Performance of Three Delivery Models”, Chapter 2 in La Forgia, G. Ed.: Health
System Innovation in Central America: lessons and impact of new approaches, Working Paper No
57, The World Bank.
Dellarocas, C., Dini, F., and Spagnolo, G. (2006). “Designing Reputation Mechanisms”, Chapter
18 of Dimitri, N., Piga, G., and Spagnolo, G. (Eds.), Handbook of Procurement, Cambridge
University Press, pp. 446-482.
Dehejia, R., and Wahba, S. (2002). “Propensity score-matching methods for non-experimental
causal studies”, The Review of Economics and Statistics, 84(1):151-161.
Eichler, R., Auxila, P., Antoine, U., and Desmangles, B. (2007). “Performance-Based Incentives
for Health: Six Years of Results from Supply-Side Programs in Haiti”, Center for Global
Development, Working Paper No 121, April 2007.
Galiani, S., Gertler, P., and Schargrodsky, E. (2005). “Water for Life: The impact of the
Privatization of Water Services on Child Mortality”, Journal of Political Economy, 113(1):83-120.
Garcia Prado, A.and Lao Peña, C. (2010). “Contracting and Providing Basic Health Care Services
in Honduras: A Comparison of Traditional and Alternative Service Delivery Models”, Human
Development Network Discussion Paper, The World Bank.
Gauri, V., Cercone, J., and Briceno, R. (2004). “Separating financing from provision: evidence
from 10 years of partnership with health cooperatives in Costa Rica”, Health Policy and Planning,
19(5): 292-301.
Gertler, P., Martinez, S., Premand, P., Rawlings, L., and Vermeersch, C. (2011). Impact
Evaluation in Practice, The World Bank.
Hart, O., Shleifer, A., and Vishny, R. (1997). “The Proper Scope of Government: Theory and an
Application to Prisons”, The Quarterly Journal of Economics, 112(4): 1127-1161.
Iossa, E. and Rey, P. (2010). “Building Reputation for Contract Renewal: Implications for
Performance Dynamics and Contract Duration, CEIS Research Paper No 155, University of Tor
Vergata.
Iossa, E., Spagnolo, G.,and Vellez, M.(2007a). “Contract Design in Public-Private Partnerships”,
Report prepared for the Sustainable Development Department, The World Bank.
24
Iossa, E., Spagnolo, G., and Vellez, M. (2007b). “Best Practices in Contract Design in Public-
Private Partnerships”, Report prepared for the Sustainable Development Department, The World
Bank.
Kesterton, A., Cleland, J., Sloggett, A., and Ronsmans, C. (2010). “Institutional delivery in rural
India: the relative importance of accessibility and economic status”, BMC Pregnancy and
Childbirth, 10:30.
Klein, B. Leffler, K. (1981). “The Role of Market Forces in Assuring Contractual Performance”,
Journal of Political Economy, 89(4): 615-641.
Lagarde, M., and Palmer N. (2009). “The impact of contracting-out on health outcomes and use of
health services in low and middle income countries”, Cochrane Database of Systematic Reviews
2009, Issue 4. Art. No.: CD008133.
Lavadenz F., Schwab, N., and Straatman, H. (2001). “Redes publicas, descentralizadas y
comunitarias de salud en Bolivia”, Pan American Journal Public Health, 9(3): 182-189.
Levin, J. and Tadelis, S. (2010). “Contacting for Government Services: Theory and Evidence from
U.S Cities”, Journal of Industrial Economics, 58(3): 507-541.
Liu, X., Hotchkiss, D., and Bose, S. (2008). “The effectiveness of contracting-out primary health
care services in developing countries: a review of the evidence”, Health Policy and Planning, 23:
1-13.
Loevinsohn, B. (2008). Performance-Based Contracting for Health Services in Developing
Countries, A Toolkit, Health, Nutrition and Population Series, The World Bank.
Loevinsohn, B. and Harding, A. (2005). “Buying results? Contracting for health service delivery
in developing countries”, The Lancet, 366: 676-681.
Loevinsohn, B., Hong, R., and Gauri, V. (2006). “Will more inputs improve the delivery of health
services?: Analysis of district vaccination coverage in Pakistan”, International Journal of Health
Planning and Management, 21: 45-54.
MacLeod, W. B. (2007). “Reputations, Relationships, and Contract Enforcement”, Journal of
Economic Literature, 45(3): 595-628.
MOH (2009). “Estudio Comparativo entre el Modelo Descentralizado y el Modelo Centralizado
en la Provisión de Servicios de Salud en Atención Primaria en Honduras”, junto con MEASURE
Evaluation y PRODIM Consultants, Marzo 2009.
Munthali, A. (2007). “Determinants Of Vaccination Coverage In Malawi: Evidence From The
Demographic And Health Surveys”, Malawi Medical Journal, 19(2): 79-82.
25
PAHO (2009). Health Systems Profile Honduras, Monitoring and Analyzing Health Systems
Change”, Pan American Health Organization, February 2009.
Rosenbaum, P. and Rubin, D. (1983). “The central role of the propensity score in observational
studies for causal effects”, Biometrika, 70(1):41-55.
Slack, K. and Savedoff, W. (2001). “Public Purchaser-Private Provider Contracting for Health
Services, Examples from Latin America and the Caribbean”, Sustainable Development
Department, Technical Papers Series, Inter-American Development Bank.
Vellez, M. (2010). “Evaluación del Modelo de Gestión Descentralizada de Atención Primaria de
Salud en Honduras”, Programa de Apoyo a la Articulación de Políticas Sociales HO-T1126,
reporte preparado para el Banco Interamericano de Desarrollo, División de Protección Social y
Salud, Agosto 2010.
Wooldridge, J. (2002). Econometric Analysis of Cross Section and Panel Data, MIT Press.
26
TABLES AND GRAPHS
Table 1. Summary of contracting-out empirical studies
Country Author Contracting-out Methodology
Performance
Dimension
Results
Bolivia
Lavadenz et al.
(2001)
Descriptive before-
after comparision with
control district
Acces s and
quality
Higher number of ambulatory visits, institutional deliveries , bed
occup ancy rates, and user satisfaction.
Cambo dia Bloom et al. (2006)
Pre-designed
experimental with
random selection of
districts
Access,
efficiency,
quality, and
health ou tcomes
Higher vitamin A supplementation, prenatal care, deliveries by
trained pers onnel, full immunization, and use of public facilities.
Lower out-of-pocket expenditures, but higher public health
spending. Patients had a poorer perception of quality. Reduced
the chance of reporting illnes s during the past month , and the
incidence of diarrhea in children, but had no effect on child
survival.
Costa Rica Gauri et al. (2004)
Panel data analysis
with control public
clinics
Acces s and
efficiency
Higher number of GP visits per capita. Fewer specialist visits
per capita, lab exams per visit, medicines per visit, and lower
expenditures per capita.
Danel and La
Forgia (2005)
Cross-s ection data
from ho usehold and
provider surveys with
control public
providers
Access,
efficiency and
quality
Higher immunization rates, prenatal care utilization, us e of
tetanus to xoid and iron su pplementation during pregnancy in
the mixed provider but are indistinguishable in the traditional
and direct model. Higher use of ORS, growht-monitoring
checkups, and user satisfaccion in mixed and direct provider.
Higher productivity of labor, and unitary and per capita costs
in mixed and direct providers than their traditional counterparts.
Cristia et al. (2011) Sercice delivery contract
Difference-in-
difference analys is
using LSMS
househo ld survey
Access
Higher prenatal care and immunization cov erage, but no impact
on family planning services.
Haiti Eichler et al. (2007)
Panel data analysis
with control NGOs
with cost-
reimbursment
payments
Access
Higher imnun ization rates and attended deliveries, but less
conclus ive results on prenatal and pos tnatal care visits.
MoH (2009)
Descriptive cros s-
section stu dy
comparing 8 contract-
out an d 8 public health
facilities
Acces s, quality,
equity, and
efficiency
Higher coverage in all the selected outpu ts, except for
immunizations rates ; better accomplishment of clinical
protocols but not difference in average waiting time; families
reported low inco me levels and poor life conditions; higher
number of visits per phys ician and lower costs per visit in the
selected outp uts.
García Prado and
Lao Peña (2010)
Descriptive cros s-
section stu dy
comparing 10 contract-
out an d 10 public
health facilities using
exit facility survey s;
probit analysis
Acces s, quality,
and efficiency
More closesly located to referrals hos pitals, more functioning
vehicles, lower waiting times , higher cleanliness, more
availability of drugs and better equipped, higher percentage of
trained pers onnel, more managerial auton omy; higher unitary
costs of medicines, lower wage costs, and higher labor
productivity. Patients were more likely to return to health
facility if this was managed by a non-state provider, the more
satisfied they were with received treatment, the more clealines s
of bathrooms, if th ey received all medicines, and waiting time
was less th an hour.
Rwanda Basinga et al.
(2010)
Pre-designed random
selection of districts,
quasi-experimental
approach using
difference in difference
specification
Acces s and
quality
Higher probability of having an institutional birth and receiving
child visits, but no impact on prenatal care and child
immunization services . Higher likelihood of receiving tetanus
vaccines in prenatal visit and getting a higher quality sco re.
Guatemala
Honduras
27
Table 2. Expansion of the contracting-out model (accumulated figures)
2005 2006 2007 2008 2009 2010
Departments
6 6 8 11 13 13
Municipalities
8 8 11 48 53 56
As a share of total (%)
3.2 3.2 4.4 19.1 21.1 22.3
Benefited population
60,133 60,133 116,521 475,385 663,182 705,452
Decentralized providers
6 6 8 21 25 25
Municipal commonwealths
1 1 3 7 9 9
Communal associations
3 3 3 8 10 10
NGOs
2 2 2 4 4 4
Major's offices
- - - 2 2 2
Health facilities
17 17 38 171 194 203
CMO
6 6 12 44 53 53
CESAR
8 8 21 112 122 127
CMI
3 3 5 15 19 23
Year
28
Table 3. Mean differences for pre-intervention municipal socioeconomic
characteristics
Contracting-
out
municipalities
Municipalities
with public
provision
(full sample)
Mean
difference
Total Population 17,153 19,418 2,265
(2,927) (4,773) (9,055)
Proportion of housing without water (%) 64.41 54.68 -9.73***
(2.55) (1.49) (1.14)
Proportion of housing without sanitation (%) 41.87 39.41 -2.46***
(0.59) (0.53) (1.04)
Proportion of rural population (%) 89.96 83.21 -6.75**
(1.88) (1.68) (3.29)
Poverty Line (% households under PL) 84.81 79.22 -5.59***
(0.91) (0.55) (1.14)
Proportion of households with UBN (%) 73.23 68.30 -4.93***
(1.09) (0.79) (1.60)
Malnutrition Index 49.25 42.56 -6.69***
(1.90) (1.03) (2.17)
Municipal Development Index 74.36 70.07 -4.29**
(1.46) (0.99) (2.02)
Illiteracy rate 46.20 39.44 -6.76***
(1.99) (0.99) (2.13)
Years of education 1.79 2.17 0.38***
(0.09) (0.06) (2.13)
Human Developent Index (1-HDI) 42.85 39.73 -3.13***
(0.57) (0.33) (0.69)
Health facilities (per 10,000 inhabitants) 3.65 3.94 -0.29
(0.19) (0.15) (0.30)
Health facilities density (per km2)2.39 2.24 -0.16
(0.24) (0.11) (0.25)
CMI (per municipality) 0.34 0.17 -0.16**
(0.07) (0.03) (0.06)
Health facilities (per municipality) 5.14 4.66 -0.48
(0.67) (0.41) (0.86)
Observations 56 195 251
Variables
Before matching
Notes: standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and 10%,
respectively.
29
Figure 1. Health care outputs and outcomes over time
1.00
1.13
1.25
1.38
1.50
2008 2009 2010
Visits per capita
Public (full sample) Public (matched sample)
Contracted-out
0.60
0.70
0.80
0.90
1.00
2008 2009 2010
Prenatal coverage
Public (full sample) Public (matched sample)
Contracted-out
1.00
1.50
2.00
2.50
3.00
3.50
4.00
2008 2009 2010
Prenatal visits per pregnancy
Public (full sample) Public (matched sample)
Contracted-out
0.20
0.40
0.60
0.80
1.00
2008 2009 2010
Institutional birth coverage
Public (full sample) Public (matched sample)
Contracted-out
30
Figure 1. Health care outputs and outcomes over time (continue)
0.20
0.35
0.50
0.65
0.80
2008 2009 2010
BCG coverage
Public (full sample) Public (matched sample)
Contracted-out
0.40
0.55
0.70
0.85
1.00
2008 2009 2010
Sabin coverage
Public (full sample) Public (matched sample)
Contracted-out
0.40
0.55
0.70
0.85
1.00
2008 2009 2010
DTP/Hep B/Hib coverage
Public (full sample) Public (matched sample)
Contracted-out
4.00
6.00
8.00
10.00
12.00
2008 2009 2010
Infant mortality rate
Public (full sample) Public (matched sample)
Contracted-out
31
Table 4. Logit estimates and mean differences for pre-intervention municipal
socioeconomic characteristics after matching
Contracting-
out
municipalities
Municipalities
with public
provision
(control group)
Mean
difference
Total Population -0.000 17,153 14,986 -2,166
(0.000) (2,927) (2,764) (4,274)
Proportion of housing without water (%) 0.017 64.41 61.56 -2.86
(0.015) (2.55) (3.35) (4.16)
Proportion of housing without sanitation (%) 0.071 41.87 41.90 0.03
(0.054) (0.59) (0.98) (1.08)
Proportion of rural population (%) -0.010 89.96 86.12 -3.85
(0.017) (1.88) (3.82) (3.85)
Poverty Line (% households under PL) -0.000 84.81 83.07 -1.74
(0.074) (0.91) (1.17) (1.48)
Proportion of households with UBN (%) -0.022 73.23 72.75 -0.48
(0.038) (1.09) (1.70) (1.93)
Malnutrition Index -0.056** 49.25 50.42 1.17
(0.023) (1.90) (2.33) (3.02)
Municipal Development Index 0.002 74.36 73.74 -0.62
(0.031) (1.46) (2.17) (2.52)
Illiteracy rate -0.005 46.20 41.49 -4.71
(0.029) (1.99) (2.58) (3.23)
Years of education -0.114 1.79 1.92 0.14
(0.537) (0.09) (0.14) (0.16)
Human Developent Index (1-HDI) 0.195* 42.85 41.84 -1.01
(0.112) (0.57) (0.85) (0.99)
Health facilities (per 10,000 inhabitants) -0.253 3.65 3.65 0.00
(0.178) (0.19) (0.22) (0.30)
Health facilities density (per km2)14.090 2.39 2.53 0.13
(15.985) (0.24) (0.29) (0.38)
CMI (per municipality) 1.070** 0.34 0.28 -0.06
(0.497) (0.07) (0.09) (0.11)
Health facilities (per municipality) 0.269* 5.14 4.75 -0.39
(0.140) (0.67) (0.78) (1.06)
Observations 251 56 36 92
After matching
Logit regression
for contracting-
out health care
Variables
Notes: standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and 10%, respectively. The logit regression
includes department fixed effects and a constant term. The dependent variable is an indicator that equals 1 if the municipality
has contracted-out health care services and 0 otherwise. LR Chi2 (75.81), Pseudo R2 (0.285), Kolmogorov-Smirnov p-value
(0.134).
32
Table 5. Impact of contracting-out health services on access
(1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4)
Contracting-out 0.222* 0.101
0.223***
0.073 0.009 0.121** 0.410 -0.151
0.931***
-0.211 -0.245 -0.231
(0.127) (0.159) (0.063) (0.079) (0.087) (0.061) (0.347) (0.341) (0.175) (0.281) (0.285) (0.496)
Contracting-out * 2009 0.118*
0.118***
0.585***
0.132
(0.069) (0.044) (0.155) (0.247)
Contracting-out * 2010 0.159 0.083
0.737***
-0.143
(0.109) (0.058) (0.165) (0.242)
Communal association 0.117 0.137*
0.941***
-0.283
(0.090) (0.078) (0.206) (0.634)
NGO
0.243***
0.089
0.823***
-0.166
(0.089) (0.086) (0.227) (0.485)
Municipal commonwealth
0.269***
0.124*
0.960***
-0.220
(0.075) (0.074) (0.223) (0.651)
Bonus 0.122** 0.187* 0.167* 0.179* 0.031 0.038 0.077 0.077 -0.027 0.255* 0.057 0.062 -0.144 -0.222 0.354 0.359
(0.055) (0.102) (0.092) (0.094) (0.034) (0.050) (0.097) (0.097) (0.131) (0.151) (0.245) (0.249) (0.133) (0.149) (0.322) (0.340)
Observations 249 249 173 173 249 249 173 173 249 249 173 173 56 56 45 45
R-squared 0.848 0.852 0.410 0.422 0.955 0.957 0.528 0.528 0.906 0.916 0.562 0.562 0.974 0.976 0.716 0.717
Estimated impact 19% 10% 19% 21-23% - 16% 16% 17-18% - 22-27% 34% 30-36% - - - -
Institutional birth coverage
Notes: Robust standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and 10%, respectively. All models include a constant, percentage of rural population,
literacy rate, and per capita income as control variables. Models (1) and (2) include municipalities and time fixed-effects, while models (3) and (4) contain department and time fixed-
effects using 2009-2010 data.
Visits per capita
Prenatal coverage
Prenatal visits per pregnancy
Control group mean was 1.17 in 2008
Control group mean was 0.75 in 2008
Control group mean was 2.7 in 2008
Control group mean was 0.27 in 2008
33
Table 5. Impact of contracting-out health services on access (continue)
(1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4)
Contracting-out
0.396***
0.347***
0.486***
0.019 -0.136* 0.100** 0.039 -0.116 0.095*
(0.103) (0.110) (0.058) (0.066) (0.080) (0.049) (0.068) (0.085) (0.049)
Contracting-out * 2009 0.050
0.171***
0.164***
(0.053) (0.062) (0.062)
Contracting-out * 2010 0.064
0.204***
0.203***
(0.052) (0.060) (0.065)
Communal association
0.424***
0.055 0.032
(0.092) (0.085) (0.085)
NGO
0.443***
0.185** 0.169**
(0.104) (0.072) (0.073)
Municipal commonwealth
0.529***
0.095 0.103*
(0.069) (0.059) (0.059)
Bonus 0.041 0.066 0.060 0.070 -0.015 0.058 0.019 0.020 -0.010 0.066 0.022 0.025
(0.037) (0.052) (0.094) (0.096) (0.039) (0.056) (0.089) (0.090) (0.039) (0.061) (0.090) (0.090)
Observations 249 249 173 173 249 249 173 173 249 249 173 173
R-squared 0.948 0.949 0.586 0.590 0.896 0.905 0.252 0.257 0.893 0.903 0.254 0.260
Estimated impact 128% 112% 157%
137-171%
- -16-24% 12% 22% - 20-24% 11% 12-20%
Notes: Robust standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and 10%, respectively. All models include a
constant, percentage of rural population, literacy rate, and per capita income as control variables. Models (1) and (2) include municipalities
and time fixed-effects, while models (3) and (4) contain department and time fixed-effects using 2009-2010 data.
BCG coverage
Sabin coverage
DTP/Hep B/Hib coverage
Control group mean was 0.31 in 2008
Control group mean was 0.84 in 2008
Control group mean was 0.84 in 2008
34
Table 6. Impact of contracting-out health services on quality of care
(1) (2) (3) (4)
Contracting-out -4.645 -6.553 1.283
(4.490) (5.424) (2.469)
Contracting-out * 2009 0.730
(2.819)
Contracting-out * 2010 2.684
(3.870)
Communal association 5.585*
(3.267)
NGO -1.619
(3.262)
Municipal commonwealth -0.675
(2.415)
Bonus 0.335 2.013 1.367 0.926
(2.215) (3.701) (2.393) (2.347)
Prenatal care coverage 5.673 6.101 -2.058 -2.944
(6.869) (6.936) (3.321) (3.320)
Prenatal visits per pregnancy 0.967 0.660 2.043* 2.054*
(1.965) (2.018) (1.138) (1.111)
BCG coverage 7.896 7.851 0.045 1.106
(5.821) (5.879) (2.853) (2.772)
Observations 249 249 173 173
R-squared 0.558 0.560 0.220 0.269
Estimated impact
- - - 54%
Infant mortality
Notes: Robust standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and
10%, respect ively. All specifications include a constant, percentage of rural population,
literacy rate, and per capita income as control variables. Models (1) and (2) include
municipalities and time fixed-effects, while models (3) and (4) contain department and time
fixed-effects using 2009-2010 data.
Control group mean was 10.2 in 2008
35
Table 7. Impact of contracting-out health services on equity
Visits per
capita
Prenatal
care
coverage
Prenatal
visits per
pregnancy
Institutional
birth
coverage
BCG
coverage
Sabin
coverage
DTP/Hep
B/Hib
coverage
Infant
mortality
Contracting-out 0.031 -0.323* -0.036 0.039 -0.068 -0.182 -0.187 2.885
(0.143) (0.167) (0.333) (0.557) (0.161) (0.145) (0.146) (3.503)
Contracting-out * High UBN 0.291* 0.443** 0.820* -0.334 0.605*** 0.315* 0.307* -2.518
(0.173) (0.218) (0.457) (0.599) (0.198) (0.170) (0.170) (3.064)
Contracting-out * Extremely High UBN 0.179 0.535*** 1.338*** 1.005 0.666*** 0.355** 0.363** -2.682
(0.150) (0.175) (0.364) (0.960) (0.166) (0.148) (0.148) (3.632)
Bonus 0.150 0.068 0.052 0.135 0.051 0.017 0.020 0.992
(0.095) (0.115) (0.260) (0.293) (0.106) (0.091) (0.091) (2.399)
Prenatal care coverage -1.130
(3.119)
Prenatal visits per pregnancy 1.990*
(1.160)
BCG coverage 1.151
(2.937)
Observations 173 173 173 45 173 173 173 173
R-squared 0.370 0.329 0.513 0.758 0.461 0.248 0.250 0.221
Estimated impact 25% -43-28% 30-50% - 195-215% 38-42% 37-43% -
Control group mean in 2008 1.17 0.75 2.7 0.27 0.31 0.84 0.84 10.2
Notes: Robust standard errors in parentheses. ***,**,* indicates significance at 1%, 5%, and 10%, respectively. All models use observations for t he period 2009-2010
and include a constant, percentage of rural population, literacy rate, department and time fixed-effect s, and UBN dummies. Reported coefficients correspond to the
interaction between cont racting-out dummy and three categories of UBN: municipalities with UBN below 67% (medium), municipalities with UBN between 67% and
75% (high), and those with UBN above 75% (extremely high). Baseline category is medium UBN.
36
APPENDIX
Table A.1. M&E performance indicators
Management area (M):
1The decentralized provider developed a working schedule based on the Operation Annual Plan (POA) for the monitoring period
2All commitments made in previous monitoring were honored during the period
3Purchases and provision of medical supplies to health facilities were made within 15 days of request
4All human resources required in the contract are employed
5Decisions about projects execution are evaluated between technical and management personnel
6At least 1 training session for employees was conducted in the monitoring period
7Statistical reports were submitted in a timely manner
8Actions towards the improvement of health facilities and promotion teams were supported by the decentralized provider
9The target of prenatal care coverage was achieved according to the contract
10 The target of postnatal care coverage was achieved according to the contract
11 At least 95% of DTP/Hep B/Hib vaccination coverage in children less than 12 months old (3er dose) was achieved
12 At least 95% of SRP vaccine coverage in children 12 to23 months old was achieved
13 All cases of maternal deaths were analyzed with a corresponding action plan
14 All cases of child deaths (under 5 years old) were analyzed with a corresponding action plan
Health promotion, prevention, and surveillance at health centers (H):
1At least 90% of the scheduled educational activities at schools were conducted in the monitoring period
2All emergency cases in the community were transported to the health facilities
3All pregnant women receiving prenatal care had a birth plan
4The decentralized provider facilitated community group meetings at the health centers (pregnant, high blood pressure.)
5At least 90% of scheduled domiciliary visits to high risk households were made
6The health facility had an up to date sketch to identify community risks (biological, social, etc.)
7All communities had complete reports and files
8All community housing satisfied good health behaviors (maternal and child care, safety water, waste management, etc.)
9At least 95% of DTP/Hep B/Hib vaccination coverage in children less than 12 months old (3er dose) was achieved
10 At least 95% of SRP vaccine coverage in children 12 to23 months old was achieved
11 At least 95% of BCG vaccination coverage in children less than 12 months old was achieved
12 At least 80% of pregnant women received prenatal care
13 At least 80% of new mother patients received postnatal care within 10 days after delivery
14 At least 80% of expected patients with respiratory conditions were treated
15 According to the Children Surveillance List (LINVI) no children had vaccines delays of more than 10 days
16 The health facility had a monthly working schedule for the monitoring period
Quality of services at health centers (Q):
1All pregnant women were registered, with risk identification and follow up actions
2All pregnant woman receiving first prenatal visit in the monitoring period were registered and evaluated according to protocols
3At least 70% of new pregnant patients received prenatal care before 12 weeks gestation
4At the third prenatal visit, all pregnant women had the second blood and urine test results archived in their clinic history
5All history clinics of postnatal women had temperature, and blood pressure records
6All postnatal visits includes information/advisory on family planning methods
7All children under 2 years old receiving care at the health facilities were registered in the Children Surveillance List (LINVI)
8All child care visits were conducting according to the Integrated Management of Childhood Diseases (AIEPI) protocols
9All children under 5 years old with respiratory conditions were treated according to protocols
10 All children under 5 years old with diarrhea conditions were treated according to protocols
QUARTERLY MONITORING
37
11 At least 80% of patients with respiratory conditions had three bacilloscopies according to protocols
12 At least 80% of smear test samples satisfied quality standards
13 All patients with smear tes t results related to swelling, STD, or malignant cells were treated accordingly
14 The health facility requested medical supplies within the first 15 days of the quarter
15 Drugs and medical supplies were available during the monitoring period
16 At least 80% of the quality improvement plan activities were performed
17 The health facility complied with biosafety measures
18 The health facility complied with management of cold chain norms
CMI (C):
1The quality improvement plan for the CMI was under execution
2The newborn and maternal-child care norms were available at the CMI
3Drugs and medical supplies were available during the monitoring period
4The CMI complied with biosafety measures
5The CMI complied with management of cold chain norms
6Prompt to deliver women living far away from the CMI received accommodation at the maternal-child residence
7In at least 90% of institutional births the partogram was used correctly
8All emergency births cases were referred to hospital
9At least 90% of births possess a complete history clinic
10 In the third stage of birth, all patients were treated according to protocols
11 All pregnant women with obstetric complications were treated according to protocols
12 After delivery, all new mothers received information/advisory on family planning methods
13 At least 90% of newborns were treated according to protocols
14 All newborns weighting 2.5kg or more received the BCG vaccine
15 All newborns with complications (low weight, infection, suffocation) were treated according to protocols
16 Improvement requests were reviewed and analyzed by the Health Committee
Annual Performance (A):
1At least 90% of statistical reports were submitted in a timely manner
2At least 2 community projects were executed
3At least 3 personnel training sessions to strengthen health care provision were conducted in the monitoring period
4At least 1 cleaning and destruction of breeding grounds were conducted with community participation
5At least 1 health exposition/meeting at each health facility was organized
6All newborns weighting 2.5kg or more received the BCG vaccine (if managed a CMI)
7At least 95% of SRP vaccine coverage in children 12 to23 months old was achieved
8At least 95% of DTP/Hep B/Hib vaccination coverage in children less than 12 months old (3er dose) was achieved
9The target of prenatal care coverage was achieved according to the contract
10 At least 70% of new pregnant patients received prenatal care before 20 weeks gestation
11 The rate of institutional births increased from previous monitoring period
12 At least 70% of new mother patients received postnatal care within 10 days after delivery
13 At least 60% of women between 30 and 69 years old received a smear testing
14 All women with positive smear testing were referred and followed up
15 Preventable infant deaths were reduced or kept in cero
16 Preventable maternal deaths were reduced or kept in cero
17 Equipment, human resources, and at least 50% of health facilities' authorization plan requirements were accomplished
18 All health facilities have a valid health license
19 At least 2 monitoring in the year had a final score above 80
20 At least 85% of patients are satisfied with the health care received
21 The Operation Annual Plan (POA) was monitored quarterly
22 A quality improvement plan was implemented at the CMI and CESAMOS
QUARTERLY MONITORING
ANNUAL EVALUATION
38
Table A.2. M&E final score formula
M&E
Final Score
Values
Deductions
and Bonus
M*0.2 + (H + Q)*0.5 + C*0.3
84 - 75 -2%
74 - 65 -4%
64 - 60 -6%
M*0.3 + (H + Q)*0.7
below 59 -15%
Annual
Evaluation
Awhere above 85 +1%
Final Score Formula
Note: The final score of the quarterly monitoring is a weighted average of decentralized providers' performance in different
areas, where (M) refers to Management, (H) is Health promotion, prevention and surveillance, (Q) is Quality of care, and
(C) are CMI indicators. If decentralized providers do not manage a CMI, the monitoring final score is then simplified.
Performance indicators of the annual evaluation are denoted by (A).
Quarterly
Monitoring
or
where
39
Table A.3. Data and sources of information
Variables
Sources
TREATMENT VARIABLE:
Contracting-out Service delivery contracts 2008-2010
PRE-INTERVENTION MUNICIPAL CHARACTERISTICS:
Total Population
Proportion of housing without water
Proportion of housing without sanitation
Proportion of rural population
Poverty Line (% households under PL)
Proportion of households with UBN
Malnutrition Index INE Census Talla 2000
Municipal Development Index INE Government 2001
Illiteracy rate
Years of education
Human Developent Index (1-HDI)
Health facilities (per 10,000 inhabitants)
Health facilities density (per km2)
CMI (per municipality)
Health facilities (per municipality)
HEALTH OUTPUTS AND OUTCOMES:
Visits per capita (Total visits/Total population)
Prenatal care coverage (1st prenatal visits/Total pregnant women)
Prenatal visits per pregnancy (Total prenatal visits/Total pregnant women)
Institutional birth coverage (Institutional deliveries/Population under 12 months)
BCG coverage (Doses/Population under 12 months)
Sabin coverage (3rd doses/Population under 12 months)
DPT/Hep B/Hib coverage (3rd doses/Population under 12 months)
Infant mortality rate (Infant deaths /Population under 12 months per 1,000)
CONTROL VARIABLES:
Income per capita (PPP)
Literacy rate
Proportion of rural population MOH Statistics Department 2008-2010
Incentive bonus Service delivery contracts 2008-2010
MOH UECF (non-state providers) and
MOH Statistics Department (public
providers) 2008-2010
INE Census 2001
UNDP 1998
MOH Statistics Department, RUPS 2001
UNDP 2007
ResearchGate has not been able to resolve any citations for this publication.
Book
Full-text available
http://siteresources.worldbank.org/EXTHDOFFICE/Resources/5485726-1295455628620/Impact_Evaluation_in_Practice.pdf
Article
Full-text available
The report presents examples to illustrate many of the key issues regarding health care contracting. This study discusses 27 examples of contracting with the private sector in Latin America and the Caribbean. The examples cover three types of services and target population: 11 are broadly-defined services for the entire population (e.g. outpatient services by private hospitals in Brazil); 8 are specific services for the entire population (e.g. high-tech services -heart surgery, kidney transplants, hip replacements, etc.- by private units in Uruguay); and 8 are specific services for a target population (e.g. a program to reduce child malnutrition in Honduras). In addition to its analysis of the cases, the study has generated a publicly available electronic database that can be corrected, updated, and expanded.
Article
Full-text available
Seller reputation is an important asset because buyers often chooses sellers on the basis of their reputation. This is particularly true when the quality of the good or service transacted are hard to measure and the parties cannot perfectly contract on the outcome of the transaction. As a consequence, the seller will be mindful of building and maintaining a good reputation through the information that buyers have about the seller, including previous transactions and the reports of other buyers. We introduce a unifying framework that embeds a number of different approaches to seller reputation, incorporating both hidden information and hidden action. We use this framework to stress that the way in which consumers learn affects both behavior and outcomes. In particular, the extent to which information is generated and socially aggregated determines the efficiency of markets. After reviewing these theoretical building blocks we discuss several applications and empirical concerns. We highlight that the environment in which a transaction is embedded can help determine whether the transaction will occur and how parties will behave. Institutions, ranging from the design of online markets to norms in a community, can be understood as ensuring that concerns for reputation lead to more efficient outcomes. Similarly, the desire to affect consumer beliefs regarding the firm's incentives can help us understand strategic firm decisions that seem unrelated to the particular transactions they wish to promote. We conclude by considering slightly different models of reputation that lie beyond the scope our framework, briefly reviewing the somewhat sparse empirical literature and highlighting and suggesting future directions for research.
Article
Full-text available
The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates. Applications include: (i) matched sampling on the univariate propensity score, which is a generalization of discriminant matching, (ii) multivariate adjustment by subclassification on the propensity score where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations, and (iii) visual representation of multivariate covariance adjustment by a two- dimensional plot.
Article
We investigate the impact of a large-scale Mexican program to replace dirt floors with cement floors on child health and adult happiness. We find that replacing dirt floors with cement significantly improves the health of young children measured by decreases in the incidence of parasitic infestations, diarrhea, and the prevalence of anemia, and an improvement in children's cognitive development. Additionally, we find significant improvements in adult welfare measured by increased satisfaction with their housing and quality of life, as well as by lower scores on depression and perceived stress scales.
Article
In order to determine whether physical resources or technical inputs can make a difference to the delivery of health services, we carried out a study that examined the large variation in district level vaccination coverage in Pakistan. Vaccination coverage was assessed by district-wise cluster surveys and the predictor variables were collected from census data and from a survey of 99 district health offices. Information was collected on basic supplies, physical infrastructure, management, training, socio-economic variables, and a variety of other indicators. Univariate and multivariate analyses were carried out. A model including female literacy rate, TV ownership, and provincial dummies explained 48% of the variation in DTP3 coverage. Very few of the other variables examined were significantly correlated to coverage. Possible explanatory variables like adequacy of syringe and vaccine supply, the number of vaccinators per capita, recent training of managers, frequency of supervision, availability of micro-plans, and turnover of managers were not correlated with coverage. While the Government of Pakistan has ensured that many physical resources and technical inputs have been provided to the district health offices, this does not appear able to explain the relatively low overall coverage or the variation between districts. Bolder initiatives and innovations are likely needed to improve delivery of basic health services.
Article
Beginning in 1997, the government of Guatemala extended coverage of a package of basic health services to 3 million people living in rural, impoverished, and primarily indigenous communities through three different delivery models: contracting of nongovernmental organizations (NGOs) to directly provide services (direct model), contracting of NGOs to act as financial managers for the Ministerio de Salud y Assistencia (Ministry of Health and Social Protection [MSPAS]) health service providers (mixed model), and strengthening of health posts operated by the MSPAS (traditional model). This chapter presents the results of household and provider surveys carried out in 2001. The household survey found that women and children in catchment areas served by the mixed model had significantly better results for many key health indicators when compared to the traditional model. Results for those served by the direct model were similar to the traditional model although these communities had only recently begun to receive services and were located much farther from health facilities. Users tended to report greater satisfaction with the new models. The provider survey found that the NGO-based models are generally more productive than the traditional model; however, they are more costly, and results on economic efficiency were mixed. We conclude that mixed and direct provider models are good alternatives to traditional health services for providing basic services in Guatemala. Contracting basic health care may be an effective strategy to reduce health inequities in other country contexts where access to health services is inadequate.
Article
The purpose of this paper is to identify groups of children in Malawi who are less or not reached by vaccination services by using data from the Demographic and Health Surveys conducted in Malawi in 1992, 1996, 2000 and 2004. These surveys have shown that the proportion of children aged 12- 23 months who were fully vaccinated by 12 months of age has been decreasing: it was 67% in 1992, then 55%, 54% and 51% in 1996, 2000 and 2004, respectively. The review has also shown that birth order of the child, residence (rural/ urban) and mother's education are major determinants of the immunization status of the child. Malawi Medical Journal Biology Vol. 19 (2) 2007: pp. 79-82