Researchers and funders need to use systems
approaches that are beginning to translate
research not only to the bedside but also to
global health programs.
Temina Madon, Karen J. Hofman,* Linda Kupfer, Roger I. Glass
care) and their delivery to communities in the
developing world. As a result, nearly 14,000
people in sub-Saharan Africa and South Asia
die daily from HIV , malaria, and diarrheal dis
ease (1), even though scientific advances have
enabled prevention, treatment, and, in some
cases, elimination of these diseases in devel
Many evidence-based innovations fail to
produce results when transferred to commu
nities in the global south, largely because
their implementation is untested, unsuitable,
or incomplete. For example, rigorous studies
have shown that appropriate use of insecti
cide-treated bed nets can prevent malaria (2),
yet, in 2002, fewer than 10% of children in 28
sub-Saharan African countries regularly slept
with this protection (3). Newer studies have
shown that malaria incidence is decreased by
distribution of free nets, but further
research is needed to promote cost-effective,
sustained access—particularly for the poor
living in rural areas (4).
The same is true of strategies to prevent
mother-to-child transmission of HIV. Al
though interventions like prophylactic anti
retroviral therapy and replacement feeding
have worked well in hospitals and clinics,
increasing coverage in rural areas (where
women have limited access to clean water and
formal health care) may require testing of
novel approaches, such as self-administration
of drugs (5, 6). Similarly, the scale-up of male
circumcision, which has been shown to pro
tect against HIV transmission in recent clini
cal trials (7), will require development of safe,
culturally acceptable, and accessible methods
for surgery and care (8).
e face a formidable gap between
innovations in health (including
vaccines, drugs, and strategies for
The Implementation Research Gap
Why is effective implementation, particularly
in resource-poor countries, such an intractable
problem? The reasons are complex. First, sci
entists have been slow to view implementa
tion as a dynamic, adaptive, multiscale phe
nomenon that can be addressed through a
Division of Advanced Science and Policy Analysis, The John
E. Fogarty International Center, U.S. National Institutes of
Health, Bethesda, MD 20892, USA.
*Author for correspondence. E-mail: email@example.com. gov
research agenda. Although randomized, con
trolled experiments are the gold standard for
testing safety and efficacy of pharmaceuti
cals, health delivery schemes are less likely to
be subject to rigorous scientific analysis.
Second, people living in poverty face a
bewildering constellation of social constraints
and health threats that make prevention and
treatment more difficult. They often have lim
ited knowledge of preventive health practices
and insufficient or sporadic access to quality
care. Their health systems are underfinanced,
underregulated, and crippled by health-worker
shortages. Even for those with access to
care, health is routinely undermined by heavy
pathogen loads, environmental exposures,
inadequate sanitation infrastructure, and socio
economic barriers to behavior change. Faced
with such challenges, it is not surprising that
public-health professionals have found it dif
ficult to successfully adapt, implement, and
sustain new interventions.
Although a few rigorous studies of imple
mentation could advance the delivery of
A doctor seeing a patient through Progresa, Mexico’s incen
tive-based development program that targets the very poor.
By improving children’s growth and nutrition, it has especially
benefited those who lack health care. Its success in improving
health outcomes depends on rigorous, scientific studies that
established the effectiveness of new strategies.
health care in low-income coun
tries, recent billion-dollar increases
in budgets for global health have
provided only limited support for
studies needed to ensure maxi
mum impact (9). Instead, planners
often assume that clinical research
findings can be immediately trans
lated into public health impact,
simply by issuing “one-size-fits
all” clinical guidelines or best prac
tices without engaging in system
atic study of how health outcomes
vary across community settings.
A Framework for Research?
Realizing the need for a quantita
tive, scientific framework to guide
health-care scale-up in developing
countries, researchers in health,
engineering, and business are build
ing interest in implementation
science (10–14). Unlike routine
applied (or operations) research,
which may identify and address
barriers related to performance of
specific projects, implementation
science creates generalizable know
ledge that can be applied across set
tings and contexts to answer central questions.
Why do established programs lose effective
ness over days, weeks, or months? Why do
tested programs sometimes exhibit unintended
effects when transferred to a new setting? How
can multiple interventions be effectively pack
aged to capture cost efficiencies and to reduce
the splintering of health systems into disease-
specific programs? Answering questions like
these will require analysis of biological, social,
and environmental factors that impact imple
mentation, both to develop and test communi
tywide, multisector interventions that are not
testable in clinical settings, and to identify how
proven clinical interventions should be modi
fied to achieve sustained health improvements
in the “real world.” A few innovative studies
have begun to appear (15).
One example is the research program coor
dinated with implementation of Mexico’s 1997
reform of health and social services. Before
reform, food subsidies and health care were
provided by the Mexican government, largely
without gains in public health and welfare.
CREDIT: INTERAMERICAN DEVELOPMENT BANK
Downloaded from www.sciencemag.org on December 20, 2007
14 DECEMBER 2007 VOL 318 SCIENCE www.sciencemag.org
Published by AAAS
POLICYFORUM? Download full-text
Frustrated with poor outcomes, the govern
ment worked with scientists to develop a multi-
sector antipoverty program, Progresa, to help
increase the uptake of existing nutrition and
The new initiative provided conditional
financial incentives for poor rural families, on
the basis of their use of prenatal, child health,
and nutritional services provided by local clin
ics. Because researchers were involved in the
initial design, they were able to build a prospec
tive, cluster-randomized experiment into the
program’s roll-out, revealing statistically sig
nificant improvements in child development as
a result of the new initiative (16). Because these
and other quantitative studies showed sustained
effectiveness (17), conditional cash transfers
have enjoyed continuous support of the
Mexican government, despite radical changes
in political leadership. Similar programs are
being adopted by policy-makers throughout
The West African Onchocerciasis Control
Programme (OCP) is another example of how
rigorous implementation research can am
plify the public health impact of proven inter
ventions. This decades-long initiative has
used established vector elimination methods
and communitywide drug treatment cam
paigns to control the nematode parasite that
causes river blindness. However, the program
is unique in that it has, from the beginning,
integrated mathematical modeling into every
aspect of implementation and ongoing opera
tion (18). Modeling of strategies has enabled
the OCP to package together tested interven
tions, without direct experimentation. It has
also helped optimize interventions to match
field conditions and has enabled scientists to
better understand parasite transmission and
Many implementation experiments—par
ticularly cluster-randomized trials and agent-
based models that compare the population-
level health impacts of different delivery
strategies—can be coupled with the planning
and roll-out of new programs by health min
istries, making the cost of research marginal.
They can also be used to model the potential
gains of health-system designs, policies, and
multisector interventions that cannot be tested
experimentally. These approaches all require
the involvement of scientists in early planning
to ensure that research questions are incorpo
rated into program design.
Identifying New Research Opportunities
Opportunities for learning about implementa
tion are particularly promising for initiatives
like the Global Fund to Fight AIDS, Tuber
culosis, and Malaria; the U.S. President’s
Emergency Plan for AIDS Relief (PEPFAR);
and the President’s Malaria Initiative. To date,
these programs have focused on trial-and-error
optimization of health services, using descrip
tive studies, process evaluations, and monitor
ing to measure program outputs. More recently,
they have expanded to include targeted evalua
tions, which use comparison groups to infer the
likely impacts of interventions on community
health. Among the questions they need to
address are those relating to behavior change
and HIV prevention; the effectiveness of
orphan care services; the risk of drug resistance
in the scale-up of antiretroviral and antimalarial
therapy; and the packaging together of inter
ventions for HIV/AIDS, tuberculosis, and
malaria. Questions that focus on health-care
providers and systems include how pay-for
performance schemes impact quality and cost
of care, and how agent-based modeling of
clinic and hospital operations can inform devel
opment of human resources for health.
Implementation Science for Global Health
There are three additional imperatives facing
the research community. First, we must
advance theoretical models and new analytic
methods that apply to resource-poor settings.
This may include, for example, developing
frameworks for implementation that rely on
existing social networks and markets for sus
tained health-care delivery, rather than the
training of health workers—a limited resource
in most developing countries. Multiple disci
plines, from systems science and computer
simulation to public health and behavioral eco
nomics, need to be integrated.
The World Health Organization’s Special
Programme for Research and Training in
Tropical Diseases (TDR) has begun to address
this need, through support of collaborative
research grants in implementation research
(19). For example, with funds from the Exxon-
Mobil Foundation, TDR researchers are
currently testing the impacts of health-care
franchising (based on a micro-enterprise busi
ness model) on access to antimalarial drugs
in Kenyan villages (20, 21). Programs like
these should be expanded. The U.S. National
Institutes of Health is actively soliciting inter
national research proposals for its ongoing
initiative in Dissemination and Implementation
Research in Health (22).
Second, we need to train a generation
of researchers who can effectively bridge the
implementation gap. This will require new cur
ricula and interdisciplinary, systems-oriented
approaches. Because some features of imple
mentation are context-specific, it also calls for
strengthening of research institutions in low-
A final imperative is for researchers to col
laborate with developing country governments,
nongovernmental organizations (NGOs), and
communities. For example, the George Wash
ington University School of Public Health and
Health Services recently announced a partner
ship with the Elizabeth Glaser Pediatric AIDS
Foundation, to help capture opportunities
to integrate research into the delivery of
HIV/AIDS prevention and treatment services
supported by the foundation (23).
Although implementation experiments and
computational modeling may be more com
plex—in terms of study design and data analy
sis—than the monitoring and observational
studies currently funded by donors, any incon
venience is outweighed by the profound ability
of scientifically rigorous findings to focus lim
ited health resources and to save more lives.
References and Notes
1. A. D. Lopez, C. D. Mathers, M. Ezzati, D. T. Jamison,
C. J. L. Murray, Eds., Global Burden of Disease and Risk
Factors (Oxford Univ. Press, New York, 2006).
2.? C. Lengeler, “Insecticide-treated bednets and curtains for
preventing malaria” (Cochrane Review, update software,
Cochrane Library, issue 4, Oxford, 2001).
3.? R. Monasch et al., Am. J. Trop. Med. Hyg. 71 (suppl.), 232
4.? A. M. Noor, A. A. Amin, W. S. Akhwale, R. W. Snow, PLoS
Med. 4, e255 (2007).
5. D. J. Jackson et al., AIDS 21, 509 (2007).
6.? J. Kagaayi et al., J. Acquir. Immune Defic. Syndr. 39, 121
7. B. Auvert et al., PLoS Med. 2, e298 (2005).
8. T. C. Quinn, Curr. Opin. Infect. Dis. 20, 33 (2007).
9.? W. D. Savedoff, R. Levine, N. Birdsall, When Will We Ever
Learn? Improving Lives Through Impact Evaluation
(Report of the Evaluation Gap Working Group, Center for
Global Development, Washington, DC, 2006).
10. M. P. Eccles, B. S. Mittman, Implement. Sci. 1, 1 (2006).
11. D. Sanders, A. Haines, PLoS Med. 3, e186 (2006).
12.? D. L. Fixsen et al., Implementation Research: A Synthesis
of the Literature [Florida Mental Health Institute (FMHI)
publ. no. 231, Louis de la Parte FHMI, University of South
Florida, The National Implementation Research Network,
Tampa, FL, 2005].
13. R. G. A. Feachem, Trop. Med. Int. Health 9, 1139 (2004).
14.? E. A. McCarthy, M. E. O’Brien, W. R. Rodrigues, PLoS Med.
3, e304 (2006).
15.? P. Buekens, G. Keusch, J. Belizan, Z. A. Bhutta, JAMA 291,
16.? E. Skoufias, PROGRESA and Its Impacts of the Welfare of
Rural Households in Mexico (Research report 139,
International Food Policy Research Institute, Washington,
17. E. Gakidou et al., Lancet 368, 1920 (2006).
18.? F. E. McKenzie, E. M. Samba, Am. J. Trop. Med. Hyg. 71?
(suppl.), 94 (2004).?
19.? Special Programme for Research and Training in Tropical
Diseases (TDR), www.who.int/tdr/topics/ir/default.htm.
20.? “Shopkeepers to deliver health to Africa? Trust the people,”
RealHealthNews (Global Forum for Health Research,
Geneva, May 2007).
21.? Additional details re (20) available at www.cfwshops.org/
22.? Announcement of request for proposals, http://grants.?
23.? Elizabeth Glaser Pediatric AIDS Foundation,?
Downloaded from www.sciencemag.org on December 20, 2007
www.sciencemag.org SCIENCE VOL 318 14 DECEMBER 2007
Published by AAAS