of the reminders was even greater for clinicians whose compli-
ance rates had been worse before the intervention. In fact, as the
study progressed, compliance rates improved. Given that scant
data exist on effective interventions to translate evidence-based
medicine into practice in developing countries,
offer a new powerful tool for improving HIV care in these
resource-limited settings. The study also highlights the impor-
tance of EHRs in these settings.
As stated by Dexter et al,
the easy sustainability of
computer-based reminder systems contrasts with the weak-
nesses of such approaches as manual reviewing of charts
physician-directed continuing medical education.
computer-generated reminders can be tailored for all levels of
providersdan approach that is particularly relevant in settings
where less-trained personnel provide a large amount of care. In
addition, CDSS allows evolving care protocols to be seamlessly,
efﬁciently, and broadly introduced into clinical practice.
As in prior reminder studies from the developed world,
12 24 27 28
we observed a variation in adherence to computer reminders
among clinicians. Even though we did not formally evaluate
reasons for non-adherence, informal questioning of reminder
recipients indicated that several factors were at play. Some
clinicians had rote practice patterns and simply disagreed with
the algorithms used in the reminder: with education about the
reminders, acceptance rates improved. Occasionally, clinicians
were right to ignore the reminders because the recommended
action was inappropriate for the particular patient during that
visit, often because the clinician had other information not
available in the computer. In fact, because computers are limited
by the data they contain, computer reminders should be
considered as care suggestions to the clinicians. The ﬁnal deci-
sion must still rest with the clinician, as clinical judgment
should always take precedence over the computer’s judgment.
Several limitations in our study deserve mention. The gener-
alizability of our ﬁndings is limited by the fact that only a few
reminders were implemented and at a single clinical site. We
observe a variation in compliance by type of CD4 reminder,
which demonstrates that not all care suggestions will be treated
equally by clinicians. Our evaluation only lasted for a short
period of time, and we cannot account for possible retardation in
efﬁciency and effectiveness of utilization over time. Another
limitation of our study is that the intervention would have an
uncertain role in settings with no EHR. However, many care
rules are based on limited data (eg, gender, age, duration of care)
and do not require fully implemented EHRsdfor example,
reminders about childhood immunizations are based solely on
the child’s age and history of prior immunizations, and the latter
can easily be maintained with simple ﬂowsheets in patients’
charts. Even where EHRs have been implemented in developing
countries, the generalizability of our intervention may be limited
by the additional level of technology required to implement
CDSS. It should however be noted that our intervention only
used a single computer at the intervention clinic. Lastly, the
comparative design (instead of a randomized study design) may
have introduced some bias. We controlled for signiﬁcant cova-
riates in the analyses.
This study provides a model through which HIV care guide-
lines can be broadly implemented in resource-limited settings.
The approach can be used to provide reminders about druge
drug interactions and known allergies,
and reminders on
overuse or underuse of diagnostic tests or medications. The
guidelines can also extend beyond HIV to encompass a broad set
of diseases, especially for conditions such as diabetes and
hypertension where particular care protocols are well accepted
as best practice. To better delineate the speciﬁc effects of
reminders, we are conducting evaluations in which patient
summaries are being presented to both intervention and control
groups, but clinical reminders presented only to the intervention
group. In the future, we hope to demonstrate the impact of
CDSS on patient outcomes and quality of care in these resource-
limited settings, and determine sustainability of the observed
impact of reminders over time.
Clinical summaries with computer-generated reminders signiﬁ-
cantly improved clinician adherence to CD4 testing guidelines in
this ﬁrst study of its kind in sub-Saharan Africa. This technology
can have broad applicability to improve quality of HIV care in
Acknowledgments We would like to thank our patients and providers at the study
clinics. Special thanks to S Masit, P Tanui, J Lelei, B McKown, B Wolfe, J Kariuki,
J Lagat, R Vreeman, and A Yeung.
Funding This work was supported by a grant from the Abbott Fund, and in part by
a grant to the USAIDeAMPATH Partnership from the United States Agency for
International Development as part of the President’s Emergency Plan for AIDS Relief
Competing interests None.
Ethics approval Ethics approval was provided by the Institutional Review Boards at
Indiana University School of Medicine in Indianapolis, Indiana and the Institutional
Review and Ethics Committee at Moi University School of Medicine in Eldoret, Kenya.
Provenance and peer review Not commissioned; externally peer reviewed.
1. Chen L, Evans T, Anand S, et al. Human resources for health: overcoming the crisis.
2. Joint United Nation s Programme on AIDS. Report on the Global AIDS Epidemic.
Geneva: UNAIDS, 2008.
3. Kober K, Van Damme W. Scaling up access to antiretroviral treatment in southern
Africa: who will do the job? Lancet 2004;364:103e7.
4. Kumar P. Providing the providersdremedying Africa’s shortage of health care
workers. N Engl J Med 2007;356:2564e7.
5. Narasimhan V, Brown H, Pablos-Mendez A, et al. Responding to the global human
resources crisis. Lancet 2004;363:1469e72.
6. Dovlo D. Using mid-level cadres as substitutes for internationally mobile health
professionals in Africa. A desk review. Hum Resour Health 2004;2:7.
7. Samb B, Celletti F, Holloway J, et al. Rapid expansion of the health workforce in
response to the HIV epidemic. N Engl J Med 2007;357:2510e14.
8. McDonald CJ. Protocol-based computer reminders, the quality of care and the
non-perfectability of man. N Engl J Med 1976;295:1351e5.
9. Nicholas DD, Heiby JR, Hatzell TA. The Quality Assurance Project: introducing
quality improvement to primary health care in less developed countries. Qual Assur
Health Care 1991;3:147e65.
10. Wikipedia: Clinical Decision Support System. http://en.wikipedia.org/wiki/Clinical_
Decision_Support (accessed 26 Oct 2009).
11. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support
tools in the ambulatory/primary care setting: a systematic review of the literature.
Inform Prim Care 2008;
12. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled
trials to evaluate computer-based clinical reminder systems for preventive care in the
ambulatory setting. J Am Med Inform Assoc 1996;3:399e409.
13. Safran C, Rind DM, Davis RB, et al. Guidelines for management of HIV infection with
computer-based patient’s record. Lancet 1995;346:341 e 6.
14. Williams F, Boren SA. The role of the electronic medical record (EMR) in care
delivery development in developing countries: a systematic review. Inform Prim Care
15. Einterz RM, Kimaiyo S, Mengech HN, et al. Responding to the HIV pandemic: the
power of an academic medical partnership. Acad Med 2007;82:812e18.
16. Tierney WM, Rotich JK, Hannan TJ, et al. The AMPATH medical record system:
creating, implementing, and sustaining an electronic medical record system to
support HIV/AIDS care in western Kenya. Stud Health Technol Inform
17. Mamlin BW, Biondich PG, Wolfe BA, et al. Cooking up an open source EMR for
developing countries: OpenMRSd a recipe for successful collaboration. AMIA Annu
Symp Proc 2006:529e33.
18. Seebregts CJ, Mamlin BW, Biondich PG, et al. The OpenMRS Implementers
Network. Int J Med Inform 2009;78:711e20.
154 J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520
Research and applications