Article

Evaluation of computer-generated reminders to improve CD4 laboratory monitoring in sub-Saharan Africa: A prospective comparative study

Indiana University School of Medicine, Indianapolis, Indiana, USA.
Journal of the American Medical Informatics Association (Impact Factor: 3.5). 03/2011; 18(2):150-5. DOI: 10.1136/jamia.2010.005520
Source: PubMed

ABSTRACT

Little evidence exists on effective interventions to integrate HIV-care guidelines into practices within developing countries. This study tested the hypothesis that clinical summaries with computer-generated reminders could improve clinicians' compliance with CD4 testing guidelines in the resource-limited setting of sub-Saharan Africa.
A prospective comparative study of two randomly selected outpatient adult HIV clinics in western Kenya. Printed summaries with reminders for overdue CD4 tests were made available to clinicians in the intervention clinic but not in the control clinic.
Changes in order rates for overdue CD4 tests were compared between and within the two clinics.
The computerized reminder system identified 717 encounters (21%) with overdue CD4 tests. Analysis by study assignment (regardless of summaries being printed or not) revealed that with computer-generated reminders, CD4 order rates were significantly higher in the intervention clinic compared to the control clinic (53% vs 38%, OR = 1.80, CI 1.34 to 2.42, p < 0.0001). When comparison was restricted to encounters where summaries with reminders were printed, order rates in intervention clinic were even higher (63%). The intervention clinic increased CD4 ordering from 42% before reminders to 63% with reminders (50% increase, OR = 2.32, CI 1.67 to 3.22, p < 0.0001), compared to control clinic with only 8% increase from prestudy baseline (CI 0.83 to 1.46, p = 0.51). Limitations Evaluation was conducted at two clinics in a single institution.
Clinical summaries with computer-generated reminders significantly improved clinician compliance with CD4 testing guidelines in the resource-limited setting of sub-Saharan Africa. This technology can have broad applicability to improve quality of HIV care in these settings.

Full-text

Available from: Sylvester Kimaiyo, Jan 04, 2016
Evaluation of computer-generated reminders to
improve CD4 laboratory monitoring in sub-Saharan
Africa: a prospective comparative study
Martin C Were,
1,2
Changyu Shen,
1,2
William M Tierney,
1,2
Joseph J Mamlin,
1,4
Paul G Biondich,
1,2
Xiaochun Li,
1,2
Sylvester Kimaiyo,
3,4
Burke W Mamlin
1,2
ABSTRACT
Objective Little evidence exists on effective interventions
to integrate HIV-care guidelines into practices within
developing countries. This study tested the hypothesis
that clinical summaries with computer-generated
reminders could improve clinicians’ compliance with CD4
testing guidelines in the resource-limited setting of
sub-Saharan Africa.
Design A prospective comparative study of two
randomly selected outpatient adult HIV clinics in western
Kenya. Printed summaries with reminders for overdue
CD4 tests were made available to clinicians in the
intervention clinic but not in the control clinic.
Measurements Changes in order rates for overdue CD4
tests were compared between and within the two
clinics.
Results The computerized reminder system identified
717 encounters (21%) with overdue CD4 tests. Analysis
by study assignment (regardless of summaries being
printed or not) revealed that with computer-generated
reminders, CD4 order rates were significantly higher in
the intervention clinic compared to the control clinic
(53% vs 38%, OR¼1.80, CI 1.34 to 2.42, p<0.0001).
When comparison was restricted to encounters where
summaries with reminders were printed, order rates in
intervention clinic were even higher (63%). The
intervention clinic increased CD4 ordering from 42%
before reminders to 63% with reminders (50% increase,
OR¼2.32, CI 1.67 to 3.22, p<0.0001), compared to
control clinic with only 8% increase from prestudy
baseline (CI 0.83 to 1.46, p¼0.51).
Limitations Evaluation was conducted at two clinics in
a single institution.
Conclusions Clinical summaries with computer-
generated reminders significantly improved clinician
compliance with CD4 testing guidelines in the resource-
limited setting of sub-Saharan Africa. This technology
can have broad applicability to improve quality of HIV
care in these settings.
INTRODUCTION
Healthcare systems in the worlds poorest places
must care for large numbers of patients with
a heavy disease burden. Paradoxically, these same
healthcare systems have too few resources and too
few skilled personnel.
1
In sub-Saharan Africa, the
challenge of providing adequate care is increasingly
magnied as large numbers of HIV-positive
patients seek treatment.
23
While HIV care in
developed countries would typically be managed by
specialist physicians, resource-limited settings often
utilize less-trained healthcare workers out of
necessity.
4e7
The combination of overworked staff
with limited training, increasingly busy clinics, the
challenges of providing chronic disease manage-
ment, and the non-perfectibility of man
8
often
results in suboptimal patient care.
9
Approaches are urgently needed to improve the
quality and outcomes of care offered to patients in
these resource-limited settings. Unfortunately, no
amount of training will overcome the limitations of
human providers in high-volume clinics with
complex care protocols. However, basic inform a-
tion management tools may help healthcare
workers do the right things. One such tool is
offered by Clinical Decision Support Systems
(CDSS).
10
CDSS use data stored in electronic health
records (EHRs) to provide care suggestions and
reminders to clinicians whenever there is a devia-
tion from the accepted standard of care, and thus
offer one of the most power ful tools in EHRs. In
the developed world, CDSS have been shown to
improve clinician behaviors and the quality of
healthcare.
11 12
In fact, a study conducted in the
USA by Safran et al demonstrated that computer-
based alerts and reminders were effective in helping
clinicians adhere to HIV care guidelines.
13
Although more and more resource-limited
settings are implementing EHRs,
14
little research
documents whether these systems affect clinician
behavior or improve the quality of care. Extrapo-
lating from CDSS success in the developed world,
we hypothesized that CDSS (implemented within
EHRs) would change clinician behavior and
improve the quality of care offered to HIV-positive
patients in the resource-limited setting of sub-
Saharan Africa. In this study, we assessed whether
a CDSS that provides clinicians in an HIV care
system in western Kenya with just-in-time clinical
summary reports and patient-specic care sugges-
tions could improve adherence to accepted CD4
testing guidelines.
METHODS
Setting
This study was conducted at two randomly selected
adult HIV clinics afliated with the partnership
between United States Agency for International
Development and the Academic Model Providing
Access to Healthcare (USAIDeAMPATH) in
Western Kenya.
15
This program provides compre-
hensive care to over 100 000 active HIV-positive
patients through 23 parent and 23 satellite clinics
<
An additional appendix is
published online only. To view
this file please visit the journal
online (http://jamia.bmj.com).
1
Indiana University School of
Medicine, Indianapolis, Indiana,
USA
2
Regenstrief Institute,
Indianapolis, Indiana, USA
3
Moi University School of
Medicine, Eldoret, Kenya
4
USAIDeAMPATH Partnership,
Eldoret, Kenya
Correspondence to
Dr Martin C Were, Regenstrief
Institute, and Indiana University
School of Medicine, 410 West
10th Street, Suite 2000,
Indianapolis, IN 46202-3012,
USA; mwere@regenstrief.org
Received 8 May 2010
Accepted 1 December 2010
Published Online First
20 January 2011
150 J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520
Research and applications
Page 1
(gure 1). The urban USAIDeAMPATH facility is located in
Eldoret, Kenya and is home to four HIV clinicsdone pediatric and
three adult clinics. All three adult clinics offer the same services in
the same building, with 90% of the patient visits handled by
nurses and clinical ofcers (equivalent to physician assistants)
without the presence of a supervising physician. Enrollment to
the clinics is based largely on the order of presentation, and there is
very little patient crossover between clinics.
Electronic medical record
Since 2004, USAIDeAMPATH clinics have used the AMPATH
Medical Record System (AMRS) to store comprehensive, longi-
tudinal, electronic patient records for all enrolled patients.
16
AMRS is the original implementation of OpenMRS, an open-
source EHRs deployed widely in the developing world.
17 18
Patient records in the system contain demographic information,
historical and physical examination data, problem lists, medi-
cations, diagnostic test results, and visit data. The clinical
information are stored largely as coded concepts (as opposed to
free-text) for easy retrieval and analysis.
19
Clinicians caring for AMPATH patients do not enter data
directly into AMRS but rather complete paper encounter forms
that contain clinical parameters and categorical observations
previously dened and encoded into the AMRS concept dictio-
nary (see appendix, available as an online data supplement at
http://www.jamia.org). Where necessary, clinicians can write
down diagnoses, test results, and other observations as free-text
if these are not included in checklists on the encounter form.
Clerks with basic computer skills and minimal medical knowl-
edge enter data from the encounter forms into the AMRS. The
encounter forms are then placed in the patients paper clinic
chart, which is available to the clinician during patient care.
Clinical summaries and reminders
With input from AMPATH clinicians, we developed a module
within OpenMRS which generated a patient-specic clinical
summary that displayed selected information from the patients
record to provide a quick reference to the most relevant data
needed by the clinicians. The module also contained CDSS
functionality which appended patient-specic care suggestions
and reminders to the bottom of the clinical summary
(gure 2).
20
Generated clinical summaries with reminders could
be printed and were typically attached to patients paper charts
at the time of a patients visit.
For the current study, we implemented ve care suggestions in
the CDSS which recommended that an overdue CD4 test be
ordered if particular criteria were met (table 1). Overdue CD4
studies were determined based on testing algorithms used for
clinical care at USAID eAMPATH. These algorithms were based
on recommendations by the WHO
21
and Kenyan Ministry of
Health,
22
and had been adopted through consensus with specic
attention to nancial constraints within the USAIDeAMPATH
setting.
Intervention
We assessed the effect of clinical summary reports with CD4
care suggestions on adherence to testing protocols in a controlled
trial at two randomly selected adult clinics afliated with
USAIDeAMPATH. One clinic was the intervention clinic for
our study, whereas the other was the control clinic. For both
study clinics, we determined baseline order rates for overdue
CD4 tests 2 months prior to the intervention using the same
algorithms for generating study CD4 care suggestions.
Our study took place in February 2009. When an adult HIV-
positive patient presented for a return visit at the intervention
clinic during the study period, a patient summary report was
generated at registration. A printout of the summary (with the
reminder section containing suggestions for CD4 testing, if
indicated) was placed at the front of the patients paper chart,
along with a blank encounter form. These were made available
to all intervention clinic providers who interacted with the
Figure 1 United States Agency for International Developmente
Academic Model Providing Access to Healthcare clinical sites.
Figure 2 United States Agency for International Developmente
Academic Model Providing Access to Healthcare clinical summary with
a reminder to order CD4 count.
J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520 151
Research and applications
Page 2
patient during that visit. The summaries with reminders could
also be viewed on a computer placed in the intervention clinic.
Clinicians recorded data on the encounter form where appro-
priate. If they wanted to order a CD4 Panel, they would check
the option for CD4 Panel in the Tests Ordered section of the
encounter form (see appendix: item 18, available as an online
data supplement at http://www.jamia.org). Data from
completed encounter forms were entered into AMRS, and the
forms placed in the patients paper chart. For the control clinic,
the computer also generated summaries with reminders, but no
printouts were made available to clinicians, and there was no
computer through which the summaries could be viewed.
Patients had to be eligible for CD4 testing (ie, have a CD4 care
suggestion generated) to be included in the study. The study was
approved by the Institutional Review Boards at Indiana
University School of Medicine in Indianapolis, Indiana and the
Institutional Review and Ethics Committee at Moi University
School of Medicine in Eldoret, Kenya.
Statistical analysis
The unit of analysis for all analyses was the individual clinic
visitdthat is, each individual clinicianepatient encounter. This
unit was chosen because patients typically saw whichever
clinician was available at the time of their visit, instead of seeing
the same clinician every time. The primary outcome of the
study was clinicians compliance rates with ordering CD4
laboratory studies. We primarily compared compliance rates
between intervention and control clinics during the study
period, controlling for confounding of measured characteristics
and clustering effects in our model. To eliminate potential
confounding due to unmeasured characteristics that differed
between the two clinics, we also conducted a preepost
comparison within each clinic.
Continuous variables were summarized by mean (SD) or
median (IQR), and categorical variables summarized by
frequency and percentage. Comparisons of continuous and
categorical variables between different groups were performed
with Wilcoxon rank-sum and Fisher exact tests, respectively.
Demographic data used for these comparisons were obtained as
part of the routine procedures for clinic registration and patient
care.
We used generalized linear mixed-effects model to account for
(1) confounding effects due to potentially non-randomized
assignment to intervention and control groups and (2) clustering
effect due to the fact that some clinicians treated more than one
patient. To identify covariates included for the purpose of (1)
above, we rst perfor med univariate analyses to identify factors
associated with intervention assignment, and all signicant
factors are included in the model in addition to the intervention
variable. It should be noted that some patients appear more than
once in these data, indicating a potential clustering effect of
patient. However, such cases are relatively few (less than 4%),
and inclusion of the additional patient clustering effect does not
change the analysis results substantially. Therefore, we chose to
report the results based on models with clustering by clinician
only. We also used generalized linear mixed-effects model to
assess changes in compliance with reminders over time. For all
analyses, a two-sided p value of <0.05 was considered statisti-
cally signicant. All analyses were performed in SAS 9.1.
RESULTS
Study subjects and clinics
Baseline evaluation conducted 2 months prior to our study
revealed that 436 (29%) of 1482 patient visits in the intervention
clinic had an overdue CD4 test compared to 489 (31%) of 1581
visits in the control clinic. During this time, baseline CD4 order
rates for the overdue tests were different between the inter-
vention clinic (42%) and control clinic (36%) (p¼0.04, OR¼1.32,
95% CI 1.01 to 1.72).
A total of 3108 patients accounted for 3405 clinic visits during
the study period in February 2009; there were 1929 visits to the
intervention clinic and 1476 visits to the control clinic. For 361
(19%) of the visits (349 unique patients) to the intervention
clinic, a CD4 test was overdue compared to 355 (24%) of visits
(341 unique patients) to the control clinic. All patients with
overdue CD4 tests were included in the study. Study patients
had a mean age of 38 years, 65% were women, and 64% were on
antiretroviral medication for HIV (table 2). Twenty-ve (3.6%)
of these patients had more than one clinical encounter during
the study periods, with no patients crossing over between the
two study clinics. Signicant differences between intervention
and control patients included WHO stage, most recent prior
CD4 count, and years since clinical enrollment. All subsequent
analyses adjusted for these characteristics.
A total of 36 clinicians (29 clinical ofcers, ve nurses, and
two medical doctors) cared for the study patients, with four
clinicians crossing over between the two clinics. At the begin-
ning of the study, clinicians at the two clinics had worked at
AMPATH for an average of 3.01 years (SD¼1.95 years), with
experience levels and gender distribution not varying signi-
cantly between the two clinics for those who did not cross over
(p¼0.38 and p¼0.09 respectively).
Effect of clinical summaries with reminders on rates of
ordering CD4 tests
Comparison between intervention and control clinics
During the study period in Februar y 2009, clinical summaries
with CD4 care suggestions were printed and made available to
clinicians only in the intervention clinic. In 39% of the patient
visits (140 out of 361), the summaries with reminders were
inadvertently not printed, and thus were not available to
intervention clinicians. This happened mostly because the
computer or printer in the clinic was not working. As such, only
Table 1 Criteria for generating CD4 care suggestions
Indication for care suggestion Care suggestion generate d
No previous CD4 count result or order Please order CD4 count now (no CD4 in system)
Only one CD4 result or order, done more than 6 months ago Please order CD4 count now (last CD4 over 6 months ago)
Only two prior CD4 results, with at least one less than 400, and no new CD4 order or
result for more than 6 months
Please order CD4 count now (one of first two CD4s was less than 400; repeat should
be in 6 months)
More than two CD4s, where last CD4 was less than 400, and no new CD4 order or
result in over 6 months
Please order CD4 count now (last CD4 was less than 400; repeat should be in
6 months)
More than two CD4s, where last CD4 was more than 400, and no new CD4 order or
result in over 12 months
Please order CD4 count now (last CD4 ordered over 12 months ago)
152 J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520
Research and applications
Page 3
221 (61%) of the clinical summaries with reminders in the
intervention clinic were printed and made available clinicians. In
the control clinic, none of the summaries with reminders were
printed.
Analysis by study assignment (regardless of summaries being
printed or not, and similar to intention-to-treat analysis in
randomized trials) revealed that clinicians ordered CD4 tests
during 53% of all visits to the intervention clinic when CD4
tests were indicated versus 38% in the control clinic (p<0.0001,
OR¼1.80, 95% CI 1.34 to 2.42). Adjusting for the number of
years a clinician had worked in AMPATH, number of years
a patient had been in AMPATH system, time since last CD4 test,
last CD4 count, WHO stage, and clinician gender (variables
correlated with intervention assignment for each clin-
icianepatient encounter) yielded an OR of 2.07 (p¼0.04) with
95% CI of 1.05 to 4.07. Taking into consideration only
encounters for which reminders were printed, order rates
between the intervention and control groups were 63% versus
38% (adjusted OR 2.90, 95% CI 1.66 to 5.05, p¼0.0002).
Conversely, for the 140 intervention-clinic visits where the
summaries with reminders were not printed, CD4 order rates
were no different to order rates in the control clinic (36% vs
38 %, OR¼0.91, 95% CI 0.60 to 1.35, p¼0.68).
When clinical summaries with reminders were available to
clinicians, the compliance rates varied depending on the
reminder type (table 3). Availability of three of the reminders led
to statistically signicant increases in CD4 order rates. The other
two reminders, though not achieving statistical signicance,
showed a trend of increasing ordering rates. Clinicians who
received the reminders varied in the frequency with which they
accepted these reminders. The distribution of ordering rates
among the seven clinicians who saw the reminders for at least
10 patients (range 12 to 45) ranged from 52% to 86%. A strong
negative correlation (<0.9) between the random intercept and
slope was observed among the seven cliniciansdthis suggests
that over the course of the intervention, there was a convergence
of compliance rates between the clinicians, with steeper
increasing acceptance by clinicians who initially had not
complied as well compared to those who did well at the
beginning of the intervention.
Change in CD4 ordering rates from baseline: preepost analysis
To eliminate potential confounding due to unmeasured charac-
teristics that differ between the two clinics, we also conducted
a pre epost comparison within each clinic. In a preepost anal-
ysis within the intervention clinic, comparing baseline data
(November 2008) with intervention data (February 2009), we
see a statistically signicant increase in the order rates for
overdue CD4 studies whether we use all study cases (42% vs
53%, p¼0.005, OR¼1.50, 95% CI 1.13 to 1.98) or exclude cases
where the clinical summaries were not printed (42% vs 63%,
p<0.0001, OR¼2.32, 95% CI 1.67 to 3.22).
Order rates for overdue CD4 counts in the control clinic
between baseline data (November 2008) and our study period
(February 2009) were not different (36% vs 38%, p¼0.51,
OR¼1.10, 95% CI 0.83 to 1.46).
DISCUSSION
The results of this study provide compelling evidence that
clinical summaries containing computer-generated care sugges-
tions can improve clinician adherence with HIV care guidelines
in the resource-limited setting of sub-Saharan Africa. The effect
Table 2 Characteristics of patients with reminders during the study period (most recent values are used
for patients with multiple visits)
Characteristics Total (n[690)
Intervention clinic
(n[349)
Control clinic
(n[341) p Value
Age (mean, SD) 38.1 (9.8) 37.8 (9.9) 38.5 (9.8) 0.14
Female gender (%) 64.6 66.5 62.8 0.34
On antiretroviral medication (%) 64.4 66.2 62.5 0.34
WHO stage (%)
1 34.6 33.2 36.1 <0.001
2 17.8 21.8 13.8
3 33.0 26.9 39.3
4 14.5 18.1 10.9
Last CD4 count (median, IQR) 338 (213e491) N¼608 318 (199e476) N ¼ 308 356 (221e512) N¼300 0.04
No of prior clinic visit, (median, IQR) 22 (15e33) N¼471 21 (15e33) N¼243 23 (15e33) N¼228 0.72
Duration since clinic enrollment in
years (median, IQR)
1.81 (0.84e2.95) 1.66 (0.81e2.79) 2.01 (0.90e2.97) 0.05
Seen by same clinician on last visit (%) 16.8 17.8 15.8 0.54
Table 3 Ordering rates of CD4 tests by type of reminder
Reminder type
Reminder
viewed?
No of
instances
No (percentage
compliance)
OR of CD4
ordering p Value
No previous CD4 count result or order (n¼44) Yes 14 5 (35.7) 2.78 0.25
No 30 5 (16.7)
Only one CD4 result or order, done more than 6 months ago (n¼102) Yes 68 39 (57.4) 3.23 0.01
No 34 10 (29.4)
Only two prior CD4 results, with at least one less than 400, and no new
CD4 order or result for more than 6 months (n¼ 57)
Yes 34 22 (64.7) 3.44 0.03
No 23 8 (34.8)
More than two CD4s, where last CD4 was less than 400, and no new CD4
order or result in over 6 months (n¼114)
Yes 77 54 (70.1) 1.43 0.40
No 37 23 (62.2)
More than two CD4s, where last CD4 was more than 400, and no new CD4
order or result in over 12 months (n¼45)
Yes 29 20 (68.9) 4.67 0.01
No 16 4 (25.0)
J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520 153
Research and applications
Page 4
of the reminders was even greater for clinicians whose compli-
ance rates had been worse before the intervention. In fact, as the
study progressed, compliance rates improved. Given that scant
data exist on effective interventions to translate evidence-based
medicine into practice in developing countries,
23
these ndings
offer a new powerful tool for improving HIV care in these
resource-limited settings. The study also highlights the impor-
tance of EHRs in these settings.
As stated by Dexter et al,
24
the easy sustainability of
computer-based reminder systems contrasts with the weak-
nesses of such approaches as manual reviewing of charts
25
and
physician-directed continuing medical education.
26
Messages in
computer-generated reminders can be tailored for all levels of
providersdan approach that is particularly relevant in settings
where less-trained personnel provide a large amount of care. In
addition, CDSS allows evolving care protocols to be seamlessly,
efciently, and broadly introduced into clinical practice.
As in prior reminder studies from the developed world,
12 24 27 28
we observed a variation in adherence to computer reminders
among clinicians. Even though we did not formally evaluate
reasons for non-adherence, informal questioning of reminder
recipients indicated that several factors were at play. Some
clinicians had rote practice patterns and simply disagreed with
the algorithms used in the reminder: with education about the
reminders, acceptance rates improved. Occasionally, clinicians
were right to ignore the reminders because the recommended
action was inappropriate for the particular patient during that
visit, often because the clinician had other information not
available in the computer. In fact, because computers are limited
by the data they contain, computer reminders should be
considered as care suggestions to the clinicians. The nal deci-
sion must still rest with the clinician, as clinical judgment
should always take precedence over the computers judgment.
Several limitations in our study deserve mention. The gener-
alizability of our ndings is limited by the fact that only a few
reminders were implemented and at a single clinical site. We
observe a variation in compliance by type of CD4 reminder,
which demonstrates that not all care suggestions will be treated
equally by clinicians. Our evaluation only lasted for a short
period of time, and we cannot account for possible retardation in
efciency and effectiveness of utilization over time. Another
limitation of our study is that the intervention would have an
uncertain role in settings with no EHR. However, many care
rules are based on limited data (eg, gender, age, duration of care)
and do not require fully implemented EHRsdfor example,
reminders about childhood immunizations are based solely on
the childs age and history of prior immunizations, and the latter
can easily be maintained with simple owsheets in patients
charts. Even where EHRs have been implemented in developing
countries, the generalizability of our intervention may be limited
by the additional level of technology required to implement
CDSS. It should however be noted that our intervention only
used a single computer at the intervention clinic. Lastly, the
comparative design (instead of a randomized study design) may
have introduced some bias. We controlled for signicant cova-
riates in the analyses.
This study provides a model through which HIV care guide-
lines can be broadly implemented in resource-limited settings.
The approach can be used to provide reminders about druge
drug interactions and known allergies,
29
and reminders on
overuse or underuse of diagnostic tests or medications. The
guidelines can also extend beyond HIV to encompass a broad set
of diseases, especially for conditions such as diabetes and
hypertension where particular care protocols are well accepted
as best practice. To better delineate the specic effects of
reminders, we are conducting evaluations in which patient
summaries are being presented to both intervention and control
groups, but clinical reminders presented only to the intervention
group. In the future, we hope to demonstrate the impact of
CDSS on patient outcomes and quality of care in these resource-
limited settings, and determine sustainability of the observed
impact of reminders over time.
CONCLUSION
Clinical summaries with computer-generated reminders signi-
cantly improved clinician adherence to CD4 testing guidelines in
this rst study of its kind in sub-Saharan Africa. This technology
can have broad applicability to improve quality of HIV care in
these settings.
Acknowledgments We would like to thank our patients and providers at the study
clinics. Special thanks to S Masit, P Tanui, J Lelei, B McKown, B Wolfe, J Kariuki,
J Lagat, R Vreeman, and A Yeung.
Funding This work was supported by a grant from the Abbott Fund, and in part by
a grant to the USAIDeAMPATH Partnership from the United States Agency for
International Development as part of the President’s Emergency Plan for AIDS Relief
(PEPFAR).
Competing interests None.
Ethics approval Ethics approval was provided by the Institutional Review Boards at
Indiana University School of Medicine in Indianapolis, Indiana and the Institutional
Review and Ethics Committee at Moi University School of Medicine in Eldoret, Kenya.
Provenance and peer review Not commissioned; externally peer reviewed.
REFERENCES
1. Chen L, Evans T, Anand S, et al. Human resources for health: overcoming the crisis.
Lancet 2004;364:1984e90.
2. Joint United Nation s Programme on AIDS. Report on the Global AIDS Epidemic.
Geneva: UNAIDS, 2008.
3. Kober K, Van Damme W. Scaling up access to antiretroviral treatment in southern
Africa: who will do the job? Lancet 2004;364:103e7.
4. Kumar P. Providing the providersdremedying Africa’s shortage of health care
workers. N Engl J Med 2007;356:2564e7.
5. Narasimhan V, Brown H, Pablos-Mendez A, et al. Responding to the global human
resources crisis. Lancet 2004;363:1469e72.
6. Dovlo D. Using mid-level cadres as substitutes for internationally mobile health
professionals in Africa. A desk review. Hum Resour Health 2004;2:7.
7. Samb B, Celletti F, Holloway J, et al. Rapid expansion of the health workforce in
response to the HIV epidemic. N Engl J Med 2007;357:2510e14.
8. McDonald CJ. Protocol-based computer reminders, the quality of care and the
non-perfectability of man. N Engl J Med 1976;295:1351e5.
9. Nicholas DD, Heiby JR, Hatzell TA. The Quality Assurance Project: introducing
quality improvement to primary health care in less developed countries. Qual Assur
Health Care 1991;3:147e65.
10. Wikipedia: Clinical Decision Support System. http://en.wikipedia.org/wiki/Clinical_
Decision_Support (accessed 26 Oct 2009).
11. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support
tools in the ambulatory/primary care setting: a systematic review of the literature.
Inform Prim Care 2008;
16:79e91.
12. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled
trials to evaluate computer-based clinical reminder systems for preventive care in the
ambulatory setting. J Am Med Inform Assoc 1996;3:399e409.
13. Safran C, Rind DM, Davis RB, et al. Guidelines for management of HIV infection with
computer-based patient’s record. Lancet 1995;346:341 e 6.
14. Williams F, Boren SA. The role of the electronic medical record (EMR) in care
delivery development in developing countries: a systematic review. Inform Prim Care
2008;16:139e45.
15. Einterz RM, Kimaiyo S, Mengech HN, et al. Responding to the HIV pandemic: the
power of an academic medical partnership. Acad Med 2007;82:812e18.
16. Tierney WM, Rotich JK, Hannan TJ, et al. The AMPATH medical record system:
creating, implementing, and sustaining an electronic medical record system to
support HIV/AIDS care in western Kenya. Stud Health Technol Inform
2007;129:372e6.
17. Mamlin BW, Biondich PG, Wolfe BA, et al. Cooking up an open source EMR for
developing countries: OpenMRSd a recipe for successful collaboration. AMIA Annu
Symp Proc 2006:529e33.
18. Seebregts CJ, Mamlin BW, Biondich PG, et al. The OpenMRS Implementers
Network. Int J Med Inform 2009;78:711e20.
154 J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520
Research and applications
Page 5
19. Were MC, Mamlin BW, Tierney WM, et al. Concept dictionary creation and
maintenance under resource constraints: lessons from the AMPATH Medical Record
System. AMIA Annu Symp Proc 2007:791e5.
20. Noormohammad SF, Mamlin BW, Biondich PG, et al. Changing course to make
clinical decision support work in an HIV clinic in Kenya. Int J Med Inform
2010;79:204e10.
21. World Health Organization. Antiretroviral Therapy for HIV Infection in Adults and
Adolescents: Recommendations for a Public Health Approach. Geneva: World Health
Organization 2006.
22. National AIDS and STI control Programme (NASCOP) MoH. Guidelines for
Antiretroviral Drug Therapy in Kenya. Kenya, 2005.
23. Siddiqi K, Newell J, Robinson M. Getting evidence into practice: what works in
developing countries? Int J Qual Health Care 2005;17:447e54.
24. Dexter PR, Perkins S, Overhage JM, et al. A computerized reminder system to increase
the use of preventive care for hospitalized patients. N Engl J Med 2001;345:965e70.
25. Morrow RW, Gooding AD, Clark C. Improving physicians’ preventive health
care behavior through peer review and financial incentives. Arch Fam Med
1995;4:165e9.
26. Davis D, O’Brien MA, Freemantle N, et al. Impact of formal continuing medical
education: do confe rences, workshops, rounds, and other traditional continuing
education activities change physicia n behavior or health care outcomes? JAMA
1999;282:867e74.
27. Overhage JM, Tierney WM, Zhou XH, et al. A randomized trial of ‘corollary orders’
to prevent errors of omission. J Am Med Inform Assoc 1997;4:364e75.
28. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an
introspective computer medical record. A two-year randomized trial. Ann Intern Med
1984;100:130
e8.
29. Cresswell KM, Sheikh A. Information technology-based approaches to reducing
repeat drug exposure in patients with known drug allergies. J Allergy Clin Immunol
2008;121:1112e17.
J Am Med Inform Assoc 2011;18:150e155. doi:10.1136/jamia.2010.005520 155
Research and applications
Page 6
  • Source
    • "Success of AMRS helped to convince the Kenyan MOH to roll out OpenMRS in over 300 MOH clinics. Beyond eHealth, AMPATH has used mobile technology for: (a) data collection as part of a homebased counseling and testing for HIV program that has reached over a million individuals; (b) computerized clinical decision support for HIV and chronic disease management; (c) mobile store-and-forward teleconsulttion; and (d) mobile learning and counseling platforms, among others[14] [15] [16] [17]. The AMPATH partnership with Moi University offers a real-world laboratory in HI, where trainees can work to complement foundational aspects of HI training. "
    [Show abstract] [Hide abstract] ABSTRACT: Current approaches for capacity building in Health Informatics (HI) in developing countries mostly focus on training, and often rely on support from foreign entities. In this paper, we describe a comprehensive and multidimensional capacity-building framework by Lansang & Dennis, and its application for HI capacity building as implemented in a higher-education institution in Kenya. This framework incorporates training, learning-by-doing, partnerships, and centers of excellence. At Moi University (Kenya), the training dimensions include an accredited Masters in HI Program, PhD in HI, and HI short courses. Learning-by-doing occurs through work within MOH facilities at the AMPATH care and treatment program serving 3 million people. Moi University has formed strategic HI partnerships with Regenstrief Institute, Inc. (USA), University of Bergen (Norway), and Makerere University (Uganda), among others. The University has also created an Institute of Biomedical Informatics to serve as an HI Center of Excellence in the region. This Institute has divisions in Training, Research, Service and Administration. The HI capacity-building approach by Moi provides a model for adoption by other institutions in resource-limited settings.
    Full-text · Article · Aug 2015 · Studies in health technology and informatics
  • Source
    • "And out of 15,924 patients who were eligible for two tests, only 1,006 (6.3%) received testing as per guideline [39] . The reasons for poor followup CD4 cell count testing could be multifactorial: providers not requesting tests, patients not coming for testing, lack of awareness from the patients' side, or a breakdown of machines or lack of reagents at facilities [40, 41]. Two factors were important in predicting immunological treatment failure. "
    [Show abstract] [Hide abstract] ABSTRACT: Immunological monitoring is part of the standard of care for patients on antiretroviral treatment. Yet, little is known about the routine implementation of immunological laboratory monitoring and utilization in clinical care in Ethiopia. This study assessed the pattern of immunological monitoring, immunological response, level of immunological treatment failure and factors related to it among patients on antiretroviral therapy in selected hospitals in southern Ethiopia. A retrospective longitudinal analytic study was conducted using documents of patients started on antiretroviral therapy. Adequacy of timely immunological monitoring was assessed every six months the first year and every one year thereafter. Immunological response was assessed every six months at cohort level. Immunological failure was based on the criteria: fall of follow-up CD4 cell count to baseline (or below), or CD4 levels persisting below 100 cells/mm3, or 50% fall from on-treatment peak value. A total of 1,321 documents of patients reviewed revealed timely immunological monitoring were inadequate. There was adequate immunological response, with pediatric patients, females, those with less advanced illness (baseline WHO Stage I or II) and those with higher baseline CD4 cell count found to have better immunological recovery. Thirty-nine patients (3%) were not evaluated for immunological failure because they had frequent treatment interruption. Despite overall adequate immunological response at group level, the prevalence of those who ever experienced immunological failure was 17.6% (n=226), while after subsequent re-evaluation it dropped to 11.5% (n=147). Having WHO Stage III/IV of the disease or a higher CD4 cell count at baseline was identified as a risk for immunological failure. Few patients with confirmed failure were switched to second line therapy. These findings highlight the magnitude of the problem of immunological failure and the gap in management. Prioritizing care for high risk patients may help in effective utilization of meager resources.
    Full-text · Article · May 2015 · PLoS ONE
  • Source
    • "The record system first developed and deployed at AMPATH became known as OpenMRS, now the world's most popular open source electronic medical record system with implementations in over 40 countries across every continent [23]–[27]. The availability of digital patient data through OpenMRS has made several other eHealth interventions possible at AMPATH, including an HIV clinical decision support system proven to improve adult and pediatric HIV care [28], [29]. "
    [Show abstract] [Hide abstract] ABSTRACT: With the aim of integrating HIV and tuberculosis care in rural Kenya, a team of researchers, clinicians, and technologists used the human-centered design approach to facilitate design, development, and deployment processes of new patient-specific TB clinical decision support system for medical providers. In Kenya, approximately 1.6 million people are living with HIV and have a 20-times higher risk of dying of tuberculosis. Although tuberculosis prevention and treatment medication is widely available, proven to save lives, and prioritized by the World Health Organization, ensuring that it reaches the most vulnerable communities remains challenging. Human-centered design, used in the fields of industrial design and information technology for decades, is an approach to improving the effectiveness and impact of innovations that has been scarcely used in the health field. Using this approach, our team followed a 3-step process, involving mixed methods assessment to (1) understand the situation through the collection and analysis of site observation sessions and key informant interviews; (2) develop a new clinical decision support system through iterative prototyping, end-user engagement, and usability testing; and, (3) implement and evaluate the system across 24 clinics in rural West Kenya. Through the application of this approach, we found that human-centered design facilitated the process of digital innovation in a complex and resource-constrained context.
    Full-text · Article · Aug 2014 · PLoS ONE
Show more