Access to this full-text is provided by Springer Nature.
Content available from Nature Medicine
This content is subject to copyright. Terms and conditions apply.
Nature Medicine | Volume 30 | January 2024 | 76–84 76
nature medicine
Article
https://doi.org/10.1038/s41591-023-02633-9
A digital health algorithm to guide antibiotic
prescription in pediatric outpatient care:
a cluster randomized controlled trial
Rainer Tan 1,2,3,4 , Godfrey Kavishe5,7, Lameck B. Luwanda2,7,
Alexandra V. Kulinkina 3,4,7, Sabine Renggli2, Chacha Mangu5, Geofrey Ashery2,
Margaret Jorram2, Ibrahim Evans Mtebene2, Peter Agrea5,
Humphrey Mhagama5, Alan Vonlanthen1, Vincent Faivre1, Julien Thabard1,
Gillian Levine3,4, Marie-Annick Le Pogam1, Kristina Keitel3,4,6, Patrick Taffé1,
Nyanda Ntinginya5,8, Honorati Masanja2,8 & Valérie D’Acremont 1,3,4,8
Excessive antibiotic use and antimicrobial resistance are major global
public health threats. We developed ePOCT+, a digital clinical decision
support algorithm in combination with C-reactive protein test, hemoglobin
test, pulse oximeter and mentorship, to guide health-care providers in
managing acutely sick children under 15 years old. To evaluate the impact
of ePOCT+ compared to usual care, we conducted a cluster randomized
controlled trial in Tanzanian primary care facilities. Over 11 months, 23,593
consultations were included from 20 ePOCT+ health facilities and 20,713
from 20 usual care facilities. The use of ePOCT+ in intervention facilities
resulted in a reduction in the coprimary outcome of antibiotic prescription
compared to usual care (23.2% versus 70.1%, adjusted dierence −46.4%,
95% condence interval (CI) −57.6 to −35.2). The coprimary outcome of day
7 clinical failure was noninferior in ePOCT+ facilities compared to usual
care facilities (adjusted relative risk 0.97, 95% CI 0.85 to 1.10). There was
no dierence in the secondary safety outcomes of death and nonreferred
secondary hospitalizations by day 7. Using ePOCT+ could help address the
urgent problem of antimicrobial resistance by safely reducing antibiotic
prescribing. Clinicaltrials.gov Identier: NCT05144763
Bacterial antimicrobial resistance (AMR) was responsible for
1.27 million deaths in 2019, with the highest burden in sub-Saharan
Africa1. This is as many deaths as malaria and human immunodeficiency
virus (HIV) combined. Inappropriate and excessive prescription
of antibiotics represents one of the primary contributors to AMR
2–4
.
In Tanzania and many resource-constrained countries, more than 50%
of sick children receive antibiotics when visiting a health facility5–8,
with 80–90% of such antibiotics prescribed at the outpatient level6,9,10
and most deemed inappropriate
5,9–11
. Antibiotic use and AMR are pro-
jected to increase over the coming years, indicating the urgency to take
Received: 19 June 2023
Accepted: 6 October 2023
Published online: 18 December 2023
Check for updates
1Centre for Primary Care and Public Health (Unisanté), University of Lausanne, Lausanne, Switzerland. 2Ifakara Health Institute, Dar es Salaam, United
Republic of Tanzania. 3Swiss Tropical and Public Health Institute, Allschwil, Switzerland. 4University of Basel, Basel, Switzerland. 5National Institute of
Medical Research – Mbeya Medical Research Centre, Mbeya, United Republic of Tanzania. 6Pediatric Emergency Department, Department of Pediatrics,
University Hospital Bern, Bern, Switzerland. 7These authors contributed equally: Godfrey Kavishe, Lameck B. Luwanda, Alexandra V. Kulinkina.
8These authors jointly supervised this work: Nyanda Ntinginya, Honorati Masanja, Valérie D’Acremont. e-mail: rainer.tan@unisante.ch
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 77
Article https://doi.org/10.1038/s41591-023-02633-9
20 clusters in the
intervention arm (ePOCT+)
20 clusters in the control
arm (routine care)
Child-level information
32,965 children screened 26,910 children screened
23,593 consultations included
21,680 initial consultations
1,913 reattendance consultations
17,985 consultations managed
using ePOCT+
20,355 consultations with day 7 data
(intention to treat for day 7 outcomes)
18,670 initial consultations
1,685 reattendance consultations
3,238 lost to follow-up
4,523 not using ePOCT+
1,685 reattendance consultations
14,396 initial consultations
in the day 7 outcome per
protocol and complete case
population
18,937 consultations
documented in eCRF
17,292 consultations with day 7 data
(intention to treat for day 7 outcome)
15,661 initial consultations
1,631 reattendance consultations
3,421 lost to follow-up
14,363 initial consultations
in the day 7 outcome per
protocol and complete case
population
Cluster-level information
1,456 eCRF not filled
1,631 reattendance consultations
5,608 consultations without
documented treatment in
ePOCT+ 1,776 consultations without
documented treatment in eCRF
9,372 excluded:
Wrong age (n = 673)
Not an acute illness (n = 719)
Declined or unable to consent (n = 4,532)
Child not registered or data issue (n = 3,448)
6,197 excluded:
Wrong age (n = 461)
Not an acute illness (n = 165)
Declined or unable to consent (n = 3,932)
Child not registered or data issue (n = 1,639)
40 health facilities randomized
259 health facilities
assessed for eligibility in
eligible councils
68 eligible health facilities
191 ineligible
122 wrong type of health facility
69 did not see enough patients per week
1,604 reattendance
consultations
1,732 reattendance
consultations
16,381 initial consultations managed
using ePOCT+ (per protocol for
day 0 outcomes)
17,205 initial consultations
documented in eCRF (per
protocol for day 0 outcomes)
20,713 consultations included
18,789 initial consultations
1,924 reattendance consultations
Fig. 1 | Health facility and patient flow diagram. Boxes highlighted in gray correspond to the coprimary outcome populations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 78
Article https://doi.org/10.1038/s41591-023-02633-9
action12–14. Accordingly, the World Health Organization has declared
AMR as “one of the biggest threats to global health, food security and
development today”
15
. In response, countries worldwide, including
Tanzania, have developed national action plans on antimicrobial resist-
ance to address this important problem16,17.
Electronic clinical decision support algorithms (CDSAs) are
digital health or mobile health tools that guide health-care providers
on what symptoms and signs to assess, advise on what tests to
perform, and propose the appropriate diagnoses, treatments and
management
18,19
. Previous efficacy studies under controlled research
conditions have shown the potential for digital CDSAs to reduce anti-
biotic prescription in children 2 to 59 months old
20,21
. However, many
close-to-real-world studies have shown little to no reduction in anti-
biotic prescription22–24. In addition, many of the close-to-real-world
studies have a number of methodological limitations as health facil-
ities were not randomized and/or safety was not evaluated18,24,25,
emphasizing the need for more evidence on the impact of CDSAs on
antibiotic prescription. Finally, poor uptake remains a challenge with
previous and existing CDSAs26,27.
We developed ePOCT+, a new CDSA with point-of-care tests, to
address these challenges28. The scope of ePOCT+ was expanded from
previous versions of the CDSA20,29 to include infants under 2 months
and children up to age 14 years, and to address syndromes and diag-
noses not considered by other CDSAs
30
. The aim of this study was to
evaluate the impact of ePOCT+ compared to usual care on antibiotic
prescription and day 7 clinical outcome in a pragmatic, cluster rand-
omized controlled trial in acutely sick children under 15 years of age
presenting to Tanzanian primary care facilities.
Result
Baseline characteristics of health facilities and patients
A total of 68 out of 259 health facilities from the participating councils
met the eligibility criteria (Fig.1). One hundred twenty-two health facili-
ties were ineligible as they were either hospitals or private dispensary or
health centers, and 69 did not see enough patients per week. A stratified
random sampling process identified 40 health facilities for inclusion
in the study (24 in the Morogoro region and 16 in the Mbeya region),
which were randomized 1:1 to ePOCT+ (intervention) or usual care
(control). Overall, 59,875 children were screened for inclusion between
1 December 2021 and 31 October 2022, and 44,306 (74%) consultations
were enrolled (23,593 in ePOCT+ health facilities and 20,713 in usual care
health facilities). The first health facilities started enrolling patients
on 1 December 2021, and the last health facilities started enrolling
patients on 13 April 2022. A total of 28,243 unique patients were enrolled
with a mean of 1.6 consultations per patient over the duration of the
study. Among those enrolled in the intervention health facilities, 17,985
Table 1 | Baseline characteristics of enrolled participants
and health facilities
Health facilities ePOCT+(n=20) Usual care (n=20)
Level of health facility, n
Dispensaries 16 16
Health centers 4 4
Region, n
Morogoro 12 12
Mbeya 8 8
Number of enrolled patients per
health facility per month, median
(IQR)
127 (101; 199) 136 (73; 163)
Service availability and readiness
assessmenta
General Service Readiness score,
% (mean±s.d.) 60.3±10.8 63.7±9.4
Pediatric score, % (mean±s.d.) 55.9±10.8 64.9±10.6
Participants ePOCT+(n=23,593) Usual care (n=20,713)
Sex: Female, % (n)51.2 (12,085) 51.3 (10,075)
Age, days, median (IQR) 583 (263; 1,202) 555 (246; 1,189)
Age group, % (n)
0 to <2months 4.0 (954) 5.0 (1,038)
2 to <60months 84.1 (19,845) 82.0 (16,984)
5 to <15years 11.8 (2,794) 13.0 (2,691)
Type of consultation, % (n)
New consultation 91.9 (21,680) 90.7 (18,789)
Reattendance 7.8 (1,841) 9.2 (1,899)
Referral from another
health facility 0.3 (72) 0.1 (25)
Positive malaria test among
those tested, % (n/N)18.4 (1,878/10,225) 19.2 (1,803/9,378)
Hospitalized in the last
14 days, % (n)0.3 (65) 0.4 (73)
Phone number available, % (n)84.0 (19,808) 83.0 (17,186)
Participant data from all enrolled patients. Values of standard deviations (s.d., after
mean values) are preceded by the ± sign. IQR, interquartile ranges (after median values).
aScores were calculated based on the proportion of prespeciied indicators that were
present in each health facility during the assessment of health facilities before the start of
the study31.
Table 2 | Presenting complaints of infants and children
under 15years old
Presenting complaints, % ePOCT+ Usual care
Infants <2months n=717 n=929
Fever, convulsions, lethargy 25.7 13.1
Respiratory 43.7 46.8
Gastrointestinal 22.2 19.7
Skin 14.0 14.5
Ear/mouth 2.5 0.9
Eye 7.0 5.4
Feeding/weight 0.3 1
Malformation 0.4 0.4
Injuries 0.4 0.1
Other 6.4 7.8
Infants and children ≥2months to <15years n=17,268 n=17,089
Fever 61.6 56.9
Respiratory (cough/dificulty breathing) 47.8 49.4
Gastrointestinal (diarrhea/vomiting) 23.4 22.3
Skin 12.5 11.9
Ear/throat/mouth 2.6 2.3
Eye 2.1 2.1
Genitourinary 1.4 3.1
Neurological (headache, stiff neck) 31.2
Accident/musculoskeletal (including burns,
wounds, poison) 1.5 2.0
Other 2.1 4.2
Data from patients for whom clinical information was entered into ePOCT+ in the intervention
arm, and in the eCRF in control health facilities (per protocol population). Patients may have
multiple complaints.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 79
Article https://doi.org/10.1038/s41591-023-02633-9
(76.2%) consultations were managed using ePOCT+, and day 7 outcome
was ascertained in 20,355 consultations (86.3%). In usual care health
facilities, 18,937 (91.4%) consultations had final treatment documented
in the electronic case report form (eCRF), and 17,292 consultations
(83.5%) had day 7 outcome ascertained. Information technology (IT)
problems and power outages were reported by research assistants on
respectively 293 (7.3%) and 245 (6.1%) health facility days in ePOCT+
facilities, and 160 (4.1%) and 245 (6.1%) health facility days in usual care
facilities. Both issues contributed to children being prevented from
enrollment in the study.
Intervention health facilities saw similar numbers of consul-
tations, but had a slightly lower Service Availability and Readiness
Assessment Pediatric score (Table1)31. Patients in both study arms were
similar in age, sex, type of consultation and previous hospitalization
(Table1). Malaria prevalence among those tested was similar in both
study arms (Table1). Young infants less than 2 months of age in the
intervention health facilities presented more frequently for fever,
convulsions or lethargy, and slightly less often for respiratory condi-
tions, whereas patients 2 months and above had a similar distribution
in presenting complaints (Table2). Age, phone availability and level of
health facility differed among patients with and without day 7 outcome
ascertained (Supplementary Tables1 and 2). Patients managed and
not managed per protocol were similar, except for the level of health
facility (Supplementary Table3).
Primary outcomes: antibiotic prescription and clinical failure
Overall antibiotic prescription at initial consultations for the per pro-
tocol analysis was 23.2% (3,806 of 16,381) in ePOCT+ health facilities
and 70.1% (12,058 of 17,205) in routine care health facilities, which cor-
responds to an adjusted absolute difference of −46.4% (95% CI −57.6
to −35.2) (Table3 and Fig.2). The adjusted analysis found a 65% reduc-
tion in the risk of prescribing an antibiotic at day 0 (adjusted relative
risk (aRR) 0.35, 95% CI 0.29 to 0.43, P < 0.001). Using a conservative
imputation analysis approach in the intention-to-treat population by
considering that all patients who were not managed per protocol were
prescribed an antibiotic, antibiotic prescription remained lower in
ePOCT+ health facilities than in usual care, with an adjusted absolute
difference of −34.2% (95% CI −42.1% to −26.4%) (Extended Data Table1).
When including reattendance cases, antibiotic prescription reduction
was similar, with an adjusted absolute difference of −45.0% (95% CI
−56.3% to −33.6%) (Supplementary Table4).
The proportion of patients with clinical failure by day 7 was nonin-
ferior in ePOCT+ health facilities (3.7%, 532 of 14,396) compared to usual
care health facilities (3.8%, 543 of 14,363), with an adjusted relative risk
of 0.97 (95% CI 0.85 to 1.10) in the per protocol complete case popula-
tion (Table3 and Fig.3). Clinical failure by day 7 was also noninferior
in the intention-to-treat complete case population (Extended Data
Table1), when including reattendance cases (Supplementary Table4)
and using unadjusted analyses (Table3 and Fig.2).
Secondary and exploratory clinical safety outcomes
There were no significant differences in the proportion of patients
who died, were subjectively worse or, were hospitalized after the day
of the initial consultation without a referral (nonreferred secondary
hospitalization), all hospitalizations by day 7 or unplanned reattend-
ance visits (Table3). There was however a significant reduction in
Table 3 | Antibiotic prescription and clinical outcomes among sick children in the DYNAMIC trial
ePOCT+,
% (n/N)Usual care,
% (n/N)Intracluster
correlation
coeicient (95% CI)
Crude
dierence
(95% CI)
Adjusted
dierence
(95% CI)
Crude
relative risk
(95% CI)
P value Adjusted
relative risk
(95% CI)
P value
Primary outcome
Antibiotic prescription at day 0 23.2%
(3,806/16,381) 70.1 %
(12,058/17,205) 0.3
(0.2; 0.4) −46.9%
(−47.8%; −45.9%) −46.4%
(−57.6%; −35.2%) 0.33
(0.32; 0.34) <0.001 0.35
(0.29; 0.43) <0.001
Clinical failure by day 7 3.7%
(532/14,396) 3.8%
(543/14,363) 0.004
(0.001; 0.006) −0.1%
(−0.5%; 0.4%) −0.1%
(−0.6%; 0.3%) 0.98
(0.87; 1.10) 0.70 0.97
(0.85; 1.10) 0.59
Secondary and exploratory
outcomes
Death by day 7 0.1%
(9/14,396) 0.1%
(11/14,363) <0.001 0.0%
(−0.1%; 0.0%) 0.0 %
(−0.1%; 0.0%) 0.82
(0.34; 1.97) 0.65 0.66
(0.24, 1.84) 0.43
Subjectively worse at day 7a0.3%
(41/14,396) 0.3%
(40/14,363) 0.002
(0.000; 0.004) 0.0%
(−0.1%; 0.1%) 0.0%
(−0.1%; 0.2%) 1 .02
(0.66 ; 1.58) 0.92 1.11
(0.71; 1.73) 0.65
Nonreferred secondary
hospitalizations by day 7 0.4%
(57/14,396) 0.4%
(50/14,363) 0.001
(0.000; 0.002) 0.0%
(−0.1%; 0.2%) 0.0%
(−0.0%; 0.2%) 1.14
(0.78; 1.66) 0.51 1.14
(0.77; 1.69) 0.52
Hospitalizations by day 7a1.0%
(145/14,396) 0.9%
(130/14,363) 0.01
(0.01; 0.02) 0.1 %
(−0.1%; 0.3%) 0.3%
(−0.0%; 0.7%) 1.11
(0.88; 1.41) 0.38 1.43
(1.00; 2.05) 0.05
Primary referrals at day 0 1.2%
(194/16,381) 1.0%
(170/17,205) 0.03
(0.01; 0.04) 0.1%
(−0.2%; 0.3%) 0.8%
(0.1%; 1.5%) 1.2
(0.98; 1.47) 0.08 2.08
(1.15; 3.74) 0.02
Referral resulting in
hospitalization by day 7b16.8%
(25/149) 20.3%
(29/143) 0.05
(0.00; 0.14) −3.5%
(−5.4%; 12.4%) −2.8%
(−11.8%; 6.2%) 0.83
(0.51; 1.34) 0.44 0.8 6
(0.53; 1.40) 0.55
Unplanned reattendance visits
by day 7c1.8%
(256/14,603) 2.9%
(425/14,723) 0.03
(0.01; 0.04) −1.1%
(−1.5%; −0.8%) −1.0%
(−2.8%; 0.9%) 0.61
(0.52; 0.71) <0.001 0.67
(0.32; 1.44) 0.31
Additional medication taken
after initial consultation up
to day 7
7.1%
(1,006/14,244) 7.2%
(1,017/14,229) 0.006 −0.1%
(−0.7%; 0.5%) − 0.9%
(−2.1%; 0.4%) 0.99
(0.91; 1.07) 0.78 0.88
(0.74; 1.05) 0.17
All data shown for day 0 outcomes are per protocol, and all data for day 7 outcomes are per protocol and complete case (day 7 outcomes assessed). Clinical failure by day 7 deined as
‘not cured’ and ‘not improved’, or unscheduled hospitalization as reported by caregivers. Nonreferred secondary hospitalizations by day 7 are hospitalizations at least a day after the initial
consultation that were not referred by a health-care provider. Unplanned reattendance visits by day 7 are return visits between day 1 and 7 that were not proposed by the initial health-care
provider. Adjusted relative risks and differences were estimated using a random effects logistic regression model adjusting for clustering (health facility and patient), as well as individual
(age, sex, complaints, availability of phone) and health facility (council of health facility, level of health facility, mean number of patients seen per month at the health facility) baseline
characteristics. Formal adjustments were not performed for multiple testing. aPost hoc exploratory outcome not prespeciied. bDenominator is based on consultations for which a primary
referral was proposed and day 7 hospitalization data were ascertained, and as such may be less than the total number of primary referrals at day 0. cIncluding unplanned outpatient and
hospitalized reattendance visits.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 80
Article https://doi.org/10.1038/s41591-023-02633-9
unplanned reattendance visits by day 7 in unadjusted analyses. The
proportion of patients who died (0.1%) or were hospitalized (1.0%) was
low in both study arms. Results in the intention-to-treat population
and when including reattendance visits were similar (Extended Data
Table1 and Supplementary Table4).
Additional medications by day 7 and antibiotic prescription
over time
At day 7 (range 6–14), additional medicines were taken after the initial
consultation in a similar proportion of patients in both study arms (7.1%
versus 7.2% in intervention versus control health facilities, Table3).
When evaluating evolution of mean antibiotic prescription rates over
time, it appears to decrease over time in ePOCT+ health facilities,
whereas no change was found in usual care facilities (Supplementary
Fig.1).
Referral and hospitalizations
Health-care providers identified 3.6% (582 of 15,799) of cases as
having a severe diagnosis in ePOCT+ facilities compared to 2.6%
(453 of 17,205) in usual care facilities (per protocol in initial cases).
The proportion of cases referred for hospitalization was higher in
ePOCT+ facilities (1.2%) than in usual care facilities (1.0%) (aRR 2.08,
95% CI 1.15 to 3.74) (Table3). The proportion of children referred that
resulted in hospitalization was low and similar in both study arms
(Table3). The proportion of cases referred to specialized outpatient
clinics (malnutrition clinic, tuberculosis investigation, HIV clinic)
was low and similar between ePOCT+ and usual care health facilities
(Supplementary Table5).
Subgroup analyses: sex, age, complaints
The effect of the intervention on antibiotic prescription at day 0
was more pronounced in children presenting with respiratory com-
plaints (absolute difference −62.1%, 95% CI −63.3% to −60.9%) and the
2–59-month age group (absolute difference −48.9%, 95% CI −49.9% to
−47.9%) (Fig.3 and Extended Data Table2). Antibiotic prescription
was reduced by at least 25 percentage points in all prespecified sub-
groups, with the smallest reduction found in infants under 2 months
old (absolute difference −25.5%, 95% CI −30.3% to −20.6%). Among
post hoc subgroup analyses, patients with a positive malaria test had a
lower reduction in antibiotic prescription between ePOCT+ and usual
care (adjusted absolute difference −18.8%, 95% CI −25.1% to −12.6%)
(Extended Data Table2). Young infants less than 2 months old had the
largest reduction in day 7 clinical failure (aRR 0.61, 95% CI 0.37 to 1.00;
P = 0.05) (Fig.3 and Extended Data Table3).
Discussion
In this cluster randomized controlled trial involving 44,306 sick
children under 15 years of age in Tanzania, the use of the ePOCT+
digital clinical decision support algorithm (CDSA) package resulted in
a close-to three-fold reduction in the likelihood of a sick child receiving
an antibiotic prescription compared to children in usual care facilities.
Despite substantially fewer antibiotic prescriptions, clinical failure
PP adjusted dierence, –46.4% (95% CI –57.6% to –35.2%)
PP
ePOCT+ Usual care
ITT
aRR (95% CI)
0.99 (0.89, 1.10)
0.97 (0.86, 1.09)
0.98 (0.86, 1.10)
0.97 (0.85, 1.10)
Not noninferiorNoninferior
1.31.2
42.0%
72.6%
70.1%
23.2%
100
a
90
80
70
60
50
40
30
Percentage of consultations
20
10
0
ePOCT+ Usual care
PP adjusted relative risk, 0.35 (95% CI 0.29 to 0.43) ITT adjusted relative risk, 0.55 (95% CI 0.47 to 0.64)
Noninferiority plot for clinical failure at day 7
Antibiotic prescription at day 0
ITT adjusted dierence, –34.2% (95% CI –42.1% to –26.4%)
1.11.00.9
ITT (unadjusted)
ITT (adjusted)
PP (unadjusted)
PP (adjusted)
b
Fig. 2 | Coprimary outcomes. a, Proportion of antibiotic prescription in
ePOCT+ and usual care health facilities; data are presented as the point
estimate and unadjusted 95% confidence intervals. Sample sizes are as
follows: PP ePOCT+ clusters n = 16,381, PP usual care clusters n = 17,205, ITT
ePOCT+ clusters n = 21,680, ITT usual care clusters n = 18,789. b, Relative risk
of day 7 clinical failure between ePOCT+ and usual care health facilities, with
noninferiority prespecified as an adjusted relative risk of <1.3. Noninferiority
plot shown on a logarithmic scale. ITT, intention to treat; PP, per protocol; aRR,
adjusted relative risk.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 81
Article https://doi.org/10.1038/s41591-023-02633-9
did not increase in intervention facilities. Such findings align with
Tanzania’s National Action Plan to reduce antibiotic use17 and are
in line with the Tanzania digital health strategy to improve quality
of care32.
The reduction of antibiotic prescription associated with the
ePOCT+ intervention in our study is consistent with our previous
research with CDSAs in Tanzania in more controlled research set-
tings20,21,33. However, the results differ from other studies evaluating
CDSAs implemented in routine health programs in Nigeria, Afghanistan,
Burkina Faso and South-Africa, and in a controlled study setting
in Uganda, which found smaller and even no reduction in antibiotic pre-
scription22–24,34,35. There are a number of differences that may explain the
divergent results. First and foremost, the clinical algorithm of ePOCT+
differs from other CDSAs. It notably has a wider scope including addi-
tional conditions and point-of-care tests such as C-reactive protein
(CRP), not included in the Integrated Management of Childhood Illness
(IMCI)
28
. A randomized controlled trial comparing two different CDSAs
found differences in the impact of antibiotic stewardship due to the
addition of CRP and other algorithm modifications, demonstrating
that not all CDSAs are equal20. Other differences that may explain
the divergent results include (1) differences in the supportive
training and mentorship provided, (2) disease epidemiology
(notably malaria prevalence) and (3) health-care provider skills and
adherence. The extent of the impact on antibiotic stewardship in our
study is also greater than that observed in other antibiotic steward-
ship studies that included one single intervention rather than an
intervention package
36,37
. ePOCT+ integrates multiple proven antibi-
otic stewardship interventions together, including clinical decision
support20,21,33, the use of point-of-care CRP tests38, pulse oximeter39
and continuous quality improvement mentorship support with
data feedback to health-care providers utilizing benchmarking of
health facilities40,41.
Clinical failure was not higher in patients managed in ePOCT+
health facilities, despite a significant reduction in antibiotic prescrip-
tion in line with other antibiotic stewardship studies
38
. Similarly the
proportions of children who died, were hospitalized without referral
or had unplanned reattendance visits were not higher. Whereas pre-
vious CDSA studies were able to demonstrate significant reductions
a
Sex (female)
Sex (male)
Age <2 months
Age 2–11 months
Age 12–59 months
Age 5–14 years
Complaint: fever
Complaint: respiratory
Complaint: gastrointestinal
Complaint: skin
All cases
All cases
Complaint: ear, nose, mouth, throat
Complaint: skin
Complaint: gastrointestinal
Complaint: respiratory
Complaint: fever
Age 5–14 years
Age 2–59 months
Age <2 months
Sex (male)
Sex (female)
–80 –60 –40 –20 0
–46.40 (–57.60, –35.20)
–45.20 (–52.80, –37.70)
–50.90 (–57.40, –44.30)
–55.40 (–62.20, –48.70)
−63.50 (−69.50, −57.40)
–46.90 (–54.30, –39.40)
–45.60 (–53.20, –38.00)
–49.50 (–57.10, –42.00)
–50.20 (–57.90, –42.50)
–31.10 (–40.20, –22.00)
–48.50 (–56.00, –40.90)
–48.50 (–57.60, –35.20)
Adjusted dierence, %
(95% CI)
aRR
(95% CI)
1.05 (0.88, 1.25)
0.91 (0.77, 1.08)
0.61 (0.37, 1.00)
1.00 (0.88, 1.15)
1.00 (0.68, 1.48)
0.95 (0.79, 1.14)
0.96 (0.79, 1.14)
1.12 (0.85, 1.47)
1.05 (0.76, 1.45)
0.71 (0.36, 1.40)
0.97 (0.85, 1.10)
Usual care betterePOCT + better
0.50
Complaint: ear, nose, mouth, throat
1.0 1.5 2.0
b
Fig. 3 | Antibiotic prescription and clinical failure by sex, age group and main
complaints. a, Data are presented as adjusted differences with 95% CI of day 0
antibiotic prescription between ePOCT+ health facilities and usual care health
facilities. All data are from the per protocol population in initial consultations.
Sample sizes for each subgroup are found in Extended Data Table2. b, Data
are presented as adjusted relative risk with 95% CI of clinical failure in ePOCT+
compared to usual care health facilities. All data are from the per protocol and
complete case population among initial consultations. Sample sizes for each
subgroup are found in Extended Data Table3.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 82
Article https://doi.org/10.1038/s41591-023-02633-9
in clinical failure, the current trial was not powered to do so20,21,23.
Nonetheless, the greatest benefit on clinical cure compared to usual
care was observed in the subgroup of infants aged under 2 months,
important results given that this population represents more than 50%
of mortality in children under 5 years old42.
Although the present findings are encouraging, it is important
to note that nearly 25% of patients were not managed using ePOCT+
in the intervention arm. Lower uptake of the tool could reduce the
positive impact of antibiotic stewardship as seen in the lower reduction
in antibiotic prescription in the intention-to-treat (ITT) population.
It is reasonable to assume that not all health providers use the digital
tool to manage all patients, just as health providers do not consult
the IMCI paper chartbook every time they see a patient. Indeed
CDSAs have been found to improve adherence to IMCI guide-
lines23,24,43,44, nonetheless many challenges in adherence to paper
guidelines remain for digital tools, notably low motivation, lack of
on-site mentoring and cognitive overload
45,46
. The use of electronic
medical record (EMR) systems in some health facilities may also
explain poor uptake, as some providers were expected to input clinical
data in ePOCT+, the EMR and a paper log, prolonging the consulta-
tion time. Integration of clinical decision support within the EMR
system instead of separate standalone systems could help and is
currently being explored. In addition to harmonization of digital
health tools, numerous other factors must be considered and are
currently being evaluated in order for ePOCT+ and similar tools to be
adequately scaled up in Tanzania and other countries. They include
a better understanding of why health providers did not use ePOCT+
and how the clinical algorithms of ePOCT+ can be further improved,
how health providers can be better supported to use the digital tools,
the impact of benchmarking and mentoring dashboards, cost and
greenhouse gas emission analyses, and acceptance by patients and
community members.
Our study possesses several strengths that contribute to its
robustness. First, we employed a cluster randomized controlled
study design, which was adequately powered to assess noninferiority
of clinical failure. Second, the implementation of our intervention
encompassed a wide range of epidemiological settings, including
both rural and urban areas, with varying levels of malaria transmis-
sion and facilities such as dispensaries and health centers. Moreover,
our study employed comprehensive patient inclusion criteria that
were designed to be inclusive. By incorporating these inclusive cri-
teria, randomly sampling health facilities for inclusion, and observ-
ing consistent effects across subgroups at both the health facility
and individual levels, our findings can be generalized to a broader
population.
There are several limitations to our study. First, antibiotic pre-
scription data relied on documentation by the health-care provider,
an approach often used in pragmatic trials
47,48
. When using a conserva-
tive imputation analysis approach in the ITT population considering
that all patients for which treatment was not documented were con-
sidered to have been prescribed an antibiotic, ePOCT+ still reduced
antibiotic prescription considerably (Extended Data Table1). Second,
despite multiple phone calls and home visits, 15% of cases were lost
to follow-up, consistent with data from similar studies (13–25%)49 –51.
To account for potential biases in loss to follow-up, we adjusted the
final model for baseline variables associated with missing outcome
data, analogous to performing multiple imputation in the case of a
single endpoint. Third, the fact that a child has not improved after day
7 sometimes reflects the natural course of the disease, rather than the
poor quality of care at the initial consultation, and may not therefore
be expected to be influenced by the intervention for all clinical situa-
tions. To show an effect on more severe outcomes such as secondary
hospitalization, death or even clinical failure at day 14 or 28 would
require a very large sample size owing to the rarity of the event at the
primary care level. Further complicating assessment of these severe
outcomes are the challenges linked to referral and quality of care at
admitting hospitals.
In conclusion, the ePOCT+ electronic clinical decision support
algorithm (CDSA) in association with point-of-care tests (CRP, hemo-
globin, pulse oximeter) and mentorship support informed by clinical
practice data, safely and substantially reduced antibiotic prescription
in sick children less than 15 years of age presenting to primary care
facilities in Tanzania. Widespread implementation of ePOCT+ could
help address the urgent problem of antimicrobial resistance by reduc-
ing excessive antibiotic prescription in sick children while maintaining
clinical safety.
Online content
Any methods, additional references, Nature Portfolio reporting sum-
maries, source data, extended data, supplementary information,
acknowledgements, peer review information; details of author contri-
butions and competing interests; and statements of data and code avail-
ability are available at https://doi.org/10.1038/s41591-023-02633-9.
References
1. Murray, C. J. L. et al. Global burden of bacterial antimicrobial
resistance in 2019: a systematic analysis. Lancet 399, 629–655
(2022).
2. Holmes, A. H. et al. Understanding the mechanisms and drivers of
antimicrobial resistance. Lancet 387, 176–187 (2016).
3. Laxminarayan, R. et al. Antibiotic resistance—the need for global
solutions. Lancet Infect. Dis. 13, 1057–1098 (2013).
4. Costelloe, C., Metcalfe, C., Lovering, A., Mant, D. & Hay, A. D.
Eect of antibiotic prescribing in primary care on antimicrobial
resistance in individual patients: systematic review and
meta-analysis. Brit. Med. J. 340, c2096 (2010).
5. van de Maat, J., De Santis, O., Luwanda, L., Tan, R. & Keitel, K.
Primary care case management of febrile children: insights
from the ePOCT Routine Care Cohort in Dar es Salaam, Tanzania.
Front. Pediatr. 9, 626386 (2021).
6. Levine, G., Bielicki, J. & Fink, G. Cumulative antibiotic exposure in
the irst ive years of life: estimates for 45 low- and middle-income
countries from demographic and health survey data. Clin. Infect.
Dis. 75, 1537–1547 (2022).
7. Fink, G., D’Acremont, V., Leslie, H. H. & Cohen, J. Antibiotic
exposure among children younger than 5 years in low-
income and middle-income countries: a cross-sectional
study of nationally representative facility-based and
household-based surveys. Lancet Infect. Dis. 20, 179–187
(2020).
8. Sulis, G. et al. Antibiotic prescription practices in primary care
in low- and middle-income countries: a systematic review and
meta-analysis. PLoS Med. 17, e1003139 (2020).
9. English Surveillance Programme for Antimicrobial Utilisation and
Resistance (ESPAUR) Report 2020 to 2021 (UK Health Security
Agency, 2021).
10. Swedres-Svarm 2021: Sales of Antibiotics and Occurrence of
Antibiotic Resistance in Sweden 2021 (Public Health Agency of
Sweden, Solna/Uppsala, 2021).
11. Ardillon, A. et al. Inappropriate antibiotic prescribing and
its determinants among outpatient children in 3 low- and
middle-income countries: a multicentric community-based
cohort study. PLoS Med. 20, e1004211 (2023).
12. Klein, E. Y. et al. Global increase and geographic convergence in
antibiotic consumption between 2000 and 2015. Proc. Natl Acad.
Sci. USA 115, E3463–E3470 (2018).
13. O’Neill, J. Tackling Drug-Resistant Infections Globally: Final Report
and Recommendations (The Review on Antimicrobial resistance,
2016); https://amr-review.org/sites/default/iles/160525_Final%20
paper_with%20cover.pdf
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 83
Article https://doi.org/10.1038/s41591-023-02633-9
14. Allwell-Brown, G. et al. Trends in reported antibiotic use among
children under 5 years of age with fever, diarrhoea, or cough
with fast or diicult breathing across low-income and middle-
income countries in 2005-17: a systematic analysis of 132 national
surveys from 73 countries. Lancet Glob. Health 8, e799–e807
(2020).
15. Antibiotic resistance. World Health Organization https://www.
who.int/en/news-room/fact-sheets/detail/antibiotic-resistance
(2022).
16. The National Action Plan on Antimicrobial Resistance 2017–2022
(The United Republic of Tanzania: Ministry of Health, Community,
Development, Gender, Elderly and Children, World Health
Organization, 2017).
17. The National Action Plan on Antimicrobial Resistance 2023–2028
(Government of the United Republic of Tanzania, World Health
Organization, 2022).).
18. Keitel, K. & D’Acremont, V. Electronic clinical decision algorithms
for the integrated primary care management of febrile children
in low-resource settings: review of existing tools. Clin. Microbiol.
Infect. 24, 845–855 (2018).
19. Pellé, K. G. et al. Electronic clinical decision support algorithms
incorporating point-of-care diagnostic tests in low-resource
settings: a target product proile. BMJ Glob. Health 5, e002067
(2020).
20. Keitel, K. et al. A novel electronic algorithm using host biomarker
point-of-care tests for the management of febrile illnesses
in Tanzanian children (e-POCT): a randomized, controlled
non-inferiority trial. PLoS Med. 14, e1002411 (2017).
21. Shao, A. F. et al. New Algorithm for Managing Childhood
Illness Using Mobile Technology (ALMANACH): a controlled
non-inferiority study on clinical outcome and antibiotic use in
Tanzania. PLoS ONE 10, e0132316 (2015).
22. Bernasconi, A. et al. Results from one-year use of an electronic
Clinical Decision Support System in a post-conlict context:
an implementation research. PLoS ONE 14, e0225634
(2019).
23. Schmitz, T. et al. Eectiveness of an electronic clinical decision
support system in improving the management of childhood
illness in primary care in rural Nigeria: an observational study.
BMJ Open 12, e055315 (2022).
24. Sarrassat, S. et al. An Integrated eDiagnosis Approach (IeDA)
versus standard IMCI for assessing and managing childhood
illness in Burkina Faso: a stepped-wedge cluster randomised trial.
BMC Health Serv. Res. 21, 354 (2021).
25. Agarwal, S. et al. Decision-support tools via mobile devices to
improve quality of care in primary healthcare settings. Cochrane
Database Syst. Rev. 7, Cd012944 (2021).
26. Shao, A. F. et al. Can smartphones and tablets improve the
management of childhood illness in Tanzania? A qualitative study
from a primary health care worker’s perspective. BMC Health Serv.
Res. 15, 135 (2015).
27. Jensen, C., McKerrow, N. H. & Wills, G. Acceptability and
uptake of an electronic decision-making tool to support the
implementation of IMCI in primary healthcare facilities in
KwaZulu-Natal, South Africa. Paediatr. Int. Child Health 40,
215–226 (2020).
28. Tan, R. et al. ePOCT+ and the medAL-suite: development of an
electronic clinical decision support algorithm and digital platform
for pediatric outpatients in low- and middle-income countries.
PLoS Digital Health 2, e0000170 (2023).
29. Rambaud-Althaus, C., Shao, A. F., Kahama-Maro, J., Genton, B. &
d’Acremont, V. Managing the sick child in the era of declining
malaria transmission: development of ALMANACH, an Electronic
Algorithm for Appropriate Use of Antimicrobials. PLoS ONE 10,
e0127674 (2015).
30. Beynon, F. et al. Digitalizing clinical guidelines: experiences
in the development of clinical decision support algorithms
for management of childhood illness in resource-constrained
settings. Glob. Health Sci. Pract. 11, e2200439 (2023).
31. Service Availability and Readiness Assessment (SARA): An Annual
Monitoring System for Service Delivery: Reference Manual (World
Health Organization, 2013).
32. Tanzania Digital Health Strategy 2019–2024 (The United Republic
of Tanzania: Ministry of Health, Community, Development,
Gender, Elderly and Children, 2019).
33. Rambaud-Althaus, C. et al. Performance of health workers using
an electronic algorithm for the management of childhood illness
in Tanzania: a pilot implementation study. Am. J. Trop. Med. Hyg.
96, 249–257 (2017).
34. Horwood, C. et al. Electronic Integrated Management of
Childhood Illness (eIMCI): a randomized controlled trial to
evaluate an electronic clinical decision-making support system
for management of sick children in primary health care facilities
in South Africa Preprint at https://www.researchsquare.com/
article/rs-2746877/v1. (2023).
35. Kapisi, J. et al. Impact of the introduction of a package of
diagnostic tools, diagnostic algorithm, and training and
communication on outpatient acute fever case management at
3 diverse sites in Uganda: results of a randomized controlled trial.
Clin. Infect. Dis. 77, S156–S170 (2023).
36. Cox, J. A. et al. Antibiotic stewardship in low- and middle-income
countries: the same but dierent? Clin. Microbiol. Infect. 23,
812–818 (2017).
37. Ya, K. Z., Win, P. T. N., Bielicki, J., Lambiris, M. & Fink, G. Association
between antimicrobial stewardship programs and antibiotic use
globally: a systematic review and meta-analysis. JAMA Netw.
Open 6, e2253806 (2023).
38. Smedemark, S. A. et al. Biomarkers as point-of-care tests to
guide prescription of antibiotics in people with acute respiratory
infections in primary care. Cochrane Database Syst. Rev. 10,
Cd010130 (2022).
39. Sylvies, F., Nyirenda, L., Blair, A. & Baltzell, K. The impact of
pulse oximetry and Integrated Management of Childhood
Illness (IMCI) training on antibiotic prescribing practices in
rural Malawi: a mixed-methods study. PLoS ONE 15, e0242440
(2020).
40. O’Riordan, F., Shiely, F., Byrne, S. & Fleming, A. Quality
indicators for hospital antimicrobial stewardship programmes:
a systematic review. J. Antimicrob. Chemother. 76, 1406–1419
(2021).
41. Deussom, R., Mwarey, D., Bayu, M., Abdullah, S. S. & Marcus, R.
Systematic review of performance-enhancing health worker
supervision approaches in low- and middle-income countries.
Hum. Resour. Health 20, 2 (2022).
42. Li, Z., Karlsson, O., Kim, R. & Subramanian, S. V. Distribution of
under-5 deaths in the neonatal, postneonatal, and child hood
periods: a multicountry analysis in 64 low- and middle-income
countries. Int. J. Equity Health 20, 109–109 (2021).
43. Bernasconi, A. et al. The ALMANACH Project: preliminary results
and potentiality from Afghanistan. Int. J. Med. Inform. 114, 130–135
(2018).
44. Mitchell, M., Hedt-Gauthier, B. L., Msellemu, D., Nkaka, M. &
Lesh, N. Using electronic technology to improve clinical care—
results from a before-after cluster trial to evaluate assessment
and classiication of sick children according to Integrated
Management of Childhood Illness (IMCI) protocol in Tanzania.
BMC Med. Inform. Decis. Mak. 13, 95 (2013).
45. Lange, S., Mwisongo, A. & Mæstad, O. Why don’t clinicians adhere
more consistently to guidelines for the Integrated Management
of Childhood Illness (IMCI)? Soc. Sci. Med. 104, 56–63 (2014).
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine | Volume 30 | January 2024 | 76–84 84
Article https://doi.org/10.1038/s41591-023-02633-9
46. Kiplagat, A., Musto, R., Mwizamholya, D. & Morona, D. Factors inluen-
cing the implementation of integrated management of childhood
illness (IMCI) by healthcare workers at public health centers &
dispensaries in Mwanza, Tanzania. BMC Public Health 14, 277 (2014).
47. Ford, I. & Norrie, J. Pragmatic trials. N. Engl. J. Med. 375, 454–463
(2016).
48. Mc Cord, K. A. et al. Routinely collected data for randomized
trials: promises, barriers, and implications. Trials 19, 29 (2018).
49. Hannigan, A., Chisale, M., Drew, R., Watson, C. & Gallagher, J.
GP133 Mobile phones for follow up in paediatric clinical studies in
Africa. 104, A84 (2019).
50. Nguhuni, B. et al. Reliability and validity of using telephone calls
for post-discharge surveillance of surgical site infection following
caesarean section at a tertiary hospital in Tanzania. Antimicrob.
Resist. Infect. Control 6, 43 (2017).
51. Christie, S. A. et al. Feasibility of a cellular telephone follow-up
program after injury in Sub-Saharan Africa. J. Am. Coll. Surg. 227,
S129–S130 (2018).
Publisher’s note Springer Nature remains neutral with regard
to jurisdictional claims in published maps and institutional
ailiations.
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing,
adaptation, distribution and reproduction in any medium or format,
as long as you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons license, and indicate
if changes were made. The images or other third party material in this
article are included in the article’s Creative Commons license, unless
indicated otherwise in a credit line to the material. If material is not
included in the article’s Creative Commons license and your intended use
is not permitted by statutory regulation or exceeds the permitted use, you
will need to obtain permission directly from the copyright holder. To view
a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
© The Author(s) 2023
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
Methods
Study design and setting
The DYNAMIC Tanzania study was a pragmatic, open-label, parallel-
group, cluster randomized trial conducted in 40 primary health facili
-
ties in Tanzania. The health facility was the unit of randomization, since
the intervention was targeted at the health facility level.
Study sites were purposefully chosen to represent a variety of
health-care and epidemiological settings within five councils in the
Mbeya and Morogoro region, with a total population in those councils
of 1,701,717 (ref. 52). Two councils were semiurban (Mbeya city and
Ifakara Town councils), whereas the three others were rural (Mbeya,
Ulanga and Mlimba district councils). Overall 42.8% of the Tanzanian
population is less than 15 years old53. The prevalence of malaria in febrile
children aged 6–59 months is 5.8% in the Morogoro region and 3.4% in
the Mbeya region
54
. In accordance with the Tanzanian national clinical
guidelines, all febrile patients should be tested for malaria using a rapid
diagnostic test at the health facility of contact
55
. HIV prevalence among
children less than 15 years old is 0.5% in both regions
56
. Health care for
acute illnesses at government or government-designated primary
health facilities is free of charge for children under 5 years, including
the cost of medications such as antibiotics. For patients older than
5 years, health-care expenses are charged to the patient, unless they
have a health insurance plan (around 10% of Tanzanians)57.
First-level health facilities included in the DYNAMIC Tanzania
study include dispensaries and health centers with the latter distin-
guished by several characteristics. Health centers are characterized
by multiple outpatient consultation rooms, potential presence of
medical doctors, occasional small inpatient wards and a broader array
of diagnostic and therapeutic capabilities compared to dispensaries.
Participants
Primary care health facilities (dispensaries or health centers) were
eligible for inclusion if they performed on average 20 or more consulta-
tions per week with children aged from 2 months to 5 years, were gov-
ernment or government-designated health facilities, and were located
less than 150 km from the research institutions. Acute outpatient care
is routinely provided by nurses and clinical officers in primary health
facilities, whereas medical doctors provide care occasionally at health
centers. Clinical officers, the principal health providers at primary
health facilities, are non-physician health professionals with 2–3 years
of clinical training following secondary school58.
Infants and children between 1 day old and 15 years old seeking
care for an acute medical or surgical condition at participating health
facilities were eligible. Children presenting solely for scheduled consul-
tations for a chronic disease (for example HIV, tuberculosis, malnutri-
tion) or for routine preventive care (for example growth monitoring,
vaccination) were not eligible. Written informed consent was obtained
from all parents or guardians of participants when attending the par-
ticipating health facility during the enrollment period.
Sampling, randomization and masking
The 40 health facilities were randomly selected from all eligible health
facilities in the participating councils following a 3:2 ratio between
health facilities from the Morogoro and Mbeya region (to include more
health facilities in the higher malaria transmission area). In addition,
to include a representative sample of health centers compared to dis-
pensaries, four health centers per region were included.
The sampled health facilities were then randomized (1:1) to
ePOCT+ (intervention) or usual care (control). Randomization was
stratified by region, council, level of health facility (health center ver-
sus dispensary) and attendance rate. An independent statistician in
Switzerland was provided with the list of all eligible health facilities
and performed computer-generated sampling and randomization.
Intervention allocation by the study team was only shared with study
investigators in Tanzania once all council leaders had confirmed the
participation of their selected health facilities. The nature of the inter-
vention did not allow for masking of the intervention to health-care
providers, patients or study implementers.
Intervention
The intervention consisted of providing ePOCT+ with the supporting
IT infrastructure, C-reactive protein (CRP) semiquantitative lateral
flow test, hemoglobin point-of-care tests (and hemoglobinometer if
not already available), pulse oximeter, training and supportive mentor-
ship (Extended Data Fig.1). If unavailable in health facilities, materials
to perform laboratory tests such as prickers, cotton swabs, gloves and
alcohol were provided. The decision to perform tests (malaria, CRP,
hemoglobin, pulse oximeter), like all clinical symptoms and signs, is
determined by the clinical algorithm behind ePOCT+, and prompted to
the health-care provider when required. The health-care provider can
decide not to follow the recommendations of ePOCT+ as they see fit.
CRP point-of-care rapid tests and hemoglobin point-of-care tests were
integrated as per usual laboratory procedures (that is, in health facili-
ties where point-of-care tests are usually performed and interpreted
in the laboratory by a laboratory technician, tests were performed
in the laboratory; in health facilities where tests are usually done in
the consultation room, they were done by the health-care provider).
The development process and details of the ePOCT+ CDSA and the
medAL-reader Android-based application used to deploy ePOCT+ have
been described in detail previously
28
. In summary the clinical algorithm
of ePOCT+ is based on previous-generation CDSAs (ALMANACH and
ePOCT)
20,29
, international and national clinical guidelines, and input
from national and international expert panels, and was adapted based
on piloting and health-care provider feedback28. Mentorship by the
implementation team included visits to health facilities every 2 to
3 months and communication by phone call or group messages three
to four times per month, to resolve issues and provide guidance and
feedback on the use of the new tools. Results from quality-of-care
dashboards were shared through group messages to give feedback
on the use of ePOCT+, a strategy often described as ‘benchmarking’,
allowing health-care providers to compare their antibiotic prescrip-
tion, uptake and other quality-of-care indicators with other health
facilities59. Control health facilities provided care as usual, with no
access to clinical data dashboards.
All participating health facilities were provided with IT infrastruc-
ture to support the tablet-based ePOCT+ CDSA or in the case of con-
trol health facilities, to support the use of tablet-based eCRFs. The
IT infrastructure included a tablet for each outpatient consultation
room, router, local server (Rasberry Pi), internet and, if needed, back up
power (battery) or solar power. In addition, weighing scales, mid-upper
arm circumference bands and thermometers were provided to health
facilities for both study arms if not already available. Health-care pro-
viders from both intervention and control health facilities received
equivalent clinical refresher training based on the IMCI chartbook. In
addition, specific training was provided on the use of the ePOCT+ CDSA
in intervention facilities and the use of the eCRF in control facilities.
Study procedures
Children seeking care at included health facilities were screened for
eligibility by a research assistant between 08:00 and 16:00 on week-
days. If eligible, demographic information was collected and entered
in the eCRF (ePOCT+ for intervention health facilities and eCRFs for
usual care facilities within the data collection system medAL-reader).
Health-care providers in the control health facilities managed the
patients as usual, but documented the main complaints, anthropo-
metrics and test results (if performed), diagnoses, treatments and
referral decision in the eCRF. To harmonize data collection across the
intervention and control facilities, the eCRF for the control facilities
was also programmed into the medAL-reader platform, but no decision
support was provided. Research questions were included in the eCRF
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
to capture whether an oral or systemic antibiotic was prescribed, and
whether the patient was referred for inpatient hospitalization or other
outpatient investigations. In intervention health facilities, in addition
to the same information collected in the eCRF, symptoms and signs
of the patients were recorded in the ePOCT+ CDSA during the con-
sultation with the patient. The symptoms and signs entered are used
by the ePOCT+ CDSA to guide the clinical consultation. Health-care
providers who documented the final treatment for a consultation in
ePOCT+ or the eCRF were categorized as having been managed per
protocol, as recording of the final treatment is required to complete
the ePOCT+ CDSA.
All patients were called or visited at their home by research assis-
tants to assess clinical outcomes and their care and treatment seeking
behavior at day 7 (range 6–14 days). Research assistants performing the
phone calls were blinded to the intervention status and were not part
of the team enrolling patients at health facilities. Home visits rather
than phone calls were conducted if the caregiver of patients did not
have a phone number or did not know somebody with a phone near
their home, or if research assistants were not able to reach the provided
phone number after five attempts. The home visits were performed by
the research assistants enrolling patients from the same health facility,
and as such they were not blinded to intervention allocation. Patients
who were still sick at follow-up were encouraged to return to a health
facility for follow-up care. Day 7 data were recorded using REDCap
web for phone calls and REDCap mobile application for home visits.
Outcomes
The coprimary outcomes measured at the individual patient level
included: (1) antibiotic prescription at the time of the initial consulta-
tion as documented by the health-care provider (superiority analysis);
and (2) clinical failure at day 7 defined as ‘not cured’ and ‘not improved’,
or unscheduled hospitalization as reported by caregivers (noninferior-
ity analysis). Secondary outcomes include unscheduled reattendance
visits at any health facility by day 7, nonreferred secondary hospitaliza-
tion by day 7, death by day 7 and referral for inpatient hospitalization at
initial consultation. Additional antibiotics prescribed on subsequent
days following the initial consultation were not part of the coprimary
outcome of antibiotic prescription; instead this is captured by phone
call on day 7, where all patients are assessed for whether additional
medication was taken after the initial consultation, and compared
between study arms as an exploratory outcome. Given patients’ and
caregivers’ difficulty in distinguishing antibiotics from other medica
-
tions
60,61
, we could not reliably assess antibiotic intake based on the
caregiver’s report; the outcome thus looked at all medications, rather
than antibiotics specifically. The intervention was deemed a success if
ePOCT+ was noninferior in terms of clinical failure and reduced anti-
biotic prescription by at least 25%. Prespecified additional outcomes
are outlined in the statistical analysis plan.
Sample size
The sample size was calculated for testing noninferiority of the clinical
failure outcome given that it would require a higher sample size than for
the antibiotic prescription coprimary outcome. We assumed a cluster
size of 900 patients per health facility (mean of 150 patients per month
per health facility multiplied by 6 months, the minimum duration of the
study) based on routine data within the national health management
information system, an intraclass correlation coefficient of 0.002 and
a clinical failure rate of 3%. To have 80% power to detect an acceptable
noninferiority margin of a relative risk of 1.3, corresponding to 3.9%,
we required 19 clusters and 17,100 patients per arm (total patients
n = 37,620 assuming 10% loss to follow-up). Given the uncertainty of
some of the assumptions, the total number of health facilities was
rounded up to 20 clusters per arm.
No interim analysis was planned; however, owing to lower enroll-
ment than expected, after 8 months of recruitment, we planned an ad
hoc sample size recalculation by an independent statistician to calcu-
late the expected power of the study based on updated parameters
(Supplementary Information Note1). The study team prespecified
the specifications and approach, documented in an update to the
statistical analysis plan.
Statistical analysis
All outcomes were evaluated using random effects logistic regression
models using the cluster (health facility) and patient as random effects,
with further adjustment using fixed effect terms for randomization
stratification factors62, and baseline characteristics hypothesized to
be associated with the outcome, imbalances between arms and imbal-
ances between characteristics among patients for whom day 7 data
were available and not available (lost to follow-up). These included
the patient characteristics of age, sex, presenting complaints (fever,
respiratory, gastrointestinal, skin) and phone availability, and the
health facility characteristics of care provision level (dispensary versus
health center), attendance rate per month and council. A partition-
ing method was used to separate within-cluster and between-cluster
effects to account for confounding by cluster63–65. In the case of too
few events, and small variance among health facilities, which did not
allow the model to converge, the health facility was incorporated in the
model as a fixed effect. Adjusted relative risk and absolute differences
were estimated based on the computed marginal probabilities of the
conditional probabilities66,67. Formal adjustments were not performed
for multiple testing, as adjustments would likely be overly conservative
given that the outcomes are not all independent68, and variable selec-
tion was not based on statistical tests of significance69. No adjustment
for baseline characteristics or for within-health-care-facility correla-
tions was used for the calculation of crude confidence intervals for
relative risk and absolute differences.
Noninferiority was determined if the upper limit of the 95% CI of
the aRR was below 1.3. All analyses based on outcomes from day 0 were
performed in the per protocol population, and outcomes determined
at day 7 were performed in the per protocol and complete case popu-
lation (only in those for which day 7 outcomes were ascertained) and
displayed accordingly unless stated otherwise. The primary analyses
were performed on the first visit for an illness, with reattendance vis-
its (a second visit to a health facility for the same illness) included in
exploratory analyses. Prespecified analyses to assess the effect of the
intervention in different population groups were performed by sex, age
group and consultation complaint categories (respiratory symptoms,
fever, gastrointestinal complaint, skin problem, ear, nose and throat
problem). All analyses were performed using Stata v.16 and v.17 (ref. 70).
Inclusion and ethics
Ethical approval was obtained in Tanzania from the Ifakara Health Insti-
tute (IHI/IRB/No: 11-2020), the Mbeya Medical Research Ethics Com-
mittee (SZEC-2439/R.A/V.1/65) and the National Institute for Medical
Research Ethics Committee (NIMR/HQ/R.8a/Vol. IX/3486 and NIMR/
HQ/R.8a/Vol. IX/3583), and in Switzerland from the cantonal ethics
review board of Vaud (CER-VD 2020-02800). The study was registered
on ClinicalTrials.gov number NCT05144763, where the trial protocol
and statistical analysis plan can be found (statistical analysis plan also
found in Supplementary Information Note 2). The study design and
implementation was developed collaboratively between the Ifakara
Health Institute, Mbeya Medical Research Centre, Swiss Tropical and
Public Health Institute and the Centre for Primary Care and Public
Health, University of Lausanne, based on feedback from stakehold-
ers, patients and health-care providers involved in our similar trials
in Tanzania20,21,33. In addition, previous work from Tanzania was used
to guide the design of the study and to develop ePOCT+, and other
work from Tanzania was taken into account in the citations for this
manuscript
32,45,46
. ePOCT+ and the medAL-suite was developed col-
laboratively by an international group of digital and global health
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
experts from Tanzania and other LMICs
28
. Specifically a Tanzanian
clinical expert group including representatives designated by the
Ministry of Health made the final decision on clinical content, and
primary care level health providers gave important feedback to develop
and improve ePOCT+, including a Delphi survey among 30 Tanzanian
health providers28. Over 100 community engagement meetings with
over 7,000 participants were conducted before and during the study,
including numerous meetings with Community and Regional Health
Management Teams in the Mbeya and Morogoro regions of Tanzania.
Reporting summary
Further information on research design is available in theNature Port-
folio Reporting Summary linked to this article.
Data availability
De-identified data can be found at https://doi.org/10.5281/
zenodo.8043523, including case, patient and health facility identifi-
cation number, study arm allocation, baseline characteristics and all
outcomes.
Code availability
The code for the medAL-reader application used to collect data entered
by health providers (including demographic, clinical, diagnosis, pre-
scription and referral data of the consultations) can be found at https://
github.com/Wavemind/liwi-medal-reader.
References
52. The 2022 Population and Housing Census: Administrative Units
Population Distribution Report (Ministry of Finance and Planning
Tanzania, National Bureau of Statistics and President’s Oice—
Finance and Planning and Oice of the Government Statistician,
Zanzibar, 2022).
53. The 2022 Population and Housing Census: Age and Sex
Distribution Report, Key Findings (Ministry of Finance and Planning
Tanzania, National Bureau of Statistics and President’s Oice—
Finance and Planning and Oice of the Government Statistician,
Zanzibar, 2022).
54. Tanzania: Demographic and Health Survey and Malaria Indicator
Survey 2022—Key Indicators Report (Ministry of Health Dodoma,
Ministry of Health Zanzibar, National Bureau of Statistics Dodoma,
Oice of Chief Government Statistician Zanzibar, The DHS
Program ICF, 2023).
55. Standard Treatment Guidelines and National Essential Medicines
List for Children and Adolescents (Tanzania Ministry of Health,
Community Development, Gender, Elderly and Children, 2018).
56. Tanzania HIV Impact Survey (THIS) 2016–2017: Final Report
(Tanzania Commission for AIDS, Zanzibar AIDS Commission, 2018).
57. Amu, H., Dickson, K. S., Kumi-Kyereme, A. & Darteh, E. K. M.
Understanding variations in health insurance coverage in Ghana,
Kenya, Nigeria, and Tanzania: evidence from demographic and
health surveys. PLoS ONE 13, e0201833 (2018).
58. Mullan, F. & Frehywot, S. Non-physician clinicians in 47
sub-Saharan African countries. Lancet 370, 2158–2163 (2007).
59. Ibrahim, O. M. & Polk, R. E. Antimicrobial use metrics and
benchmarking to improve stewardship outcomes: methodology,
opportunities, and challenges. Infect. Dis. Clin. 28, 195–214 (2014).
60. Do, N. T. T. et al. Community-based antibiotic access and use in
six low-income and middle-income countries: a mixed-method
approach. Lancet Glob. Health 9, e610–e619 (2021).
61. Monnier, A. A. et al. Is this pill an antibiotic or a painkiller?
Improving the identiication of oral antibiotics for better use.
Lancet Glob. Health 11, e1308–e1313 (2023).
62. Kahan, B. C. & Morris, T. P. Reporting and analysis of trials using
stratiied randomisation in leading medical journals: review and
reanalysis. BMJ 345, e5840 (2012).
63. Localio, A. R., Berlin, J. A. & Have, T. R. T. Confounding due to
cluster in multicenter studies—causes and cures. Health Serv.
Outcomes Res. Methodol. 3, 195–210 (2002).
64. Begg, M. D. & Parides, M. K. Separation of individual-level and
cluster-level covariate eects in regression analysis of correlated
data. Stat. Med. 22, 2591–2602 (2003).
65. Neuhaus, J. M. & Kalbleisch, J. D. Between- and within-cluster
covariate eects in the analysis of clustered data. Biometrics 54,
638–645 (1998).
66. Mu, S., Held, L. & Keller, L. F. Marginal or conditional regression
models for correlated non‐normal data? Methods Ecol. Evol. 7,
1514–1524 (2016).
67. Miglioretti, D. L. & Heagerty, P. J. Marginal modeling of nonnested
multilevel data using standard software. Am. J. Epidemiol. 165,
453–463 (2007).
68. Armstrong, R. A. When to use the Bonferroni correction.
Ophthalmic Physiol. Opt. 34, 502–508 (2014).
69. Savitz, D. A. & Olshan, A. F. Multiple comparisons and related
issues in the interpretation of epidemiologic data. Am. J.
Epidemiol. 142, 904–908 (1995).
70. Stata Statistical Software v.16 (StataCorp LLC, 2019).
Acknowledgements
We acknowledge the contributions of the research assistants at the
Ifakara Health Institute and Mbeya Medical Research Centre—National
Institute of Medical Research, who assisted in the data collection, the
Information Technology sta at Unisanté (S. Schaufelberger and G.
Martin) and sta at Wavemind (E. Barchichat, A. Fresco and Q. Girard)
for their work on the medAL-suite during the study, the health-care
providers in the participating health facilities and the Community
Health Management Team members in the ive participating councils
in Tanzania for their collaboration in implementing the study. We
acknowledge researchers of the Tools of Integrated Management
of Childhood Illness and DYNAMIC Rwanda project for their
contributions to ePOCT+ and the many common research activities
(F. Beynon, T. Glass, L. Matata, H. Langet, L. Boole, L. Cobuccio,
V. Rwandacu, R. Moshiro, M. Norris and L. Cleveley). We thank the
Tanzania expert panel for their valuable work in helping to develop
the ePOCT+ Tanzania algorithm (K. Manji, N. Salim, H. Naburi, J.
Massaro, D. Moshi, W. Ndembeka, L. Chirande and E. Mbao). We
thank M.-A. Hartley for her work on the initial project proposal and
guidance on various aspects of the DYNAMIC TZ project, A. Chaouch
for his help in conducting the independent sample size recalculation
and A. Chandna for his valuable comments on the manuscript. We
especially thank all the patients and caregivers who participated
in the study. Finally we acknowledge the work of I. Masanja on the
initial development of the study, who regrettably passed away before
the start of the study. This work was supported by a grant from the
Fondation Botnar, Switzerland (grant number 6278) (V.D.A.) and from
the Swiss Development Cooperation (project number 7F-10361.01.01)
(V.D.A.). The study sponsor (Centre for Primary Care and Public Health,
Unisanté, University of Lausanne) led the study design, the writing of
the report and the decision to submit the article for publication. The
funders of the study had no role in study design, data collection, data
analysis, data interpretation or writing of the report.
Author contributions
R.T., A.V.K., K.K., N.N., H. Masanja and V.D.A. were responsible for study
design. R.T., A.V.K. and V.D.A. came up with the statistical analysis
plan. R.T. was responsible for data curation. R.T. and P.T. carried out the
formal analysis. K.K., N.N., H. Masanja and V.D.A. acquired funding. R.T.,
G.K., L.B.L., A.V.K., S.R., C.M., G.A., M.J., I.E.M., P.A. and H. Mhagama
carried out the investigation. A.V.K., L.B.L., C.M., S.R. and V.F. were
responsible for project administration. R.T., A.V.K., L.B.L., C.M., S.R.,
V.F., J.T., N.N., H. Masanja. and V.D.A. supervised the project. R.T. wrote
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
the original draft of the paper. R.T., G.K., L.B.L., A.V.K., S.R., C.M., G.A.,
M.J., I.E.M., P.A., H. Mhagama., A.V., V.F., J.T., G.L., M.-A.L.P., K.K., P.T.,
N.N., H. Masanja. and V.D.A. wrote, reviewed and edited the inal
paper. R.T. had full access to all of the data in the study and takes
responsibility for the integrity of the data and the accuracy of the
data analysis.
Competing interests
The authors declare no competing interests.
Additional information
Extended data is available for this paper at
https://doi.org/10.1038/s41591-023-02633-9.
Supplementary information The online version contains supplementary
material available at https://doi.org/10.1038/s41591-023-02633-9.
Correspondence and requests for materials should be addressed to
Rainer Tan.
Peer review information Nature Medicine thanks David Bates, Sumanth
Gandra, Raphael Sangeda and the other, anonymous, reviewer(s) for
their contribution to the peer review of this work. Primary Handling
Editor: Lorenzo Righetto, in collaboration with the Nature Medicine team.
Reprints and permissions information is available at
www.nature.com/reprints.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
Extended Data Fig. 1 | Description of ePOCT+ and supportive mentorship
intervention. The intervention included the provision of the ePOCT+ Clinical
Decision Support Algorithm (CDSA), C-Reactive Protein & Hemoglobin point-
of-care tests, and pulse oximeter. The use of these additional tests and tools are
proposed within the ePOCT+ CDSA. The intervention also included the sharing of
quality of care indicators within dashboards which allowed healthcare providers
to see their performance compared to other health facilities. Finally mentorship
support in the form of messages, phone calls, and visits were conducted to
answer questions and support the use of ePOCT+. Both intervention and control
health facilities received equivalent Integrated Management of Childhood Illness
(IMCI) training, and Information Technology (IT) support. If required the health
facilities also received a weighing scale, mid-upperarm circumference (MUAC)
band, and thermometer.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
Extended Data Table 1 | Primary, secondary and exploratory outcomes in initial cases in the intention-to treat population
Secondary and exploratory outcomes that rely on data collected on day 0 are not shown as this information is not available for all ITT patients (non-referred secondary hospitalizations by day
7, completed referral, primary referrals at day 0). All outcomes that rely on day 7 outcomes are complete case analyses. Adjusted relative risks and differences were estimated using a random
effects logistic regression model adjusting for clustering (health facility, and patient), and individual and health facility baseline characteristics that were available for all ITT cases (age, sex,
availability of phone, council of health facility, level of health facility, average number of patients seen per month at the health facility). Formal adjustments were not performed for multiple
testing. *Considering that all cases that were not managed per protocol were prescribed an antibiotic; **Post-hoc exploratory outcomes
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
Extended Data Table 2 | Stratiied subgroup analysis for antibiotic prescription at D0 in initial consultations in the per
protocol population
Adjusted relative risks and differences were estimated using a random effects logistic regression model adjusting for clustering (health facility and patient), and individual and health
facility baseline characteristics (age, sex, complaints, phone availability, council of health facility, level of health facility, average number of patients seen per month at the health
facility). Formal adjustments were not performed for multiple testing. aPost-hoc exploratory subgroup analyses bMalaria risk deined as per IMCI: Low risk = <5% malaria positivity rate in
febrile children; High risk > = 5% malaria positivity rate in febrile children
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Nature Medicine
Article https://doi.org/10.1038/s41591-023-02633-9
Extended Data Table 3 | Subgroup analysis for clinical failure at day 7 in the per protocol, complete case population among
initial consultations
Clinical failure by day 7 deined as ‘not cured’ and ‘not improved’, or unscheduled hospitalization as reported by caregivers. Adjusted relative risks were estimated using a random effects
logistic regression model adjusting for clustering (health facility and patient), and individual and health facility baseline characteristics (age, sex, complaints, availability of phone, council
of health facility, level of health facility, average number of patients seen per month at the health facility) but omitting the variable as an adjustment variable in the model for the subgroup
analyzed. Formal adjustments were not performed for multiple testing. aPost-hoc exploratory subgroup analyses bMalaria risk deined as per IMCI: Low risk = <5% malaria positivity rate in
febrile children; High risk > = 5% malaria positivity rate in febrile children
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Content courtesy of Springer Nature, terms of use apply. Rights reserved
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-scale
personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By accessing,
sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these purposes, Springer
Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription (to
the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue, royalties,
rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal content cannot be
used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any other, institutional
repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or content on
this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature may revoke this
licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied with
respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law, including
merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed from
third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
