Access to this full-text is provided by PLOS.
Content available from PLOS Digital Health
This content is subject to copyright.
RESEARCH ARTICLE
A cluster randomized trial assessing the effect
of a digital health algorithm on quality of care
in Tanzania (DYNAMIC study)
Rainer TanID
1,2,3,4☯
*, Godfrey Kavishe
5☯†
, Alexandra V. Kulinkina
3,4
, Sabine Renggli
2
,
Lameck B. Luwanda
2
, Chacha Mangu
5
, Geofrey Ashery
2
, Margaret Jorram
2
, Ibrahim
Evans Mtebene
2
, Peter Agrea
5
, Humphrey Mhagama
5
, Kristina Keitel
3,4,6
, Marie-Annick Le
PogamID
1
, Nyanda Ntinginya
5‡
, Honorati Masanja
2‡
, Vale
´rie D’Acremont
1,3,4‡
1Centre for Primary Care and Public Health (Unisante
´), University of Lausanne, Lausanne, Switzerland,
2Ifakara Health Institute, Dar es Salaam, United Republic of Tanzania, 3Swiss Tropical and Public Health
Institute, Allschwil, Switzerland, 4University of Basel, Basel, Switzerland, 5National Institute of Medical
Research–Mbeya Medical Research Centre, Mbeya, United Republic of Tanzania, 6Pediatric Emergency
Department, Department of Pediatrics, University Hospital Bern, Bern, Switzerland
☯These authors contributed equally to this work.
† Deceased.
‡ NN, HM and VD also contributed equally to this work.
*rainer.tan@unisante.ch
Abstract
Digital clinical decision support tools have contributed to improved quality of care at primary
care level health facilities. However, data from real-world randomized trials are lacking. We
conducted a cluster randomized, open-label trial in Tanzania evaluating the use of a digital
clinical decision support algorithm (CDSA), enhanced by point-of-care tests, training and men-
torship, compared with usual care, among sick children 2 to 59 months old presenting to pri-
mary care facilities for an acute illness in Tanzania (ClinicalTrials.gov NCT05144763). The
primary outcome was the mean proportion of 14 major Integrated Management of Childhood
Illness (IMCI) symptoms and signs assessed by clinicians. Secondary outcomes included
antibiotic prescription, counseling provided, and the appropriateness of antimalarial and antibi-
otic prescriptions. A total of 450 consultations were observed in 9 intervention and 9 control
health facilities. The mean proportion of major symptoms and signs assessed in intervention
health facilities was 46.4% (range 7.7% to 91.7%) compared to 26.3% (range 0% to 66.7%) in
control health facilities, an adjusted difference of 15.1% (95% confidence interval [CI] 4.8% to
25.4%). Only weight, height, and pallor were assessed statistically more often when using the
digital CDSA compared to controls. Observed antibiotic prescription was 37.3% in intervention
facilities, and 76.4% in control facilities (adjusted risk ratio 0.5; 95% CI 0.4 to 0.7; p<0.001).
Appropriate antibiotic prescription was 81.9% in intervention facilities and 51.4% in control
facilities (adjusted risk ratio 1.5; 95% CI 1.2 to 1.8; p = 0.003). The implementation of a digital
CDSA improved the mean proportion of IMCI symptoms and signs assessed in consultations
with sick children, however most symptoms and signs were assessed infrequently. Nonethe-
less, antibiotics were prescribed less often, and more appropriately. Innovative approaches to
overcome barriers related to clinicians’ motivation and work environment are needed.
PLOS DIGITAL HEALTH
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 1 / 18
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Tan R, Kavishe G, Kulinkina AV, Renggli
S, Luwanda LB, Mangu C, et al. (2024) A cluster
randomized trial assessing the effect of a digital
health algorithm on quality of care in Tanzania
(DYNAMIC study). PLOS Digit Health 3(12):
e0000694. https://doi.org/10.1371/journal.
pdig.0000694
Editor: Shannon Freeman, University of Northern
British Columbia, CANADA
Received: June 25, 2024
Accepted: November 7, 2024
Published: December 23, 2024
Peer Review History: PLOS recognizes the
benefits of transparency in the peer review
process; therefore, we enable the publication of
all of the content of peer review and author
responses alongside final, published articles. The
editorial history of this article is available here:
https://doi.org/10.1371/journal.pdig.0000694
Copyright: ©2024 Tan et al. This is an open access
article distributed under the terms of the Creative
Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in
any medium, provided the original author and
source are credited.
Data Availability Statement: De-identified data can
be found at https://zenodo.org/records/10849644.
Author summary
Digital health tools have been created to help healthcare workers provide better care, but
their real-world impact is still uncertain. This study evaluated the use of ePOCT+, a digital
health clinical decision support algorithm for health providers, combined with point-of-
care diagnostic tools (CRP tests, pulse oximetry), training, and mentorship. The study
compared 9 primary care health facilities using ePOCT+ with 9 facilities providing care as
usual. The study found that using the digital health intervention improved the assessment
of important symptoms and signs in children aged 2 to 59 months with acute illnesses.
However, many symptoms and signs were still not frequently assessed. In addition, antibi-
otic prescriptions were halved in facilities using the tool, and the appropriateness of pre-
scriptions improved significantly. Despite these benefits, challenges related to healthcare
workers’ motivation and work environments must be explored to fully realize the poten-
tial of such tools. These findings suggest that digital health tools can improve the quality
of care and address issues like overprescription of antibiotics. However, broader strategies
are needed to support healthcare workers in delivering comprehensive, high-quality care
in similar settings globally.
Introduction
Millions of preventable deaths are attributed to suboptimal healthcare quality [1]. Factors such
as staff shortages, inadequate budget allocation, poor clinical knowledge, and limited access to
quality medical education, mentorship and supervision collectively contribute to this issue [2–
4]. In response to this challenge, and to reduce childhood mortality, the World Health Organi-
zation developed the Integrated Management of Childhood Illness (IMCI) Chartbook [5].
Since its inception, over 100 countries have implemented the guidelines, and IMCI may reduce
mortality and improve quality of care [6,7]. However, poor adherence to IMCI is common,
limiting its benefits [8–10].
Digital Clinical Decision Support Algorithms (CDSAs) were devised to enhance adherence
to clinical guidelines. These tools, typically operating on electronic tablets or mobile phones,
guide healthcare providers through the consultation process, by prompting the evaluation of
symptoms, signs, and recommended diagnostic tests, to finally propose the appropriate diag-
nosis and treatment [11,12]. While several studies have found that using these digital CDSAs
improve adherence to IMCI, a noteworthy research gap is that many of these investigations
were conducted in controlled study settings, and most lacked randomization [13–20].
ePOCT+, a digital CDSA, was developed based on insights from two previous generations
of CDSAs [21,22], specifically addressing challenges by our CDSAs and others, such as limited
scope and information technology difficulties [23]. The present study aimed to assess whether
this CDSA associated with point-of-care tests, training, and mentorship, would improve the
quality of care for sick children compared to usual care, by comparing adherence to IMCI in a
pragmatic cluster randomized trial.
Methods
Study design
The present study is an open-label, parallel-group, cross-sectional cluster randomized trial
within the DYNAMIC Tanzania project. An external clinical researcher observed a sample of
consultations from health facilities from both study arms documenting adherence of
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 2 / 18
Funding: This study was supported by a grant
from the Fondation Botnar Switzerland (https://
www.fondationbotnar.org/) grant number 6278 to
V.D.A., and from the Swiss Development
Cooperation (https://www.fdfa.admin.ch/sdc)
project number 7F-10361.01.01 to V.D.A.. The
study sponsor (Centre for Primary Care and Public
Health, Unisante
´, University of Lausanne) led the
study design, the writing of the report and the
decision to submit the article for publication. The
funders of the study had no role in study design,
data collection, data analysis, data interpretation or
writing of the report.
Competing interests: The authors have declared
that no competing interests exist.
healthcare providers to quality-of-care indicators. The study was a planned ancillary study
within a larger cluster randomized trial conducted between 1 December 2021 and 31 October
2022 using a sample of the clusters [24]. A cluster design was chosen since the intervention
was targeted at the health facility and healthcare provider. The trial design and rationale are
outlined in the protocol available in the parent trial registration on ClinicalTrials.gov number
NCT05144763 and in the supplementary materials (S1 File). The detailed statistical analysis
plan for this ancillary study is also available in the supplementary materials (S2 File).
The study design and implementation were collaboratively executed between both Tanza-
nian (Ifakara Health Institute, National Institute for Medical Research—Mbeya Medical
Research Centre) and Swiss (Centre for Primary Care and Public Health [Unisante
´]–Univer-
sity of Lausanne, and Swiss Tropical and Public Health Institute) partners. The design was
guided by input from patients, and health providers during the implementation of similar tri-
als in Tanzania [14,22,25]. Over 100 community engagement meetings involving over 7,000
participants were conducted before and during the study. These meetings included discussions
with Community and Regional Health Management Teams in Tanzania.
Participants
The health facility was the unit of randomization since the intervention targeted both the
healthcare provider and health facility. Primary care health facilities (dispensaries or health
centers) were eligible for inclusion if they performed on average 20 or more consultations with
children 2 months to 5 years per week, were government or government-designated health
facilities, and were located less than 150 km from the research institutions. Specific to this
study, consultations were only included if healthcare providers had been trained to use ePOCT
+, and the ePOCT+ tool was functioning on the day of observation (no IT issues related to
power outages, or crashes reported).
In contrast to the larger trial that included children aged 1 day to 14 years, this ancillary
study included only children aged 2 to 59 months old presenting for an acute medical or surgi-
cal condition at participating health facilities. Children presenting solely for scheduled consul-
tations for a chronic disease (e.g. HIV, tuberculosis, malnutrition), for routine preventive care
(e.g. growth monitoring, vaccination), or a follow-up consultation were excluded.
The study was conducted in 5 councils within the Mbeya and Morogoro regions of Tanza-
nia, with two councils being semi-urban and three rural. Malaria prevalence in febrile children
was low in three councils, and moderately high in two. HIV prevalence among children less
than 5 years old in Tanzania is 0.4% [26]. Healthcare for children under 5 years of age is free
for all acute illnesses at government or government-designated primary health facilities,
including the cost of medications [27]. Nurses and clinical officers routinely provide outpa-
tient care in dispensaries, while in health centers medical doctors sometimes provide care as
well. Clinical Officers, the predominant healthcare providers at primary health facilities, are
non-physician health professionals with 2–3 years of clinical training following secondary
school [28].
Interventions
The intervention involved equipping health facilities with ePOCT+, an electronic clinical deci-
sion support algorithm on an Android based tablet (Fig 1), along with associated point-of-care
tests (C-Reactive Protein, Hemoglobin, pulse oximetry), training, and mentorship. ePOCT+
prompts the healthcare provider to answer questions about demographics, symptoms, signs,
and tests [23]. Based on the answers, ePOCT+ proposes one or more diagnoses, treatments,
and management plans including referral recommendation. Healthcare providers had the
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 3 / 18
possibility to deviate from ePOCT+ recommendations, and had the option to not use the tool.
In order to move forward within the different sections of the digital tool, it was mandatory to
respond to all IMCI symptoms and signs, except for height and mid-upper arm circumference
(MUAC) which was optional. The tool allowed some signs to be estimated (temperature, respi-
ratory rate) or based on recent measurements (weight). Detailed description on the develop-
ment process and features of ePOCT+ and the medAL-reader application can be found in
separate publications [23,29].
The implementation team provided mentorship to intervention health facilities. This men-
toring consisted of regular visits to health facilities every 2–3 months, and frequent communi-
cation via phone calls or group messages (3–4 times per month) to address issues, offer
guidance, and gather feedback on the new tools. Quality-of-care dashboards were shared
through group messages, enabling healthcare providers to compare their antibiotic prescrip-
tion rates, uptake, and other quality-of-care indicators with other facilities (benchmarking).
Control health facilities continued with usual care, did not have access to clinical data dash-
boards, and only received visits from the implementation team to help resolve issues related to
the electronic case report forms (eCRFs).
Fig 1. Screen-shots of different stages of ePOCT+ running on the medAL-reader application. Stages are shown in
order of appearance, however not all stages are shown in the figure.
https://doi.org/10.1371/journal.pdig.0000694.g001
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 4 / 18
The infrastructure provided to all health facilities (control and intervention) included a tab-
let for each outpatient consultation room, a router, a local server (Raspberry Pi), internet con-
nectivity, and backup power (battery or solar system if needed). If unavailable weighing scales,
mid-upper arm circumference (MUAC) bands, and thermometers were provided to all health
facilities. Healthcare providers from both intervention and control facilities underwent equiva-
lent clinical refresher training on IMCI and concepts of antibiotic stewardship. Additionally,
specific training was provided on the use of the ePOCT+ CDSA in intervention facilities and
the use of the eCRF in control facilities.
Outcomes
The primary outcome was the mean proportion of 14 pre-identified major IMCI symptoms
and signs assessed by the healthcare provider, as observed by an external clinical research assis-
tant. The included symptoms were fever, cough or difficult breathing, convulsions in this ill-
ness, diarrhea, ear pain or discharge, child unable to drink or breastfeed, and child vomits
everything. The included signs were measurement of temperature, respiratory rate, pallor,
weight, mid-upper arm circumference (MUAC), height, and skin turgor. In specific circum-
stances, some patients were not included in the denominator for specific signs as they were
not clinically indicated as defined by IMCI: they include MUAC in children less than 6 months
old, respiratory rate in the absence of cough or difficult breathing, and skin turgor in the
absence of diarrhea. If cough or difficult breathing was not assessed, then we took the most
conservative approach assuming that respiratory rate should have been measured, and the
same for diarrhea and skin turgor. Of note “lethargic and unconscious” was considered as
assessed if the clinician asked the caregiver if it was present during the illness, and not based
on observation of the child as being “lethargic or unconscious”.
Secondary outcomes include the proportion of consultations during which each major IMCI
symptom and sign were assessed, the proportion of which other symptoms and signs were
assessed (S2 File), the proportion of consultations where different IMCI counseling was con-
ducted, and proportion of consultations for which antibiotics were prescribed. The rationale for
the distinction between “major” IMCI and “other” symptoms and signs are described in detail in
the statistical analysis plan (S2 File). Prescription of antibiotics was assessed by the research assis-
tant by observing the actual prescription prescribed. The appropriateness of antibiotic prescription
in relation to the retained diagnosis was also assessed. An antibiotic prescription was considered
appropriate if one of the retained diagnoses required an antibiotic as per IMCI or the WHO hospi-
tal pocket book, and the absence of a prescription if no diagnosis required an antibiotic [30,31].
An appropriate antimalarial prescription was a prescription of any antimalarial if there was a posi-
tive malaria test. Assessment of appropriateness was conducted blinded to the study arm.
All outcomes pertained to the cluster level (health facility), and were assessed by an external
clinical research assistant who observed the consultations in the consultation room without
interfering with the consultation. The external clinical research assistants were clinical officers
with experience in primary care consultations for children. Data was collected using a struc-
tured and pre-tested observation form programmed on ODK, and collected on an Android-
based tablet. The observation form was based on the 2012 DHS Service Provision Assessment
Survey form [32]. Of note, a more recent modification to this survey was developed after the
initial planning of this study [32]. Modifications were made to the survey form, to shorten the
duration of the evaluation and align it more closely with the aim of the evaluation, to incorpo-
rate additional signs and symptoms in line with IMCI 2014 guidelines such as duration of
symptoms and the symptomatic assessment of lethargic or unconscious, and additions used by
similar evaluations conducted previously [17].
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 5 / 18
Initially, antibiotic prescription was considered a co-primary outcome alongside the cur-
rent primary outcome (proportion of 14 major IMCI symptoms and signs) but was later
reclassified as a secondary outcome. We made this change to focus the analysis on quality of
care, given that antibiotic prescription was already the primary outcome of the large longitudi-
nal cluster randomized trial [24].
Sample size
The original sample size calculation was based on the previous co-primary outcome of antibi-
otic prescription. To detect a 25% absolute decrease in mean antibiotic prescription from a
baseline of 50%, using an intraclass correlation coefficient (ICC) of 0.10 and an alpha of 0.05, a
sample size of 25 patients in 9 clusters (health facilities) per arm was required to have 80%
power. The ICC was based on studies evaluating prescription variations among different
health care facilities/practices, ranging from 0.07 to 0.10 [33–36].
Expecting a high variability in baseline values of symptoms and signs assessed by a clinician
and between clinicians [8,13,17,37], the above sample size would have 67–93% power to detect
a 30% absolute increase in the assessment of major IMCI symptoms and signs, considering a
baseline value of 40–60%, an ICC of 0.15–0.25, and an alpha of 0.05.
Randomization
Within the parent trial, health facilities were randomized 1:1 by an independent statistician,
stratified by monthly attendance, type of health facility (dispensary or health center), region,
and council [24]. For the present ancillary study, another independent statistician sampled 18/
40 facilities to be included. This included 8/8 health centers (4 intervention, and 4 control),
and 10/32 dispensaries. Among the 32 dispensaries, 10 were randomly sampled, stratified by
study arm and region (following the same 3:2 ratio in favor of the Morogoro region as done in
the parent trial). Due to the nature of the intervention, it was not feasible to blind the health-
care providers, patients, study implementers, or external clinical research assistants (observers)
to the intervention.
A convenience sampling was employed, whereby the external clinical researcher observed
all eligible consultations while present at the health facility during normal standard working
hours (Monday to Friday, 8:00 to 15:00).
Statistical methods
All analyses were performed using an intention-to-treat approach, i.e. all children with a
recorded outcome were included in the analysis regardless if the intervention, ePOCT+, was
used or not. The eCRF was designed to prevent missing data, as such all data was complete. All
analyses were performed using a clustered-level analysis approach instead of an individual-
level analysis due to the small number of clusters included [38,39]. Such an approach has been
shown to be robust in cluster randomized trials with less than 30 clusters, with cluster sizes of
>10, when assessing outcomes with a prevalence of >10%, and performs best when cluster
sizes are similar, and in case of high ICC [38,39]. The analysis was performed using a two-
stage approach as outlined by Hayes et al [39,40], to adjust for both the cluster-level and indi-
vidual-level covariates. In the first stage, we used a logistic regression model for binary out-
comes and a linear regression model for continuous outcomes adjusting for covariates and
ignoring clustering and trial arm. Cluster-level residuals were then calculated for each cluster.
In the second stage, the residuals were compared to estimate risk ratios for the binary out-
comes and mean risk difference for the continuous outcome (including the primary outcome)
between study arms. Pre-specified cluster-level covariates were the type of health facility,
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 6 / 18
council, healthcare worker cadre and healthcare worker years of experience. Pre-specified
individual-level covariates of patients were age and sex. Covariates were pre-specified based on
their potential influence on the study outcomes. All analyses were performed using Stata v16,
v17 and v18 [41].
Ethics
Written informed consent was obtained from all parents or guardians of participants when
attending the participating health facility during the enrollment period. We also requested,
written informed consent from all healthcare providers for which their consultations were
observed during this ancillary study. Ethical approvals were granted from the Ifakara Health
Institute (IHI/IRB/No: 11–2020), the Mbeya Medical Research Ethics Committee (SZEC-2439/
R.A/V.1/65), the National Institute for Medical Research Ethics Committee (NIMR/HQ/R.8a/
Vol. IX/3486 and NIMR/HQ/R.8a/Vol. IX/3583) in Tanzania and from the cantonal ethics
review board of Vaud (CER-VD 2020–02800) in Switzerland.
Results
Baseline characteristics of health facilities, healthcare providers, and
patients
Between 23 March 2022 to 3 June 2022, 225 consultations were observed in 9 intervention
facilities, and 225 consultations in 9 control facilities (Fig 2). The type of health facility, urban/
rural localization, and region were well distributed between both study arms (Table 1). A total
of 17 healthcare providers saw patients in the control arm, and 22 in the intervention arm dur-
ing the study. Distribution of sex, age, working experience and cadre of healthcare providers
were similar in both study arms. The number of healthcare providers with less than 3 years of
experience was slightly higher in the control arm. Among the included patients, there were
slightly more female patients and median age was slightly higher in the intervention arm
Fig 2. Health facility and patient flow diagram.
https://doi.org/10.1371/journal.pdig.0000694.g002
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 7 / 18
compared to the control. Within the intervention arm, ePOCT+ was used throughout the
whole consultation in 213/225 (95%) of consultations, partially used in 5/225 (2%), used after
the consultation in 6/225 (3%), and not used at all in 1/225 (0.4%) of consultations.
Assessment of symptoms and signs, and counseling
The primary outcome of mean proportion of major IMCI symptoms and signs assessed was
higher by an adjusted difference of 15.1% (95% confidence interval [CI] 4.8% to 25.4%), p-
value 0.007) in intervention health facilities (mean of 46.4%, range 7.7% to 91.7%) compared
to control health facilities (mean of 26.3%, range 0% to 66.7%) (Table 2 and Fig 3). Weight,
mid-upper arm circumference (MUAC) and pallor were the only individual assessments
Table 1. Characteristics of health facilities, healthcare providers, and patients.
Characteristics of health facilities Control (N = 9) Intervention (N = 9)
Type of facility Dispensary n (%) 5 (56%) 5 (56%)
Health Center n (%) 4 (44%) 4 (44%)
Geographical distribution Urban n (%) 4 (44%) 3 (33%)
Rural n (%) 5 (56%) 6 (67%)
Region Mbeya n (%) 4 (44%) 4 (44%)
Morogoro n (%) 5 (56%) 5 (56%)
Characteristics of healthcare providers Control (N = 17) Intervention(N = 22)
Sex Female n (%) 9 (53%) 9 (41%)
Male n (%) 8 (47%) 13 (59%)
Age Years (Median; IQR) 32 (28,36) 34 (30,38)
20- <30 years n (%) 5 (29%) 4 (18%)
30- <40 years n (%) 9 (53%) 14 (64%)
40- <50 years n (%) 2 (12%) 3 (14%)
50- <60 years n (%) 1 (6%) 1 (5%)
Experience*<= 3 years 7 (41%) 6 (27%)
3–5 years 3 (18%) 6 (27%)
5–10 years 3 (18%) 6 (27%)
>10 years 4 (24%) 4 (18%)
Cadre Medical Doctor 3 (18%) 4 (18%)
Assistant Medical Officer 1 (6%) 1 (5%)
Clinical Officer 9 (53%) 10 (46%)
Clinical Assistant 0 (0%) 1 (5%)
Registered or Enrolled Nurse 4 (24%) 4 (18%)
Medical Attendant 0 (0%) 2 (9%)
Characteristics of patients Control (N = 225) Intervention (N = 225)
Sex Female n (%) 108 (48%) 123 (55%)
Male n (%) 117 (52%) 102 (45%)
Age Months (Median; IQR) 14 (7,28) 19 (9,31)
2–11 months n (%) 98 (44%) 69 (31%)
12–23 months n (%) 54 (24%) 72 (32%)
24–35 months n (%) 33 (15%) 36 (16%)
36–47 months n (%) 24 (11%) 26 (12%)
48–59 months n (%) 16 (7%) 22 (10%)
IQR: Interquartile range
*Experience: Years of working experience
https://doi.org/10.1371/journal.pdig.0000694.t001
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 8 / 18
among the pre-identified major IMCI symptoms and signs that showed a statistically signifi-
cant difference in the adjusted risk ratio (Table 2 and Fig 4). Among other symptoms and
signs assessed, there was a statistically significant difference in the proportion of patients for
which the duration of cough, duration of diarrhea, mother’s HIV status were assessed, and the
proportion of children who were undressed for the physical examination (Table 3). There was
no statistical difference in the proportion of counseling topics covered by healthcare providers
between study arms (Table 4). For most outcomes the intraclass correlation coefficient (ICC)
was relatively high suggesting high variability in adherence to IMCI between health facilities.
For the primary outcome of major IMCI symptoms and signs assessed, the ICC was higher in
intervention health facilities (0.608) compared to control health facilities (0.354). This differ-
ence can also be seen when visualizing the mean proportion of major IMCI symptoms and
signs assessed from each health facility in a scatterplot (Fig 3).
Antibiotic and antimalarial prescription
Antibiotic prescription as observed by the external clinical researcher was lower in interven-
tion health facilities compared to control health facilities with an adjusted risk ratio of 0.5, 95%
CI 0.4 to 0.7 p-value <0.001 (Table 5 and Fig 5). The documented antibiotic prescription by
healthcare providers (in the ePOCT+ tool for intervention facilities, and eCRF in control
Table 2. Major IMCI Symptoms and Signs Assessed.
Primary Outcome Control, mean %
(range)
Intervention, mean %
(range)
Intraclass correlation
coefficient
Adjusted mean difference with 95%
CI
a
p-value
Primary outcome
Major IMCI Symptoms and
Signs
26.3% (0%; 66.7%) 46.4% (7.7%; 91.7%) 0.652 15.1% (4.8%; 25.4%) 0.007
Secondary Outcome Control, n/N
a
(%) Intervention, n/N
b
(%))Intraclass correlation
coefficient
Adjusted risk ratio with 95% CI
a
p-
value
IMCI Symptoms assessed:
Convulsions in this illness
b
16/225 (7.1%) 75/225 (33.3%) 0.442 2.1 (0.6; 7.0) 0.208
Unable to drink or
breastfeed
b
47/225 (20.9%) 107/225 (47.6%) 0.275 2.3 (0.8; 6.4) 0.109
Vomiting everything
b
41/225 (18.2%) 81/225 (36.0%) 0.524 1.7 (0.5; 5.9) 0.363
Fever 196/225 (87.1%) 204/225 (90.7%) 0.060 1.0 (1.0; 1.14) 0.342
Cough or difficulty breathing 188/225 (83.6%) 189/225 (84.0%) 0.098 1.0 (0.9; 1.2) 0.847
Diarrhea 111/225 (49.3%) 130/225 (57.8% 0.289 1.1 (0.5; 2.4) 0.872
Ear problem 11/225 (4.9%) 37/225 (16.4%) 0.504 1.0 (0.3; 3.5) 0.951
IMCI Signs assessed
Weight 38/225 (16.9%) 128/225 (56.9%) 0.582 4.9 (1.9; 12.9) 0.004
Height 1/225 (0.4%) 3/225 (1.3%) 0.059 0.3 (0.1; 2.1) 0.225
MUAC 4/195 (2.1%) 131/202 (64.9%) 0.719 5.5 (1.7; 17.6) 0.008
Temperature 95/225 (42.2%) 148/225 (65.8%) 0.586 1.9 (0.6; 5.6) 0.227
Pallor 13/225 (5.8%) 72/225 (32.0%) 0.324 4.1 (1.6; 10.4) 0.005
Respiratory rate 15/182 (8.2%) 47/164 (28.7%) 0.280 1.9 (0.6; 6.1) 0.230
Skin turgor 4/153 (2.6%) 16/141 (11.4%) 0.142 2.1 (0.9; 5.0) 0.087
CI: Confidence interval; MUAC: Mid-upper arm circumference
a
The differences and relative risk were adjusted by type of health facility, council, healthcare worker cadre, healthcare worker years of experience, patient age and sex
b
Denominator based on patients for which this sign is clinically indicated to measure, i.e. for MUAC only children age 6 months and above, for respiratory rate only
patients for which cough or difficulty is present or not asked, etc
c
IMCI Danger sign
https://doi.org/10.1371/journal.pdig.0000694.t002
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 9 / 18
Fig 3. Scatter-plot of the mean proportion of the major IMCI symptoms and signs assessed.
https://doi.org/10.1371/journal.pdig.0000694.g003
Fig 4. Proportion of individual major IMCI symptoms and signs assessed.
https://doi.org/10.1371/journal.pdig.0000694.g004
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 10 / 18
facilities) during the same 5-week period collected by the external clinical researchers in the
same health facilities of the present analysis, was slightly lower than that observed by the exter-
nal clinical researchers. The adjusted risk ratio remained nonetheless similar, 0.4, 95% CI 0.2
to 0.7, p-value 0.005.
81.9% of antibiotic prescriptions were appropriate in the intervention arm, compared to
51.4% in the control arm, adjusted relative risk of 1.5, 95% CI 1.2–1.5, p-value 0.003. All
patients with malaria appropriately received an antimalarial treatment, and no patient without
malaria received an antimalarial in both study arms.
Table 3. Other Symptoms and Signs Assessed.
Symptoms & signs assessed Control, n/N (%) Intervention, n/N (%) Intraclass correlation coefficient Adjusted risk ratio with 95% CI
a
p-value
Lethargic or Unconscious
b
8/225 (3.6%) 14/225 (6.2%) 0.086 1.4 (0.6, 3.3) 0.476
Duration of fever
c
96/147 (65.3%) 141/166 (84.9%) 0.176 1.6 (0.9, 2.6) 0.086
Duration of cough
c
87/145 (60.0%) 110/128 (85.9%) 0.103 1.7 (1.1, 2.6) 0.021
Duration of diarrhea
c
28/39 (71.8%) 43/46 (93.5%) 0.107 1.3 (1.0, 1.6) 0.036
Mother’s HIV status 5/225 (2.2%) 162/225 (72.0%) 0.701 11.5 (4.1, 32.5) 0.002
Tuberculosis household contact 2/225 (0.9%) 60/225 (26.7%) 0.422 1.9 (0.5, 7.7) 0.361
Neck stiffness 2/147 (1.4%) 2/166 (1.2%) 0.010 n/a
d
Felt behind ear
c
1/4 (25.0%) 2/3 (66.7%) n/a n/a
d
Looked in mouth 4/225 (1.8%) 20/225 (8.9%) 0.305 1.9 (0.5, 7.7) 0.363
Pulse oximetry n/a 24/164 (14.6%) 0.717 n/a
Lung auscultation 26/183 (14.2%) 15/164 (9.2%) 0.235 1.6 (0.5, 5.1) 0.373
Undressed the child 43/225 (19.1%) 99/225 (44.0%) 0.379 3.4 (1.4; 8.3) 0.011
Checked health card 68/225 (30.2%) 89/225 (39.6%) 0.552 1.5 (0.5, 5.0) 0.465
CI: Confidence interval; HIV: Human Immunodeficiency Virus; n/a: not applicable
a
The risk ratio was adjusted by type of health facility, council, healthcare worker cadre, healthcare worker years of experience, patient age and sex
b
IMCI Danger sign
c
Denominator based on patients for which clinically relevant to assess, i.e. duration of fever in those with reported fever; duration of cough/difficulty breathing, pulse
oximetry or lung auscultation in those with cough or difficult breathing; duration of diarrhea in those with diarrhea; felt behind ear in those with an ear problem; neck
stiffness in those with fever
d
Adjusted risk ratio not calculated when fewer than 5 events
https://doi.org/10.1371/journal.pdig.0000694.t003
Table 4. Counseling.
Counseling topic Control, n/N (%) Intervention, n/N (%) Intraclass correlation
coefficient
Adjusted risk ratio with 95%
CI
a
p-value
Informed diagnosis 92/225 (40.9%) 105/225 (46.7%) 0.454 1.4 (0.5, 4.0) 0.557
Feeding habit when not ill 40/225 (17.8%) 33/225 (14.7%) 0.427 0.7 (0.2, 2.7) 0.609
Feeding when not ill 32/225 (14.2%) 27/225 (12.0%) 0.258 0.7 (0.2, 1.8) 0.391
Extra fluids during current illness 12/225 (5.3%) 21/225 (9.3%) 0.198 1.2 (0.4, 4.0) 0.748
Continue feeding and breastfeeding when
ill
25/225 (11.1%) 33/225 (14.7%) 0.292 1.0 (0.3, 3.1) 0.987
Danger signs to return to health facility 13/225 (5.8%) 17/225 (7.6%) 0.152 0.6 (0.2, 1.3) 0.182
Discussed growth chart 14/225 (6.2%) 15/225 (6.7%) 0.206 1.1 (0.3, 4.0) 0.826
Discuss follow up visit 18/225 (8.0%) 8/225 (3.6%) 0.052 0.9 (0.4, 2.1) 0.703
Opportunity to ask questions 87/225 (38.7%) 122/225 (54.2%) 0.717 1.0 (0.2, 4.9) 0.998
CI: Confidence interval
a
The relative risk was adjusted by type of health facility, council, healthcare worker cadre, healthcare worker years of experience, patient age and sex
https://doi.org/10.1371/journal.pdig.0000694.t004
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 11 / 18
Discussion
This cluster randomized controlled trial in Tanzania found that the use of ePOCT+, a digital
clinical decision support algorithm, for the management of sick children aged 2–59 months
old, moderately increased (by 15%) the mean proportion of major IMCI symptoms and signs
assessed by healthcare providers in primary care level health facilities. The overall proportion
of IMCI symptoms and signs assessed at each individual consultation however remained low.
The overall increase in the mean proportion of major IMCI symptom and signs assessed by
healthcare providers aligns with previous studies in Tanzania [13], Afghanistan [37], Nigeria
[17], and Burkina Faso [16], but in contrast to findings in South Africa [42], which did not
find improvements in the assessment of IMCI symptoms and signs. While these findings sug-
gest that digital clinical decision support algorithms can positively enhance quality of care for
Table 5. Antibiotic prescription and appropriate antibiotic and antimalarial prescription.
Control Intervention Intraclass correlation
coefficient
Adjusted risk ratio with
95% CI
p-value
Antibiotic prescription as observed by external clinical
researchers
172/225
(76.4%)
84/225 (37.3%) 0.146 0.5 (0.4, 0.7) <0.001
Antibiotic prescription as reported in ePOCT+ or eCRF by
healthcare providers
ab
162/239
(67.8%)
90/294 (30.6%) 0.214 0.4 (0.2, 0.7) 0.005
Appropriate antibiotic prescription 112/218
(51.4%)
158/193
(81.9%)
0.165 1.5 (1.2; 1.8) 0.003
Appropriate antimalarial prescription 2/2 (100%) 30/30 (100%) n/a n/a n/a
a
Antibiotic prescription among new cases in patients aged 2–59 months, where antibiotic prescription was documented during the same days and same health facilities
as the ancillary study
b
Analysis performed using the same model including the same individual and cluster level adjustment factors, except for those related to the healthcare providers
(education, and cadre)
https://doi.org/10.1371/journal.pdig.0000694.t005
Fig 5. Reported and observed antibiotic prescription and appropriateness. Panel A presents the antibiotic
prescription as observed by the external clinical researchers in 225 consultations in the control arm and 225
consultations in the intervention arm. It also presents the reported antibiotic prescription as documented by the
healthcare providers in the ePOCT+ digital health tool among 239 new consultations in patients aged 2–59 months in
intervention health facilities, and the documented antibiotic prescription within the eCRF in the control health
facilities during the same days and same health facilities when the external clinical researchers were present. Panel B
presents the appropriateness of antibiotic prescription according to the diagnoses documented or stated by the
healthcare providers in the 225 control arm consultations and 225 intervention arm consultations.
https://doi.org/10.1371/journal.pdig.0000694.g005
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 12 / 18
sick children, there is much room for improvement. Indeed, the proportion of most individual
IMCI symptoms and signs assessed in both control and intervention arms was low, much
lower than in previous studies, and many symptoms and signs were not assessed more fre-
quently in intervention health facilities. Differences compared to other studies, may be
explained by the more pragmatic nature of the study compared to other studies conducted in
more controlled settings (shorter pilot studies) where the Hawthorne effect may have a greater
influence on healthcare provider behavior [13]. In addition, our study found much poorer
adherence to IMCI in the control arm compared to most other similar studies; which could
partly be explained by differences in the setting and healthcare providers, notably in the fre-
quency and type of IMCI training provided to healthcare providers [16,17]. The low propor-
tion of children assessed for danger signs in order to identify children at highest risk of
mortality is most concerning (33% for convulsions, 48% for unable to drink or breastfeed, and
36% for vomiting everything in the intervention arm). Respiratory rate was also infrequently
assessed (29% in intervention arm), despite being an essential sign to distinguish children with
cough or difficulty breathing requiring antibiotics or not [30].
The statistically significant increase in the assessment of weight (aRR 4.9 [95% CI 1.9 to
12.9]), and MUAC (aRR 5.5 [95% CI 1.7 to 17.6]) is noteworthy, as they are critical anthropo-
metric measures to identify children with malnutrition, a condition that contributes signifi-
cantly to childhood morbidity and mortality. Systematically measuring weight and MUAC to
identify and manage severe malnutrition can indeed improve clinical outcomes as well as the
long-term health status of children [43]. The low proportion of children assessed for height/
length reflects the difficulty and constraints of this measurement in particular [44], and the
impact of not requiring the measurement to be mandatory within the digital tool.
While improved adherence to the assessment of IMCI symptoms and signs would likely be
beneficial, translation to improved clinical outcomes should not automatically be assumed.
Healthcare providers often integrate a number of clinical cues that may allow them to distin-
guish which child would truly require danger signs to be assessed, or respiratory rate to be
measured. For example, a 2 year old child presenting to a primary care health facility smiling
and playing in the consultation room, with complaints of cough and runny nose for the past 2
days without difficulty breathing or fever, is unlikely to have danger signs and very often not
have fast breathing or chest indrawing. This raises the important question on how to best
assess quality of care, and whether it can be done without assessment of clinical outcome.
The large variation in mean proportion of major IMCI symptoms and signs between inter-
vention health facilities provide clues to higher potential benefits of the intervention. The three
best health facilities have a mean score of 60% or above, and the three worst below 30%. While
clinical decision support algorithms may help improve knowledge and information on what
symptoms and signs to assess, it does not completely address the other barriers and bad habits
linked to poor adherence to IMCI [9,10]. As with many complex health interventions, imple-
mentation of new interventions, and or guidelines may often not succeed with training alone,
instead concomitant and meaningful mentorship must accompany it [45]. Indeed training,
mentorship, and dashboards integrating benchmarking were part of the current intervention
package, however these supportive interventions were not targeted towards assuring adher-
ence to the assessment of many IMCI symptoms and signs but rather targeted on antibiotic
stewardship, and overall uptake on the use of the tools. Adaptations to these supportive tools
to target specific IMCI quality of care measures may help [46–48]. Further qualitative investi-
gations are underway to better understand healthcare provider perspectives on barriers in
adhering to ePOCT+ and the IMCI chartbook.
The study also revealed a two-fold reduction in antibiotic prescriptions (adjusted relative
risk 0.5, 95% CI 0.4 to 0.7), and 50% improvement in the appropriate use of antibiotics in
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 13 / 18
health facilities using ePOCT+ (adjusted relative risk 1.5 (95% CI 1.2 to 1.8). These are critical
findings given the global concern for bacterial antimicrobial resistance [49], and validating the
results found in the parent trial [24]. Similar reductions were found in the parent trial using
the intention-to-treat results over the full 11 month trial period (adjusted relative risk 0.6 (95%
CI 0.5 to 0.6) [24], and in the same 5-week period as the present trial (adjusted relative risk 0.4,
95% CI 0.2 to 0.7). However, the documented antibiotic prescription as observed by external
clinical researchers was slightly higher than that documented by the healthcare providers in
ePOCT+ (intervention health facilities) and the eCRF (control health facilities), suggesting
that some healthcare providers may under report antibiotic prescription in ePOCT+ and the
eCRF.
There were several limitations to this study. First, the Hawthorne effect, the presence of an
external clinical researcher observing the consultation may have influenced the healthcare pro-
vider’s practice in both study arms. Indeed the uptake of the ePOCT+ tool in this study was
higher compared to the parent trial (95% versus 76%), likely due to the presence of the
researchers. Despite this, adherence to the IMCI chartbook was relatively low, and substan-
tially lower compared to other studies, suggesting that the Hawthorne effect may not have had
such a big impact, and the desirability bias minimal. Second, sample size; while the study was
sufficiently powered for the primary outcome, interpretation of the secondary outcomes
would have benefited from a higher sample size. Indeed many secondary outcomes did not
show statistical significance despite relatively high effect size, likely due to the higher than
expected heterogeneity between health facilities as indicated by the high ICC. Third, the inter-
vention package included not only the digital tool, but also mentorship and benchmarking
quality of care dashboards. It is thus not possible to understand what part, and to what extent
the intervention package impacted quality of care. Finally as discussed in previous paragraphs,
adherence to IMCI is an imperfect proxy for the measurement of quality of care.
In conclusion, a digital clinical decision support algorithm package can help improve qual-
ity of care, however adherence to IMCI remained low for many symptoms and signs in a close
to real world assessment. Further efforts including innovative approaches to improve quality
of care are highly needed. The implementation of multiple interventions, such as the develop-
ment and improvement of supportive mentorship of clinicians, better healthcare provider
incentives, task-shifting, ongoing training and performance accountability may help address
the many barriers to quality of care.
Supporting information
S1 Checklist. CONSORT Checklist.
(DOCX)
S1 File. DYNAMIC Tanzania study protocol.
(PDF)
S2 File. Statistical analysis plan.
(PDF)
Acknowledgments
We would like to first thank all the participating healthcare providers, patients and caregivers.
We acknowledge the contributions of the research assistants at the Ifakara Health Institute and
Mbeya Medical Research Centre–National Institute of Medical Research, who assisted in the
data collection, the Information Technology staff at Unisante
´(Sylvain Schaufelberger, Greg
Martin), and staff at Wavemind (Emmanuel Barchichat, Alain Fresco, and Quentin Girard)
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 14 / 18
for their work on the medAL-suite during the study, and Community Health Management
Team members in the 5 participating councils in Tanzania for their collaboration in imple-
menting the study. We acknowledge researchers of the Tools of Integrated Management of
Childhood Illness and DYNAMIC Rwanda project for their contributions to ePOCT+ and the
many common research activities (Dr Fenella Beynon, Dr Lena Matata, Dr Helene Langet, Dr
Ludovico Cobuccio, Dr Victor Rwandacu, Dr Robert Moshiro). We would also like to
acknowledge Janet Urquhart-Ducharme for her assistance in proofreading a version of this
manuscript. We would like to acknowledge Dr Irene Masanja for her work on the initial devel-
opment of the study, who regrettably passed away before the start of the study. Dr Godfrey
Kavishe passed away before the submission of the final version of this manuscript. Rainer Tan
accepts responsibility for the integrity and validity of the data collected and analyzed.
Author Contributions
Conceptualization: Rainer Tan, Godfrey Kavishe, Alexandra V. Kulinkina, Lameck B.
Luwanda, Kristina Keitel, Marie-Annick Le Pogam, Honorati Masanja, Vale
´rie
D’Acremont.
Data curation: Rainer Tan, Godfrey Kavishe, Sabine Renggli.
Formal analysis: Rainer Tan, Godfrey Kavishe.
Funding acquisition: Nyanda Ntinginya, Honorati Masanja, Vale
´rie D’Acremont.
Methodology: Rainer Tan, Godfrey Kavishe, Marie-Annick Le Pogam.
Project administration: Godfrey Kavishe, Alexandra V. Kulinkina, Sabine Renggli, Lameck B.
Luwanda, Chacha Mangu, Geofrey Ashery, Nyanda Ntinginya, Honorati Masanja.
Software: Rainer Tan, Godfrey Kavishe, Sabine Renggli, Ibrahim Evans Mtebene, Peter Agrea.
Supervision: Rainer Tan, Godfrey Kavishe, Alexandra V. Kulinkina, Sabine Renggli, Lameck
B. Luwanda, Chacha Mangu, Geofrey Ashery, Margaret Jorram, Humphrey Mhagama,
Marie-Annick Le Pogam, Nyanda Ntinginya, Honorati Masanja, Vale
´rie D’Acremont.
Writing – original draft: Rainer Tan, Godfrey Kavishe.
Writing – review & editing: Rainer Tan, Godfrey Kavishe, Alexandra V. Kulinkina, Sabine
Renggli, Lameck B. Luwanda, Chacha Mangu, Geofrey Ashery, Margaret Jorram, Ibrahim
Evans Mtebene, Peter Agrea, Humphrey Mhagama, Kristina Keitel, Marie-Annick Le
Pogam, Nyanda Ntinginya, Honorati Masanja, Vale
´rie D’Acremont.
References
1. Kruk ME, Gage AD, Joseph NT, Danaei G, Garcı
´a-Saiso
´S, Salomon JA. Mortality due to low-quality
health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137
countries. The Lancet. 2018; 392(10160):2203–12. https://doi.org/10.1016/S0140-6736(18)31668-4
PMID: 30195398
2. Di Giorgio L, Evans DK, Lindelow M, Nguyen SN, Svensson J, Wane W, et al. Analysis of clinical knowl-
edge, absenteeism and availability of resources for maternal and child health: a cross-sectional quality
of care study in 10 African countries. BMJ Glob Health. 2020; 5(12). Epub 2020/12/24. https://doi.org/
10.1136/bmjgh-2020-003377 PMID: 33355259; PubMed Central PMCID: PMC7751199.
3. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health sys-
tems in the Sustainable Development Goals era: time for a revolution. The Lancet Global Health. 2018;
6(11):e1196–e252. https://doi.org/10.1016/S2214-109X(18)30386-3 PMID: 30196093
4. World Health Organization. World Health Report, Working Together for Health. Geneva: World Health
Organization; 2006.
5. World Health Organization. Integrated Management of Childhood Ilness Chartbook. Geneva: 2014.
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 15 / 18
6. Boschi-Pinto C, Labadie G, Dilip TR, Oliphant N.L. Dalglish S, Aboubaker S, et al. Global implementa-
tion survey of Integrated Management of Childhood Illness (IMCI): 20 years on. BMJ open. 2018; 8(7):
e019079. https://doi.org/10.1136/bmjopen-2017-019079 PMID: 30061428
7. Gera T, Shah D, Garner P, Richardson M, Sachdev HS. Integrated management of childhood illness
(IMCI) strategy for children under five. Cochrane Database Syst Rev. 2016;(6):Cd010123. Epub 2016/
07/06. https://doi.org/10.1002/14651858.CD010123.pub2 PMID: 27378094; PubMed Central PMCID:
PMC4943011.
8. Kru¨ger C, Heinzel-Gutenbrunner M, Ali M. Adherence to the integrated management of childhood ill-
ness guidelines in Namibia, Kenya, Tanzania and Uganda: evidence from the national service provision
assessment surveys. BMC Health Serv Res. 2017; 17(1):822. Epub 2017/12/15. https://doi.org/10.
1186/s12913-017-2781-3 PMID: 29237494; PubMed Central PMCID: PMC5729502.
9. Kiplagat A, Musto R, Mwizamholya D, Morona D. Factors influencing the implementation of integrated
management of childhood illness (IMCI) by healthcare workers at public health centers & dispensaries
in Mwanza, Tanzania. BMC public health. 2014; 14:277. Epub 2014/03/29. https://doi.org/10.1186/
1471-2458-14-277 PMID: 24666561; PubMed Central PMCID: PMC3987128.
10. Lange S, Mwisongo A, Mæstad O. Why don’t clinicians adhere more consistently to guidelines for the
Integrated Management of Childhood Illness (IMCI)? Soc Sci Med. 2014; 104:56–63. Epub 2014/03/04.
https://doi.org/10.1016/j.socscimed.2013.12.020 PMID: 24581062.
11. Beynon F, Gue
´rin F, Lampariello R, Schmitz T, Tan R, Ratanaprayul N, et al. Digitalizing Clinical Guide-
lines: Experiences in the Development of Clinical Decision Support Algorithms for Management of
Childhood Illness in Resource-Constrained Settings. Global Health: Science and Practice. 2023.
https://doi.org/10.9745/GHSP-D-22-00439 PMID: 37640492
12. Keitel KD’Acremont V. Electronic clinical decision algorithms for the integrated primary care manage-
ment of febrile children in low-resource settings: review of existing tools. Clinical microbiology and infec-
tion: the official publication of the European Society of Clinical Microbiology and Infectious Diseases.
2018; 24(8):845–55. Epub 2018/04/24. https://doi.org/10.1016/j.cmi.2018.04.014 PMID: 29684634.
13. Mitchell M, Hedt-Gauthier BL, Msellemu D, Nkaka M, Lesh N. Using electronic technology to improve
clinical care—results from a before-after cluster trial to evaluate assessment and classification of sick
children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC
medical informatics and decision making. 2013; 13:95. Epub 2013/08/29. https://doi.org/10.1186/1472-
6947-13-95 PMID: 23981292; PubMed Central PMCID: PMC3766002.
14. Rambaud-Althaus C, Shao A, Samaka J, Swai N, Perri S, Kahama-Maro J, et al. Performance of Health
Workers Using an Electronic Algorithm for the Management of Childhood Illness in Tanzania: A Pilot
Implementation Study. The American journal of tropical medicine and hygiene. 2017; 96(1):249–57.
Epub 2017/01/13. https://doi.org/10.4269/ajtmh.15-0395 PMID: 28077751; PubMed Central PMCID:
PMC5239703.
15. Horwood C, Haskins L, Mapumulo S, Connolly C, Luthuli S, Jensen C, et al. Electronic Integrated Man-
agement of Childhood Illness (eIMCI): a randomized controlled trial to evaluate an electronic clinical
decision-making support system for management of sick children in primary health care facilities in
South Africa. BMC Health Services Research. 2024; 24(1):177. https://doi.org/10.1186/s12913-024-
10547-6 PMID: 38331824
16. Sarrassat S, Lewis JJ, Some AS, Somda S, Cousens S, Blanchet K. An Integrated eDiagnosis
Approach (IeDA) versus standard IMCI for assessing and managing childhood illness in Burkina Faso:
a stepped-wedge cluster randomised trial. BMC health services research. 2021; 21(1):354–. https://doi.
org/10.1186/s12913-021-06317-3 PMID: 33863326.
17. Bernasconi A, Crabbe
´F, Adedeji AM, Bello A, Schmitz T, Landi M, et al. Results from one-year use of
an electronic Clinical Decision Support System in a post-conflict context: An implementation research.
PLoS One. 2019; 14(12):e0225634. Epub 2019/12/04. https://doi.org/10.1371/journal.pone.0225634
PMID: 31790448; PubMed Central PMCID: PMC6886837.
18. Bernasconi A, Crabbe
´F, Rossi R, Qani I, Vanobberghen A, Raab M, et al. The ALMANACH Project:
Preliminary results and potentiality from Afghanistan. International journal of medical informatics. 2018;
114:130–5. https://doi.org/10.1016/j.ijmedinf.2017.12.021 PMID: 29330009
19. Beynon F, Salzmann T, Faye PM, Thiongane A, Ndiaye O, Luwanda LB, et al. Paediatric digital clinical
decision support for global health. Revue Medicale Suisse. 2023; 19(836):1398–403.
20. Agarwal S, Glenton C, Tamrat T, Henschke N, Maayan N, Fønhus MS, et al. Decision-support tools via
mobile devices to improve quality of care in primary healthcare settings. Cochrane Database Syst Rev.
2021; 7(7):Cd012944. Epub 2021/07/28. https://doi.org/10.1002/14651858.CD012944.pub2 PMID:
34314020; PubMed Central PMCID: PMC8406991.
21. Rambaud-Althaus C, Shao AF, Kahama-Maro J, Genton B, d’Acremont V. Managing the Sick Child in
the Era of Declining Malaria Transmission: Development of ALMANACH, an Electronic Algorithm for
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 16 / 18
Appropriate Use of Antimicrobials. PLoS One. 2015; 10(7):e0127674. Epub 2015/07/15. https://doi.org/
10.1371/journal.pone.0127674 PMID: 26161753; PubMed Central PMCID: PMC4498609.
22. Keitel K, Kagoro F, Samaka J, Masimba J, Said Z, Temba H, et al. A novel electronic algorithm using
host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-
POCT): A randomized, controlled non-inferiority trial. PLoS medicine. 2017; 14(10):e1002411. Epub
2017/10/24. https://doi.org/10.1371/journal.pmed.1002411 PMID: 29059253; PubMed Central PMCID:
PMC5653205.
23. Tan R, Cobuccio L, Beynon F, Levine GA, Vaezipour N, Luwanda LB, et al. ePOCT+ and the medAL-
suite: Development of an electronic clinical decision support algorithm and digital platform for pediatric
outpatients in low- and middle-income countries. PLOS Digital Health. 2023; 2(1):e0000170. https://doi.
org/10.1371/journal.pdig.0000170 PMID: 36812607
24. Tan R, Kavishe G, Luwanda LB, Kulinkina AV, Renggli S, Mangu C, et al. A digital health algorithm to
guide antibiotic prescription in pediatric outpatient care: a cluster randomized controlled trial. Nature
Medicine. 2024; 30(1):76–84. https://doi.org/10.1038/s41591-023-02633-9 PMID: 38110580
25. Shao AF, Rambaud-Althaus C, Samaka J, Faustine AF, Perri-Moore S, Swai N, et al. New Algorithm for
Managing Childhood Illness Using Mobile Technology (ALMANACH): A Controlled Non-Inferiority
Study on Clinical Outcome and Antibiotic Use in Tanzania. PLoS One. 2015; 10(7):e0132316. Epub
2015/07/15. https://doi.org/10.1371/journal.pone.0132316 PMID: 26161535; PubMed Central PMCID:
PMC4498627.
26. Tanzania Commission for AIDS (TACAIDS), (ZAC) ZAC. Tanzania HIV Impact Survey (THIS) 2016–
2017: Final Report. Dar es Salaam, Tanzania 2018.
27. Amu H, Dickson KS, Kumi-Kyereme A, Darteh EKM. Understanding variations in health insurance cov-
erage in Ghana, Kenya, Nigeria, and Tanzania: Evidence from demographic and health surveys. PLoS
One. 2018; 13(8):e0201833. https://doi.org/10.1371/journal.pone.0201833 PMID: 30080875
28. Mullan F, Frehywot S. Non-physician clinicians in 47 sub-Saharan African countries. The Lancet. 2007;
370(9605):2158–63. https://doi.org/10.1016/S0140-6736(07)60785-5 PMID: 17574662
29. Cobuccio LG, Faivre V, Tan R, Vonlanthen A, Beynon F, Barchichat E, et al. medAL-suite: a software
solution for creating and deploying complex clinical decision support algorithms. https://zenodo.org/
records/13889290
30. World Health Organization. Integrated Management of Childhood Ilness Chart Booklet. Geneva: 2014
2014. Report No.
31. World Health Organization. Pocket Book of Hospital Care for Children: Guidelines for the Management
of Common Childhood Illnesses. 2nd ed. Geneva: World Health Organization; 2013.
32. Demographic and Health Suveys (DHS). Service Provision Assessments (SPA) 2024 [cited 2024
05.02.2024]. Available from: https://dhsprogram.com/Methodology/Survey-Types/SPA.cfm.
33. Gilroy K, Winch PJ, Diawara A, Swedberg E, Thie
´ro F, Kane
´M, et al. Impact of IMCI training and lan-
guage used by provider on quality of counseling provided to parents of sick children in Bougouni District,
Mali. Patient Education and Counseling. 2004; 54(1):35–44. https://doi.org/10.1016/S0738-3991(03)
00189-7 PMID: 15210258
34. Rowe AK, Onikpo F, Lama M, Osterholt DM, Rowe SY, Deming MS. A multifaceted intervention to
improve health worker adherence to integrated management of childhood illness guidelines in Benin.
Am J Public Health. 2009; 99(5):837–46. Epub 2009/03/19. https://doi.org/10.2105/AJPH.2008.134411
PMID: 19299681.
35. Lemiengre MB, Verbakel JY, Colman R, De Burghgraeve T, Buntinx F, Aertgeerts B, et al. Reducing
inappropriate antibiotic prescribing for children in primary care: a cluster randomised controlled trial of
two interventions. The British journal of general practice: the journal of the Royal College of General
Practitioners. 2018; 68(668):e204–e10. Epub 2018/02/12. https://doi.org/10.3399/bjgp18X695033
PMID: 29440016.
36. Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored
interventions to improve the management of urinary tract infections in women and sore throat. BMJ.
2002; 325(7360):367. https://doi.org/10.1136/bmj.325.7360.367 PMID: 12183309
37. Bernasconi A, Crabbe
´F, Raab M, Rossi R. Can the use of digital algorithms improve quality care? An
example from Afghanistan. PLoS One. 2018; 13(11):e0207233–e. https://doi.org/10.1371/journal.pone.
0207233 PMID: 30475833.
38. Thompson JA, Leyrat C, Fielding KL, Hayes RJ. Cluster randomised trials with a binary outcome and a
small number of clusters: comparison of individual and cluster level analysis method. BMC Medical
Research Methodology. 2022; 22(1):1–15.
39. Hayes RJ, Moulton LH. Cluster randomised trials: CRC press; 2017.
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 17 / 18
40. Thompson JA, Leurent B, Nash S, Moulton LH, Hayes RJ. Cluster randomized controlled trial analysis
at the cluster level: The clan command. The Stata Journal. 2023; 23(3):754–73. https://doi.org/10.1177/
1536867X231196294 PMID: 37850046
41. StataCorp. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC.; 2019.
42. Horwood C, Haskins L, Mapumulo S, Connolly C, Luthuli S, Jensen C, et al. Electronic Integrated Man-
agement of Childhood Illness (eIMCI): a randomized controlled trial to evaluate an electronic clinical
decision-making support system for management of sick children in primary health care facilities in
South Africa. 2023. https://doi.org/10.21203/rs.3.rs-2746877/v1
43. Tan R, Kagoro F, Levine GA, Masimba J, Samaka J, Sangu W, et al. Clinical Outcome of Febrile Tanza-
nian Children with Severe Malnutrition Using Anthropometry in Comparison to Clinical Signs. American
Journal of Tropical Medicine and Hygiene. 2020; 102(2):427–35. https://doi.org/10.4269/ajtmh.19-0553
WOS:000512881500035. PMID: 31802732
44. Myatt M, Khara T, Collins S. A Review of Methods to Detect Cases of Severely Malnourished Children
in the Community for Their Admission into Community-Based Therapeutic Care Programs. Food and
Nutrition Bulletin. 2006; 27(3_suppl3):S7–S23. https://doi.org/10.1177/15648265060273S302 PMID:
17076211
45. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of
interventions to improve professional practice. CMAJ: Canadian Medical Association journal = journal
de l’Association medicale canadienne. 1995; 153(10):1423–31. Epub 1995/11/15. PMID: 7585368;
PubMed Central PMCID: PMC1487455.
46. Odendaal WA, Anstey Watkins J, Leon N, Goudge J, Griffiths F, Tomlinson M, et al. Health workers’
perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a
qualitative evidence synthesis. Cochrane Database Syst Rev. 2020; 3(3):Cd011942. Epub 2020/03/28.
https://doi.org/10.1002/14651858.CD011942.pub2 PMID: 32216074; PubMed Central PMCID:
PMC7098082.
47. Kruse C, Betancourt J, Ortiz S, Valdes Luna SM, Bamrah IK, Segovia N. Barriers to the Use of Mobile
Health in Improving Health Outcomes in Developing Countries: Systematic Review. J Med Internet
Res. 2019; 21(10):e13263. Epub 2019/10/09. https://doi.org/10.2196/13263 PMID: 31593543; PubMed
Central PMCID: PMC6811771.
48. Renggli S, Mayumana I, Mboya D, Charles C, Maeda J, Mshana C, et al. Towards improved health ser-
vice quality in Tanzania: An approach to increase efficiency and effectiveness of routine supportive
supervision. PLoS One. 2018; 13(9):e0202735. https://doi.org/10.1371/journal.pone.0202735 PMID:
30192783
49. Murray CJL, Ikuta KS, Sharara F, Swetschinski L, Robles Aguilar G, Gray A, et al. Global burden of bac-
terial antimicrobial resistance in 2019: a systematic analysis. The Lancet. 2022. https://doi.org/10.1016/
S0140-6736(21)02724-0 PMID: 35065702
PLOS DIGITAL HEALTH
DYNAMIC Tanzania QoC cRCT
PLOS Digital Health | https://doi.org/10.1371/journal.pdig.0000694 December 23, 2024 18 / 18
