Validation of a clinical algorithm to identify neonates with severe illness during routine household visits in rural Bangladesh.

Abstract

To validate a clinical algorithm for community health workers (CHWs) during routine household surveillance for neonatal illness in rural Bangladesh.
Surveillance was conducted in the intervention arm of a trial of newborn interventions. CHWs assessed 7587 neonates on postnatal days 0, 2, 5 and 8 and identified neonates with very severe disease (VSD) using an 11-sign algorithm. A nested prospective study was conducted to validate the algorithm (n=395). Physicians evaluated neonates to determine whether newborns with VSD needed referral. The authors calculated algorithm sensitivity and specificity in identifying (1) neonates needing referral and (2) mortality during the first 10 days of life.
The 11-sign algorithm had sensitivity of 50.0% (95% CI 24.7% to 75.3%) and specificity of 98.4% (96.6% to 99.4%) for identifying neonates needing referral-level care. A simplified 6-sign algorithm had sensitivity of 81.3% (54.4% to 96.0%) and specificity of 96.0% (93.6% to 97.8%) for identifying referral need and sensitivity of 58.0% (45.5% to 69.8%) and specificity of 93.2% (92.5% to 93.7%) for screening mortality. Compared to our 6-sign algorithm, the Young Infant Study 7-sign (YIS7) algorithm with minor modifications had similar sensitivity and specificity.
Community-based surveillance for neonatal illness by CHWs using a simple 6-sign clinical algorithm is a promising strategy to effectively identify neonates at risk of mortality and needing referral to hospital. The YIS7 algorithm was also validated with high sensitivity and specificity at community level, and is recommended for routine household surveillance for newborn illness. ClinicalTrials.gov no. NCT00198627.

Full text

Page 1

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-300591 1140
ABSTRACT
Background To validate a clinical algorithm for
community health workers (CHWs) during routine
household surveillance for neonatal illness in rural
Bangladesh.
Methods Surveillance was conducted in the
intervention arm of a trial of newborn interventions.
CHWs assessed 7587 neonates on postnatal days 0, 2,
5 and 8 and identifi ed neonates with very severe disease
(VSD) using an 11-sign algorithm. A nested prospective
study was conducted to validate the algorithm
(n=395). Physicians evaluated neonates to determine
whether newborns with VSD needed referral. The
authors calculated algorithm sensitivity and specifi city
in identifying (1) neonates needing referral and (2)
mortality during the fi rst 10 days of life.
Results The 11-sign algorithm had sensitivity of 50.0%
(95% CI 24.7% to 75.3%) and specifi city of 98.4% (96.6%
to 99.4%) for identifying neonates needing referral-level
care. A simplifi ed 6-sign algorithm had sensitivity of
81.3% (54.4% to 96.0%) and specifi city of 96.0% (93.6%
to 97.8%) for identifying referral need and sensitivity
of 58.0% (45.5% to 69.8%) and specifi city of 93.2%
(92.5% to 93.7%) for screening mortality. Compared
to our 6-sign algorithm, the Young Infant Study 7-sign
(YIS7) algorithm with minor modifi cations had similar
sensitivity and specifi city.
Conclusion Community-based surveillance for
neonatal illness by CHWs using a simple 6-sign clinical
algorithm is a promising strategy to effectively identify
neonates at risk of mortality and needing referral to
hospital. The YIS7 algorithm was also validated with
high sensitivity and specifi city at community level, and
is recommended for routine household surveillance for
newborn illness.
ClinicalTrials.gov no. NCT00198627.
INTRODUCTION
An effective strategy to decrease neonatal mortality
in low resource settings is to introduce community-
level interventions with linkages to the healthcare
system for treatment of severe illness.1 2 In many
rural resource-poor settings, trained community
health workers (CHWs) can: promote essential
newborn care practices at home3–5; improve care
seeking for severe neonatal illness by providing
parental education in recognition of signs of illness;
identify signs of illness through direct assessments
at routine surveillance visits and refer sick infants
to a health facility6 7; and manage illness at home
when a referral is not complied with or facility-
level referral is not feasible.3 8–10
Accurate assessment by CHWs is an important
prerequisite for successful community-level man-
agement of neonatal illness, and programmes have
used various clinical algorithms for illness identi-
fi cation.3 7 8 11 WHO Integrated Management of
Childhood Illness (IMCI) protocols have been
evaluated at facility level and include an algo-
rithm for children <2 months of age.12 However,
protocols evaluated at the facility level may not
necessarily have the same validity, or be program-
matically feasible, in community settings where
the algorithm will be applied to a large proportion
of well children.12 Recently, we validated the abil-
ity of CHWs to assess newborns for the presence
of clinical signs and classifi cation of illness against
assessment by physicians in populations with rel-
atively moderate13 and high burdens of disease.14
▶ Additional supplementary
tables are published online
only at http://adc.bmj.com/
content/96/12.toc
1Department of International
Health, Bloomberg School
of Public Health, Johns
Hopkins University, Baltimore,
Maryland, USA
2Public Health Sciences
Division, ICDDR,B, Dhaka,
Bangladesh
3Department of Pediatrics,
Kumudini Women’s Medical
College, Mirzapur, Bangladesh
4Department of Microbiology,
Bangladesh Institute of Child
Health, Dhaka Shishu Hospital,
Dhaka, Bangladesh
Correspondence to
Dr Gary L Darmstadt, Family
Health Division, Global Health
Program, Bill & Melinda Gates
Foundation, PO Box 23350,
Seattle, WA 98102, USA;
gary.darmstadt@
gatesfoundation.org
Accepted 3 September 2011
Published Online First
30 September 2011
Validation of a clinical algorithm to identify neonates
with severe illness during routine household visits in
rural Bangladesh
Gary L Darmstadt,1 Abdullah H Baqui,1 Yoonjoung Choi,1 Sanwarul Bari,2 Syed
M Rahman,2 Ishtiaq Mannan,1 A S M Nawshad Uddin Ahmed,3 Samir K Saha,4
Habibur Rahman Seraji,2 Radwanur Rahman,2 Peter J Winch,1 Stephanie Chang,1
Nazma Begum,2 Robert E Black,1 Mathuram Santosham,1 Shams El Arifeen2; for the
Bangladesh Projahnmo-2 (Mirzapur) Study Group
What is already known on this topic
▶ Community health workers (CHWs) are
capable of validly identifying sick newborns.
▶ The Young Infant Study 7-sign algorithm
(YIS7) can be used by healthcare workers
at health facilities to identify sick newborns
needing urgent care in hospital.
▶ No validated clinical algorithm exists for
assessment of newborns by frontline workers
during routine household surveillance.
What this study adds
▶ We identifi ed a simple, valid, 6-sign clinical
algorithm for use by CHWs to assess new-
borns for illness during routine household
visits.
▶ The algorithm was valid for identifying new-
borns who were sick and at risk of death.
▶ The YIS7 algorithm performed similarly and
is also recommended for screening young
infants for illness during routine home visits.
group.bmj.com on August 1, 2012 - Published by adc.bmj.com Downloaded from

Page 2

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-3005911141
Building on these analyses, the primary purpose of this paper
was to validate the clinical algorithm itself that was used dur-
ing routine household surveillance in Mirzapur, Bangladesh7
in identifying neonates needing urgent referral-level evalua-
tion for severe illness. Study objectives were (1) to validate the
clinical algorithm against physicians’ algorithm-independent
judgement of need for referral-level evaluation as a gold stan-
dard, (2) to compare the validity of current and further simpli-
fi ed algorithms and (3) to assess the validity of the algorithms
in identifying neonatal mortality.
METHODS
Study design and CHW surveillance
The clinical algorithm validation study was a prospective
study nested within the PROJAHNMO-2 trial in Mirzapur,
Bangladesh, described previously.7 13 15 In the intervention
arm, CHWs conducted surveillance to identify pregnant
women in a population of approximately 4000, made two
prenatal visits at home to promote birth and newborn care
preparedness, and visited the mother and newborn infant
on the day of delivery and each live born infant at home on
postnatal days 2, 5 and 8. During the postnatal visits, CHWs
assessed neonates for the presence of severe illness using a
clinical algorithm adapted from the Bangladesh Young Infant
IMCI protocol for the management of sick children <2 months
of age at fi rst-level health facilities, and recommended urgent
referral of neonates with severe illness to Kumudini Hospital,
a 750-bed, private, referral-level hospital. CHWs ascertained
the presence of 16 historical factors and 28 clinical signs, and
conducted a detailed assessment of breastfeeding, as described
previously.13
Mirzapur CHW clinical algorithm
The primary purpose of the clinical algorithm was to iden-
tify neonates with very severe disease (VSD) requiring urgent
referral to the hospital for further evaluation and treatment.
Initially, a neonate was categorised as having VSD if she/he
had one or more of eight signs observed by a CHW (see online
supplementary table 1). In 2005, based on high case death
rates from preliminary analyses, the algorithm was revised to
include three additional signs (weak, abnormal or absent cry;
lethargic or less than normal movement; and not able to feed or
not able to suck at all based on the breastfeeding assessment),
for a total of 11 signs, for the classifi cation of VSD (online
supplementary table 1).13 Previously, we showed that CHWs’
classifi cation of neonates with severe illness using the 11-sign
algorithm had high sensitivity and specifi city compared to
classifi cation by physicians.13
The CHW clinical algorithm validation study
Study physicians from Kumudini Hospital randomly selected
one of the project CHWs each day, and conducted a complete
assessment of all the neonates seen by that CHW in a 24 h
period, except those being seen in follow-up by the CHW after
a hospital visit. Physicians completed the same standardised
newborn assessment as the CHWs, except for the feeding
assessment which was conducted by a female nurse if the phy-
sician was male, due to cultural sensitivity. In addition, physi-
cians categorised neonates as to whether they needed urgent
referral-level evaluation based on their clinical discretion,
independent of the algorithm (hereafter referred to as referral
need). Physicians assessed neonates less than 12 h after the
CHWs’ assessments either at home (for well babies and referral
failures) or at the hospital (for successfully referred neonates)
and were blinded to the CHWs’ evaluation results. The average
time between CHW and physician assessment was 3.0 h (SD
1.6 h, median 2.8 h), and 96% of the neonates were assessed at
home by both physicians and CHWs. All neonates had com-
plete assessments by both a CHW and a physician. We did not
measure interobserver reliability in assessment among physi-
cians or CHWs.
Data and analysis
A target sample size of 395 was calculated to achieve a pre-
determined agreement between CHW (n=44) and physician
(n=8) assessments.16 We assumed a VSD prevalence of 5%
identifi ed by CHWs, 5% prevalence of neonatal illness requir-
ing referral-level evaluation/management as determined by
physicians, and a κ statistic of 0.90 with ±0.1 precision. Given
the calculated sample size, the expected 95% CIs for a conser-
vatively estimated sensitivity of 70% and specifi city of 80%
would be 50% to 90% and 76% to 84%, respectively.
Table 1 Sensitivity, specifi city, PPV and NPV of different screening algorithms for VSD (n=395)
Prevalence,
% (95% CI) % (95% CI)

Sensitivity, Specifi city,
% (95% CI)
PPV,
% (95% CI)
NPV,
% (95% CI)
κ Statistic
(p value)
VSD computed using community health workers’ assessment
Mirzapur original 8-sign
Mirzapur revised 11-sign
SEARCH, 2 out of 7 signs
SEARCH, 1 out of 7 signs
YIS7
VSD computed using physicians’ assessment
Mirzapur original 8-sign
Mirzapur revised 11-sign
SEARCH, 2 out of 7 signs
SEARCH, 1 out of 7 signs
YIS7
2.5 (1.2 to 4.6)
3.5 (2.0 to 5.9)
0.0 (0.0 to 0.9)*
2.0 (0.9 to 4.0)
6.8 (4.6 to 9.8)
37.5 (15.2 to 64.6)
50.0 (24.7 to 75.3)
–
6.3 (0.2 to 30.2)
62.5 (35.4 to 84.8)
98.9 (97.3 to 99.7)
98.4 (96.6 to 99.4)
–
98.2 (96.2 to 99.3)
95.5 (92.9 to 97.4)
60.0 (26.2 to 87.8)
57.1 (28.9 to 82.3)
–
12.5 (0.3 to 52.7)
37.0 (19.4 to 57.6)
97.4 (95.3 to 98.7)
97.9 (95.9 to 99.1)
–
96.1 (93.7 to 97.8)
98.4 (96.5 to 99.4)
0.44 (0.00)
0.52 (0.00)
–
0.06 (0.11)
0.44 (0.00)
2.3 (1.0 to 4.3)
2.8 (1.4 to 4.9)
1.3 (0.4 to 2.9)
5.3 (3.3 to 8.0)
14.4 (11.1 to 18.3)
43.8 (19.8 to 70.1)
50.0 (24.7 to 75.3)
31.3 (11.0 to 58.7)
50.0 (24.7 to 75.3)
56.3 (29.9 to 80.2)
99.5 (98.1 to 99.9)
99.2 (97.7 to 99.8)
100.0 (99.0 to 100.0)
96.6 (94.2 to 98.2)
87.3 (83.6 to 90.5)
77.8 (40.0 to 97.2)
72.7 (39.0 to 94.0)
100.0 (47.8 to 100.0)
38.1 (18.1 to 61.6)
15.8 (7.5 to 27.9)
97.7 (95.6 to 98.9)
97.9 (95.9 to 99.1)
97.2 (95.0 to 98.6)
97.9 (95.8 to 99.1)
97.9 (95.8 to 99.2)
0.55 (0.00)
0.58 (0.00)
0.47 (0.00)
0.41 (0.00)
0.20 (0.00)
Screening was conducted to identify need for urgent referral-level evaluation for severe neonatal diseases. Lists of signs and symptoms for algorithms are presented in
online supplementary table 1. Validity measures were not estimated.
*97.5% one-sided CI.
NPV, negative predictive value; PPV, positive predictive value; VSD, very severe disease; SEARCH, Society for Education, Action and Research in Community Health; YIS7,
Young Infant Study 7-sign.
Kappa statistic between the computed VSD based on child’s assessment and referral need.
group.bmj.com on August 1, 2012 - Published by adc.bmj.comDownloaded from

Page 3

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-3005911142
During the validation study period (November 2005 to
December 2006), 4226 live births occurred and 3038 of them
were assessed by CHWs at least once. A total of 395 neonates
were randomly selected for validation of the clinical algorithm
in identifying neonates requiring referral-level care (fi gure 1).
To assess the validity of the algorithm in identifying neonates
at risk of mortality, we analysed data from 6924 neonates who
were assessed by CHWs at least once during the fi rst 10 days of
life throughout the entire study period (fi gure 1).
Modifi cations of the algorithm for VSD
We further modifi ed the revised 11-sign VSD algorithm in
order to explore simplifi ed, alternate algorithms for identi-
fying VSD by CHWs, since an 11-sign algorithm poses sig-
nifi cant challenges for training and supervision of CHWs.
Since the algorithm was adopted from a widely used IMCI
algorithm, we aimed to explore modifi cations of the cur-
rent revised 11-sign VSD algorithm rather than conduct an
exhaustive examination of associations between referral need
and all individual signs and symptoms.12 17 Thus, the algo-
rithm was modifi ed based on clinical signifi cance, assessed
using case death rates (results not shown), and the practical-
ity of assessment of the signs. Online supplementary table 1
presents signs and symptoms included in seven sequentially
modifi ed algorithms. We also applied a community-based
algorithm which was used by the Society for Education,
Action and Research in Community Health (SEARCH) to
screen for neonates with suspected serious infection11 and
a recently validated WHO Young Infant Study 7-sign (YIS7)
algorithm used to screen for severe illness requiring hospital
admission, excluding jaundice, among those who visited out-
patient facilities (online supplementary table 1).12
Statistical analysis
The validation study sample of 395 neonates was analysed to
assess associations between physicians’ judgement of referral
Figure 1 Flowchart of the surveillance (January 2004 to December 2006) and selection of the validation study sample. CHW, community health
worker.
Table 2 Sensitivity, specifi city, PPV and NPV of various modifi cations to the revised Mirzapur 11-sign algorithm for VSD (n=395)
Prevalence,
% (95% CI)% (95% CI)

Sensitivity, Specifi city,
% (95% CI)
PPV,
% (95% CI)
NPV,
% (95% CI)
κ Statistic
(p value)
Modifi cation of revised 11-sign algorithm
Modifi cation A
Modifi cation B
Modifi cation C
Modifi cation D
Modifi cation E
Modifi cation F
Modifi cation G
Modifi cation of YIS7 algorithm
Modifi cation X
Modifi cation Y
Modifi cation Z
4.3 (2.5 to 6.8)
4.1 (2.3 to 6.5)
7.3 (5.0 to 10.4)
7.3 (5.0 to 10.4)
9.1 (6.5 to 12.4)
7.1 (4.8 to 10.1)
5.1 (3.1 to 7.7)
62.5 (35.4 to 84.8)
62.5 (35.4 to 84.8)
81.3 (54.4 to 96.0)
81.3 (54.4 to 96.0)
81.3 (54.4 to 96.0)
81.3 (54.4 to 96.0)
68.8 (41.3 to 89.0)
98.2 (96.2 to 99.3)
98.4 (96.6 to 99.4)
95.8 (93.2 to 97.6)
95.8 (93.2 to 97.6)
93.9 (91.0 to 96.1)
96.0 (93.6 to 97.8)
97.6 (95.5 to 98.9)
58.8 (32.9 to 81.6)
62.5 (35.4 to 84.8)
44.8 (26.4 to 64.3)
44.8 (26.4 to 64.3)
36.1 (20.8 to 53.8)
46.4 (27.5 to 66.1)
55.0 (31.5 to 76.9)
98.4 (96.6 to 99.4)
98.4 (96.6 to 99.4)
99.2 (97.6 to 99.8)
99.2 (97.6 to 99.8)
99.2 (97.6 to 99.8)
99.2 (97.6 to 99.8)
98.7 (96.9 to 99.6)
0.59 (0.00)
0.61 (0.00)
0.55 (0.00)
0.55 (0.00)
0.47 (0.00)
0.57 (0.00)
0.59 (0.00)
7.3 (5.0 to 10.4)
5.3 (3.3 to 8.0)
7.3 (5.0 to 10.4)
68.8 (41.3 to 89.0)
68.8 (41.3 to 89.0)
81.3 (54.4 to 96.0)
95.3 (92.6 to 97.2)
97.4 (95.2 to 98.7)
95.8 (93.2 to 97.6)
37.9 (20.7 to 57.7)
52.4 (29.8 to 74.3)
44.8 (26.4 to 64.3)
98.6 (96.8 to 99.6)
98.7 (96.9 to 99.6)
99.2 (97.6 to 99.8)
0.46 (0.00)
0.58 (0.00)
0.55 (0.00)
Screening was conducted to identify need for urgent referral-level evaluation for severe neonatal diseases. Lists of signs and symptoms for algorithms are presented in
online supplementary table 1.
NPV, negative predictive value; PPV, positive predictive value; VSD, very severe disease; YIS7, Young Infant Study 7-sign.
Kappa statistic between the computed VSD based on child’s assessment and referral need.
group.bmj.com on August 1, 2012 - Published by adc.bmj.comDownloaded from

Page 4

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-3005911143
need and a computed VSD categorisation based on individual
signs and symptoms assessed by CHWs; the validity of a
computed VSD categorisation using assessment of signs and
symptoms by physicians was also examined and found to pro-
duce similar results. Consistent with the YIS7 used to defi ne
IMCI guidelines,12 and to avoid the infl uence of treatment,
we utilised physicians’ judgement of need for referral to hos-
pital as the gold standard to calculate sensitivity, specifi city,
positive predictive value (PPV) and negative predictive value
(NPV). κ Statistics were also calculated to determine agree-
ment between the computed VSD classifi cation and physi-
cians’ judgement, and were considered as poor (<0), slight
(0.0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial
(0.61–0.80) and almost perfect agreement (0.81–1.00).18 95%
CI was calculated for all estimates, and we compared 95% CIs
across algorithms in order to examine statistical signifi cance
in differential estimates.
In addition, to assess validity in identifying mortality, vari-
ous VSD algorithms were applied to the 6924 neonates who
were assessed by CHWs at least once during the fi rst 10 days of
life (fi gure 1), using mortality at the end of the 10-day period as
the gold standard outcome. A neonate was categorised as hav-
ing VSD if he/she had VSD at any one or more of CHW assess-
ments during the period. We estimated sensitivity, specifi city,
PPV and NPV. We further calculated the likelihood ratio for a
positive result, a measure useful in clinical settings, summaris-
ing both sensitivity and specifi city. In addition, bivariate anal-
yses were conducted to estimate the OR of mortality based
on logistic regression models and the population attributable
fraction (PAF) of mortality risk by each VSD classifi cation.
PAF was further calculated for selected individual signs and
symptoms. PAF estimates the proportion of deaths that would
be prevented following elimination of a condition, assuming
the condition is causal.19 20
Binomial exact 95% CIs were calculated for all proportion
estimates. We compared 95% CIs across algorithms in order
to examine statistical signifi cance in differential estimates.
STATA 9.0 statistical software (Stata, College Station, Texas,
USA) was used for all analyses. This study was approved by
the Committee on Human Research at the Johns Hopkins
Bloomberg School of Public Health, and the Ethical Review
Committee and Research Review Committee at ICDDR,B,
and was registered at clinicaltrials.gov (no. NCT00198627).
RESULTS
Validity in identifying neonates with VSD
The validation sample of newborns was comparable to the
overall population of newborns in the parent trial (data not
shown). About 71% and 98% of the validation sample were
assessed during the fi rst 7 and 10 days of life, respectively.
Study physicians reported a referral need prevalence of
4.1%. Based on CHWs’ assessments, the revised 11-sign algo-
rithm was able to correctly identify 50.0% of neonates with
VSD and 98.4% of those without VSD (table 1). The YIS7 algo-
rithm produced slightly higher sensitivity (62.5%) but slightly
lower specifi city (95.8%). The SEARCH algorithm could not
be evaluated, since no observation in our data set met the VSD
criteria of the SEARCH algorithm. There was no statistically
signifi cant difference in validity measures between computed
VSD classifi cations using CHWs’ and physicians’ assessment
results.
The inclusion of jaundice (modifi cation A, see online supple-
mentary table 1) improved sensitivity (table 2). Substituting
observed specifi c feeding problems with a history of a generic
feeding problem did not change sensitivity (modifi cation B).
Expanding fever and hypothermia cut-offs (from >101.0°F to
≥100.0°F and from <95.5ºF to <97.5ºF, respectively) increased
sensitivity substantially (modifi cation C). Eliminating respira-
tory rate did not change sensitivity (modifi cation F). Finally,
using WHO IMCI fever and hypothermia cut-offs (≥99.5°F
and <95.9 ºF, respectively) decreased sensitivity (modifi cation
G). Specifi city remained high and relatively similar through-
out modifi cations, ranging from 94% to 98%, implying that
the choice of the optimal algorithm would be largely based on
sensitivity.
We applied similar modifi cations to YIS7 algorithms (table
2); results were qualitatively comparable to those for the
Mirzapur 11-sign algorithm modifi cations. The fi nal modi-
fi cation (modifi cation Z: including jaundice, excluding fast
breathing and altering fever and hypothermia cut-offs (online
supplementary table 1)) showed a sensitivity of 81%, compa-
rable to that of the Mirzapur algorithm modifi cation F, as the
two algorithms are identical except that history of convulsion
is included in modifi cation Z.
Finally, sensitivity analyses using only 380 neonates who
were assessed at home resulted in similar, non-signifi cantly
lower estimates of validity, compared to the estimates using
the full sample (results not shown).
Validity in identifying deaths
Among 6924 neonates who were assessed by CHWs during
the fi rst 10 days in the parent trial, 69 died within the 10-day
period. About 86% of the deaths occurred within 3 days fol-
lowing the last CHW assessment. Sensitivity and specifi city did
not vary signifi cantly across the Mirzapur 11-sign and modi-
fi ed algorithms (results not shown). The Mirzapur and the YIS7
Table 3 Sensitivity, specifi city, likelihood ratio positive, PPV and NPV of identifying mortality during the fi rst 10 days of life by VSD algorithm
(n=6924)
Prevalence,
% (95% CI)% (95% CI)
VSD algorithm
Sensitivity, Specifi city, %
(95% CI)
Likelihood ratio
positive (95% CI)
PPV,
% (95% CI)
NPV,
% (95% CI)
Mirzapur revised 11-sign
Modifi cation F of the Mirzapur revised 11-sign
YIS7
Modifi cation Z of the YIS7
SEARCH, 2 out of 7 signs
SEARCH, 1 out of 7 signs
5.6 (5.1 to 6.2)
7.4 (6.7 to 8.0)
7.4 (6.8 to 8.1)
7.4 (6.8 to 8.1)
0.3 (0.2 to 0.5)
2.3 (1.9 to 2.6)
58.0 (45.5 to 69.8)
58.0 (45.5 to 69.8)
56.5 (44.0 to 68.4)
58.0 (45.5 to 69.8)
2.9 (0.4 to 10.1)
15.9 (8.2 to 26.7)
94.9 (94.4 to 95.4)
93.2 (92.5 to 93.7)
93.1 (92.4 to 93.6)
93.1 (92.5 to 93.7)
99.7 (99.5 to 99.8)
97.9 (97.5 to 98.2)
11.5 (9.1 to 14.3)
8.5 (6.8 to 10.5)
8.1 (6.5 to 10.2)
8.4 (6.7 to 10.4)
9.9 (2.4 to 41.7)
7.5 (4.3 to 13.2)
10.3 (7.5 to 13.8)
7.9 (5.7 to 10.5)
7.6 (5.4 to 10.2)
7.8 (5.6 to 10.4)
9.1 (1.1 to 29.2)
7.0 (3.6 to 12.2)
99.6 (99.4 to 99.7)
99.5 (99.4 to 99.7)
99.5 (99.3 to 99.7)
99.5 (99.4 to 99.7)
99.0 (98.8 to 99.2)
99.1 (98.9 to 99.3)
Screening was conducted among all neonates who were assessed at least once during the period, in the intervention arm of the PROJAHNMO-II study. Lists of signs and
symptoms for algorithms are presented in online supplementary table 1.
NPV, negative predictive value; PPV, positive predictive value; SEARCH, Society for Education, Action and Research in Community Health; VSD, very severe disease; YIS7,
Young Infant Study 7-sign.
group.bmj.com on August 1, 2012 - Published by adc.bmj.com Downloaded from

Page 5

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-3005911144
algorithms showed remarkably comparable results (table 3)
with a sensitivity of 57–58% and specifi city of 93–95%. The
likelihood ratio for a positive result suggested that neonates
who died within the 10-day period were 8–12 times more
likely to have been identifi ed with VSD using the algorithms,
compared to those who survived (table 3). The SEARCH algo-
rithm showed lower sensitivity (2.9% for the algorithm requir-
ing the presence of two signs) and slightly higher specifi city
(99.6%) compared to the Mirzapur or the YIS7 algorithms. The
Mirzapur 11-sign and the Young Infant Study (YIS) algorithms
had a PAF of mortality risk of 53–56% (table 4). The sign ‘mod-
erate to severe hypothermia’ alone had a PAF of 46%.
DISCUSSION
IMCI protocols for young infants under 2 months of age have
been validated in the past at facility level, thus potentially
introducing care seeking bias in study samples.12 21 Moreover,
the recent, multicentre, facility-based YIS was largely affected
by two sites (Dhaka and Karachi), and the authors highlighted
the importance of external validation of the algorithm for
neonates in the fi rst week of life, particularly for use during
routine household surveillance.12 In our study, algorithms
were validated not only at the community level during routine
household surveillance, but also primarily among neonates in
the fi rst week of life.
The Mirzapur 11-sign clinical algorithm administered
by CHWs had a sensitivity of 50% for identifying neonates
needing referral to hospital. Since a screening algorithm is
aimed at identifying subjects who could potentially benefi t
from further identifi cation and management of illness, sensi-
tivity is of paramount importance, and thus, the sensitivity
of the 11-sign algorithm was deemed to be unacceptably low,
prompting further analysis to identify potential improvements
to the algorithm. Modifi cations of the algorithm increased
sensitivity but did not affect the initial high specifi city, indi-
cating potential improvement of the algorithm in identifying
severely ill neonates without burdening the healthcare system
with falsely identifi ed, non-ill newborns. Sensitivity was 81%
in a simplifi ed algorithm with only six signs and symptoms
which are relatively easy to ascertain. Exclusion of respiratory
rate measurement and a detailed feeding assessment, and reli-
ance instead on maternal reporting of feeding problems,13 did
not compromise algorithm performance and would reduce the
time and complexity of neonatal assessment substantially.
The revised 11-sign and further simplifi ed 6-sign algorithms
had a sensitivity of 58% and specifi city of about 93–95% in
identifying mortality during the fi rst 10 days. Further, neo-
nates identifi ed as having VSD, across all algorithm modifi -
cations, had signifi cantly increased odds of neonatal death
compared to those not meeting criteria for VSD. PAF mortal-
ity risk analyses imply that about 55% of deaths during the 10
days may be reduced if VSD can be identifi ed and successfully
managed. Although the algorithm was developed primarily
to identify neonates with severe morbidity (ie, VSD) in the
Table 4 OR from univariable logistic regression models and PAF of mortality during the fi rst 10 days of
life by VSD classifi cation and selected individual signs (n=6924)
Prevalence, % (95% CI) OR (95% CI)PAF, % (95% CI)*
VSD algorithm
Mirzapur revised 11-sign
Modifi cation F of the Mirzapur revised 11-sign
YIS7
Modifi cation Z of the YIS7
SEARCH, 2 out of 7 signs
SEARCH, 1 out of 7 signs
Individual signs and symptoms
Convulsion†
Respiratory rate ≥70/min†
Severe chest in-drawing present†‡§
Severe fever (axillary temperature >101°F)†
Moderate to severe fever (axillary temperature
≥100°F)‡
Moderate to severe hypothermia
(axillary temperature <97.5°F)‡
Severe hypothermia (axillary temperature
<95.5°F)†
Week, abnormal or absent cry†
Unconscious†§
Lethargic or less than normal movement†‡
Not able to feed or not suck at all, observed†
History of feeding problem‡
Severe skin infection†§
Umbilical redness extending to skin†§
Jaundice‡
5.6 (5.1 to 6.2)
7.4 (6.7 to 8.0)
7.4 (6.8 to 8.1)
7.4 (6.8 to 8.1)
0.3 (0.2 to 0.5)
2.3 (1.9 to 2.6)
25.9 (15.8 to 42.2)
18.8 (11.5 to 30.6)
17.4 (10.7 to 28.3)
18.6 (11.4 to 30.2)
10.2 (2.3 to 44.5)
8.7 (4.5 to 16.9)
55.5 (41.7 to 66.0)
54.6 (40.5 to 65.4)
53.0 (38.8 to 63.9)
54.6 (40.4 to 65.4)
2.6 (−1.3 to 6.3)
14.0 (5.0 to 22.2)
0.14 (0.07 to 0.27)
1.36 (1.10 to 1.66)
0.17 (0.09 to 0.30)
0.29 (0.18 to 0.45)
1.14 (0.90 to 1.42)
25.5 (5.3 to 122.6)
4.6 (1.6 to 13.0)
–
18.3 (5.2 to 63.9)
5.6 (2.0 to 15.7)
2.8 (−0.9 to 6.3)
4.5 (−1.1 to 9.8)
4.1 (−0.5 to 8.4)
4.7 (−0.9 to 10.0)
3.61 (3.18 to 4.08)28.0 (17.2 to 45.8)45.9 (32.7 to 56.5)
1.26 (1.01 to 1.55)37.9 (21.2 to 67.7)26.6 (16.2 to 35.8)
0.77 (0.57 to 1.00)
0.03 (0.00 to 0.10)
1.29 (1.03 to 1.58)
1.85 (1.54 to 2.19)
0.53 (0.38 to 0.74)
0.95 (0.74 to 1.21)
0.17 (0.09 to 0.30)
2.11 (1.78 to 2.48)
16.7 (7.3 to 38.5)
–
20.7 (10.9 to 39.4)
27.6 (16.0 to 47.7)
5.8 (1.4 to 24.7)
–
–
2.1 (0.7 to 6.9)
9.5 (2.5 to 15.9)
17.8 (8.5 to 26.1)
29.1 (17.9 to 38.8)
2.4 (−1.6 to 6.2)
2.3 (−2.7 to 7.0)
Screening was conducted among all neonates who were assessed at least once during the period, in the intervention arm of
the PROJAHNMO-II study. Lists of signs and symptoms for algorithms are presented in online supplementary table 1.
*PAF estimated for signs with signifi cant OR only.
†Signs included in the 11-sign revised algorithm.
‡Signs and symptoms included in modifi cation F of the 11-sign revised algorithm.
§Absence of the sign predicted mortality perfectly and OR was not estimated.
PAF, population attributable fraction; VSD, very severe disease; YIS7, Young Infant Study 7-sign.
group.bmj.com on August 1, 2012 - Published by adc.bmj.comDownloaded from

Page 6

Global child health
Arch Dis Child 2011;96:1140–1146. doi:10.1136/archdischild-2011-3005911145
context of an intervention promoting facilitated referral,13 the
algorithm included signs associated with high mortality risk
and may be a useful tool in identifying risk for mortality as
well, although this needs further validation.
The WHO YIS7 algorithm also performed well in a commu-
nity setting. Moreover, the performance of the YIS7 algorithm
was further improved with minor modifi cations. Considering
the cost of introducing a new algorithm and the cost and com-
plexity of training on the relatively minor variations between
the Mirzapur and YIS7 algorithms, the WHO YIS7 algorithm
developed for use at primary healthcare level appeared suit-
able in its current form for use at community level. Further
validation of the YIS7 algorithm is needed in other settings,
however, particularly when used during routine household
surveillance.
We applied the SEARCH algorithm to our data since it is,
to our knowledge, the only widely recognised community-
based algorithm for use by CHWs to identify neonatal illness.
The algorithm showed low sensitivity in our study sample,
but it was developed to identify ‘death due to probable sep-
sis’, which is more specifi c than ‘neonates with VSD needing
urgent referral’.11 Moreover, the two study populations had
different population-to-worker ratios and distinctively differ-
ent epidemiological characteristics among the study neonates
assessed by CHWs that infl uence their performance (online
supplementary table 2),7 11 22 highlighting challenges in devel-
oping a community-level clinical algorithm for diverse popula-
tions and varying programme designs. Overall, however, we
believe that the study design in Mirzapur may more closely
resemble programmes in which skilled attendance at birth is
not ensured, which remains the case in many high mortality
areas. Moreover, as births and care seeking for illness increas-
ingly take place in health facilities, the Mirzapur and YIS7
algorithms become increasingly germane.
There are two major limitations of the study. First, there
was about a 3 h interval between the CHW and physician
assessments due to logistical reasons, during which clinical
signs might have changed. Second, while physicians were
blinded to the results of CHWs’ assessments, the location of
the assessment could have biased the physician’s judgment.
However, the vast majority (96%) were assessed in the com-
munity by both physicians and CHWs. When we excluded the
15 newborns who were assessed by physicians at the hospital,
the results (data not presented) showed that sensitivity and
PPV were slightly lower across algorithms in the subsample
(n=380) compared to those in the entire study sample of 395
neonates. However, relative validity among the various algo-
rithms was comparable with the main results. Further, given
the interval between assessments, it would have been unethi-
cal to require sick neonates to stay home to wait for the study
physicians’ arrival.
In conclusion, considering the simplicity of having the same
algorithm for community and facility use, we recommend the
YIS7 algorithm for use at the community level in screening
for neonates who need referral-level care. Further community-
based validation of the YIS7 algorithm in populations with
different disease burdens will be needed.
Bangladesh Projahnmo-2 (Mirzapur) Study Group: A S M Nawshad Uddin
Ahmed, Saifuddin Ahmed, Nabeel Ashraf Ali, Abdullah H Baqui, Nazma Begum,
Robert E Black, Sanwarul Bari, Atique Iqbal Chowdhury, Gary L Darmstadt, Shams
El-Arifeen, A K M Fazlul Haque, Zahid Hasan, Amnesty LeFevre, Ishtiaq Mannan,
Anisur Rahman, Radwanur Rahman, Syed Moshfi qur Rahman, Taufi qur Rahman,
Samir K Saha, Mathuram Santosham, Habibur Rahman Seraji, Ashrafuddin Siddik,
Hugh Waters, Peter J Winch and K Zaman.
Acknowledgements This study was supported by the Wellcome Trust–
Burroughs Wellcome Fund Infectious Disease Initiative 2000 and the Offi ce of
Health, Infectious Diseases and Nutrition, Global Health Bureau, US Agency for
International Development through the Global Research Activity Cooperative
agreement with the Johns Hopkins Bloomberg School of Public Health (award
HRN-A-00-96–90006-00). Support for data analysis and manuscript preparation
was provided by the Saving Newborn Lives programme through a grant by the Bill
& Melinda Gates Foundation to Save the Children, US. The funders had no role in
study design, data collection and analysis, decision to publish or preparation of the
manuscript.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Ethics approval The study was approved by the Committee on Human Research
at the Johns Hopkins Bloomberg School of Public Health, and the Ethical Review
Committee and Research Review Committee at ICDDR,B.
Contributors GLD, AHB, PJW, REB, MS and SEA were primarily responsible for
study design and securing funding for the study. SB, SMR, IM, ASMNUA, HRS
and RR were responsible for day-to-day management of the project, including
data collection. SC and NB were responsible for project data management.
YC was primarily responsible for data analysis, and YC and GLD were primarily
responsible for preparing the manuscript. All authors reviewed and approved the
manuscript.
REFERENCES
1. Darmstadt GL, Bhutta ZA, Cousens S, et al. Evidence-based, cost-
effective interventions: how many newborn babies can we save? Lancet
2005;365:977–88.
2. Darmstadt GL, Black RE, Santosham M. Research priorities and postpartum care
strategies for the prevention and optimal management of neonatal infections in
less developed countries. Pediatr Infect Dis J 2000;19:739–50.
3. Baqui AH, El-Arifeen S, Darmstadt GL, et al. Effect of community-based
newborn-care intervention package implemented through two service-delivery
strategies in Sylhet district, Bangladesh: a cluster-randomised controlled trial.
Lancet 2008;371:1936–44.
4. Osrin D, Mesko N, Shrestha BP, et al. Implementing a community-based
participatory intervention to improve essential newborn care in rural Nepal.
Trans R Soc Trop Med Hyg 2003;97:18–21.
5. Darmstadt GL, Kumar V, Yadav R, et al. Introduction of community-based skin-
to-skin care in rural Uttar Pradesh, India. J Perinatol 2006;26:597–604.
6. Bari S, Mannan I, Rahman MA, et al. Trends in use of referral hospital services
for care of sick newborns in a community-based intervention in Tangail District,
Bangladesh. J Health Popul Nutr 2006;24:519–29.
7. Darmstadt GL, El Arifeen S, Choi Y, et al. Household surveillance of severe
neonatal illness by community health workers in Mirzapur, Bangladesh: coverage
and compliance with referral. Health Policy Plan 2010;25:112–24.
8. Bang AT, Bang RA, Baitule SB, et al. Effect of home-based neonatal care and
management of sepsis on neonatal mortality: fi eld trial in rural India. Lancet
1999;354:1955–61.
9. Bang AT, Bang RA, Reddy HM, et al. Reduced incidence of neonatal morbidities:
effect of home-based neonatal care in rural Gadchiroli, India. J Perinatol
2005;25(Suppl 1):S51–61.
10. Bang AT, Bang RA, Stoll BJ, et al. Is home-based diagnosis and
treatment of neonatal sepsis feasible and effective? Seven years of
intervention in the Gadchiroli fi eld trial (1996 to 2003). J Perinatol 2005;25
(Suppl 1):S62–71.
11. Bang AT, Bang RA, Reddy MH, et al. Simple clinical criteria to identify sepsis or
pneumonia in neonates in the community needing treatment or referral.
Pediatr Infect Dis J 2005;24:335–41.
12. The Young Infants Clinical Signs Study Group. Clinical signs that predict
severe illness in children under age 2 months: a multicentre study. Lancet
2008;371:135–42.
13. Darmstadt GL, Baqui AH, Choi Y, et al. Validation of community health workers’
assessment of neonatal illness in rural Bangladesh. Bull World Health Organ
2009;87:12–19.
14. Baqui AH, Arifeen SE, Rosen HE, et al. Community-based validation of
assessment of newborn illnesses by trained community health workers in Sylhet
district of Bangladesh. Trop Med Int Health 2009;14:1448–56.
15. Darmstadt GL, Choi Y, Arifeen SE, et al. Evaluation of a cluster-randomized
controlled trial of a package of community-based maternal and newborn
interventions in Mirzapur, Bangladesh. PLoS ONE 2010;5:e9696.
16. Flack VF, Afi fi AA, Lachenbruch PA. Sample size determinations for the two rater
Kappa statistic. Psychometrika 1988;53:321–5.
17. The WHO Young Infants Study Group. Clinical prediction of serious bacterial
infections in young infants in developing countries. Pediatr Infect Dis J
1999;18:S23–31.
group.bmj.com on August 1, 2012 - Published by adc.bmj.com Downloaded from

Page 7

Global child health
1146
Arch Dis Child December 2011 Vol 96 No 12
18. Landis JR, Koch GG. The measurement of observer agreement for categorical
data. Biometrics 1977;33:159–74.
Rockhill B, Newman B, Weinberg C. Use and misuse of population attributable
fractions. Am J Public Health 1998;88:15–19.
Greenland S, Drescher K. Maximum likelihood estimation of the attributable
fraction from logistic models. Biometrics 1993;49:865–72.
19.
20.
21. Kalter HD, Schillinger JA, Hossain M, et al. Identifying sick children requiring
referral to hospital in Bangladesh. Bull World Health Organ 1997;75(Suppl 1):65–75.
Bang AT, Reddy HM, Bang RA, et al. Why do neonates die in rural Gadchiroli,
India? (Part II): estimating population attributable risks and contribution of
multiple morbidities for identifying a strategy to prevent deaths. J Perinatol
2005;25(Suppl 1):S35–43.
22.
group.bmj.com on August 1, 2012 - Published by adc.bmj.comDownloaded from

Page 8

doi: 10.1136/archdischild-2011-300591
September 30, 2011
2011 96: 1140-1146 originally published onlineArch Dis Child

Gary L Darmstadt, Abdullah H Baqui, Yoonjoung Choi, et al.
household visits in rural Bangladesh
neonates with severe illness during routine
Validation of a clinical algorithm to identify

http://adc.bmj.com/content/96/12/1140.full.html
Updated information and services can be found at:
These include:
Data Supplement

http://adc.bmj.com/content/suppl/2011/09/30/archdischild-2011-300591.DC1.html
"Web Only Data"
References

http://adc.bmj.com/content/96/12/1140.full.html#ref-list-1
This article cites 22 articles, 1 of which can be accessed free at:
service
Email alerting
the box at the top right corner of the online article.
Receive free email alerts when new articles cite this article. Sign up in
Collections
Topic
(314 articles)Clinical trials (epidemiology)
?
(288 articles)Screening (public health)
?
(288 articles)Screening (epidemiology)
?
(894 articles)Epidemiologic studies
?

Articles on similar topics can be found in the following collections
Notes

http://group.bmj.com/group/rights-licensing/permissions
To request permissions go to:

http://journals.bmj.com/cgi/reprintform
To order reprints go to:

http://group.bmj.com/subscribe/
To subscribe to BMJ go to:
group.bmj.com on August 1, 2012 - Published by adc.bmj.com Downloaded from