Validity and reliability of the 'Ten Questions' questionnaire for detecting moderate to severe neurological impairment in children aged 6-9 years in rural Kenya.

Abstract

The 'Ten Questions' Questionnaire (TQQ) is used to detect severe neurological impairment in children living in resource-poor countries. Its usefulness has been established in Asia and the Caribbean, but there are a few published studies from Africa. We evaluated the TQQ as part of a larger study of neurological impairment in a rural community, on the coast of Kenya.
The study was conducted in two phases from June 2001 to May 2002; in phase one, a community household screening of 10,218 children aged 6-9 years using the TQQ was performed. Phase two involved a comprehensive clinical and psychological assessment of all children testing positive on the TQQ (n = 810) and an equivalent number of those testing negative (n = 766). Data were interpreted using the impairment-specific approach.
Overall, the sensitivity rates for screening the different impairments were: cognitive (70.0%), motor (71.4%), epilepsy (100%), hearing (87.4%) and visual (77.8%). All the specificity rates were greater than 96%. However, the positive predictive values were low, and ranged from 11 to 33%.
These results are similar to those from other continents and provide evidence that the TQQ can be used to compare the epidemiology of moderate/severe impairment in different parts of the world. Furthermore, the TQQ can be used to screen for moderately/severely impaired children in resource-poor countries; however, the low positive predictive values mean that other assessments are required for confirmation.

Full text

Page 1

Original Paper
Neuroepidemiology 2004;23:67–72
DOI: 10.1159/000073977
Validity and Reliability of the ‘Ten Questions’
Questionnaire for Detecting Moderate to
Severe Neurological Impairment in Children
Aged 6–9 Years in Rural Kenya
V. Mung’ala-OderaaR. MeehanaP. NjugunaaN. MturiaK. Alcockb
J.A. Cartera,cC.R.J.C. Newtona,c
aCenter for Geographic Medicine-Coast, Kenya Medical Research Institute, Kilifi, Kenya; bDepartment of
Psychology, City University, and cNeurosciences Unit, Institute of Child Health, University College, London, UK
Victor Mung’ala-Odera
Center for Geographic Medicine-Coast
KEMRI, PO Box 428, Kilifi (Kenya)
Tel. +254 41 522063, Fax +254 41 522390
E-Mail vodera@kilifi.mimcom.net
ABC
Fax +41 61 306 12 34
E-Mail karger@karger.ch
www.karger.com
© 2004 S. Karger AG, Basel
0251–5350/04/0232–0067$21.00/0
Accessible online at:
www.karger.com/ned
Key Words
Validity W Reliability W Neurological impairment
Abstract
Background: The ‘Ten Questions’ Questionnaire (TQQ) is
used to detect severe neurological impairment in chil-
dren living in resource-poor countries. Its usefulness has
been established in Asia and the Caribbean, but there are
a few published studies from Africa. We evaluated the
TQQ as part of a larger study of neurological impairment
in a rural community, on the coast of Kenya. Methods:
The study was conducted in two phases from June 2001
to May 2002; in phase one, a community household
screening of 10,218 children aged 6–9 years using the
TQQ was performed. Phase two involved a comprehen-
sive clinical and psychological assessment of all children
testing positive on the TQQ (n = 810) and an equivalent
number of those testing negative (n = 766). Data were
interpreted using the impairment-specific approach. Re-
sults: Overall, the sensitivity rates for screening the
different impairments were: cognitive (70.0%), motor
(71.4%), epilepsy (100%), hearing (87.4%) and visual
(77.8%). All the specificity rates were greater than 96%.
However, the positive predictive values were low, and
ranged from 11 to 33%. Conclusions: These results are
similar to those from other continents and provide evi-
dence that the TQQ can be used to compare the epidemi-
ology of moderate/severe impairment in different parts
of the world. Furthermore, the TQQ can be used to
screen for moderately/severely impaired children in re-
source-poor countries; however, the low positive predic-
tive values mean that other assessments are required for
confirmation.
Copyright © 2004 S. Karger AG, Basel
Background
There are few studies on the prevalence of neurological
impairment (NI) in children in resource-poor countries,
yet most of the neurological disorders are thought to occur
in these areas [1]. In this region, identification of children
with NI is difficult, since the major sources of referral
used in western countries, i.e. schools and medical ser-
vices, do not provide enough information to detect im-
pairment [2]. Other techniques such as national census
interviews or key informant techniques often underesti-
mate the number of children with NI [3].
A ‘Ten Questions’ Questionnaire (TQQ) was devel-
oped to rapidly screen children aged 2–9 years for several
types of impairment. Studies have been reported from
Jamaica, Pakistan, Saudi Arabia and Bangladesh [4–9],
but the use of the TQQ in other continents has not been

Page 2

68
Neuroepidemiology 2004;23:67–72
Mung’ala-Odera/Meehan/Njuguna/Mturi/
Alcock/Carter/Newton
Table 1. Definitions of moderate and severe impairment
ImpairmentModerate Severe
Cognitive some delay in attaining growth milestones, difficulty in
speech as well as moderate cognitive deficit
fine motor deficits, delay in speech and in attaining growth
milestones, as well as significant cognitive deficit
Motordifficulty in holding implements, dressing and sitting
upright, able to move around with help
inability to walk and absence of functional use of hands
Epilepsy more than one nonfebrile seizure per monthmore than one nonfebrile seizure per week
Hearinga 41- to 70-dB loss in the best ear and difficulty in hearing
even with a hearing aid
more than 70 dB loss in the best ear, no useful hearing
Visionvision loss of 6/18 mvisual acuity worse than 6/60, only light perception
Source: WHO, 1980; Procedure Manual, 1987.
widely assessed. As part of a study to identify children
with NI in a rural Kenyan community, we assessed the
validity and reliability of the TQQ as a tool for identifying
children with moderate to severe NI (table 1).
Methods
Study Setting
This study was conducted in a demarcated study area in the Kilifi
District on the coast of Kenya. The area is subdivided into 87 enu-
meration zones, with sketch maps of each zone drawn in 1992 and
updated in 2000, showing major landmarks, footpaths and home-
steads, with their relative positions and survey numbers (fig. 1). The
sketch maps were used to relocate each household both during the
census in October 2000 and the NI survey between June 2001 and
March 2002.
Population
A study population of 10,218 children aged 6–9 years was drawn
from a population of about 100,000 people. This rural population
consists mainly of the Mijikenda ethnic group, in which the Giriama
subgroup predominates. This age group was chosen because it is
more difficult to identify impairments in children younger than 6
years (particularly hearing and visual impairment), and due to the
lack of culturally appropriate cognitive assessment tools for children
below 6 years of age. Furthermore, since one of the aims of this study
was to identify acquired NI, the most likely causes (bacterial menin-
gitis and cerebral malaria) will only have occurred by the age of 6
years. Children in this age group were identified through the census
conducted in October 2000. Only children who had been residing in
the area for at least 6 months preceding the survey were included.
TQQ Screening Instrument
The TQQ consists of 10 questions (Appendix 1): 1 each address-
ing the child’s vision, hearing, movement and seizures, 6 on cogni-
tion competence, and 1 extra question regarding other serious health
problems. The questionnaire was translated into Kigiriama and back
to English to ensure the intended meaning remained the same, before
being used. A screening test was considered positive if there was a
positive response to any one of the 10 questions.
Pilot Survey
A pilot survey was performed on 102 children who had earlier
undergone both neurological and cognitive impairment as part of
another study investigating the impact of malaria and seizures on
epilepsy [Carter et al., in prep.]. Twenty-nine (28%) of these children
had impairments.
Initially, 45% of the children tested positive on the TQQ, with
global sensitivity and specificity values for any impairment of 47 and
55%, respectively. We explored the reasons for the highly positive
result by checking the Kigiriama translation and the inter-interview-
er administration of the questionnaire. In 4 focus group surveys, with
28 mothers from the community, questions 1, 3 and 4 were found to
have been misunderstood. In question 1, the difficulty occurred in
the concept of delay in developmental milestones. In this communi-
ty, mothers tend to compare their child’s development with those of
other siblings; for instance, walking at the age of 11 months might not
be a significant delay; however, a mother whose other children
walked by the age of 8 months may consider this a delayed milestone.
In question 3, a direct translation of the question to the local dialect
resulted in two interpretations pertaining to hearing and inattention
(an inattentive child). The Kigiriama translation was therefore
revised to reflect the aspect of hearing alone. Likewise, question 4
had two interpretations: the child’s understanding of simple instruc-
tions and a behavioral aspect (obeying of instructions).
After the revision, the questionnaire was again piloted on guar-
dians of 439 children aged 6–9 years, before being administered to
the target population. Eleven percent of the children tested positive,
with questions 1 and 3 accounting for most of the positive re-
sponses.
Study Design
To test the validity of the TQQ, a two-stage design was followed.
Stage one involved the screening of the entire population of children
aged 6–9 years residing in the study area, using the TQQ tool. In
stage two, all the children testing positive on the TQQ underwent
comprehensive clinical and psychological assessments. Based upon

Page 3

Detection of Neurological Impairment in
Children Aged 6–9 Years in Rural Kenya
Neuroepidemiology 2004;23:67–72
69
Fig. 1. Country and area of the study.
the pilot data and estimates from other studies [4–8], we chose to
select every twelfth child who tested negative on the questionnaire to
undergo similar assessments. The assessments focused on detecting
impairments of vision, hearing, motor, and cognition, and the diag-
nosis of epilepsy. Assessments were performed within 1 week of the
screening. None of the clinicians and assessors involved in the study
knew the result of the TQQ screen at the time of the examination.
Household Screening
Five trained field interviewers fluent in Kigiriama performed the
household screening after undergoing a week’s training in field meth-
ods and questionnaire administration. The fieldworkers adminis-
tered the TQQ to the guardians of between 80 and 100 children on
each day. All those with at least 1 positive response and an equal
number of children testing negative were referred to the research cen-
ter for detailed assessment.
Psychological and Neurological Assessment
A team of 3 clinicians and 5 psychosocial assessors performed
assessments. The assessments included a vision test, with the Sonk-
sen-Silver Acuity system for measuring acuity distance [10]. Hearing
was measured with a Kamplex screening audiometer to establish the
hearing thresholds of the children. The thresholds were measured at
500, 1,000, 2,000 and 4,000 Hz [11]. Finally, motor impairments
(clinical examination) were assessed. The diagnosis of epilepsy was
based upon history elicited by clinicians, and an electroencephalo-
gram used to classify the type of epilepsy. The electroencephalogram
was performed on children who had a history of epilepsy (more than
1 seizure), with at least 1 seizure within the last 12 months, or had
partial seizures. Local adaptations of assessments of cognition,
speech and language were used. The cognitive assessment involved a
7-item battery, which included ‘Panga Mutu’ (testing intellectual and
developmental maturity of a child through observing how a child
deploys a set of basic skills to represent his/her knowledge of the
human form) [12], a local adaptation of the matching familiar figures
(assessing information processing speed and impulsivity) [13], digit
span (auditory short-term memory) [14], a construction task using
wooden sticks (simultaneous processing, visuomotor coordination,
visuospatial perception and reasoning) [15], category fluency (execu-
tive function) [14], information questions [15] and picture vocabula-
ry test (receptive vocabulary, verbal comprehension, achievement
and association of pictures and words) [15]. The speech and language
assessment battery included measures of comprehension, expression
and the child’s phonological system [16]. The neurological assess-
ment involved eliciting birth, developmental and medical history,
clinical and physical examination and observation of function and
anthropometric measurement (height, weight, head and mid-upper
arm circumferences).

Page 4

70
Neuroepidemiology 2004;23:67–72
Mung’ala-Odera/Meehan/Njuguna/Mturi/
Alcock/Carter/Newton
Table 2. Screening results for children testing positive on the TQQ
and the test-retest reliability kappa coefficients
n%
positive
Kappa
coefficients
Total
Positive on any question
1 Dev. milestones
2 Vision
3 Hearing
4 Cognitive
5 Motor
6 Seizure
7 Cognitive
8 Cognitive, speech
9 Cognitive, speech
10 Cognitive
10,218
955
417
–
9.3
4.1
0.4
3.0
0.5
0.5
1.2
0.2
0.5
1.0
1.9
–
–
0.2 (0.1–0.3)
1.0
0.3 (0.2–0.4)
1.0
1.0
1.0
1.0
1.0
1.0
0.7 (0.6–0.8)
40
307
52
50
125
17
53
103
197
Figures in parentheses indicate 95% CIs.
TQQ Reliability
Three months after the survey, the TQQ was readministered to
the guardians of 270 children by a second interviewer to test for inter-
rater agreement of the questions.
Data Storage and Analysis
All phase 1 and 2 data were double-entered and verified with the
Fox-pro version 4 software. Impairment-specific interpretations [8]
were adopted, in which a child with a given impairment had to be
positive for at least 1 screening question specific to that impairment
in order to be considered a true-positive. For example, a child with a
cognitive impairment has to test positive for at least 1 question on
cognitive competence for him/her to be considered to have a cogni-
tive impairment.
Sensitivity, specificity, positive and negative predictive values
were calculated to measure the validity of the TQQ [17]. Cohen’s
kappa scores were used to measure the test-retest reliability [17].
Results
A total of 10,218 children were screened, of whom 955
(9.3%) were positive (table 2). Delayed milestones and
hearing problems were the most common types of prob-
lems reported, while learning and visual problems were
the least reported (table 2). There was no difference be-
tween the sexes in those screened.
In total, 1,576 underwent assessment, and 810 (51.4%)
of them were those who tested positive on the TQQ. Half
of these children were (51.4%) boys. Overall the sensitivi-
ty of the TQQ to detect the different impairments (mod-
erate/severe and severe only) was greater than or equal to
70% for all domains, with lowest for cognitive impair-
ment (70.0%) and highest for epilepsy (100%; table 3). In
this study, specificities were high (171%) for all the
impairments; however, positive predictive values (PPV)
were low. There was a difference in sensitivity rates
between boys and girls in the cognition (63.4 vs. 48.1%)
and motor (66.7 vs. 80.0%) domains.
In total, 322 (20%) of the children assessed had some
form of impairment. Twenty percent of them were those
who tested negative on the TQQ. From this group (false-
negatives) 45% were aged 7 years and 66.2% were fe-
males. The associated impairments for the false-negatives
were cognition in 70.9% of the children, epilepsy in 24.6%
and hearing and vision in 4.5%.
Reliability of the TQQ
The test-retest reliability of the TQQ was excellent for
questions on vision, motor, seizures, speech and 4 of the
questions on cognition (table 2). It was fair on the general
questions about developmental milestones and hearing.
Psychological and Clinical Assessments
The interrater agreement of the clinical examinations
was good (kappa 0.40–0.75), with that of epilepsy being
excellent (kappa = 0.78). For the cognitive and language
tests, the statistical measurements of interrater reliability
were at an acceptable level of congruence between asses-
sors, with the mean difference in scores being less than
10% of the mean score [18].
Discussion
This study demonstrates that the TQQ is reliable and
useful for detecting moderate/severe impairment in chil-
dren aged 6–9 years in rural Africa. The high sensitivity
ensures that most cases of NI are identified, while the very
high specificity means that resources can be targeted
towards assessing these children. The low PPV suggests
that a large number of the children who screen positive on
the TQQ are false-positives for moderate or severe im-
pairment. Many of these children may have mild impair-
ments. This suggests that the TQQ alone is insufficient for
use in case finding in epidemiological studies of severe
impairment, but still remains useful as a screening tool in
selecting cases for further assessment.
Unlike in the other studies where only 6–10% of chil-
dren who screened negative were evaluated, findings from
our study are stronger in that an equivalent number of
those that screened negative were evaluated [4–8]. These
studies had comparable sensitivities for cognition and

Page 5

Detection of Neurological Impairment in
Children Aged 6–9 Years in Rural Kenya
Neuroepidemiology 2004;23:67–72
71
Table 3. Estimated sensitivity, specificity, positive and negative predictive values of the TQQ screen for moderate to severe impairment in
10,218 children screened and 1,576 assessed
Impair-
ment
Sensitivity
moderate/severesevere
Specificity
moderate/severe severe
Positive predictive value
moderate/severe severe
Negative predictive value
moderate/severesevere
Cognitive70.0 (63.2–76.0)79.1 (72.2–84.9) 71.4 (69.1–73.6)71.2 (69.0–73.4)23.9 (20.7–27.6)22.8 (19.6–26.4)94.8 (93.5–96.0)96.9 (95.8–97.8)
Motor71.4 (47.7–87.8)100 (56.1–100)98.3 (97.5–98.8)98.0 (97.2–98.6)32.6 (20.0–48.1)18.4 (8.3–34.9)99.7 (99.2–99.9)100 (99.9–100)
Epilepsy 100 (73.2–100)100 (67.9–100)92.9 (91.5–94.1)92.8 (91.4–94.0)11.2 (6.5–18.4)8.8 (4.7–15.6) 100 (99.7–100)100 (99.7–100)
Hearing87.4 (78.1–93.2) 92.9 (75.0–98.8)85.5 (83.6–87.1) 84.3 (82.6–86.0)24.8 (20.1–30.0) 8.5 (5.7–12.3)99.2 (98.5–99.6)99.9 (99.5–100)
Vision 77.8 (40.2–96.1)80.0 (29.9–98.9) 98.0 (97.4–98.7)98.0 (97.2–98.5)17.5 (7.9–33.4) 9.8 (3.2–24.1) 99.9 (99.5–100)99.9 (99.6–100)
Figures in parentheses indicate 95% CIs.
Table 4. A comparison of sensitivities and
specificities for moderate/severe
impairment for Kilifi (Kenya), Bangladesh
and Pakistan
Rural
Kilifi
(Kenya)
Rural
Bangladesh
[6]
Urban
Karachi
(Pakistan) [8]
Screened subjects
Age of children, years
10,218
6–9
7,635
2–9
6,365
2–9
Cognitivesensitivity
specificity
70.0
71.4
65.0
91.3
76.0
91.0
Motor sensitivity
specificity
71.4
98.3
100
91.4
75.0
96.0
Epilepsysensitivity
specificity
100
92.9
100
91.0
93.0
96.0
Hearingsensitivity
specificity
87.4
85.5
100
91.2
54.0
99.0
Visualsensitivity
specificity
77.8
98.0
100
91.1
34.0
99.0
epilepsy. The sensitivity of the TQQ in Kilifi was not as
high as that reported from a rural Bangladeshi population,
for motor, hearing and vision, but higher than that from a
urban Pakistani population for hearing and vision (ta-
ble 4). All these studies used impairment-specific inter-
pretation of the data. The differences in the sensitivity
and specificity could have been caused by different cultur-
al perceptions of impairment, differences in the develop-
ment of the questionnaire or population groups (rural vs.
urban). It is clear that the TQQ does require considerable
development, piloting and reliability tests before being
utilized.
Vision, hearing and cognitive impairments are often
underreported by parents [19]. In our study, the TQQ was
able to detect moderate/severe impairments in vision and
hearing. The low sensitivity for detection of cognitive
impairment may have arisen from the poor reliability of
question number 1 on the child’s development. Further-
more, the assessments of cognitive impairment were not
comprehensive, but were developed to test the main
domains of cognition within an hour. A more detailed
assessment of cognition may indicate that the TQQ has a
higher sensitivity for cognitive impairment.
Reliability data for the TQQ were shown to be consis-
tent over time for all the questions except for the ques-
tions on milestones (question number 1) and hearing
(question number 3) where the reliability was fair [17].
The lower level of reliability on the hearing question may
have been due to misunderstandings about the nature of
persisting hearing impairment, within this culture, partic-

Page 6

72
Neuroepidemiology 2004;23:67–72
Mung’ala-Odera/Meehan/Njuguna/Mturi/
Alcock/Carter/Newton
ularly parents’ perception of a child’s inattention com-
pared with actual deafness. Furthermore, fluctuating
hearing levels due to ear infections or upper respiratory
tract infections may have contributed to this result. How-
ever, the results suggest that the TQQ is a reliable tool, a
finding that was also demonstrated by the Pakistani
study, which established reliability coefficients in the
range of 0.6–0.8 [20].
We have found that the TQQ is a useful screen for
moderate/severe NI in epidemiological studies of children
because of its ability to identify more than 70% of the
seriously impaired children. However, the low PPV
means that it should not be used alone to detect such
impairments.
Appendix 1
The ‘Ten Questions’ Questionnaire
(1) Compared with other children, did the child have any serious
delay in sitting, standing or walking?
(2) Compared with other children, does the child have difficulty
in seeing, either in the daytime or at night?
(3) Does the child appear to have difficulty in hearing?
(4) When you tell the child to do something, does he/she seem to
understand what you are saying?
(5) Does the child have difficulty in walking or moving his/her
arms or does he/she have weakness and/or stiffness in the arms or
legs?
(6) Does the child sometimes have fits, become rigid, or lose con-
sciousness?
(7) Does the child learn to do things like other children of his/her
age?
(8) Does the child speak at all (can he/she make himself/herself
understood in words; can he/she say some recognizable words)?
(9) Is the child’s speech in any way different from normal (not
clear enough to be understood by people other than his/her imme-
diate family)?
(10) Compared with other children of his/her age, does the child
appear in any way mentally backward, dull or slow?
Acknowledgements
The Wellcome Trust, UK and Kenya Medical Research Institute
(KEMRI) supported this study. We thank the mapping and census
team, field staff and assessors who made this study possible. In par-
ticular, we thank Joseph Gona, Godfrey Otieno, Elizabeth Obiero,
Khamis Katana, Kenneth Rimba, Gladys Murira, Judy Tumaini,
Francis Yaa, Douglas Konde, Mary Karisa, Francis Kanyetta, Silas
Haro, Karen Konde and Janet Chea. We also thank Dr. Penny Hold-
ing for the assistance in designing and development of assessment
tools, and Prof. Kevin Marsh and Dr. Norbert Peshu for their advice
on the study design. This paper is published with the permission of
the director of KEMRI. Dr. C.R.J.C. Newton holds a Wellcome
Trust Career Post in Clinical Tropical Medicine (No. 050533).
References
1 Bergen DC, Silberberg S: Nervous system dis-
orders: A global epidemic. Arch Neurol 2002;
59:1194–1196.
2 Durkin M, Khan N: Framework prevalence; in
Zinkin P, McConachie H (eds): Disabled Chil-
dren and Developing Countries. London, Mac
Keith Press, 1995, pp 1–9.
3 Chamie M: Development of Statistics of Dis-
abled Persons: Case studies. United Nations
Department of International Economic and So-
cial Affairs, Statistics on Special Population
Groups. New York, UN, 1986, series Y, No 2.
4 Paul TJ, Desai P: The prevalence of childhood
disability and related medical diagnosis in
Clarendon, Jamaica. West Indian Med J 1992;
41:8–11.
5 Zaman SS, Khan NZ, Islam S, et al: Validity of
the ‘Ten Questions’ for screening serious child-
hood disability: Results from urban Bangla-
desh. Int J Epidemiol 1990;19:613–620.
6 Zaman SS, Khan N, Islam S, Durkin M: Child-
hood Disabilities in Bangladesh: Report on
Rapid Epidemiological Assessment of Child-
hood Disabilities in Bangladesh. Dhaka, Ban-
gladesh Protibondhi Foundation, 1992.
7 Thorburn M, Desai P, Paul TJ, Malcolm L,
Durkin M, Davidson L: Identification of child-
hood disability in Jamaica: The ten questions
screen. Int J Rehabil Res 1992;15:115–127.
8 Durkin MS, Hasan MZ, Hasan KZ: The ten
questions screen for childhood disabilities: Its
uses and limitations in Pakistan. J Epidemiol
Community Health 1995;49:431–436.
9 Milaat WA, Ghabrah TM, Al-Bar HMS, Abalk-
hail BA, Kordy, MN: Population-based survey
of childhood disability in Eastern Jeddah using
the ten questions tool. Disabil Rehabil 2001;
23:199–203.
10 Salt AT, Sonksen PM, Wade A, Jayatunga R:
The maturation of linear acuity and com-
pliance with the Sonksen-Silver Acuity System
in young children. Dev Med Child Neurol
1995;37:505–514.
11 Kamplex Screening Audiometer. London, PC
Werth.
12 Kathuria R, Serpell R: Standardization of the
Panga Muntu test: A nonverbal cognitive test
developed in Zambia. J Negro Educ 1999;67:
228–241.
13 Kagan J: Matching Familiar Figures Test.
Cambridge, Harvard University Press, 1964.
14 Baddeley A, Meeks-Gardner J, Graham-
McGregor S: Cross-cultural cognition: Devel-
oping tests for developing countries: Special
issues: Donald Broadbent and Applied Cogni-
tive Psychology. Appl Cogn Psychol 1995;9:
173–195.
15 Holding P, Taylor HG, Kazungu AD, et al:
Assessing cognitive outcome in a rural African
population: Development of a neuropsycholog-
ical battery. J Int Neuropsychol Soc, in press.
16 Carter JA, Murira GM, Ross AJ, Mung’ala-
Odera V, Newton CRJC: Speech and language
sequalae of severe malaria in Kenyan children.
Brain Inj 2003;17;217–224.
17 Landis JR, Koch GG: The measurement of
observer agreement for categorical data. Bio-
metrics 1977;13:159–174.
18 Carter JA: Epilepsy and developmental impair-
ment following severe malaria in Kenyan chil-
dren; unpubl. dissertation, University College
of London, 2002.
19 Durkin MS, Davidson LL, Desai P, et al: Valid-
ity of the ten questions screened for childhood
disability: Results from population-based stud-
ies in Bangladesh, Jamaica and Pakistan. Epi-
demiology 1994;5:283–289.
20 Durkin MS, Wang W, Shrout PE, et al: Evalu-
ating a ten questions screen for childhood dis-
ability: Reliability and internal structure in dif-
ferent cultures. J Clin Epidemiol 1995;48:657–
666.