Pediatric delirium: Evaluating the gold standard
GABRIELLE SILVER, M.D.,1JULIA KEARNEY, M.D.,2CHANI TRAUBE, M.D.,1
THOMAS M. ATKINSON, PH.D.,2KATARZYNA E. WYKA, PH.D.,1,3AND JOHN WALKUP, M.D.1
1Weill Cornell Medical College, New York, New York
2Memorial Sloan-Kettering Cancer Center, New York, New York
3City University of New York, New York, New York
(RECEIVED December 26, 2013; ACCEPTED January 23, 2014)
Objective: Our aim was to evaluate interrater reliability for the diagnosis of pediatric delirium
by child psychiatrists.
Method: Critically ill patients (N ¼ 17), 0–21 years old, including 7 infants, 5 children
with developmental delay, and 7 intubated children, were assessed for delirium using the
Diagnostic and Statistical Manual–IV (DSM–IV) (comparable to DSM–V) criteria. Delirium
assessments were completed by two psychiatrists, each blinded to the other’s diagnosis, and
interrater reliability was measured using Cohen’s k coefficient along with its 95% confidence
Results: Interrater reliability for the psychiatric assessment was high (Cohen’s k ¼ 0.94,
CI [0.83, 1.00]). Delirium diagnosis showed excellent interrater reliability regardless of age,
developmental delay, or intubation status (Cohen’s k range 0.81–1.00).
Significance of results: In our study cohort, the psychiatric interview and exam, long
considered the “gold standard” in the diagnosis of delirium, was highly reliable, even in
extremely young, critically ill, and developmentally delayed children. A developmental
approach to diagnosing delirium in this challenging population is recommended.
KEYWORDS: Delirium, Child psychiatry, Pediatric critical care
Delirium, acute brain dysfunction, is recognized as a
serious medical problem in the adult critical care
population (Barr et al., 2013). Evaluation and treat-
ment of delirium has only recently garnered atten-
tion in the world of pediatrics (Janssen et al., 2011;
Schieveld et al., 2007; Silver et al., 2012; Smith
et al., 2009; 2011).
The “gold standard” diagnosis for pediatric delir-
ium is an assessment by child psychiatrists using
the Diagnostic and Statistical Manual–IV (DSM–
IV) criteria. Expert consensus supports presentation
of delirium in children over the age of two years as
comparable to adult delirium, and clinical diagnosis,
based on the DSM–IV criteria, has been found to
be valid (Karnik et al., 2007; Leentjens et al., 2008;
Turkel et al., 2003; 2006). Preverbal children under
two years of age and developmentally delayed
children may be very difficult to evaluate for altera-
tions in awareness, consciousness, and cognition,
leading some to question the validity of diagnosing
delirium in this population. At the same time, some
report infant presentation of delirium with recogniz-
able deficits in awareness, cognition, and arousal
when evaluated within a developmental framework
by experienced practitioners (Madden et al., 2011;
Schieveld et al., 2010; Silver et al., 2010; Turkel
et al., 2013). Due to the lack of objective research
addressing the consistencyof the “gold standard,” es-
pecially in infants and children with developmental
delays, we conducted a study to test the interrater re-
liability of child psychiatrists’ assessments.
Address correspondence and reprint requests to: Gabrielle Sil-
ver, Consultation Liaison Child Psychiatry, Weill Cornell Medical
College/New York Presbyterian Hospital, 525 East 68th Street,
Box 140, New York, New York 10065. E-mail: firstname.lastname@example.org-
Palliative and Supportive Care, page 1 of 4, 2014.
#Cambridge University Press, 2014 1478-9515/14 $20.00
Thirty-eight delirium assessments were completed
by two psychiatrists, each blinded to the other’s diag-
nosis. They assessed all consented pediatric inten-
sive care unit (PICU) patients present that day as
the initial part of a validation study of the Cornell
Assessment of Pediatric Delirium (CAPD) (Traube
et al., 2013). Psychiatrists were also blinded to the
nursing CAPD scores. The study was conducted
over three weeks during March of 2012 and took
place in a 20-bed general PICU in atertiary-care aca-
demic medical center in New York City.
All patients were eligible unless they had a sedation
scoreequal toorlowerthan –3(deeplysedatedorun-
arousable) on the Richmond Agitation and Sedation
Scale (RASS) (Sessler et al., 2002).
Children of varying developmental abilities were
included. They were described as having “significant
clinical developmental delay” if developmental pro-
blems were the cause of an impairment in a child’s
age-appropriate ability to communicate (a symptom
that could affect psychiatrists’ assessment of the
child’s baseline in relevant symptom domains) just
prior to their critical illness.
After informed consent was obtained, two child
psychiatrists conducted diagnostic interviews and
exams on each subject to evaluate for delirium.
Each psychiatrist was blinded as to the other’s con-
clusion. After both assessments were completed, if
a child was diagnosed with delirium by either psy-
chiatrist, this was reported to the medical team car-
ing for the child so that appropriate interventions
could be made. Individual subjects were assessed up
to four times. The study was approved by the insti-
Interrater reliability was quantified using Cohen’s k
coefficient along with its 95% confidence interval.
A two-hour initial training session for the six child
psychiatrist evaluators, led by the first author, was
completed to establish consistency in concepts and
vocabulary among the group. In three subsequent
training sessions, “thinking developmentally” about
delirium, including keeping normal milestones in
mind and a broad range of expectations regarding
opmental stages during critical illness, was empha-
sized. Manifestations of alterations in attention,
consciousness, and cognition were discussed in order
to understand clinical experiences with critically ill
children. This framework was discussed in relation
to each item on our psychiatric assessment work-
sheet, which was based on the Delirium Rating Scale
(DRS-98) (Trzepacz et al., 2001), with the addition of
descriptors, expansion of some categories, and ad-
dition of an item to denote a change in cognition
When a total of 38 assessments were completed, in-
cluding 17 patients, the psychiatric diagnoses were
compared. Our sample included 7 infants under 2
years of age (13/38 assessments, 34%), 4 children
aged 2–5 years (11/38 assessments, 29%), 4 children
Table 1. Psychiatrist interrater reliability (Cohen’s kappa k) for pediatric delirium by assessment (N ¼ 38)
n (%) Two Raters (n)
Delirium Dx by Disputed Delirium Dx
by One Rater (n)**
k (95% CI)
38 (100%)121 0.94 (0.83–1.00)
Abbreviations: m ¼ months; yr ¼ years; Dx ¼ diagnosis; CI ¼ confidence interval; devel. delay ¼ developmental delay.
* 38 paired assessments were administered across 17 patients, with 65% (11/17) undergoing multiple assessments.
** The discordant assessment occurred in a one-month-old non-intubated patient with age-appropriate development.
Silver et al.
aged 6–12 years (11/38 assessments, 29%), and 2
adolescents aged 13–21 years (3/38 assessments,
8%). Of these, 5 had moderate to severe developmen-
tal delay (11/38 assessments, 29%) and 7 were intu-
bated (15/38 assessments, 39%).
Overall(seeTable1),delirium was identified in se-
ven patients, including three infants below two years
of age and three patients with moderate to severe de-
velopmental delay. Among the three children who
underwent multiple assessments and were diag-
nosed with delirium at least once, two showed a fluc-
tuating delirium course. The child psychiatrists
diagnosed delirium in 34% (13/38) of evaluations.
The interrater reliability of the 38 separate psychia-
tric assessments was high, with k ¼ 0.94 (95% CI ¼
When analyzed by age, in children under 2 (13
assessments), Cohen’s k was 0.81 (95% CI ¼ 0.45–
Cohen’s k was 1.00. Likewise, interrater reliability
was high forchildren with and without developmental
delay (k ¼ 1.00 and 0.90 [95% CI, 0.71–1.00], respect-
ively), and regardless of whether or not they were
intubated (k ¼ 1.00 and 0.89 [95% CI ¼ 0.70–1.00],
Diagnosing delirium in children, particularly infants
and children with developmental delay, can be chal-
lenging. The ability to diagnose delirium requires
recognition of a disturbance in consciousness and
set and fluctuating course (criterion C) with a link-
age to a physiologic cause (criterion D) (APA, 2000).
The list of differential diagnoses includes many pre-
sentations in pediatrics and child psychiatry (Smith
servers over a period of time to differentiate delirium
from other clinical issues (Esseveld et al., 2013;
Schieveld et al., 2007; Smith et al., 2009). Delirium
has many comorbidities (e.g., pain, premorbid or
situational anxiety, behavioral regression) that com-
plicate the clinical picture. However, “thinking
developmentally” about infants and children and ob-
taining specific information about each child’s base-
line cognition, communication, and behavior make
this a valid and useful diagnosis.
The one discordant assessment came veryearlyon in
the study and was the case of a one-month-old infant
with acute respiratory failure due to respiratory syn-
cytial viral pneumonia. Sleeping excessively, with no
differentiation of day and night, she was generally
quiet, minimally reactive, and not responding much
to her parents. One psychiatrist diagnosed her de-
creased activity and fluctuating mental status as hy-
poactive delirium. The second psychiatrist felt that
this activity was consistent with a “sick baby” and
did not meet the criteria for delirium. By the follow-
ing day, both psychiatrists independently agreed
that she did not meet the criteria for a delirium diag-
nosis, thougheachnoted intheirassessmentthatthe
patient may have had “subclinical delirium” and nee-
ded to be watched closely. Her deliriogenic medi-
cations has been reduced, and within 24 hours her
awareness, activity, and reactivity improved. On
third assessment, both psychiatrists again agreed
that she was not delirious. This case highlights the
difficulty that may exist in making a definitive diag-
nosis of delirium in young infants, hypoactive delir-
ium at any age, and subclinical cases. However, the
consideration of a diagnosis of delirium led to a clini-
cal pathway that reduced potentiallyoffending medi-
cations, ruled out medical causes of delirium, and
implemented positive environmental interventions,
all of which may have benefited this child (Schieveld
et al., 2009; Smith et al., 2013).
Although it has been generallyaccepted that the psy-
chiatric interview is the “gold standard” for diagnos-
ing delirium, reliability for this clinical diagnosis in
children has not been previously reported. Given
that the diagnosis of delirium in preverbal and devel-
suring that we were able to show a high interrater
reliability for psychiatric diagnosis. With specific at-
tention to normal development in each symptom do-
main, consulting child psychiatrists and other
clinicians can be equipped to diagnose delirium in
medically ill children of nearly all developmental
ages and trajectories.
American Psychiatric Association (APA) (2000). Diagnostic
and Statistical Manual of Mental Disorders: DSM–IV-
Barr, J., Fraser, G.L., Puntillo, K., et al. (2013). Clinical
practice guidelines for the management of pain, agita-
tion, and delirium in adult patients in the intensive
care unit. Critical Care Medicine, 41, 263–306.
Esseveld, M.M., Leroy, P.L.M.N., Leue, C., et al. (2013).
Catatonia and refractory agitation in an updated flow-
chart for the evaluation of emotional–behavioral dis-
turbances in severely ill children. Intensive Care
Medicine, 39, 528–529.
Janssen, N.J., Tan, E.Y., Staal, M., et al. (2011). On the uti-
lity of diagnostic instruments for pediatric delirium in
Pediatric delirium: Evaluating the gold standard
critical illness: An evaluation of the Pediatric Anesthe- Download full-text
sia Emergence Delirium Scale, the Delirium Rating
Scale 88, and the Delirium Rating Scale–Revised R-
98. Intensive Care Medicine, 37, 1331–1337.
Karnik, N.S., Joshi, S.V., Paterno, C., et al. (2007). Sub-
types of pediatric delirium: A treatment algorithm. Psy-
chosomatics, 48, 253–257.
Leentjens, A.F., Schieveld, J.N., Leonard, M., et al. (2008).
A comparison of the phenomenology of pediatric, adult,
and geriatric delirium. Journal of Psychosomatic Re-
search, 64, 219–223.
Madden, K., Turkel, S., Jacobson, J., etal. (2011). Recurrent
delirium after surgery for congenital heart disease in an
infant. Pediatric Critical Care Medicine, 12, e413–415.
Schieveld, J.N., Leroy, P.L., van Os, J., et al. (2007). Pedi-
correlates and treatment response in 40 cases in the
pediatric intensive care unit. Intensive Care Medicine,
Schieveld, J.N., van der Valk, J.A., Smeets, I., et al. (2009).
Diagnostic considerations regarding pediatric delirium:
tensive care units. Intensive Care Medicine, 35,
Schieveld, J.N., Staal, M., Voogd, L., et al. (2010). Refrac-
tory agitation as a marker for pediatric delirium in
very young infants at a pediatric intensive care unit. In-
tensive Care Medicine, 36, 1982–198.
Sessler, C.N., Gosnell, M.S., Grap, M.J., et al. (2002). The
Richmond Agitation–Sedation Scale: Validity and re-
liability in adult intensive care unit patients. American
Journal of Respiratory and Critical Care Medicine, 166,
Silver, G.H., Kearney, J.A., Kutko, M.C., et al. (2010). In-
fant delirium in pediatric critical care settings. Ameri-
can Journal of Psychiatry, 167, 1172–1177.
Silver, G., Traube, C., Kearney, J., et al. (2012). Detecting
pediatric delirium: Development of a rapid observa-
tional assessment tool. Intensive Care Medicine, 38,
Smith, H.A., Fuchs, D.C., Pandharipande, P.P., et al.
(2009). Delirium: An emerging frontier in the manage-
ment of critically ill children. Critical Care Clinics, 25,
Smith, H.A.B.,Boyd, J., Fuchs, D.C., et al. (2011). Diagnos-
ing delirium in critically ill children: Validity and re-
liability of the pediatric confusion assessment method
for the intensive care unit. Critical Care Medicine, 39,
Smith, H.A.B., Brink, E., Fuchs, D.C., et al. (2013). Pedi-
atric intensive care unit. Pediatric Clinics of North
America, 60, 741–760.
Traube, C., Silver, G., Kearney, J., et al. (2013). Cornell As-
sessment of Pediatric Delirium: Avalid, rapid, observa-
tional tool for screening delirium in the PICU. Critical
Care Medicine, 42(3), 656–663.
Trzepacz, P.T., Mittal, D., Torres, R., et al. (2001). Vali-
dation of the Delirium Rating Scale–Revised-98: Com-
parison with the delirium rating scale and the
cognitive test for delirium. The Journal of Neuropsy-
chiatry and Clinical Neurosciences, 13, 229–242.
Turkel, S.B., Braslow,K., Tavare, C.J., etal. (2003). Thede-
lirium rating scale in children and adolescents. Psycho-
somatics, 44, 126–129.
Turkel, S.B., Trzepacz, P.T. & Tavare, C.J. (2006). Compar-
ing symptoms of delirium in adults and children. Psy-
chosomatics, 47, 320–324.
Turkel, S.B., Jacobson, J.R. & Tavare ´, C.J. (2013). The di-
agnosis and management of delirium in infancy. Jour-
nal of Child and Adolescent Psychopharmacology, 23,
Silver et al.