Content uploaded by Irina Castellanos
Author content
All content in this area was uploaded by Irina Castellanos on Feb 21, 2018
Content may be subject to copyright.
Content uploaded by Irina Castellanos
Author content
All content in this area was uploaded by Irina Castellanos on Nov 15, 2017
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=hapc20
Applied Neuropsychology: Child
ISSN: 2162-2965 (Print) 2162-2973 (Online) Journal homepage: http://www.tandfonline.com/loi/hapc20
Questionnaire-based assessment of executive
functioning: Psychometrics
Irina Castellanos, William G. Kronenberger & David B. Pisoni
To cite this article: Irina Castellanos, William G. Kronenberger & David B. Pisoni (2018)
Questionnaire-based assessment of executive functioning: Psychometrics, Applied
Neuropsychology: Child, 7:2, 93-109, DOI: 10.1080/21622965.2016.1248557
To link to this article: https://doi.org/10.1080/21622965.2016.1248557
Published online: 14 Nov 2016.
Submit your article to this journal
Article views: 78
View related articles
View Crossmark data
Citing articles: 1 View citing articles
APPLIED NEUROPSYCHOLOGY: CHILD
2018, VOL. 7, NO. 2, 93–109
http://dx.doi.org/10.1080/21622965.2016.1248557
Questionnaire-based assessment of executive functioning: Psychometrics
Irina Castellanosa, William G. Kronenbergerb, and David B. Pisonic
aDepartment of Otolaryngology - Head and Neck Surgery, Ohio State University, Columbus, Ohio, USA; bDepartment of Psychiatry, Indiana
University School of Medicine, Indianapolis, Indiana, USA; cDepartment of Psychological and Brain Sciences, Indiana University Bloomington,
Bloomington, Indiana, USA
ABSTRACT
The psychometric properties of the Learning, Executive, and Attention Functioning (LEAF) scale
were investigated in an outpatient clinical pediatric sample. As a part of clinical testing, the LEAF
scale, which broadly measures neuropsychological abilities related to executive functioning and
learning, was administered to parents of 118 children and adolescents referred for psychological
testing at a pediatric psychology clinic; 85 teachers also completed LEAF scales to assess reliability
across different raters and settings. Scores on neuropsychological tests of executive functioning
and academic achievement were abstracted from charts. Psychometric analyses of the LEAF scale
demonstrated satisfactory internal consistency, parent-teacher inter-rater reliability in the small to
large effect size range, and test–retest reliability in the large effect size range, similar to values for
other executive functioning checklists. Correlations between corresponding subscales on the LEAF
and other behavior checklists were large, while most correlations with neuropsychological tests of
executive functioning and achievement were significant but in the small to medium range. Results
support the utility of the LEAF as a reliable and valid questionnaire-based assessment of delays and
disturbances in executive functioning and learning. Applications and advantages of the LEAF and
other questionnaire measures of executive functioning in clinical neuropsychology settings are
discussed.
KEYWORDS
Assessments; attention;
behavioral ratings; executive
function; learning; working
memory
Executive functioning (EF) refers to top-down neurop-
sychological processes responsible for the active regu-
lation of controlled attention, emotion, and planned
behavior in the service of goal attainment (Banich,
2009; Barkley, 2012). Although there is no single,
universally agreed-upon definition of EF, most concep-
tualizations of EF include several related but separate
processing domains primarily mediated by neural
circuits in the prefrontal cortex (Barkley, 2012;
Luria, 1966). These core executive functions include
self-directed attention (considered the central executive
because it serves a gatekeeping role for subsequent
executive functions), working memory (simultaneous
processing and storage of a stimulus/event), response
inhibition (controlled suppression of a prepotent or
automatic response to a stimulus/event), cognitive
flexibility (shifting between mental states, responses or
tasks), and fluency (rapid processing under con-
centration demands; Awh, Vogel, & Oh, 2006; Barkley,
2012; McAuley & White, 2011; Miyake, Friedman,
Emerson, Witzki, & Howerter, 2000). Neuropsycholo-
gists have been interested in early identification and
interventions for EF deficits for many years, because
delays and disturbances in EF are present at elevated
rates in many disorders resulting from central nervous
system disorder or injury such as spina bifida, cerebral
palsy, epilepsy, traumatic brain injury, and cancer (Daly
& Brown, 2007; Horton, Soper, & Reynolds, 2010;
O’Hara & Holmbeck, 2013; Parrish et al., 2007;
Weierink, Vermeulen & Boyd, 2013).
Poor EF is a significant clinical health issue not only
due to its prevalence in neurological injury and disorder,
but also due to the influence of poor EF on academic
outcomes such as memory problems, educational failure
(Barkley, 2012) and learning disabilities (Jerman,
Reynolds, & Swanson, 2012). EF delays, as well as disor-
ders characterized by poor EF, are commonly associated
with academic underachievement, learning deficits, and
related problems with learning and memory (Barkley,
2012). Because of the significant role of EF for learning,
memory, and academic outcomes, assessment of EF
delays that might impact significantly on school success
is a common component of neuropsychological evalu-
ation of children with significant academic deficits.
CONTACT Irina Castellanos Irina.Castellanos@osumc.edu Buckeye Center for Hearing and Development, 915 Olentangy River Road, Room 4000,
Columbus, OH 43212.
© 2016 Taylor & Francis Group, LLC
Core executive functions and related
cognitive abilities
In recent years, researchers have attempted to separate
skills that constitute core executive functions from
closely related cognitive abilities such as learning and
memory that are dependent on EF but that are not
central to the EF construct. Skills such as working
memory, inhibition, and flexibility are universally
accepted as core EF abilities because they are necessary
for focused, goal-directed activity and have been
supported by empirical research such as factor analysis
(Miyake et al., 2000). Other core domains of EF include
controlled attention, sustained sequential processing
(planning and execution of goal-directed behavior), novel
problem-solving, and organization, all of which are neces-
sary in order to initiate and complete purposeful, planned,
goal-directed activities and have been validated as core
components of EF in prior research (Barkley, 2011b;
Diamond, 2013; Naglieri & Goldstein, 2013). Organiza-
tion, planning, working memory, flexibility, controlled
attention, and inhibition skills, for example, are routinely
assessed using questionnaire-based measures of EF such
as the BRIEF, CEFI, and BDEFS-CA (Barkley, 2011b;
Gioia, Isquith, Guy, & Kenworthy, 2000; Naglieri &
Goldstein, 2013).
On the other hand, cognitive processing skills
deployed during learning that are not core to EF but
are closely related to (and heavily dependent on) EF
include concept formation (Barkley, 2012), comprehen-
sion (Gathercole & Baddeley, 1993), factual memory
(Buckner, 2004), and academic functioning (Diamond,
2016; Gathercole, Pickering, Knight, & Stegmann,
2004). These learning-related areas of cognitive func-
tioning are an integral part of academic learning and
higher-order cognitive processing that are common
weakness areas for individuals with EF delays. For
example, factual memory is dependent on the active
deployment of working memory (a core EF) in learning
situations, while concept formation requires flexibility
and novel problem-solving (core EF) skills. Because
these learning-related domains of cognitive functioning
are dependent on EF and because of the critical role of
EF in learning and academic success, these learning-
related domains of cognitive functioning are often
included in assessments of EF and are considered to be
at risk in children who have EF delays (Barkley, 2012).
Assessment of executive functioning using
behavior checklists
Clinical evaluation of EF typically includes an office-
based visit involving administration of a battery of
behaviorally-based neuropsychological assessment
instruments. Decades of research support the validity,
utility, and diagnostic value of these instruments for
the measurement of EF in children and adults across a
wide range of disorders (Lezak, 2012). Despite their
advantages, however, individually-administered neuro-
psychological measures of EF have two primary limita-
tions: First, in most cases, they must be individually
administered and scored by a technician or professional
in an office setting, which limits their utility for screen-
ing or brief assessment purposes. Second, relations
between office-based neuropsychological measures of
EF and actual behavior in the daily environment are
modest (Barkley, 2012), leading to some caution when
applying neuropsychological test results to conclusions
about behavioral outcomes. As a result of these limita-
tions of office-based neuropsychological tests of EF, par-
ent- and teacher-report behavior checklist measures of
EF have been developed for both screening purposes
and to complement the results of performance-based
neuropsychological testing by providing reports of EF
behavior in daily life (Barkley, 2011b; Gioia et al.,
2000; Naglieri & Goldstein, 2013). These checklists have
the advantage of good psychometrics, strong ecological
validity, and high clinical utility as a result of their ease
of administration, scoring, and interpretation. Further-
more, questionnaire-based assessment of EF may be
especially valuable in busy applied multidisciplinary
settings (such as pediatric specialty clinics) because of
the potential importance of EF in coping, adaptive
behavior, learning, adjustment, and self-management
skills in response to medical and neuropsychological
conditions. Therefore, EF behavior checklists offer
the potential to enhance clinical practice in pediatric
neuropsychology and to promote the application of EF
research in the clinical setting.
One of the first questionnaire-based measures of EF
was the Behavior Rating Inventory of Executive Func-
tioning (BRIEF; Gioia et al., 2000), recently revised as
the BRIEF-2 (Gioia, Isquith, Guy, & Kenworthy, 2016).
The BRIEF and BRIEF-2 forms are parent-, teacher-,
and self-report behavior checklists for children and
adolescents. BRIEF-2 EF subscales assess areas including
inhibition, self-monitoring, shifting, emotional control,
initiation, task completion, working memory, planning/
organizing, task monitoring, and organization of materi-
als. In addition to the BRIEF, other checklist measures of
EF exist, including the Comprehensive Executive Func-
tion Inventory (CEFI, for children aged 5–18 years;
Naglieri & Goldstein, 2013) and the Barkley Deficits in
Executive Functioning Scale (BDEFS for adults; Barkley,
2011a; BDEFS-CA for children and adolescents; Barkley,
2011b). The CEFI measures EF domains such as
94 I. CASTELLANOS ET AL.
attention, emotion regulation, flexibility, inhibitory con-
trol, initiation, organization, planning, self-monitoring,
and working memory. The BDEFS measures EF in daily
life activities such as time management, organization and
problem solving, self-restraint, self-motivation, and self-
regulation of emotions. These EF checklists have been
effectively used in clinical and research settings as screen-
ing measures of EF delays and disturbances, as primary
measures of EF in clinical populations, and as comp-
lementary measures in addition to traditional individu-
ally-administered neuropsychological tests (Ebrahimi
et al., 2015; García, Rodriguez, González-Castro, &
Areces, 2014; Naglieri & Goldstein, 2014).
Current behavior checklist measures of EF such as the
BRIEF, CEFI, and BDEFS-CA have been demonstrated
to be reliable, valid, and useful in both clinical and
research settings, but each of these existing EF scales
focuses on a specific set of core EF skills that excludes
some cognitive functioning domains that are considered
to be core or related to EF. For example, the BRIEF and
CEFI include subscales measuring flexibility-shifting,
whereas BDEFS-CA does not have a flexibility-shifting
subscale. Conversely, the BRIEF does not have a con-
trolled attention subscale (most of the attention items
on the BRIEF fall in the Working Memory subscale),
while the CEFI has an attention subscale. Additionally,
the major established behavior checklist measures of
EF do not include scales measuring related cognitive
domains that are crucial for learning such as compre-
hension and declarative-factual memory, and in fact,
there are very few behavior checklist measures that
evaluate both core EF domains and learning-related
domains that are dependent on EF. Therefore, there is
an unmet need for an EF checklist that evaluates a
broader set of EF-related functions (those that are
grounded in EF capabilities and prefrontal cortical
activity), encompassing both a broad set of core EF
domains and a set of learning and higher-order cognitive
processing domains that are related (but not core) to EF.
Furthermore, a significant need exists for an EF behavior
checklist that is based on a highly simplified administra-
tion and scoring methodology that allows for very easy
use in clinical settings.
A new behavioral checklist of executive
functioning: The Learning, Executive, and
Attention Functioning (LEAF) scale
We sought to create a reliable and valid instrument to
meet these needs for comprehensive EF-related assess-
ment and simplified administration and scoring: the
Learning, Executive, and Attention Functioning (LEAF)
scale. The primary purpose of the LEAF is the
measurement of executive functioning and related
learning skills. Unlike other EF scales, the LEAF was
developed primarily to assist with EF assessment when
cognitive and learning factors are a core component of
concern. As a result, the LEAF falls at the intersection
of EF and learning abilities and includes EF components
that are closely related to learning, as well as learning
domains that are vulnerable to EF delays.
The LEAF fills an important niche in the clinical
assessment of EF in neuropsychological settings that is
not fully addressed by the few existing questionnaire
measures of EF, in the following ways: (1) The LEAF
evaluates a broad set of core cognitive EF abilities as well
as related cognitive learning and academic abilities,
which are frequently seen in clinical populations with
central nervous system disease or injury and are not
addressed by existing questionnaires. Core cognitive
EF areas assessed by the LEAF include attention, proces-
sing speed under conditions requiring concentration,
organization (including visual-spatial organization
skills), sustained sequential processing to achieve goals
(e.g., planning and executing goal-directed behavior),
working memory, and novel problem-solving (Barkley,
2012; McAuley & White, 2011; Zelazo, Carter, Reznick,
& Frye, 1997). Related cognitive learning areas assessed
by the LEAF include comprehension and concept
formation, declarative/factual memory, and academic
functioning. For example, previous research indicates
that comprehension and concept formation are heavily
dependent on working memory skills (Castellanos
et al., 2015), and retrieving information from long-term
declarative/factual memory is dependent on initial atten-
tion and working memory skills during encoding and
processing (Ragland et al., 1998). Additionally, previous
research indicates that core EF skills (working memory
and inhibition) predict academic competence and
achievement in school-age children (Gathercole et al.,
2004). Consequently, the LEAF contains Academic
subscales assessing reading, writing, and math fluency,
abilities, which are supported in part by the development
of core EF skills. (2) The LEAF emphasizes EF domains
related to everyday learning and cognitive functioning as
opposed to behavioral psychopathology and psychiatric
diagnoses, using item wordings that focus more on
information processing than on behaviorally-related
problems. For example, LEAF working memory items
are conceptualized as the child’s ability to retain and
process complex information received from the environ-
ment under conditions of concurrent cognitive load
(“Gets overwhelmed if required to learn or attend to a
lot of information,” see Appendix), as opposed to BRIEF
Working Memory scale items, which emphasize
attention problems (Gioia et al., 2000). (3) The LEAF
APPLIED NEUROPSYCHOLOGY: CHILD 95
was constructed to meet all of Levy, Kronenberger, and
Dunn’s (2013) characteristics of a clinically useful
behavior checklist: brevity in administration, breadth
of additional relevant content, efficiency of scoring and
interpretation, and ease of availability for use. For
example, LEAF items are grouped by subscale, and all
subscales have the same number of items, in order to
facilitate rapid scoring in the busy clinical setting,
without the need for templates or a computer. By
addressing these characteristics, the LEAF was con-
structed to enhance the feasibility of routine
questionnaire-based EF assessment in clinical settings.
In this present paper, we describe the development
and psychometrics of the LEAF as a questionnaire
measure of EF that complements existing measures
and can be used to assess a broad and inclusive set of
EF components and related learning and academic
constructs.
Methods
Participants
Participants were 118 children and adolescents between
the ages of 6 and 17 years (M age ¼11.80 years, SD ¼3.23
years; 71 males, 47 females; 104 White, 7 African-
American, 3 Asian, 2 Hispanic, 1 Indian-Asian, and 1
of unknown ethnic background). The sample was
obtained from consecutive referrals of patients who were
seen for psychological testing at an outpatient clinic at a
pediatric hospital in the Midwest region and who
provided a LEAF scale completed by one parent. Across
primary, secondary, and tertiary referral questions coded
by the primary clinician, the most common reasons for
referral were attention/concentration problems (N ¼76),
learning problems (N ¼97), aggression, anger, or opposi-
tionality (N ¼13), and hyperactivity (N ¼8).
Participants in the sample were clinically diagnosed
by the primary testing clinician using criteria from the
Diagnostic and Statistical Manual of Mental Disorders,
Fourth Edition (DSM-IV; American Psychiatric
Association, 1994). Based on psychological testing and
interview results, 55 (47%) participants were diagnosed
with Attention-Deficit/Hyperactivity Disorder (ADHD)
(25 with predominantly Inattentive subtype, 30 with
Combined subtype), 44 (37%) participants were diag-
nosed with a Learning Disorder, 33 (28%) participants
were diagnosed with an Anxiety Disorder (including
adjustment disorders with an anxiety component), 28
(24%) participants were diagnosed with a Pervasive
Developmental Disorder, 22 (19%) participants were
diagnosed with a Depressive Disorder (including adjust-
ment disorders with a depression component), 19 (16%)
participants were diagnosed with a Disruptive Behavior
Disorder (oppositional-defiant disorder or conduct
disorder), 7 (6%) participants were diagnosed with an
Adjustment Disorder, and 3 (3%) participants were
diagnosed with an Elimination Disorder.
Nearly half (N ¼53, 45%) of sample participants had
a pediatric condition, and in 31 (26%) cases, that con-
dition was closely related to the reason for referral for
testing (i.e., cognitive functioning difficulties possibly
secondary to the condition). Neurological conditions
were the most common (N ¼28, 24%; e.g., spina bifida
[N ¼7, 6%], seizure disorder [N ¼7, 6%], brain malfor-
mations /tumors/ cysts [N ¼5, 4%]; cerebral palsy
[N ¼3, 3%]; other neurological conditions [N ¼6,
5%]). Other physical conditions in the sample were
respiratory (apnea, N ¼1, 1%; asthma, N ¼7, 6%),
gastrointestinal (chronic constipation, N ¼1, 1%;
irritable bowel, N ¼3, 3%; recurrent abdominal pain,
N ¼1, 1%), endocrine (diabetes, N ¼1, 1%; hypothyr-
oid, N ¼2, 2%), autoimmune (arthritis, N ¼1, 1%;
lupus, N ¼2, 2%), long-term complications of prema-
turity and very low birthweight (N ¼3, 3%), primary
pain syndromes (fibromyalgia, N ¼1, 1%; chronic head-
ache, N ¼1, 1%), cleft palate (N ¼1, 1%), hearing loss
(N ¼1, 1%), and hyperalaninemia (N ¼1, 1%)
(numbers add to more than the sample size because of
comorbidity). Of these latter non-neurological
conditions, only hearing loss and lupus (rule out neu-
ropsychiatric lupus) were related to the primary reason
for psychological testing.
Primary parent respondents to the LEAF scale were
103 mothers and 15 fathers. The primary focus in this
paper is on the psychometrics of parent-completed LEAF
scales. Teachers of 85 participants also completed the
LEAF scale, and their data are included for inter-rater
reliability and construct validity analyses comparing
LEAF scores and other behavior checklists. Twenty-seven
mothers completed two LEAF scales within the same
35-day period (range ¼10–35 days), providing data to
assess test–retest reliability. Children were tested by a
psychologist or trainee (psychology intern or graduate
student) at a clinic visit that occurred on the same day
that the primary LEAF scale and other parent-report
behavior checklists were completed; LEAF scales and
other behavior checklists completed by teachers were
requested by mail or hand delivery by the parent prior
to the clinic testing session and were received by
mail or fax.
Procedure
Data for the present study were obtained using chart-
review methods approved by the university Institutional
96 I. CASTELLANOS ET AL.
Review Board. Performance data from neuropsychologi-
cal tests, demographic data, and clinically assigned
diagnoses and reasons for referral (as coded by the
clinician responsible for the testing) were abstracted
from test reports and other data in the clinical charts.
LEAF scales and other behavioral checklists, which are
routinely completed by parents and teachers to provide
clinical assessment and monitoring information,
were also abstracted from the clinical charts of all
participants.
Measures
Learning, Executive, and Attention Functioning
(LEAF) scale
The LEAF scale measures executive functioning and
related learning skills in children and adolescents aged
6–17 years. The content-development process consisted
of a literature review pertaining to tests, disorders, and
interventions involving executive functioning and the
impact of executive functioning and related processes
on learning and behavioral adjustment. We identified
traditional components of executive functioning and a
broad range of abilities related to and/or dependent
upon executive functioning, including working memory
(Gioia et al., 2000), sustained sequential processing
(Conway, Pisoni, & Kronenberger, 2009), organization,
comprehension, and novel problem solving (Rourke,
1995). This process generated two Cognitive-Learning
content areas reflecting learning, memory and
reasoning skills closely related to EF but not core to
the EF construct:
1. Comprehension and Conceptual Learning (tracking
and understanding information), and
2. Factual Memory (memorization and retention of
facts); and six Cognitive-EF content areas reflecting
core EF domains:
3. Attention (sustained focus);
4. Processing Speed (speed of completing cognitive
and behavioral tasks that involve a component of
focus and concentration);
5. Visual-Spatial Organization (organization and
visual-constructive skills);
6. Sustained Sequential Processing (planning and
sustaining effort in order to follow and complete
multistep directions and sequences);
7. Working Memory (remembering and processing
multiple things at the same time); and
8. Novel Problem Solving (initiating effort toward
processing new or unfamiliar information). An
additional three Academic content areas were added
in order to enhance the clinical utility of the scale,
because executive functioning deficits are
frequently related to academic achievement diffi-
culties (Gathercole et al., 2004):
9. Mathematics Skills (math calculation difficulty or
dysfluency);
10. Basic Reading Skills (reading/phonics difficulty or
dysfluency); and
11. Written Expression Skills (limited/impoverished or
slow/effortful written expression).
Based on prior work showing that five-item behavior
checklist subscales can be completed, scored, and inter-
preted rapidly in busy clinical settings while providing
good psychometrics (Levy et al., 2013), five items were
created for each of these 11 content areas (items are
grouped by content area, in the order of the 11 content
areas previously described), resulting in a total of 55
items (see Appendix).
Individual items are rated on a 0–3 scale, and a raw
subscale score for each of the 11 content areas is created
by summing the 5 constituent items, such that higher
scores indicate more problems. Because respondents
often have difficulty consistently anchoring response
choices such as “Never,” “Sometimes,” and “Often,”
behavioral descriptors were provided for each of the
response choices: Response choices were anchored by
the following descriptors, “0 - Never: Not a problem;
Average for age,” “1 - Sometimes: A little more than
average; Not a big problem,” “2 - Often: Causes prob-
lems; Happens almost every day,” and “3 - Very Often:
Major daily problem.” Consistent with other rating
scales (Faries, Yalcin, Harder, & Heiligenstein, 2001;
Gadow & Sprafkin, 1997), ratings of “2” (Often) are
encouraged for behaviors that cause problems, whereas
ratings of “1” (Sometimes) are used to reflect behaviors
that may occur more than average but that do not cause
big problems. Therefore, an average rating of “2” for the
five items comprising a subscale would indicate that
behaviors for that subscale were rated, on average, as
causing problems and happening almost every day.
Three criterion-referenced interpretation ranges
(0–4 ¼“No Problem Range”; 5–9 ¼“Borderline Prob-
lem Range”; 10–15 ¼“Problem Range”) were created
for the LEAF.
1
A LEAF raw score of less than 5 indicates
that the average item was rated less than “1,” which has
the anchor statement of “Sometimes; A little more than
average; Not a big problem.” Hence, a raw score of
less than 5 indicates that subscale items were rated, on
average, as sporadic and not a big problem. Subscale
raw scores of 5–9 fall into an intermediate range, with
1
This criterion-referenced strategy for interpretation is not the same as
norm-based scores, since elevated norm scores provide information about
abnormality, and not necessarily functional levels of problems in the
environment.
APPLIED NEUROPSYCHOLOGY: CHILD 97
some problematic behaviors and some behaviors not
rated as problems; scores in this range are therefore
characterized as falling in the “Borderline Problem
Range.” Subscale raw scores of 10 or higher are likely
to indicate moderate to severe problems and fall in
the “Problem Range.” Norm-based scores are not yet
available for the LEAF.
Neuropsychological measures
Participants were also administered several gold-
standard performance-based measures of executive
functioning (attention and concentration) and academic
achievement as a part of their clinical testing. All
assessments selected have well-validated psychometric
properties, and most have published test manuals.
Attention and concentration were assessed using
the Stroop Color and Word Test (SCWT; Golden,
1978; N ¼103), the Counting Interference Test (CIT;
Hummer et al., 2011; N ¼107), and the Conners’
Continuous Performance Test (CPT; Conners & MHS
Staff, 2000; N ¼97). Successful performance on the
SCWT and the CIT requires components of controlled
attention to information relevant to the task, as well
as inhibition of distractors not relevant to the task.
The SCWT measures the ability to inhibit a highly
overlearned/automatic process (word reading) in favor
of a more effortful/controlled process (naming ink
color) for a series of color words (i.e., red, blue, green)
that are printed in ink colors that are incongruent with
the words. The CIT is a counting Stroop-like test for
which participants must state the number of numerals
present in a series of one-, two-, or three-digit numbers
(e.g., 222, 11, 3), suppressing numeral naming in favor
of identifying the number of digits present in the
display. The CPT is a computer-administered test that
measures timing and accuracy of responses to visu-
ally-presented targets versus nontargets. Raw scores
on the color–word condition of the SCWT, the
number-count condition of the CIT (number of accu-
rate responses in 45 seconds), and the Hit Reaction
Time Standard Error (RTSE) score of the CPT (standard
deviation of response speed for all correctly answered
items, which has been shown to be one of the most
sensitive measures of attention problems on the CPT;
Conners & MHS Staff, 2000) were used as measures
of focused attention and concentration.
Academic achievement was assessed using two
subtests and one composite score of the Woodcock-
Johnson III Tests of Achievement, Third Edition (WJ-III;
Woodcock, McGrew, & Mather, 2001), which corre-
sponded to the three academic subscales of the LEAF.
The WJ-III Basic Reading Skills score (a composite of
scores from the Letter-Word Identification and Word
Attack subtests, reflecting reading phonics and word
identification/reading skills; N ¼111) was selected to
correspond to the LEAF Basic Reading Skills subscale.
The WJ-III Calculation subtest (a measure of formal
arithmetic knowledge and math calculation skills;
N ¼112) was selected to correspond to the LEAF
Mathematics Skills subscale. The WJ-III Writing
Samples subtest (a measure of written expression
for sentences that either stand alone or that are
embedded within paragraphs; N ¼106) was selected to
correspond to the LEAF Written Expression Skills
subscale.
Behavior checklists
In addition to the LEAF scale, parents and teachers
completed the BRIEF (Gioia et al., 2000) and the
Conduct-Hyperactive-Attention Problem-Oppositional
Symptom (CHAOS; Levy et al., 2013) scales. The
CHAOS scale is clinically useful for the evaluation of
children diagnosed with ADHD and includes four
subscales, including Attention Problems, Hyperactivity-
Impulsivity, Oppositional Behavior, and Conduct
Problems (only the CHAOS Attention Problems
subscale, which measures carelessness, disorganization,
distractibility, and difficulty sustaining and controlling
attention, was used in this study; Levy et al., 2013).
Similar to the BRIEF, the CHAOS Attention Problems
subscale has high internal consistency (ranging from
.90 .91 for teacher and parent forms), medium-to-
high inter-rater reliability (r ¼.58 for mother-father;
r ¼.41 for parent-teacher), and satisfactory test–retest
reliability over 10–26 weeks (r ¼.78; Levy et al., 2013).
Furthermore, all items on the LEAF, BRIEF, and
CHAOS are rated on a severity scale, such that higher
scores indicate greater symptom severity.
In the present study, the LEAF was compared to five
BRIEF subscales reflecting domains of EF that overlap
in content with LEAF subscales: Initiate (initiation of
tasks/activities, generating ideas or problem solving
strategies; corresponding LEAF subscale is Novel Prob-
lem Solving), Working Memory (short-term working
memory for completing tasks; corresponding LEAF
subscales are Working Memory and Attention), Plan-
Organize (managing current and future tasks demands,
setting goals, organizing oral and written material;
corresponding LEAF subscale is Sustained Sequential
Processing), Organization of Materials (orderliness of
belongings; corresponding LEAF subscale is Visual-
Spatial Organization), and Monitor (self-monitoring;
no single corresponding LEAF subscale, but content
overlaps somewhat with LEAF Attention and Sustained
Sequential Processing subscales). In addition, LEAF
results were compared with the Attention Problems
98 I. CASTELLANOS ET AL.
subscale of the CHAOS (carelessness, disorganization,
distractibility, inattention; corresponding LEAF
subscale is Attention).
Data analyses
All data analyses for the present study, other than
interrater reliability and correlations with other
behavior checklists, used parent-reported LEAF scales,
because the focus of this study was on parent-reports
of EF in daily living. Psychometric evaluation of the
LEAF was performed using widely accepted statistical
techniques (DeVellis, 1991). Item- and subscale-level
analyses and descriptive statistics were reported first,
in order to provide information about the distribution
of LEAF items and scores in this clinically-referred
sample.
Secondly, principal axis factoring was performed on
the 5 items of each LEAF subscale in order to evaluate
the unidimensionality of each LEAF subscale. The
eigenvalue >1 convention was used to evaluate the
number of factors comprising each LEAF subscale, such
that subscales producing only a single factor with an
eigenvalue >1 were considered to be unidimensional.
For subscales yielding more than 1 factor, oblimin
rotation (used when factors are assumed to be
correlated) was used to identify item loadings (see Gioia
et al., 2000 for a similar example of this approach
to investigate unidimensionality of subscales and
relatedness of items).
Next, three measures of reliability were obtained for
LEAF subscales: internal consistency, parent-teacher
inter-rater reliability, and test–retest reliability. Internal
consistency of LEAF subscales was evaluated using
Cronbach’s ∝ >.70 (DeVellis, 2012; DeVon et al.,
2007; Hair, Black, Babin, & Anderson, 2010). For con-
sistency with psychometrics reported for other behavior
checklists of EF, inter-rater reliability and test–retest
reliability were evaluated with Pearson correlational
analyses. Test–retest scores at or above .70 were con-
sidered satisfactory (Nestor & Schutt, 2015). Construct
validity was evaluated with Pearson correlational
analyses between LEAF subscales and other behavior
checklists completed by the same respondent. Positive
correlations between LEAF, BRIEF, and CHAOS
subscales suggest that children, across checklists, are
similarly rated as having greater problems. Construct
validity was also examined using Pearson correlational
analyses between LEAF subscales and the perfor-
mance-based neuropsychological tests. Negative corre-
lations between LEAF subscales and SCWT and CIT
scores (higher SCWT and CIT scores indicate better
EF) suggest that children who perform better on these
neuropsychological assessments are also rated as having
fewer problems on the LEAF; in contrast, positive
correlations between LEAF subscales and CPT Reaction
Time Standard Error scores (higher CPT Reaction Time
Standard Error scores indicate more variability in
response times during the test, which is an indication
of poor EF) suggest that children who perform better
on the CPT are rated as having fewer problems on the
LEAF. In line with published conventions, correlations
(r values) were operationally defined as small (r ¼.10),
medium (r ¼.30), or large (r ¼.50; Cohen, 1992).
Finally, the three LEAF criterion-referenced
interpretation ranges (0–4 ¼“No Problem Range”;
5–9 ¼“Borderline Problem Range”; 10–15 ¼“Problem
Range”) were validated using ANOVAs by comparing
mean norm-based scores for BRIEF and WJ-III
subscales/subtests that corresponded to LEAF subscales
across the 3 LEAF interpretation ranges. Significant
ANOVA tests were further evaluated with Pairwise
t-tests. Mean BRIEF and WJ-III norm-based scores for
participants with LEAF scores in the “No Problem
Range” were expected to fall near the mean norm value
(T ¼50 for BRIEF subscales; Standard Score ¼100 for
WJ-III subtests), while those for participants with LEAF
scores in the “Problem Range” were expected to fall at
least 1–2 SD out of the norm range. Mean BRIEF and
WJ-III norm-based scores for participants with LEAF
scores in the “Borderline Problem Range” were expected
to fall between those for participants with LEAF scores in
the “No Problem Range” and “Problem Range.”
Results
Item- and subscale-level descriptive analyses
The mean scores for all 55 parent-report LEAF items fell
between 0.5 and 2.5 on the 0–3 rating scale of response
choices, and for 47 of the 55 items (85%), the mean
score fell between 1 and 2 (see Table 1 for all item-level
analyses). Skewness values for all items were between -1
and þ1, and kurtosis values for all items were less than
0. All 220 item-level response choices (e.g., 55 items,
each of which could be rated 0, 1, 2, or 3) were endorsed
by at least 5% of the sample, 211 of the 220 response
choices (96%) were endorsed by at least 10% of the
sample, and in 97% of cases (213 of the 220 response
choices) at least N ¼10 observations were present for
each LEAF item response choice (Linacre, 2002). These
findings demonstrate that all LEAF item response
choices were endorsed by a reasonable number (at least
5% or more, and in 97% of cases, N ¼10 or more) of
participants in the clinical sample and that item
distributions were not severely skewed.
APPLIED NEUROPSYCHOLOGY: CHILD 99
Table 1. LEAF item-level statistics.
LEAF subscale/item Mean SD Skewness Kurtosis % responding “0” % responding “3”
Cognitive-learning:
Comprehension and conceptual learning
Item 1 0.94 0.88 0.65 0.30 36 6
Item 2 1.31 0.97 0.28 0.85 22 15
Item 3 1.22 1.06 0.30 1.15 32 14
Item 4 1.27 0.94 0.29 0.78 22 12
Item 5 1.30 0.95 0.17 0.89 23 11
Factual memory
Item 6 1.24 1.00 0.34 0.94 27 14
Item 7 1.20 0.99 0.38 0.88 28 13
Item 8 1.04 0.95 0.65 0.42 32 10
Item 9 1.32 0.94 0.31 0.75 20 14
Item 10 1.35 0.93 0.10 0.85 20 11
Cognitive-EF:
Attention
Item 11 1.81 1.01 0.37 0.97 13 31
Item 12 1.97 0.96 0.49 0.82 8 36
Item 13 1.93 0.92 0.33 0.92 6 33
Item 14 2.07 0.90 0.56 0.67 5 39
Item 15 1.46 0.84 0.27 0.51 10 13
Processing speed
Item 16 1.71 1.02 –0.12 1.16 13 29
Item 17 1.44 1.02 0.26 1.04 18 21
Item 18 1.51 1.08 0.02 1.27 22 24
Item 19 1.92 1.06 0.50 1.06 13 40
Item 20 1.94 0.95 0.36 1.00 7 36
Visualspatial organization
Item 21 2.07 1.03 0.71 0.75 10 46
Item 22 2.06 1.04 0.69 0.82 10 46
Item 23 0.85 0.98 0.97 0.11 47 10
Item 24 1.51 1.17 0.04 1.49 26 30
Item 25 1.02 0.92 0.63 0.42 33 9
Sustained sequential processing
Item 26 1.77 1.02 0.27 1.06 13 30
Item 27 1.26 1.07 0.32 1.13 30 17
Item 28 2.20 0.94 0.80 0.58 5 51
Item 29 1.91 1.00 0.39 1.01 9 36
Item 30 1.64 1.02 0.03 1.15 14 26
Working memory
Item 31 1.77 0.96 0.22 0.96 10 27
Item 32 2.10 1.03 0.78 0.69 10 48
Item 33 1.83 0.97 0.39 0.84 11 29
Item 34 1.23 1.08 0.40 1.11 31 18
Item 35 1.64 1.15 0.16 1.42 22 32
Novel problem-solving
Item 36 1.58 0.98 0.05 1.00 15 20
Item 37 1.77 1.02 0.25 1.11 13 31
Item 38 1.14 1.15 0.50 1.21 40 20
Item 39 1.11 1.05 0.54 0.92 36 14
Item 40 0.86 1.02 0.89 0.42 49 10
Academic:
Mathematics skills
Item 41 1.50 1.19 0.31 1.52 28 30
Item 42 1.47 1.20 0.07 1.53 30 29
Item 43 1.65 1.10 0.16 1.31 20 30
Item 44 1.19 1.21 0.45 1.38 41 24
Item 45 1.56 1.17 0.03 1.47 25 31
Basic reading skills
Item 46 1.26 1.09 0.37 1.16 30 19
Item 47 1.07 1.02 0.56 0.84 36 12
Item 48 1.15 1.09 0.45 1.11 36 15
Item 49 1.17 1.03 0.46 0.92 31 14
Item 50 1.28 1.07 0.33 1.13 28 18
Written expression skills
Item 51 1.50 1.10 0.03 1.30 22 24
Item 52 1.71 1.18 0.28 1.42 22 35
Item 53 1.88 1.01 0.38 1.03 10 34
Item 54 1.85 1.10 0.50 1.09 17 36
Item 55 1.83 1.07 0.39 1.13 14 34
Note. For all items, range of answers was 0–3. SD ¼Standard Deviation; % responding “0” and % responding “3” ¼percentage of sample responding
“0” (Never: not a problem; average for age) or “3” (Very Often: major daily problem), respectively, to the item. Values are not reported for % responding
“1” and % responding “2” because these item responses were endorsed by 10% or more of the sample for all items.
100 I. CASTELLANOS ET AL.
Descriptive data for parent-report LEAF subscales by
age (6–11 vs. 12–17 years) are depicted in Table 2. All
LEAF subscales had mean raw scores between 5 and 10
in this clinically referred sample, reflecting an average
item endorsement of approximately 1 (sometimes) to 2
(often). Large standard deviations for subscale raw
scores (3 to 5 raw score points) suggest considerable
variability in LEAF subscale scores within the sample.
No differences were found between the age groups
(p >0.10), reflecting the fact that all participants were
clinically referred for psychological testing.
Factor and reliability analysis of LEAF subscales
For 10 of the 11 factor analyses of LEAF subscales (all
except the Visual-Spatial Organization subscale), one
factor accounted for more than 50% of the variance
(e.g., eigenvalue >2.5), and a single-factor solution
was supported by the eigenvalue >1 convention and
inspection of scree plots (table of results available upon
request from the authors). Hence, factor analysis results
supported the unidimensionality of these 10 LEAF
subscales.
However, for the Visual-Spatial Organization
subscale, two factors had eigenvalues greater than 1
(eigenvalues for the five factors were 2.27, 1.16, 0.78,
0.53, and 0.26). Oblimin rotation of a two-factor solution
resulted in two correlated factors (r ¼0.373). Factor 1
consisted of Items 21 (Poor organization; Factor 1
Loading ¼0.77, Factor 2 Loading ¼0.13) and 22 (Room,
desk, locker, work area is very messy; Factor 1 ¼0.95,
Factor 2 ¼ 0.13). Factor 2 consisted of Items 23 (Not
very good with puzzles or putting things together; Factor
1 ¼ 0.08, Factor 2 ¼0.59) and 24 (Drawing and/or
handwriting poor; Factor 1 ¼0.08, Factor 2 ¼0.76).
Item 25 (Doesn’t pay attention to visual details in the
environment) loaded on both factors about equally
(0.23 and 0.27, respectively).
These findings suggest that the Visual-Spatial
Organization subscale consists of two related groups
of items reflecting organization and visual-spatial skills.
Although the medium correlation between these groups
of items and the medium-to-large corrected item-
to-total correlations of all items on the subscale (ranging
from 0.31 to 0.60) support the aggregation of items into
a single subscale score, the factor analysis result also
indicates that the Visual-Spatial Organization subscale
consists of two distinct but related factors. Therefore,
the total subscale score for Visual-Spatial Organization
is included in all of the analyses in this paper with the
caveat that two related subgroups of items may exist
on this subscale.
Table 3 provides values for internal consistency,
inter-rater, and test–retest reliability. Internal consist-
ency values were .79 or higher for all LEAF subscales
with the exception of Visual-Spatial Organization (.69).
Corrected item-to-total correlations for each LEAF item
with its respective subscale score (e.g., item correlation
with the sum of the other 4 items on each subscale) were
large (>0.50) for 50 of the 55 LEAF items (a table with
item-total correlations is available from the authors);
for 3 items, item-to-total correlations were 0.40–0.49,
and for 2 items, item-to-total correlations were
0.30–0.39. No LEAF item had a corrected item-to-total
correlation of less than 0.30, and all corrected item-
to-total correlations were statistically significant
(p <0.05).
Table 2. Parent-report LEAF subscale scores by age.
LEAF subscale
6–11 Years 12–17 Years
t Mean SD Mean SD
Cognitive-learning:
Comprehension and conceptual
learning
6.33 3.86 5.82 3.93 0.71
Factual memory 6.02 3.98 6.25 4.12 0.31
Cognitive-EF:
Attention 9.94 4.01 8.72 4.07 1.63
Processing speed 8.78 3.72 8.33 3.89 0.64
Visual-spatial organization 7.57 3.41 7.42 3.49 0.24
Sustained sequential processing 9.06 3.86 8.58 4.15 0.64
Working memory 8.98 3.95 8.27 4.20 0.94
Novel problem-solving 6.61 4.03 6.34 4.36 0.34
Academic:
Mathematics skills 7.41 5.30 7.34 4.99 0.07
Basic Reading skills 6.45 4.79 5.25 4.84 1.34
Written Expression skills 8.98 4.18 8.22 4.97 0.88
Note. Values are raw scores. SD ¼Standard Deviation. T-tests are
comparisons of the two age groups; zero t-tests were statistically
significant (df ¼116, all p >0.10).
Table 3. LEAF subscale reliability and validity.
LEAF subscale
Internal
consistency
Interrater
reliability
parent–
teacher,
N ¼85
Test–retest
reliability
mother
10–35 days,
N ¼27
Cognitive-Learning:
Comprehension and
conceptual learning
.87 .46*** .82***
Factual memory .90 .47*** .74***
Cognitive-EF:
Attention .92 .37*** .88***
Processing speed .79 .33** .75***
Visual-spatial organization .69 .54*** .77***
Sustained sequential
processing
.86 .34*** .82***
Working memory .84 .28** .78***
Novel problem-solving .86 .37*** .83***
Academic:
Mathematics skills .92 .57*** .83***
Basic reading skills .95 .51*** .88***
Written expression skills .88 .41*** .88***
Note. Values for internal consistency are Cronbach’s ∝. Values for interrater
and test–retest reliability are Pearson correlation coefficients.
**p <0.01.
***p <0.001.
APPLIED NEUROPSYCHOLOGY: CHILD 101
Parent–teacher inter-rater reliability reached statisti-
cal significance for all of the LEAF subscales, and most
parent-teacher correlations were in the medium to large
effect size range (e.g., correlations fell between 0.33 and
0.57 for 10 of the 11 LEAF subscales). Test–retest
correlations for mothers completing two LEAF scales
within the same 10–35 day period were statistically sig-
nificant and ranged from 0.74 to 0.88 for all subscales
(see Table 3).
Construct validity
Behavior checklists
Correlations between the LEAF, CHAOS, and BRIEF
are reported in Table 4. The parent-reported LEAF
Cognitive-Learning and Cognitive-EF subscales corre-
lated significantly with CHAOS Attention Problems
and BRIEF subscales in the expected direction for all
correlations except one (LEAF Factual Memory with
BRIEF Organization of Materials). Nearly 2/3 of the
correlations exceeded r ¼.50. Importantly, parent-
reported LEAF Cognitive-Learning and Cognitive-EF
subscales showed especially large correlations with
CHAOS or BRIEF subscales measuring corresponding
domains of executive functioning: LEAF Attention with
CHAOS Attention Problems (r ¼.84, p <.001) and
BRIEF Working Memory (r ¼.74, p <.001); LEAF
Visual-Spatial Organization with BRIEF Plan/Organize
(r ¼.71, p <.001) and BRIEF Organization of Materials
(r ¼.60, p <.001); LEAF Sustained Sequential Proces-
sing with BRIEF Plan/Organize (r ¼.67, p <.001);
LEAF Working Memory with CHAOS Attention Prob-
lems (r ¼.70, p <.001) and BRIEF Working Memory
(r ¼.67, p <.001); and LEAF Novel Problem-Solving
with BRIEF Initiate (r ¼.61, p <.001). On the other
hand, correlations for LEAF Cognitive-Learning and
Cognitive-EF subscales that had no corresponding
subscale on the CHAOS or BRIEF were generally
much lower, although most were statistically
significant: Comprehension and Conceptual Learning
(all r <0.55; median r ¼0.50), Factual Memory
(all r <0.52; median r ¼0.42), and Processing Speed
(all r <0.56; median r ¼0.44). Correlations between
the LEAF Academic subscales and CHAOS and BRIEF
subscales were also substantially lower than correla-
tions between the corresponding LEAF and CHAOS/
BRIEF subscales. Most teacher-reported LEAF
Cognitive-Learning and Cognitive-EF subscales also
showed large correlations with teacher-reported
BRIEF and CHAOS subscales (Table 4). As with
parent-reported LEAF subscales, teacher-reported
LEAF subscales typically showed the highest correla-
tions with CHAOS and BRIEF subscales measuring
the same (or similar) type of cognitive/executive
function.
Neuropsychological tests
Correlations between the LEAF Cognitive-Learning and
Cognitive-EF subscales and individually-administered
neuropsychological measures of executive functioning
(Stroop Color-Word, CIT Number-Count, CPT) are
reported in Table 5. Five out of eight LEAF Cognitive-
Learning and Cognitive-EF subscales (Comprehension
and Conceptual Learning, Attention, Processing Speed,
Visual-Spatial Organization, and Working Memory)
were consistently correlated with Stroop Color-Word
and CIT Number-Count scores. Conners’ CPT Reaction
Time Standard Error scores were less consistently corre-
lated with LEAF Cognitive-EF subscale scores, with
small but significant correlations found only for the
LEAF Attention and Working Memory subscales. For
the WJ-III Tests of Achievement, LEAF Academic sub-
scales (Mathematics, Basic Reading, and Written
Expression) correlated most strongly with correspond-
ing WJ-III composite and subtest scores (Calculation,
Basic Reading, and Writing Samples; Table 6).
BRIEF and WJ-III scores for LEAF criterion-
referenced interpretation ranges
Table 7 depicts the means and standard deviations for
BRIEF and WJ-III scores for participants falling into
the three LEAF criterion-referenced interpretation
ranges for each LEAF subscale. For each LEAF subscale,
a corresponding BRIEF or WJ-III score was selected
based on similarity of the construct measured: BRIEF
Working Memory for LEAF Attention and LEAF
Working Memory; BRIEF Organization of Materials
for LEAF Visual-Spatial Organization; BRIEF Plan/
Organize for LEAF Sustained Sequential Processing;
BRIEF Initiate for LEAF Novel Problem-Solving; WJ-III
Calculation for LEAF Mathematics Skills; WJ-III Basic
Reading for LEAF Basic Reading Skills; and WJ-III
Writing Samples for LEAF Written Expression Skills.
Three LEAF subscales (Comprehension and Conceptual
Learning, Factual Memory, and Processing Speed) did
not have corresponding BRIEF or WJ-III subtest/
subscale scores and were excluded from analysis.
ANOVAs comparing participants in the three
LEAF interpretation ranges were statistically significant
(p <0.001, Table 7) for all tested LEAF subscales.
Follow-up pairwise t-tests showed that participants in
the “No Problem Range” scored significantly lower than
those in the other two ranges and that participants in
the “Problem Range” scored significantly higher than
those in the other two ranges (p <0.05 for all pairwise
102 I. CASTELLANOS ET AL.
Table 4. Relationships between LEAF, CHAOS, and BRIEF.
LEAF Subscale
CHAOS Subscale BRIEF subscale
Attention problems Initiate Working memory Plan-organize Organization of materials Monitor
N Parent 117 Teacher 74 Parent 114 Teacher 26 Parent 114 Teacher 26 Parent 114 Teacher 26 Parent 114 Teacher 26 Parent 114 Teacher 26
Cognitive-learning:
Comprehension and Conceptual Learning .54*** .49*** .47*** .54** .52*** .60** .45*** .38a .22* .22 .53*** .43*
Factual Memory .41*** .39** .43*** .51** .51*** .55** .45*** .35a .16a .16 .39*** .29
Cognitive-EF:
Attention .84*** .79*** .59*** .82*** .74*** .85*** .57*** .80*** .39*** .60** .61*** .82***
Processing Speed .55*** .51*** .44*** .56** .52*** .58** .43*** .59** .26** .45* .33*** .44*
Visual-Spatial Organization .72*** .67*** .61*** .72*** .60*** .63** .71*** .63** .60*** .64*** .73*** .66***
Sustained Sequential Processing .78*** .71*** .59*** .77*** .68*** .73*** .67*** .70*** .47*** .58** .59*** .73***
Working Memory .70*** .59*** .59*** .69*** .67*** .60** .58*** .50** .41*** .35a .59*** .54**
Novel Problem-Solving .56*** .47*** .61*** .61** .49*** .48* .49*** .33a .29** .18 .53*** .44*
Academic:
Mathematics Skills .18
a
.40*** .29** .58** .20* .57** .23* .49* .06 .31 .29** .38a
Basic Reading Skills .36*** .30* .24* .37a .31** .37a .31** .20 .20* .08 .26** .29
Written Expression Skills .41*** .47*** .34*** .62** .34*** .57** .41*** .52** .22* .24 .44*** .52**
Note. CHAOS ¼Conduct-Hyperactive-Attention Problem-Oppositional Symptom. BRIEF ¼Behavior Rating Inventory of Executive Function. Complete sample includes 118 parent-completed and 85 teacher-completed LEAF
forms. Values are Pearson correlation coefficients between behavior checklist measures completed by the same respondent.
a
p <0.10.
*p <0.05.
**p <0.01.
***p <0.001.
103
comparisons). For the “No Problem” group on all LEAF
subscales, mean BRIEF T-scores and WJ-III standard
scores were within ½ SD of the normative mean,
indicating average behaviors and skills. For the “Prob-
lem Group” on all LEAF subscales except Visual-Spatial
Organization, mean BRIEF subscale scores were over 2
SDs above the normative mean (e.g., T >70), and mean
WJ-III scores were approximately 1 SD below the
normative mean (e.g., Standard Score of approximately
85). These findings are indicative of problems in these
areas measured by BRIEF and WJ-III norm-referenced
scores. As expected, participants scoring in “Borderline
Problem” LEAF ranges had BRIEF and WJ-III scores
that fell between those for the “No Problem” and
“Problem” groups (Table 7).
Discussion
EF is an umbrella term used to describe a broad cluster
of related cognitive and emotional abilities necessary for
self-directed goal attainment (Banich, 2009; Barkley,
2012). The present study provides evidence supporting
the reliability and validity of the LEAF scale as a
behavior checklist measure of broad executive and
related functioning in children and adolescents.
Reliability analyses demonstrated the unidimension-
alty of subscales (with the possible exception of
Visual-Spatial Organization, which showed a hierarchi-
cal structure of two sub-factors contributing to a larger
global subscale score), strong internal consistency
(Cronbach’s ∝ >.70 for all subscales except one with
∝ ¼0.69), adequate inter-rater reliability between par-
ents and teachers (median r ¼0.41 for the 11 subscales),
and strong test–retest reliability (median r ¼0.82 for
the 11 subscales). Validity analyses showed strong
relations within respondents between LEAF subscale
scores and corresponding subscale scores from other
questionnaire measures of executive functioning (BRIEF
and CHAOS). Relations between LEAF subscale
scores and corresponding neuropsychological measures
of executive and academic functioning were statistically
significant in most cases.
Factor analyses of 10 of the 11 subscales produced
results indicating that the subscales were reflected by a
single dimension. These findings suggest that test
items on each subscale measure the same underlying
construct. Furthermore, internal consistency values
were satisfactory for all LEAF subscales, especially
considering the small number of items (5) per subscale.
In developing LEAF subscales, we chose to emphasize
internal consistency analyses instead of factor analyses
for two reasons: First, the LEAF is intended for use as
a clinically meaningful instrument. In order to
maximize clinical utility, subscale content should match
conceptually and clinically meaningful areas of EF and
related abilities. As a result, we sought to retain the
clinically meaningful subscale content as long as items
were sufficiently interrelated (as shown by the
Cronbach’s ∝ statistic). Second, the sample size was
too small for a factor analysis of all 55 test items to be
Table 5. Relationships between parent-reported LEAF and
neuropsychological measures of executive functioning.
LEAF subscale Stroop CIT CPT
Color-
word
Number-
count
Reaction time
standard Error
N 103 107 97
Cognitive-learning:
Comprehension and
conceptual learning .25* .21* .14
Factual Memory .18a .08 .18a
Cognitive-EF:
Attention .33** .30** .22*
Processing speed .24* .23* .02
Visualspatial organization .22* .25** .16
Sustained sequential processing .31** .18a .06
Working memory .28** .27** .25*
Novel problem-solving .18
a
.17a .14
Academic:
Mathematics skills .27** .22* .32**
Basic reading skills .35*** .33*** .05
Written expression skills .43*** .38*** .12
Note. Stroop ¼Stroop Color and Word Test. CIT ¼Counting Interference
Test. CPT ¼Continuous Performance Test. Values are Pearson correlation
coefficients.
a
p <0.10.
*p <0.05.
**p <0.01.
***p <0.001.
Table 6. Relationships between the parent-reported LEAF and
measures of academic achievement.
LEAF subscale WJ-III tests of achievement
Basic reading
skills
composite
Calculation
subtest
Writing
samples
subtest
N 111 112 106
Cognitive-learning:
Comprehension and
conceptual learning .35*** .40*** .48***
Factual memory .43*** .37*** .41***
Cognitive-EF:
Attention .10 .10 .29**
Processing speed .25** .12 .32**
Visualspatial organization .19* .18
a
.34***
Sustained sequential
processing .15 .14 .30**
Working memory .24* .28** .41***
Novel problemsolving .28** .30** .36***
Academic:
Mathematics skills .34*** .51*** .35***
Basic reading skills .48*** .23* .41***
Written expression skills .36*** .30** .52***
Note. WJ-III ¼Woodcock-Johnson III Tests of Achievement. The WJ-III Basic
Reading Skills score is a composite of the Letter-Word Identification and
Word Attack subtests. Values are Pearson correlation coefficients.
a
p <0.10.
*p <0.05.
**p <0.01.
***p <0.001.
104 I. CASTELLANOS ET AL.
stable. Use of an internal consistency methodology and
factor analysis of each subscale separately therefore pro-
vided statistical evidence of the homogeneity of subscale
item content while also allowing us to retain concep-
tually and clinically meaningful groupings of items
(Comrey, 1988).
For the Visual-Spatial Organization subscale, internal
consistency and factor analysis results suggest that two
related factors comprise the subscale score. One factor
consisted of items reflecting organization skills and
behavior in the environment, while a second factor com-
prised visual-organization skills in writing and puzzle
construction. Most EF questionnaire subscales of organi-
zation skills ask only about organization of materials in
the environment, but we chose to add questions about
visual-spatial organization because of the close corre-
spondence of organization of materials and visual-spatial
organization in disorders of executive functioning and
learning (Rourke, 1995). Our results suggest that these
two subdomains of organization are related (e.g., factors
correlated r ¼0.37, and ∝ ¼0.69) but distinct. Future
research should investigate whether these two domains
of organization are present in nonreferred samples and
in samples with other clinical concerns such as hearing
loss (Kronenberger, Beer, Castellanos, Pisoni, &
Miyamoto, 2014). Based on this research, it may be
warranted to investigate the two organization domains
separately as well as to use the total score. Pending
this additional research, scores on the Visual-Spatial
Organization subscale should be interpreted with
caution.
Correlations between the LEAF and corresponding
subscales from the BRIEF and CHAOS indicate
excellent construct validity across behavior checklists
completed by the same respondents. Five of the eight
LEAF Cognitive-Learning and Cognitive-EF subscales
contained content that corresponded directly to
subscales on the BRIEF and CHAOS; correlations
between these corresponding subscales were large. The
remaining three LEAF Cognitive-Learning and
Cognitive-EF subscales contained content that over-
lapped much less with BRIEF or CHAOS subscales
(Comprehension and Conceptual Learning, Factual
Memory, and Processing Speed), and, as expected,
correlations of these LEAF subscales with BRIEF and
CHAOS subscales were generally weaker.
Most LEAF Cognitive-EF subscales correlated
significantly with neuropsychological measures of EF,
particularly for Stroop-like tasks (SCWT and CIT).
Neuropsychological measures of EF tend to be corre-
lated modestly, if at all, with results obtained from
observation, interview, or questionnaire measures of
everyday behavior (Barkley, 2012). The correlations
found between the LEAF Cognitive-EF subscales and
the Stroop scores in this study were in the medium
range, consistent with findings for other behavioral
observation, interview, or questionnaire measures
(Barkley, 2012), supporting the validity of LEAF sub-
scales as satisfactory measures of EF. Correlations
between the LEAF Academic subscales and results from
the WJ-III Tests of Achievement were also quite high for
corresponding academic areas (r ¼0.48 to 0.52),
providing additional converging support for the value
of the LEAF behavior checklist as a screening measure
of academic problems in children with suspected EF
delays.
In order to facilitate interpretation of LEAF raw
scores, three criterion-referenced interpretation ranges
were created based on the anchors for response choices
of LEAF items. Specifically, LEAF subscale raw scores
of 0–4 fall within the “No Problem Range” and
indicate that the average item was answered with a
Table 7. BRIEF or WJ-III scores corresponding to parent-report LEAF criterion-based interpretation ranges.
LEAF subscale (Corresponding BRIEF/WJ-III score) No problem Borderline problem Problem F
Cognitive-Learning:
Comprehension and Conceptual Learning
(No Corresponding BRIEF subscale)
N ¼41 N ¼53 N ¼24 –
Factual Memory (No Corresponding BRIEF subscale) N ¼42 N ¼53 N ¼23 –
Cognitive-EF:
Attention (BRIEF Working Memory) 50.6 (9.6)
a
, N ¼16 65.4 (8.4)
b
, N ¼43 74.4 (8.8)
c
, N ¼59 48.1***
Processing Speed (No Corresponding BRIEF subscale) N ¼19 N ¼52 N ¼47 –
Visual-Spatial Organization (BRIEF Organization of Materials) 47.8 (11.1)
a
, N ¼24 59.8 (8.8)
b
, N ¼58 65.9 (6.5)
c
, N ¼36 30.2***
Sustained Sequential Processing (BRIEF Plan/Organize) 53.4 (9.2)
a
, N ¼19 65.1 (10.0)
b
, N ¼46 73.0 (8.9)
c
, N ¼53 30.8***
Working Memory (BRIEF Working Memory) 54.0 (10.4)
a
, N ¼21 67.2 (10.4)
b
, N ¼44 74.2 (8.0)
c
, N ¼53 34.0***
Novel problem-solving (BRIEF initiate) 53.8 (10.0)
a
, N ¼43 64.7 (11.1)
b
, N ¼46 70.5 (9.1)
c
, N ¼29 25.0***
Academic:
Mathematics Skills (WJ-III Calculation) 104.6 (16.2)
a
, N ¼41 90.0 (15.4)
b
, N ¼33 83.6 (17.7)
c
, N ¼44 16.3***
Basic Reading Skills (WJ-III Basic Reading) 102.8 (12.6)
a
, N ¼58 94.9 (12.7)
b
, N ¼32 85.7 (13.4)
c
, N ¼28 16.2***
Written Expression Skills (WJ-III Writing Samples) 107.4 (13.0)
a
, N ¼27 96.5 (12.0)
b
, N ¼36 87.1 (12.4)
c
, N ¼55 21.9***
Note. Values (unless otherwise indicated) are mean (SD) for BRIEF or WJ-III subscale/subtest scores. df for F-tests is (2,111) for BRIEF subtests, (2,109) for WJ-III
Calculation, (2,108) for Basic Reading, and (2,103) for WJ-III Writing Samples. Ns are for entire sample. Values with different superscripts are significantly
different (p <0.05) between LEAF Interpretation Ranges.
***p <0.001 for F-test.
APPLIED NEUROPSYCHOLOGY: CHILD 105
response choice of less than “1” per item (e.g., with 5
items per scale, a score of 4 or less would indicate that
the average item score is 0.80 or less). LEAF subscale
raw scores of 5–9 fall within the “Borderline Problem
Range” and indicate that the average item was
answered with a response choice of at least “1” per
item but less than “2” per item. Finally, scores of 10
or greater fall within the “Problem Range” and indicate
a per-item average rating of “2.” We caution clinicians
that interpretation of the LEAF should take into
account that different patterns of item endorsement
might result in the same subscale raw score, therefore
attention to individual item scores is important for
subscales that fall within the “Borderline Problem
Range” or “Problem Range.”
These three LEAF criterion-referenced interpretation
ranges were validated in analyses using BRIEF and
WJ-III norm-referenced subscale/subtest scores.
Participants in the “No Problem Range” consistently
scored very close to the normative mean on BRIEF
and WJ-III scores, whereas those in the “Problem
Range” scored 1–2 SD on average in the direction of
problems on BRIEF and WJ-III scores. Participants in
the “Borderline Problem” range fell between these
extremes and differed significantly from both of the
extreme groups on BRIEF and WJ-III scores. Thus,
the LEAF criterion-referenced interpretation ranges
corresponded as expected to norm-based scores on
well-established measures of similar constructs and dif-
fered significantly from each other on norm-referenced
scores from these measures.
As no behavior checklist is free of error, several lim-
itations should be taken into account when interpreting
the present results. First, inter-rater reliability analyses
showed moderate agreement (correlations in the
medium to large range) between parents and teachers
on all LEAF subscales. Correlations between parent
and teacher behavioral ratings on behavior checklists
are universally found to fall in this range and may reflect
how the child’s behavior changes between the home and
school environments (Gioia et al., 2000). Future
research is recommended to investigate additional
factors contributing to parent-teacher differences in
rating child EF, factors influencing teacher ratings of
EF, and how teacher-reported LEAF scores compare
to other behavior checklists and neuropsychological
assessments. In the present study, we found large
correlations between teacher-ratings on the LEAF and
corresponding teacher-ratings on other EF-related
behavior checklists, and further investigation of EF
behaviors in the classroom may reveal additional
clinically-relevant contributors to and sequelae of EF
delays as reported by teachers.
Second, although the internal consistency of LEAF
subscales was high (Cronbach’s ∝ ranged from .69 to
.95) and factor analyses supported the unidimensional
nature of 10 of the 11 LEAF subscales, the Visual-Spatial
Organization subscale was found to consist of two
related groups of items. Additional investigation and
replication of this result will be important for under-
standing and supporting this subscale. Third, while
factor structure was evaluated at the individual subscale
level, the sample was too small to evaluate the factor
structure of all 55 LEAF items in a single analysis.
Further research is needed with a larger sample of
children to determine the factor structure of all LEAF
items. Finally, a limitation of the present study was
the lack of a nonreferred normative sample. Although
the LEAF criterion-referenced interpretation ranges
were supported by analyses showing correspondence to
BRIEF and WJ-III norm-referenced scores, norms for
the LEAF will provide additional information about
the degree to which scores are abnormal compared to
typically developing children and adolescents. We are
currently in the process of obtaining samples of
nonreferred children and adolescents to address this
question.
In summary, delays and disturbances in executive
functioning are associated with several pediatric disor-
ders affecting brain functioning and can contribute
to at-risk long-term outcomes, particularly related to
learning and higher-order cognitive processing
(Kronenberger et al., 2014). Therefore, early identifi-
cation and intervention is critically important in pedi-
atric clinical settings. The LEAF is a broad behavior
checklist of EF and related learning and academic skills
that was developed to augment behaviorally-based
performance assessments of EF and learning in children
and adolescents and to serve as a screening tool for
possible problems in EF. As such, it may serve as a help-
ful clinical instrument for pediatric neuropsychologists
to encourage screening for EF and related problems
in at-risk populations and to provide additional
information to complement the results of conventional
neuropsychological assessment of EF, yielding broader
multisource-multitrait data. The clinical utility of the
LEAF is enhanced by several key factors: it is a freely
accessible brief behavior checklist that is easy to admin-
ister, score, and interpret. For example, most other EF
checklists have varying numbers of items per subscale
and have items distributed randomly throughout the
checklist. This makes the checklist difficult to score
without a scoring key. The LEAF, however, has the same
number of items (5) per subscale, does not have a very
large number of items per subscale, groups items by
subscale, and uses criterion-referenced scores in order
106 I. CASTELLANOS ET AL.
to allow for very efficient use without scoring keys or
computers (e.g., see Levy et al., 2013). When interpreting
results from the LEAF it is important to note the content
and goals of the scale – to provide a reliable and valid
measure of cognitive EF and related learning skills.
Our intent was not to replicate the content of the BRIEF
or other broad EF scales; therefore, the LEAF does
not include EF measures of emotional control and
behavioral inhibition.
Future research should address characteristics of the
LEAF in larger and more diverse populations to investi-
gate normative scores and characteristics associated with
different pediatric and neuropsychological conditions.
As noted earlier, additional research concerning teacher-
report LEAF scales may also enhance the use of the
LEAF as a teacher-report measure. Finally, in order to
better understand the interrelations of LEAF items and
subscales, investigation of the higher-order factor
structure of LEAF subscales and items, using much lar-
ger samples, is an important next-step for understanding
LEAF psychometrics.
References
American Psychiatric Association. (1994). Diagnostic and
statistical manual of mental disorders (4th ed.). Washington,
DC: American Psychiatric Publishing.
Awh, E., Vogel, E. K., & Oh, S. H. (2006). Interactions
between attention and working memory, Neuroscience,
139, 201–208. doi:10.1016/j.neuroscience.2005.08.023
Banich, M. T. (2009). Executive function: the search for an
integrated account. Current Directions in Psychological
Science, 18(2), 89–94. doi:10.1111/j.1467-8721.2009.01615.x
Barkley, R. A. (2011a). Barkley Deficits in Executive
Functioning Scale (BDEFS). New York, NY: Guilford Press.
Barkley, R. A. (2011b). Barkley Deficits in Executive
Functioning Scale - Children and Adolescents (BDEFS-CA).
New York, NY: Guilford Press.
Barkley, R. A. (2012). Executive functions: What they are, how
they work, and why they evolved. New York, NY: Guilford
Press.
Buckner, R. L. (2004). Memory and executive function in
again and AD: Multiple factors that cause decline and
reserve factors that compensate, Neuron, 44, 195–208.
doi:10.1016/j.neuron.2004.09.006
Castellanos, I., Kronenberger, W. G., Beer, J., Colson, B. G.,
Henning, S. C., Ditmars, A., & Pisoni, D. B. (2015).
Concept formation skills in long-term cochlear implant
users, Deaf Studies and Deaf Education, 20, 27–40.
doi:10.1093/deafed/enu039
Cohen, J. (1992). A power primer, Psychological Bulletin, 112,
155–159. doi:10.1037/0033-2909.112.1.155
Comrey, A. L. (1988). Factor-analytic methods of scale
development in personality and clinical psychology, Journal
of Consulting and Clinical Psychology, 56, 754–761.
doi:10.1037/0022-006X.56.5.754
Conners, C. K., & MHS Staff. (2000). Conners’ Continuous
Performance Test II (CPT II v. 5). North Tonawanda, NY:
Multi-Health Systems.
Conway, C. M., Pisoni, D. B., & Kronenberger, W. G. (2009).
The importance of sound for cognitive sequencing abilities
the auditory scaffolding hypothesis, Current Directions in
Psychological Science, 18, 275–279. doi:10.1111/j.1467-8721.
2009.01651.x
Daly, B. P., & Brown, R. T. (2007). Scholarly literature review:
Management of neurocognitive late effects with stimulant
medication, Journal of Pediatric Psychology, 32, 1111–1126.
doi:10.1093/jpepsy/jsm012
DeVellis, R. F. (1991). Scale development: Theory and
applications. London, UK: Sage.
DeVellis, R. F. (2012). Scale development: Theory and
applications (3rd ed.). Los Angeles, CA: Sage.
DeVon, H. A., Block, M. E., Moyle-Wright, P., Ernst, D. M.,
Hayden, S. J., Lazzara, D. J. & Kostas-Polston, E. (2007).
A psychometric toolbox for testing validity and reliability,
Journal of Nursing Scholarship, 39, 155–164. doi:10.1111/
j.1547-5069.2007.00161.x
Diamond, A. (2013). Executive functions, Annual Review
of Psychology, 64, 135–168. doi:10.1146/annurev-psych-
113011-143750
Diamond, A. (2016). Why improving and assessing executive
functions early in life is critical. Executive function in
preschool age children: Integrating measurement, neurodeve-
lopment and translational research. Washington, DC:
American Psychological Association.
Ebrahimi, A. A., Kassani, A., Menati, R., Abedi, A.,
Yarmohammadian, A., & Faramarzi, S. (2015). The
assessment of validity and reliability of behavior rating
inventory of executive functions in Iranian pre-school
children. International Journal of School Health, 2(4),
18–24. doi:10.17795/intjsh29022
Faries, D. E., Yalcin, I., Harder, D., & Heiligenstein, J. H.
(2001). Validation of the ADHD rating scale as a clirli-
cian administered and scored instrument, Journal of
Attention Disorders, 5, 107–115. doi:10.1177/108705470
100500204
Gadow, K. D., & Sprafkin, J. (1997). Child symptom inventory-4
norms manual. Stony Brook, NY: Checkmate Plus.
García, T., Rodriguez, C., González-Castro, P., & Areces, D.
(2014). Executive functions and their assessment in
children and adolescents, Cognitive Sciences, 9, 101.
Gathercole, S. E., & Baddeley, A. D. (1993). Working memory
and language. Hillsdale, NJ: Lawrence Erlbaum.
Gathercole, S. E., Pickering, S. J., Knight, C., & Stegmann, Z.
(2004). Working memory skills and educational attain-
ment: Evidence from national curriculum assessments at
7 and 14 years of age, Applied Cognitive Psychology,
18, 1–16. doi:10.1002/acp.934
Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L.
(2000). Behavior rating inventory of executive function
(BRIEF). Lutz, FL: Psychological Assessment Resource.
Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L.
(2016). Behavior rating inventory of executive function,
Second Edition (BRIEF-2). Lutz, FL: Psychological Assess-
ment Resource.
Golden, C. J. (1978). Stroop color and word rest. Chicago, IL:
Stoelting.
APPLIED NEUROPSYCHOLOGY: CHILD 107
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. (2010).
Multivariate data analysis (7th ed.). Upper Saddle River,
NJ: Prentice-Hall.
Horton, A. M., Jr., Soper, H. V., & Reynolds, C. R. (2010).
Executive functions in children with traumatic brain injury,
Applied Neuropsychology, 17, 99–103. doi:10.1080/
09084281003708944
Hummer, T. A., Kronenberger, W. G., Wang, Y., Dunn, D. W.,
Mosier, K. M., Kalnin, A. J., & Mathews, V. P. (2011).
Executive functioning characteristics associated with ADHD
comorbidity in adolescents with disruptive behavior disor-
ders, Journal of Abnormal Child Psychology, 39, 11–19.
doi:10.1007/s10802-010-9449-3
Jerman, O., Reynolds, C., & Swanson, H. L. (2012). Does
growth in working memory span or executive processes
predict growth in reading and math in children with
reading disabilities? Learning Disability Quarterly, 35,
144–157. doi:10.1177/0731948712444276
Kronenberger, W. G., Beer, J., Castellanos, I., Pisoni, D. B.,
& Miyamoto, R. T. (2014). Neurocognitive risk in
children with cochlear implants, JAMA Otolaryngology-
Head & Neck Surgery, 140, 608. doi:10.1001/jamaoto.
2014.757
Levy, J. D., Kronenberger, W. G., & Dunn, D. W. (2013).
Development of a very brief measure of ADHD: The
CHAOS scale. Journal of Attention Disorders. doi:10.1177/
1087054713497792
Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D.
(2012). Neuropsychological assessment (5th ed.). New York,
NY: Oxford University Press.
Linacre, J. M. (2002). Optimizing rating scale category
effectiveness, Journal of Applied Measurement, 3, 85–106.
Luria, A. R. (1966). Higher cortical functions in man.
New York, NY: Basic Books.
McAuley, T., & White, D. A. (2011). A latent variables
examination of processing speed, response inhibition, and
working memory during typical development, Journal of
Experimental Child Psychology, 108, 453–468. doi:10.1016/
j.jecp.2010.08.009
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., &
Howerter, A. (2000). The unity and diversity of executive
functions and their contributions to complex “frontal lobe”
tasks: A latent variable analysis, Cognitive Psychology,
41, 49–100. doi:10.1006/cogp.1999.0734
Naglieri, J. A., & Goldstein, S. (2013). Comprehensive
executive function inventory (CEFI) (pp. 1–8). Toronto,
CA: Multi-Health Systems.
Naglieri, J. A., & Goldstein, S. (2014). Using the comprehen-
sive executive function inventory (CEFI) to assess executive
function: From theory to application. In S. Goldstein &
J. A. Naglieri (Eds.), Handbook of executive functioning
(pp. 223–244). New York, NY: Springer.
Nestor, P. G., & Schutt, R. K. (2015). Research methods in
psychology: Investigating human behavior. Los Angeles,
CA: Sage.
O’Hara, L. K., & Holmbeck, G. N. (2013). Executive functions
and parenting behaviors in association with medical
adherence and autonomy among youth with spina bifida,
Journal of Pediatric Psychology, 38, 675–687. doi:10.1093/
jpepsy/jst007
Parrish, J., Geary, E., Jones, J., Seth, R., Hermann, B., &
Seidenberg, M. (2007). Executive functioning in childhood
epilepsy: Parent-report and cognitive assessment,
Developmental Medicine and Child Neurology,
49, 412–416. doi:10.1111/j.1469-8749.2007.00412.x
Ragland, J. D., Gur, E. C., Glahn, D. C., Censits, D. M., Smith,
R. J., Lazarev, M. G. & Gur, R. E. (1998). Frontotemporal
cerebral blood flow change during executive and
declarative memory tasks in Schizophrenia: A Positron
Emission Tomography study, Neuropsychology, 12,
399–414. doi:10.1037/0894-4105.12.3.399
Rourke, B. P. (1995). Syndrome of nonverbal learning
disabilities: Neurodevelopmental manifestations. New York,
NY: Guilford Press.
Wechsler, D., Kaplan, E., Fein, D., Kramer, J., Morris, R.,
Delis, D., & Maerlender, A. (2004). Wechsler intelligence
scale for children – Fourth Edition – Integrated. San
Antonio, TX: Harcourt Assessment.
Weierink, L., Vermeulen, R. J., & Boyd, R. N. (2013). Brain
structure and executive functions in children with cerebral
palsy: A systematic review, Research in Developmental
Disabilities, 34, 1678–1688. doi:10.1016/j.ridd.2013.01.035
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001).
Woodcock-Johnson III tests of achievement. Itasca, IL:
Riverside.
Zelazo, P. D., Carter, A., Reznick, J. S., & Frye, D. (1997).
Early development of executive function: A problem-
solving framework, Review of General Psychology,
1, 198–226. doi:10.1037/1089-2680.1.2.198
Appendix
Learning, Executive, and Attention Functioning
(LEAF) scale items and instructions
Instructions: Please answer the following questions
based on this child’s behavior during the LAST WEEK.
Please circle your answers, and answer all questions
(make your best guess if you are uncertain).
108 I. CASTELLANOS ET AL.
Never (Not a
problem;
Average
for age)
Sometimes (A little
more than
average;
Not a big problem)
Often (Causes
problems;
Happens almost
every day)
Very often
(Major daily
problem)
Cognitive Learning:
Comprehension and Conceptual Learning
0 1 2 3 Doesn’t seem to understand things that are said to him/her
0 1 2 3 Has difficulty following long conversations or explanations.
0 1 2 3 Poor comprehension of reading material.
0 1 2 3 Doesn’t “get the point” of what is being said.
0 1 2 3 Doesn’t really understand new learning materials.
Factual Memory
0 1 2 3 Difficulty memorizing information.
0 1 2 3 Doesn’t retain facts well.
0 1 2 3 Has poor memory.
0 1 2 3 Forgets things that he/she has just learned.
0 1 2 3 Remembers the main idea but forgets details.
Cognitive-EF:
Attention
0 1 2 3 Poor attention span.
0 1 2 3 Mind seems to drift or wander when he/she is suppose to concentrate.
0 1 2 3 Does not stay focused on learning material.
0 1 2 3 Easily distracted.
0 1 2 3 Doesn’t listen when others are teaching or talking to him/her.
Processing Speed
0 1 2 3 Has trouble completing work quickly, even when motivated to do well.
0 1 2 3 Works deliberately and slowly on schoolwork and homework.
0 1 2 3 Writes and/or reads slowly.
0 1 2 3 Needs extra time to complete tests or other work.
0 1 2 3 Is slow to get started on things.
Visual-Spatial Organization
0 1 2 3 Poor organization.
0 1 2 3 Room, desk, locker, work area, etc. is very messy.
0 1 2 3 Not very good with puzzles or putting things together.
0 1 2 3 Drawing and/or handwriting is poor.
0 1 2 3 Doesn’t pay attention to visual details in the environment.
Sustained Sequential Processing
0 1 2 3 Doesn’t plan ahead.
0 1 2 3 Doesn’t learn from punishment or other past experiences.
0 1 2 3 Has trouble with long assignments or multistep directions.
0 1 2 3 Loses track of step-by-step directions.
0 1 2 3 Doesn’t complete tasks in proper order; Haphazard in approaching
problems.
Working Memory
0 1 2 3 Can’t do more than one thing at a time.
0 1 2 3 Gets overwhelmed if required to learn or attend to a lot of information.
0 1 2 3 If distracted, loses track of what he/she was doing/learning.
0 1 2 3 Forgets things that he/she knew how to do a few hours or days before.
0 1 2 3 Gets upset or “shuts down” when challenged with learning.
Novel Problem-Solving
0 1 2 3 Struggles when learning new materials.
0 1 2 3 Does not solve problems independently (needs help on new problems).
0 1 2 3 Resists learning anything that is unfamiliar, new, or different.
0 1 2 3 Has difficulty with new situations, new people, or unfamiliar settings.
0 1 2 3 Avoids new experiences.
Academic:
Mathematics Skills
0 1 2 3 Math is difficult for him/her.
0 1 2 3 Slow at math.
0 1 2 3 Makes mistakes during arithmetic calculations or counting.
0 1 2 3 Prefers reading and language subjects to math.
0 1 2 3 Takes a long time to learn new mathematics operations or concepts.
Basic Reading Skills
0 1 2 3 Reading is slow.
0 1 2 3 Has trouble sounding out new words when reading.
0 1 2 3 Reading is hesitant or choppy; doesn’t read smoothly.
0 1 2 3 Makes mistakes when sounding out or pronouncing words in reading.
0 1 2 3 Skips or mistakes words during reading.
Written Expression Skills
0 1 2 3 Writing is slow.
0 1 2 3 Written expression is very simple (basic, immature).
0 1 2 3 Makes errors in grammar, punctuation, or spelling when writing sentences.
0 1 2 3 Has difficulty explaining things or expressing self in writing.
0 1 2 3 Struggles with subjects that require writing.
Note. Subscales are indicated on this version of the LEAF for illustrative purposes. Please contact the authors for the administration version of the LEAF scale.
Criterion-referenced interpretation ranges (0–4 ¼“No Problem Range”; 5–9 ¼“Borderline Problem Range”; 10–15 ¼“Problem Range”).
APPLIED NEUROPSYCHOLOGY: CHILD 109
