Content uploaded by Amanda R Rabinowitz
Author content
All content in this area was uploaded by Amanda R Rabinowitz on Jan 08, 2014
Content may be subject to copyright.
Intraindividual Cognitive Variability Before and After
Sports-Related Concussion
Amanda R. Rabinowitz
University of Pennsylvania School of Medicine
Peter A. Arnett
The Pennsylvania State University
Objective: Inconsistent performance is associated with cognitive dysfunction in a number of clinical
populations. However, intraindividual cognitive variability in healthy individuals is poorly understood.
Inconsistency poses a challenge to clinicians when interpreting change over time. This study examined
intraindividual cognitive variability within a sample of college athletes tested at baseline and postcon-
cussion. Method: Athletes (n ⫽ 71) and control participants (n ⫽ 42) were tested with a comprehensive
neuropsychological battery at baseline and postconcussion (athletes) or one month later (controls). A
subset of indices with high internal consistency was used to calculate overall performance and perfor-
mance variability. A k-means cluster analysis of baseline and postconcussion performance variability
examined heterogeneity within the sample. Results: In the athlete sample, performance variability was
significantly greater than zero, and was negatively correlated with overall performance at both time
points (p ⬍ .001). Wechsler Test of Adult Reading Full Scale IQ estimate was significantly correlated
with overall performance (p ⬍ .01), but not with performance variability. Cluster analysis revealed
low-variability (n ⫽ 46) and high-variability (n ⫽ 25) cluster groups. Whereas the low-variability cluster
group exhibited a pattern of performance similar to that of control participants, membership in the
high-variability cluster group was associated with postconcussion cognitive dysfunction. Conclusion:
These findings suggest that normative cognitive performance in college athletes is characterized by
significant intraindividual variation across tests. Cross-test intraindividual variability may impart clini-
cally meaningful information, as higher levels of variability were related to poorer overall performance
and postconcussion cognitive dysfunction.
Keywords: concussion, intraindividual variability, traumatic brain injury
Clinical and research protocols designed to assess sports-related
concussion and track its recovery have been implemented at the
professional, collegiate, and high school levels over the past 20
years. Barth and colleagues (1989) at the University of Virginia
were the first to develop the Sports as a Laboratory Assessment
Model as an attempt to better characterize the neuropsychological
impact of concussion and the trajectory of recovery. This was the
first program to systematically conduct preconcussion (baseline)
neuropsychological assessment on athletes, a method that allows
for the impact of the injury to be assessed directly (as change from
baseline performance) rather than simply inferred based on cutoff
scores or assumptions regarding premorbid functioning. Today,
baseline testing is becoming the gold standard in concussion
management as an objective measure of athletes’ premorbid cog-
nitive abilities provides an ideal comparison standard for postcon-
cussion assessment.
However, some authors have drawn attention to issues that
complicate straightforward pre–postinjury test comparisons, such
as motivation toward testing and psychometric issues (Bailey,
Echemendia, & Arnett, 2006; Ragan & Kang, 2007; Randolph,
McCrea, & Barr, 2005). Inconsistency in an individual’s cognitive
performance is another factor that poses a challenge to clinicians
when interpreting change across pre- and postinjury test sessions.
Intraindividual variability is a term that has been used to describe
moment-to-moment fluctuations in state, or inconsistencies (Ram,
Rabbitt, Stollery, & Nesselroade, 2005). Cognitive variability
could be related to true idiosyncratic strengths and weaknesses,
fluctuations in effort due to premorbid neurological vulnerabilities,
or fluctuations in effort due to suboptimal motivation. Distinguish-
ing between these potential sources of cognitive inconsistency is
crucial for valid assessment.
In the field of psychology, a science that is dominated by central
tendency, intraindividual variability has traditionally been charac-
terized as error, instability, and noise. However, within the neuo-
psychological literature, recent research has shown that fluctua-
tions in cognitive performance reflect more than random error and
measurement unreliability, and, in fact, are often negatively cor-
related with mean levels of performance (Hultsch & MacDonald,
2004; Hultsch, Strauss, Hunter, & MacDonald, 2008). Research
has linked increased variability with a number of clinical phenom-
ena including aging, depression and anxiety, traumatic brain in-
jury, attention-deficit/hyperactivity disorder, and schizophrenia
(see MacDonald, Li, & Bäckman, 2009, for review). Furthermore,
there is some evidence suggesting that variability may be, in fact,
Amanda R. Rabinowitz, Department of Neurosurgery, University of
Pennsylvania School of Medicine; Peter A. Arnett, Psychology Depart-
ment, The Pennsylvania State University.
Correspondence concerning this article should be addressed to Amanda
R. Rabinowitz, Department of Neurosurgery, University of Pennsylvania
School of Medicine, 370 Stemmler Hall, Philadelphia, PA 19104. E-mail:
rabinowitz.a@gmail.com
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Neuropsychology © 2013 American Psychological Association
2013, Vol. 27, No. 4, 481– 490 0894-4105/13/$12.00 DOI: 10.1037/a0033023
481
more sensitive than mean level of performance in detecting cog-
nitive decline. In a longitudinal study of cognitive aging, Lövdén,
Li, Shing, and Lindenberger (2007) found that changes in vari-
ability preceded measurable deficits in mean level of performance.
These findings suggest that intraindividual variability is a prom-
ising indicator of cognitive dysfunction. Theoretical proposals
have been put forth to explain the cognitive and neurobiological
mechanisms that may lead to inconsistency in performance in
order to elucidate the link between variability and dysfunction. At
the cognitive level, variability has been attributed to lapses in
attention (Bunce, Warr, & Cochrane, 1993) and failure to maintain
executive control (West, Murphy, Armilio, Craik, & Stuss, 2002).
At the neuroanatomical level, variability is thought to arise from
inefficiencies in central nervous system functioning. Central ner-
vous system inefficiencies may be related to disrupted neural
connectivity (Kelly, Uddin, Biswal, Castellanos, & Milham,
2008), reduced efficacy of neurotransmitter systems (Bäckman,
Nyberg, Lindenberger, Li, & Farde, 2006), structural damage to
gray matter (Sowell et al., 2003; Stuss, Murphy, Binns, & Alex-
ander, 2003), or loss of white-matter integrity (Anstey et al., 2007;
Britton, Meyer, & Benecke, 1991; Walhovd & Fjell, 2007).
Investigators examining intraindividual inconsistencies in cog-
nitive performance have operationalized performance variability in
a number of ways. Some work has used intraindividual standard
deviations taken from multiple trials within a single task (Ram et
al., 2005). Other studies have considered variability in perfor-
mance on the same task on separate occasions (Hultsch, MacDon-
ald, Hunter, Levy-Bencheton, & Strauss, 2000). A few investiga-
tors have taken a cross-domain approach to intraindividual
variability by examining within-person inconsistency in perfor-
mance across different neuropsychological tests administered as
part of a neuropsychological test battery (Holtzer, Verghese,
Wang, Hall, & Lipton, 2008; Kliegel & Sliwinski, 2004; Schretlen,
Munro, Anthony, & Pearlson, 2003).
A cross-test approach to variability considers performance on a
battery of neuropsychological tests, standardizes these tests with
reference to a common sample, and indexes the extent to which
these scores deviate from each other using a variance parameter,
such as standard deviation. For example, cross-test variability for
a neuropsychological battery including tests of verbal memory,
visual memory, processing speed, and attention—with partici-
pants’ scores of 98, 105, 111, and 102 in standard score units—
could be indexed by spread (13 standard score points), variance
(30 standard score points), or standard deviation (5.48 standard
score points).
A cross-test approach to intraindividual variability is appealing
because indices of cross-test variability can be derived from any
neuropsychological battery. However, to interpret inconsistencies
across a test battery as abnormal or meaningful, it must first be
established that normal performance is consistent. From a theoret-
ical perspective, there are reasons to posit that a given individual’s
test performance should have low variability under typical condi-
tions, granted that an individual is noninjured, healthy, and devel-
opmentally normal. The concept of general intelligence (or Spear-
man’s g) supports this assumption. Intraindividual consistency in
cognitive behavior, subserved by IQ, is an assumption underlying
the deficit measurement paradigm in clinical neuropsychological
assessment—a rubric for determining the presence of neuropsy-
chological impairment by comparing an individual’s test scores
with a comparison standard. This comparison standard may be
estimated based on population norms, premorbid test data, per-
sonal historical information such as academic performance or
occupation, or indirectly from the current test findings and obser-
vations. The expectation that premorbid cognitive performance
should be consistent justifies the use of a unitary comparison
standard.
However, many investigators have noted that environmental
factors and personal characteristics can lead to intraindividual
differences in cognitive skills. Research using a cross-test variabil-
ity approach has found evidence of marked performance variabil-
ity in healthy adults (Schretlen et al., 2003). Specialization of
interests and activities, socialization experiences, personal expec-
tations, educational limitations, or emotional disturbance could
lead to inconsistencies in performance across cognitive domains
(Halpern, 1997). Furthermore, research has demonstrated that, for
normal healthy individuals, performance in the impaired range on
at least one test index from a large neuropsychological battery is
psychometrically normal (Binder, Iverson, & Brooks, 2009). This
suggests that some level of cross-test variability is normative.
Furthermore, there are many exceptions to the generality that
cognitive behavior is consistent across situations. Few persons
consistently function at their maximum potential for a variety of
reasons, including illness, educational deficiencies, impulsivity,
test anxiety, or disinterest.
With these considerations in mind, the present study examined
cognitive inconsistency in college athletes tested at baseline and
postconcussion by examining the degree of intraindividual vari-
ability across cognitive tests administered as part of a sports
concussion assessment battery. Because noninjured athletes are
tested at baseline (before the start of team activities when they first
arrive at the university) and also postconcussion, these data pro-
vide an opportunity to address the following questions: (1) How
variable is cognitive performance in healthy noninjured persons?
(2) What is the effect of cerebral concussion on intraindividual
variability? (3) What factors are related to cross-domain cognitive
inconsistency before and after head injury?
Method
Participants
Participants in the present study were 71 college athletes par-
ticipating in a multisport program at a large state university that
assesses participants prior to and following sports-related concus-
sion, as well as 42 control participants (see Table 1 for character-
istics of the sample). Control participants were students from the
same university who participate in athletic activities at the intra-
mural level but do not participate in the concussion management
program. This program is designed according to the Sports as a
Laboratory Assessment Model (Barth et al., 1989) paradigm to
provide objective neuropsychological test data to team physicians
to inform return-to-play decisions. Eight athletic programs regu-
larly participate in baseline testing: football, men’s and women’s
lacrosse, men’s and women’s soccer, men’s and women’s basket-
ball, men’s ice hockey, and wrestling. Postconcussion testing is
provided for any athlete who sustains a head injury. Whenever
possible, postconcussion assessment takes place within 48 hr of
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
482
RABINOWITZ AND ARNETT
injury. Only athletes who had undergone both baseline and post-
concussion assessment were included in the present analyses.
Measures
The test battery consists of a number of measures that assess
cognitive, physiological, and affective functioning, including the
Hopkins Verbal Learning Test—Revised (HVLT–R; Benedict,
Schretlen, Groninger, & Brandt, 1998), the Brief Visuospatial
Memory Test—Revised (BVMT–R; Benedict, 1997), the Symbol-
Digit Modalities Test (SDMT; Smith, 1982), the Digit Span Test
(Wechsler, 1997), the Pennsylvania State University (PSU) Can-
cellation Task (Echemendia & Julian, 2001), the Stroop Color-
Word Test (SCWT; Trenerry, Crosson, DeBoe, & Leber, 1989),
Controlled Oral Word Association Test (COWAT; Ruff, Light,
Parker, & Levin, 1996), Comprehensive Trail-Making Test
(CTMT; Reynolds, 2002), the Vigil Continuous Performance Task
(Cegalis & Bowlin, 1991), and the Wechsler Test of Adult Read-
ing (WTAR; Wechsler, 2001). For athletes who underwent multi-
ple evaluations (baseline testing and one or more postconcussion
evaluations), alternative forms of the HVLT–R, BVMT–R,
SDMT, and PSU Cancellation Task were used (see Benedict,
1997; Benedict et al., 1998; Smith, 1982, for alternative form
reliability information). In the case of the CTMT, Trails 2 and 4
were used as simple and complex trail-making trials at one time
point, and Trails 3 and 5 were administered at the other. With the
exception of the WTAR, all of these tests have been shown to be
sensitive to traumatic brain injury in prior work (Bailey, Echemen-
dia, & Arnett, 2005; Bohnen et al., 1992; Bruce & Echemendia,
2003; Ponsford & Kinsella, 1992; Vanderploeg, Curtiss, &
Belanger, 2005).
The Immediate Post-Concussion Assessment and Cognitive
Testing computerized battery (ImPACT; Lovell, Collins, Podell,
Powell, & Maroon, 2000) was also used. Although complete
validity data for the use of this test battery in sports-related
concussion have not yet been published, it assesses cognitive
domains typically affected following such injury. The ImPACT is
a computerized test battery that was designed to offer a time-
effective and standardized method for collecting data to assist in
concussion assessment and management. The battery consists of
three main parts: demographic data, neuropsychological tests, and
the Post-Concussion Symptom Scale (PCSS). Six neuropsycholog-
ical tests are included, designed to target attention, memory, pro-
cessing speed, and reaction time. From the six tests, five composite
scores are derived: verbal memory, visual memory, visuomotor
speed, reaction time, and impulse control. Studies in high school
and college athletes have demonstrated that ImPACT performance
is correlated with performance on similar paper-and-pencil neuro-
psychological tests (Iverson, Lovell, & Collins, 2005); further-
more, ImPACT performance is sensitive to the acute effects of
concussion (Schatz, Pardini, Lovell, Collins, & Podell, 2006).
The WTAR is a test of reading recognition that was designed for
premorbid IQ estimation. It was developed and conormed with the
Wechsler Adult Intelligence Scale—Third Edition and Wechsler
Memory Scale—Third Edition in both the United States and
United Kingdom using the same large, representative sample of
normally functioning adults. Using the normative data from the
conorming sample, WTAR scores can be converted to Full Scale
IQ (FSIQ) estimates. Research has demonstrated that the WTAR
has strong correlations (.70 –.80) with Wechsler Adult Intelligence
Scale—Third Edition FSIQ scores for a wide age range of WTAR
scores, and WTAR performance is relatively resistant to the effects
of traumatic brain injury (Wechsler, 2001). The Computerized
Assessment of Response Bias is a digit-recognition task that was
administered as a test of effort. Accuracy below 89% is indicative
of inadequate effort (Conder, Allen, & Cox, 1992). In addition to
the neuropsychological tests, the battery includes a self-report
measure of symptoms. The PCSS is a list of symptoms that are
commonly associated with concussion, including headache, nau-
sea, dizziness, trouble concentrating, and feeling “in a fog.” Ath-
letes rate the extent to which they are currently experiencing each
symptom on a scale from 0 (absence of the symptom)to6(extreme
distress from that symptom).
Procedure
All 71 athletes had baseline and postconcussion testing, includ-
ing the instruments described in detail above. Athletes may have
been tested a number of times postinjury, depending on the course
of their recovery and the team physician’s referral. The majority of
initial postconcussion assessments took place within 1 week of the
head injury and within 48 hr whenever possible (mean days
postconcussion ⫽ 7.2, SD ⫽ 10.8, min ⫽ 0 days, max ⫽ 210
days). For the purpose of the present study, only the initial post-
concussion evaluation was considered. Control participants were
administered the same test battery at two time points 1 month
apart. Evaluations were conducted by a PhD-level clinical neuro-
psychologist or a graduate or undergraduate assistant who had
been trained by a PhD-level clinical neuropsychologist. Testing
sessions took approximately 1.5 hr/evaluation and also required
approximately 30 min for paperwork, debriefing, and administra-
tion of other instruments not included in the present study. Written
informed consent was obtained from study participants. The study
Table 1
Characteristics of the Sample
Characteristic Athlete Control
Female, % 21 51
Caucasian, % 72 95
African American, % 25 1
ADHD diagnosis, % 5 0
LD, % 6 0
Mean (SD) age at baseline (years) 18.6 (0.8) 18.5 (0.8)
Previous head injuries, %
Denied 51 100
1340
2 or more 14 0
Sport, %
Football 38
Men’s ice hockey 14
Men’s lacrosse 13
Women’s soccer 10
Women’s lacrosse 7
Men’s basketball 3
Men’s soccer 4
Wrestling 4
Women’s basketball 3
Note. ADHD ⫽ attention-deficit/hyperactivity disorder (self-report);
LD ⫽ learning disorder (self-report).
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
483
COGNITIVE VARIABILITY AND CONCUSSION
received approval from The Pennsylvania State University Insti-
tutional Review Board and was conducted in accordance with its
ethical guidelines.
Data Analysis
Indices of overall performance and performance variability.
Because the intent of the present study was to examine cross-test
variability as an individual difference variable, the index of vari-
ability should assess variability that is meaningful with respect to
the individual and eliminate, to the greatest extent possible, vari-
ability that is related to properties of the tests. For this purpose, a
principal components analysis was performed on the following test
indices as assessed at baseline: Vigil—Total Omissions, Vigil—
Total Commissions, Vigil—Average Delay, ImPACT Verbal
Memory composite, ImPACT Visual Memory composite, Im-
PACT Visuomotor Speed composite, ImPACT Reaction Time
composite, ImPACT Impulse Control composite, BVMT–R Total
Immediate Recall, BVMT–R Total Delayed Recall, HVLT–R To-
tal Immediate Recall, HVLT–R Total Delayed Recall, SDMT—
Total Correct, SDMT—Incidental Memory, Digit Span Forwards,
Digit Span Backwards, SCWT—Word Time, SCWT—Color-
Word Time, PSU Cancellation Test Total Correct, COWAT Total,
CTMT—Trial 1 Time, and CTMT—Trial 2 Time. A priori, it was
decided that all test indices with component loadings on the first
component greater than .4 would be retained for subsequent anal-
ysis and Cronbach’s alpha would be consulted to confirm the
internal consistency of the retained indices at the aggregate level.
These retained indices were then used to calculate measures of
overall performance and performance variability. First, all test
indices were put on the same metric by converting them to stan-
dard scores (SSs). SSs have a mean of 100 and a standard deviation
of 15. SS units have been chosen because these are the metric
employed by many commonly used neuropsychological tests, in-
cluding the Wechsler Adult Intelligence Scale FSIQ score. The
sample mean and sample standard deviation from the athletes
tested at baseline were used to calculate SSs. Test indices for
which higher scores indicate poorer performance (i.e., Stroop—
Color-Word Time) were calculated by subtracting the observed
score from the sample mean, so that for all SSs, higher scores
indicate better performance.
Overall performance at baseline was calculated by taking the
mean across all SS-converted retained test indices for that indi-
vidual at baseline. Similarly, overall performance postconcussion
was calculated by taking the mean across all SS-converted retained
test indices for that individual postconcussion. Performance vari-
ability at baseline was calculated by taking the standard deviation
across all SS-converted retained test indices for that individual at
baseline. Similarly, performance variability postconcussion was
calculated by taking the standard deviation across all SS-converted
retained test indices for that individual postconcussion.
To examine potential subgroups in the sample of 71 athletes, we
entered baseline and postconcussion performance variability indi-
ces into a k-means cluster analysis where k was set equal to 2.
Reliable decline. Reliability statistics were calculated based
on the control group. A 90% confidence interval was selected as
the criterion for determining reliable decline in performance post-
concussion. An index of total declined scores was calculated by
summing the total number of tests for which an individual dem-
onstrated reliable decline.
Results
In the athlete sample, 85% of concussions were Cantu Grade II
injuries, characterized by loss of consciousness less than 5 min in
duration or amnesia lasting between 30 min and 24 hr. To evaluate
a possible influence of days since concussion on likelihood of
exhibiting cognitive decline postconcussion, athletes who were
tested within 48 hr of their injury were compared with those who
were tested longer than 48 hr since their injury. These groups were
not significantly different in total number of scores to reliably
decline postinjury, t(69) ⫽⫺0.70, p ⫽ .47.
A list of the tests included and their component loadings can be
found in Table 2. Cronbach’s alpha was calculated using the
retained indices to confirm consistency of these items at the
aggregate level. This analysis suggested that the retained items had
good internal consistency (␣⫽.82).
Based on those retained tests, indices of overall performance
and performance variability were calculated as described above.
Performance variability was significantly greater than zero at base-
line, M ⫽ 12.05, t(70) ⫽ 23.20, p ⬍ .001, and postconcussion,
M ⫽ 12.39, t(70) ⫽ 21.30, p ⬍ .001. To examine the relationship
between performance variability and other indices of cognitive
functioning, we examined bivariate correlations among perfor-
mance variability, overall performance, and WTAR FSIQ esti-
mate. At baseline, overall performance was negatively correlated
with performance variability (r ⫽⫺.44, p ⬍ .001), indicating that
better performance was associated with less variability. This in-
verse relationship between performance variability and overall
performance was also significant postconcussion (r ⫽⫺.52, p ⬍
.001). WTAR FSIQ estimate was significantly correlated with
overall performance, but not performance variability at both time
points (see Table 3).
Paired-sample t tests revealed that overall performance and
performance variability did not differ across time points in the
athlete sample as a whole. For overall performance, the mean
difference between baseline and postconcussion assessments was
0.33 SS points higher on the postconcussion assessment, t(70) ⫽
⫺0.30, p ⫽ .77. Similarly, for performance variability, the mean
difference between time points was 0.33 SS points higher on the
Table 2
Retained Test Indices and Their Component 1 Loadings
Test index Loading
Comprehensive Trail-Making Trial 1 (Trail 2 or 3) .41
Comprehensive Trail-Making Trial 2 (Trail 4 or 5) .57
ImPACT Verbal Memory composite .41
ImPACT Visual Memory composite .64
ImPACT Visual Motor Speed composite .76
ImPACT Reaction Time composite .67
BVMT–R Delayed Recall .62
HVLT–R Delayed Recall .46
Symbol-Digit Modalities Test .54
Stroop Color-Word Time .63
Note. ImPACT ⫽ Immediate Post-Concussion Assessment and Cognitive
Testing; BVMT–R ⫽ Brief Visual Memory Test—Revised; HVLT–R ⫽
Hopkins Verbal Memory Test—Revised.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
484
RABINOWITZ AND ARNETT
postconcussion assessment, t(70) ⫽⫺0.53, p ⫽ .60. In the control
sample, overall performance significantly increased across assess-
ments, whereas performance variability significantly decreased.
For overall performance, the mean difference between Time 1 and
Time 2 assessments was 3.0 SS points higher on the Time 2
assessment, t(41) ⫽⫺4.2, p ⬍ .001; for performance variability,
the mean difference between time points was 1.5 SS points lower
on the Time 2 assessment, t(41) ⫽ 2.1, p ⫽ .04 (see Figure 1).
Cluster Analysis
Cluster analysis was also used to examine performance variabil-
ity in the athlete sample. Baseline and postconcussion performance
variability indices were entered into a k-means cluster analysis
where k was set equal to 2. This approach allowed for examination
of potential subgroups with distinct longitudinal variability pro-
files. Final cluster centers for Cluster 1 (n ⫽ 46) were baseline
performance variability ⫽ 10.4 SS points and postconcussion
performance variability ⫽ 9.6 SS points. Cluster centers for Clus-
ter2(n ⫽ 25) were baseline performance variability ⫽ 15.3 SS
points and postconcussion performance variability ⫽ 17.6 SS
points. These are referred to subsequently as the low-variability
cluster and high-variability cluster, respectively. A repeated mea-
sures analysis of variance (ANOVA) was conducted to examine a
potential cluster by time point interaction, which was confirmed,
F(1, 69) ⫽ 5.77, p ⬍ .05. The nature of the effect was such that the
low-variability cluster group demonstrated a slight decrease in
performance variability over time, whereas the high-variability
cluster group demonstrated an increase in performance variability
postconcussion.
To further examine the effect of concussion on neuropsycho-
logical performance and symptom reporting in the low- and high-
variability cluster groups, we ran separate repeated measures
ANOVAs using overall performance and PCSS score as dependent
variables. Results indicated a significant main effect of cluster
group on overall performance, F(1, 69) ⫽ 21.18, p ⬍ .001, as well
as a significant cluster group by time point interaction, F(1, 69) ⫽
6.38, p ⬍ .05. The nature of this effect was such that the low-
variability cluster group demonstrated a slight increase in overall
performance from baseline to postconcussion, whereas the high-
variability cluster group demonstrated a decrease in overall per-
formance. Figures 1 and 2 illustrate the cluster by time point
interaction; Time 1 and Time 2 means for the control group are
also depicted for context. With regard to PCSS score, there were
no significant main effect of time point and no significant time
Table 3
Correlations
Variable BL OP BL PV PC OP PC PV
Baseline overall performance (BL OP) — — — —
Baseline performance variability (BL PV) ⫺.44
ⴱⴱ
———
Postconcussion overall performance (PC OP) .61
ⴱⴱ
⫺.30
ⴱ
——
Postconcussion performance variability (PC PV) ⫺.27 .36
ⴱ
⫺.52
ⴱⴱ
—
WTAR FSIQ estimate .38
ⴱⴱ
⫺.03 .46
ⴱⴱ
⫺.14
Note. Overall performance is the mean of standard score-transformed indices from the following measures:
Comprehensive Trail-Making Test, Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT
Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction Time composites), Brief Visual Memory
Test—Revised Delayed Recall, Hopkins Verbal Memory Test—Revised Delayed Recall, Symbol-Digit Modal-
ities Test, and Stroop Color-Word Time. Performance variability is the standard deviation of the indices listed
above. WTAR FSIQ ⫽ Wechsler Test of Adult Reading Full Scale IQ.
ⴱ
Significant at the .05 level.
ⴱⴱ
Significant at the .01 level.
Figure 1. Cluster by time point interaction for performance variability.
Intraindividual variability is in standard score (SS) units. SSs have a mean
of 100 and a standard deviation of 15. Repeated measures analysis of
variance, F(1, 69) ⫽ 5.77, p ⬍ .05. Post hoc paired-samples t test revealed
a trend toward postconcussion increase in performance variability from
baseline for the high-variability cluster group: t(24) ⫽ 1.96, p ⫽ .06.
Figure 2. Cluster by time point interaction for overall performance. Mean
performance is in standard score (SS) units. SSs have a mean of 100 and
a standard deviation of 15. Repeated measures analysis of variance,
F(1, 69) ⫽ 6.38, p ⬍ .05. Post hoc paired-samples t test revealed signif-
icant postconcussion improvement from baseline in overall performance
for the low variability cluster group: t(45) ⫽ 2.14, p ⬍ .05.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
485
COGNITIVE VARIABILITY AND CONCUSSION
point by cluster group interaction. There was a trend toward a main
effect of cluster group on PCSS score, F(1, 69) ⫽ 3.25, p ⫽ .08.
Post hoc paired-sample t tests were conducted to further exam-
ine the significant cluster group by time point interactions. This
analysis revealed that the low-variability cluster group exhibited a
significant increase in overall performance postconcussion,
t(45) ⫽ 2.14, p ⬍ .05, but no significant change in performance
variability, t(45) ⫽⫺1.12, p ⫽ .27. In contrast, the high-
variability cluster group exhibited a trend toward a significant
increase in performance variability, t(24) ⫽ 1.96, p ⫽ .06, but no
significant change in overall performance postconcussion, t(24) ⫽
⫺1.44, p ⫽ .16.
The high-variability and low-variability cluster groups were
not significantly different from each other with respect to
self-reported previous head injuries (p ⫽ .97), WTAR FSIQ
estimate (p ⫽ .16), or diagnosis of attention-deficit/hyperactiv-
ity disorder or learning disability (p ⫽ .14, and p ⫽ .29,
respectively). Scores for individual test indices at both time
points by group are provided in Table 4.
Reliable Decline
To examine the potential clinical significance of these findings,
we calculated reliable decline on one or more, two or more, and
three or more test indices. Individuals in the high-variability clus-
ter group were more likely than individuals in the low-variability
cluster or control groups to exhibit reliable decline on one or more
test,
2
(2) ⫽ 8.5, p ⬍ .05, two or more tests,
2
(2) ⫽ 14.6, p ⬍
.005, and three or more tests,
2
(2) ⫽ 20.5, p ⬍ .001 (see Table 5
and Figure 3).
Discussion
The purpose of the present study was to examine intraindividual
variability across cognitive tests in college athletes tested pre- and
postinjury. These findings suggest that normative cognitive per-
formance in college athletes is characterized by significant intra-
individual variation across tests on the order of nearly 1 standard
deviation (12 SS points). Greater performance variability was
unrelated to an estimate of intellectual functioning (WTAR FSIQ
estimate), but was significantly related to lower mean level of
performance on the test battery.
In the athlete sample as a whole, there was no effect of concus-
sion on either performance variability or overall performance. This
is a counterintuitive finding for which there are multiple explana-
tions. It is likely that injury effects on specific tests are obscured
in analyses considering the summary indices. However, the cluster
analysis strongly suggests that the null finding in the overall
sample is at least, in part, due to heterogeneity in the neurocogni-
tive effects of concussion. Despite the absence of an injury effect
in the athlete sample overall, cluster membership predicted pre-
and postinjury change in overall performance and performance
variability. Furthermore, compared with the low-variability cluster
and control groups, the high-variability cluster group was signif-
icantly more likely to display reliable declines on individual tests.
Roughly two thirds of the sample exhibited cognitive performance
that was characterized by relatively low variability. Membership in
this low-variability cluster group predicted a decrease in perfor-
mance variability and an increase in overall performance postcon-
cussion. In contrast, roughly one third of the sample exhibited
cognitive performance that was characterized by relatively high
variability. Membership in the high-variability cluster group pre-
dicted an increase in performance variability and a decrease in
overall performance postconcussion.
Control participants who were administered the same test bat-
tery at two time points 1 month apart demonstrated a pattern of
performance similar to athletes in the low-variability cluster
group—that is, controls showed a decrease in performance vari-
ability and an increase in overall performance from Time 1 to
Time 2. It is likely that this pattern of performance reflects a
practice effect—that is, control participants and low-variability
athletes demonstrated the ability to benefit from prior exposure to
the test battery. By contrast, the high-variability cluster group did
not demonstrate a benefit from practice.
There are at least two likely explanations for this finding. It
could be that high-variability athletes were poorly motivated at
baseline, and hence, failed to benefit from prior exposure to the
test battery. Alternatively, this group of athletes may be experi-
encing cognitive dysfunction as a result of the recent head injury,
and their increased performance inconsistency may represent the
neurocognitive consequence of their concussion.
We examined the likelihood of exhibiting reliable decline on the
test battery to evaluate the clinical utility of these findings, and
also to help distinguish between the two explanations for the
high-variability cluster group’s failure to exhibit a practice effect:
(1) poor motivation versus (2) concussion-related cognitive dys-
function. Lower incidence of reliable decline in the high-
variability athletes would support the notion that these athletes
were poorly motivated at baseline; hence, their baseline perfor-
mance represented an underestimate their true premorbid abilities.
Conversely, higher incidence of reliable decline from baseline
performance would support the hypothesis that these athletes ex-
perienced concussion-related cognitive dysfunction. This latter
explanation was supported by our results: Athletes in the high-
variability cluster group were more likely than the other groups to
exhibit reliable decline from their baseline performance. We note
that these two explanations are not mutually exclusive; it is, of
course, possible that high-variability athletes were both poorly
motivated at baseline and also experiencing neurocognitive se-
quelae of their concussion. However, the fact that these athletes
exhibited reliable decline from their baseline performance suggests
that intraindividual variability could impart clinically meaningful
information regarding which athletes are likely to exhibit
concussion-related cognitive dysfunction.
Taken together, these results suggest that there is a relationship
between cross-test intraindividual variability and mean level of
performance such that as performance improves, cross-test vari-
ability decreases. This relationship between performance and vari-
ability on a cross-domain level is consistent with previous work
demonstrating a link between cross-test intraindividual variability
and cognitive decline (Holtzer et al., 2008; Kliegel & Sliwinski,
2004), as well as a larger body of research that has reported an
inverse relationship between mean level of performance and in-
traindividual inconsistency within a single task (Hultsch & Mac-
Donald, 2004; Hultsch et al., 2008). The replication of this finding
at the cross-domain level has potential clinical relevance because
cross-test variability can be easily calculated in typical neuropsy-
chological batteries administered for clinical purposes.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
486
RABINOWITZ AND ARNETT
Table 4
Scores for Individual Test Indices by Group
Test
High variability
(n ⫽ 25)
Low variability
(n ⫽ 46)
Control
(n ⫽ 42)
BL/T1
between groups
PC/T2
between groups
BL
M (SD)
PC
M (SD)
Within
group
BL
M (SD)
PC
M (SD)
Within
group
T1
M (SD)
T2
M (SD)
Within
group
t(24) pt(45) pt(41) pF(2, 109) pF(2, 110) p
CTMT
Trial 1 Z-score
(Trail 2 or 3) ⫺0.56 (1.68) ⫺0.21 (0.21) ⫺0.8 .43 0.26 (0.68) 0.46 (0.16) ⫺2.3 .02 ⫺0.23 (1.18) 0.23 (0.16) ⫺4.4 ⬍.01 4.8 .01 4.5 .01
Trial 2 Z-score
(Trail 4 or 5) ⫺0.26 (1.09) ⫺0.25 (0.18) 0.3 .74 0.06 (0.81) 0.40 (0.14) ⫺2.5 .01 ⫺0.12 (0.83) 0.36 (0.14) ⫺4.0 ⬍.01 1.1 .33 6.3 ⬍.01
ImPACT
Verbal Memory composite 79.8 (9.3) 74.4 (2.0) 2.8 .01 85.6 (10.9) 86.0 (1.5) ⫺0.2 .88 88.4 (9.0) 88.3 (1.5) 0.1 .95 5.3 .01 17.3 ⬍.01
Visual Memory composite 70.9 (12.4) 69.0 (2.7) 0.7 .47 77.8 (12.6) 79.1 (2.0) ⫺1.0 .31 75.6 (16.2) 78.7 (2.0) ⫺1.4 .17 1.4 .25 6.5 ⬍.01
Visual Motor Speed
composite 35.1 (9.9) 34.4 (1.3) 0.5 .62 36.6 (6.6) 37.7 (1.0 ⫺1.4 .17 38.3 (6.9) 38.5 (1.0) ⫺0.0 .99 1.4 .25 3.0 .05
Reaction Time composite 0.59 (0.09) 0.61 (0.02) ⫺0.3 .80 0.59 (0.06) 0.58 (0.01) 0.7 .48 0.59 (0.09) 0.59 (0.02) 0.2 .88 0.1 .94 0.6 .58
BVMT–R Delayed Recall raw
score 9.5 (2.6) 8.8 (0.3) 1.0 .31 10.8 (1.4) 10.8 (0.3) 0.1 .90 11.2 (0.9) 11.4 (0.3) ⫺1.2 .22 9.1 ⬍.01 19.2 ⬍.01
HVLT–R Delayed Recall raw
score 8.8 (2.4) 7.6 (0.4) 2.1 .05 9.7 (1.5) 9.7 (0.3) 0.2 .87 9.7 (1.6) 9.9 (0.3) ⫺0.4 .72 2.2 .11 14.2 ⬍.01
SDMT raw score 57.1 (10.8) 54.5 (2.5) 1.5 .16 61.3 (9.6) 63.6 (1.9) ⫺1.9 .06 62.8 (11.2) 68.9 (1.9) ⫺5.5 ⬍.01 2.3 .10 12.0 ⬍.01
Stroop Color-Word Time 111.3 (18.7) 119.4 (5.2) ⫺0.4 .69 111.3 (21.3) 105.3 (4.0) 3.6 ⬍.01 102.3 (17.7) 98.6 (4.0) 1.6 .11 3.7 .03 4.6 .01
CARB % failure 0.0 12.0 6.5 0.0 0.0 2.4
Note.BL⫽ Baseline; T1 ⫽ Time 1; PC ⫽ postconcussion; T2 ⫽ Time 2; CTMT ⫽ Comprehensive Trail-Making Test; ImPACT ⫽ Immediate Post-Concussion Assessment and Cognitive Testing;
BVMT–R ⫽ Brief Visuospatial Memory Test—Revised; HVLT–R ⫽ Hopkins Verbal Learning Test—Revised; SDMT ⫽ Symbol-Digit Modalities Test; CARB ⫽ Computerized Assessment of
Response Bias. Within-group t tests compare means for BL and PC or for T1 and T2. Between-groups analyses of variance compare means across three groups: concussed athletes in the high-variability
cluster group, concussed athletes in the low-variability cluster group, and controls.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
487
COGNITIVE VARIABILITY AND CONCUSSION
Importantly, although performance inconsistency was signifi-
cantly correlated with mean level of performance, it was not
significantly correlated with an estimate of intellectual function-
ing. This suggests that the association between performance in-
consistency and performance level may be independent of IQ;
however, a more thorough IQ assessment (rather than a
performance-based estimate like the WTAR) would be necessary
to test this hypothesis. Furthermore, all participants in this study
were college students at the same institution. Hence, there were no
gross educational discrepancies among these participants. These
observations suggest that intraindividual variability may be index-
ing an important cognitive process that cannot be fully explained
by IQ or education. It is possible that this process is related to
lapses in attention (Bunce et al., 1993), failure to maintain exec-
utive control (West et al., 2002), or inefficiencies in central ner-
vous system functioning (Holtzer et al., 2008).
Although there was no effect of concussion on cognitive per-
formance in the sample as a whole, cluster membership based on
performance variability at baseline was related to response to
injury, with the high-variability cluster group demonstrating signs
of concussion-related neurocognitive dysfunction. There are a
number of possible explanations for this finding. Intraindividual
variability in the high-variability cluster group may indicate a level
of premorbid vulnerability to injury. That is, this group may be
more likely to experience neurological decline as a result of mild
traumatic brain injury. The high- and low-variability cluster groups
were not significantly different from each other with regard to
self-report of previous head injury, attention-deficit/hyperactivity
disorder diagnosis, or learning disability diagnosis. These results
suggest that any group differences in vulnerability to injury may
not be related to these particular premorbid conditions; however,
inaccuracies in self-report may hamper the validity of these vari-
ables.
As discussed previously, athletes’ approach to testing could be
another possible explanation for the difference between clusters. It
may be the case that higher levels of intraindividual variability are
related to poor effort toward testing. At baseline, athletes may
undervalue the importance of putting forth adequate effort. They
may fail to see the significance of baseline concussion testing or
regard it as an imposition. It is also possible that some athletes may
purposefully underperform at baseline to decrease their chances of
being evaluated as impaired postconcussion. By contrast, return-
to-play decisions provide a powerful incentive for athletes to
exhibit maximal effort toward testing postconcussion—perhaps
even greater effort than patients typically seen in other neuropsy-
chological assessment settings. This notion has been supported in
work by Bailey et al. (2006) demonstrating that some athletes
exhibit markedly improved cognitive performance in response to
concussion.
Alternatively, the low-variability cluster group may be highly
motivated at baseline. In this group, practice effects may mask any
neurocognitive impairment related to mild and uncomplicated con-
cussions. This notion is supported by the fact that participants in
the control group demonstrated improved performance suggestive
of practice effects, whereas the low-variability athletes exhibited
stable performance. Prior research has demonstrated that, within a
single task, intraindividual variability in reaction time decreases
with repeated exposure to the task, suggesting that decreased
inconsistency is a hallmark of practice (Ram et al., 2005). We note,
however, that the results of formal effort testing do not support this
hypothesis, as three athletes in the low-variability cluster group
and no athletes in the high-variability cluster group failed formal
effort testing (see Table 4). However, this does not preclude the
possibility that more subtle motivational forces, perhaps not de-
tectible with standard symptom validity indicators, may be at play.
There are limitations to the present study. First, with 71 partic-
ipants in the athlete group, this study may have been underpowered
to detect small-to-moderate effects that may be meaningful, par-
ticularly with regard to group differences between the high-
Table 5
Number of Cases Exhibiting Reliable Decline on One or More, Two or More, and Three or More Test Indices
Test
Group
2
(2)
p
High variability Low variability Control
Decline Stable Decline Stable Decline Stable
1 or more 16 (64%) 9 (36%) 15 (33%) 31 (67%) 13 (31%) 29 (69%) 8.5 .014
2 or more 12 (48%) 13 (52%) 8 (17%) 38 (83%) 4 (10%) 38 (90%) 14.6 .001
3 or more 9 (36%) 16 (64%) 4 (9%) 42 (91%) 0 (0%) 42 (100%) 20.5 ⬍.001
Note. Reliable change statistics were calculated based on control group scores. A 90% confidence interval was selected as the criterion for determining
reliable decline in performance postconcussion. An index of total declined scores was calculated by summing the total number of tests for which an
individual demonstrated reliable decline.
Figure 3. The percentage of cases to exhibit reliable decline by group.
Reliable change statistics were calculated based on control group scores. A
90% confidence interval was selected as the criterion for determining
reliable decline in performance postconcussion. An index of total declined
scores was calculated by summing the total number of tests for which an
individual demonstrated reliable decline.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
488
RABINOWITZ AND ARNETT
variability and low-variability cluster groups. Furthermore, self-
reports of important clinical variables such as history of premorbid
conditions, previous concussions, and postconcussion symptoms
are subject to biases. A more objective examination of the rela-
tionship between these variables and intraindividual variability is
warranted. Finally, the present study is exploratory in nature.
Intraindividual variability in cognitive performance is a relatively
new area of research, and cross-domain inconsistency has not thus
far been evaluated for its potential clinical and theoretical rele-
vance in concussion.
The aim of the present study was twofold: (1) to describe
cross-test intraindividual variability in a sample of college athletes
before and after sports-related head injury, and (2) to begin to
explore potential correlates of this cognitive inconsistency. Future
studies evaluating some of the specific hypotheses discussed above
(i.e., cognitive inconsistency as an index of previous head trauma
and cognitive inconsistency as an index of suboptimal motivation)
are necessary to determine the potential theoretical significance
and clinical utility of cross-test intraindividual variability in sports-
related concussion assessment.
References
Anstey, K. J., Mack, H. A., Christensen, H., Li, S. C., Reglade-Meslin, C., Maller,
J.,...Sachdev, P. (2007). Corpus callosum size, reaction time speed and
variability in mild cognitive disorders and in a normative sample.
Neuropsychologia, 45, 1911–1920. doi:10.1016/j.neuropsychologia
.2006.11.020
Bäckman, L., Nyberg, L., Lindenberger, U., Li, S. C., & Farde, L. (2006).
The correlative triad among aging, dopamine, and cognition: Current
status and future prospects. Neuroscience & Biobehavioral Reviews, 30,
791– 807. doi:10.1016/j.neubiorev.2006.06.005
Bailey, C. M., Echemendia, R. J., & Arnett, P. A. (2005, February). Visual
memory predicts concussion status independent of verbal memory and
post-mild traumatic brain injury symptoms in college athletes. Paper
presented at the 33rd annual meeting of the International Neuropsycho-
logical Society, St. Louis, MO.
Bailey, C. M., Echemendia, R. J., & Arnett, P. A. (2006). The impact of
motivation on neuropsychological performance in sports-related mild
traumatic brain injury. Journal of the International Neuropsychological
Society, 12, 475– 484. doi:10.1017/S1355617706060619
Barth, J. T., Alves, W. M., Ryan, T. V., Macciocchi, S. N., Rimel, R. W.,
Jane, J. A., & Nelson, W. E. (1989). Mild head injury in sports:
Neuropsychological sequelae and recovery of function. In H. S. Levin,
H. M. Eisenberg, & A. L. Benton (Eds.), Mild head injury (pp. 257–
275). New York, NY: Oxford University Press.
Benedict, R. H. B. (1997). Brief Visuospatial Memory Test—Revised:
Professional manual. Odessa, FL: Psychological Assessment Resources.
Benedict, R. H. B., Schretlen, D., Groninger, L., & Brandt, J. (1998).
Hopkins Verbal Learning Test—Revised: Normative data and analysis
of inter-form and test–retest reliability. Clinical Neuropsychologist, 12,
43–55. doi:10.1076/clin.12.1.43.1726
Binder, L. M., Iverson, G. L., & Brooks, B. L. (2009). To err is human:
“Abnormal” neuropsychological scores and variability are common in
healthy adults. Archives of Clinical Neuropsychology, 24, 31–46. doi:
10.1093/arclin/acn001
Bohnen, N., Twijnstra, A., Jolles, J., Moller, J. T., Cluitmans, P., Rasmus-
sen, L. S., . . . Metsemakers, J. F. M. (1992). Performance in the Stroop
Color-Word Test in relationship to the persistence of symptoms follow-
ing mild head injury. Acta Neurologica Scandinavica, 85, 116 –121.
doi:10.1111/j.1600-0404.1992.tb04009.x
Britton, T. C., Meyer, B. U., & Benecke, R. (1991). Variability of cortically
evoked motor responses in multiple sclerosis. Electroencephalography
and Clinical Neurophysiology, 81, 186 –194. doi:10.1016/0168-
5597(91)90071-5
Bruce, J. M., & Echemendia, R. J. (2003). Delayed-onset deficits in verbal
encoding strategies among patients with mild traumatic brain injury.
Neuropsychology, 17, 622– 629. doi:10.1037/0894-4105.17.4.622
Bunce, D. J., Warr, P. B., & Cochrane, T. (1993). Blocks in choice
responding as a function of age and physical fitness. Psychology and
Aging, 8, 26 –33. doi:10.1037/0882-7974.8.1.26
Cegalis, J., & Bowlin, J. (1991). VIGIL: Software for the assessment of
attention. Nashua, NH: Forthought.
Conder, R., Allen, L., & Cox, D. (1992). Computerized Assessment of
Response Bias test manual. Durham, NC: Cognisyst.
Echimendia, R. J., & Julian, L. J. (2001). Mild traumatic brain injury in
sports: Neuropsychology’s contribution to a developing field. Neuropsy-
chology Review, 11, 69 – 88. doi:10.1023/A:1016651217141
Halpern, D. F. (1997). Sex differences in intelligence: Implications for
education. American Psychologist, 52, 1091–1102. doi:10.1037/0003-
066X.52.10.1091
Holtzer, R., Verghese, J., Wang, C., Hall, C. B., & Lipton, R. B. (2008).
Within-person across-neuropsychological test variability and incident
dementia. Journal of the American Medical Association, 300, 823– 830.
doi:10.1001/jama.300.7.823
Hultsch, D. F., & MacDonald, S. W. S. (2004). Intraindividual variability
in performance as a theoretical window onto cognitive aging. In R. A.
Dixon, L. Bäckman, & L. Goran-Nillson (Eds.), New frontiers in cog-
nitive aging (pp. 65– 88). Oxford, UK: Oxford University Press. doi:
10.1093/acprof:oso/9780198525691.003.0004
Hultsch, D. F., MacDonald, S. W. S., Hunter, M. A., Levy-Bencheton, J.,
& Strauss, E. (2000). Intraindividual variability in cognitive perfor-
mance in older adults: Comparison of adults with mild dementia, adults
with arthritis, and healthy adults. Neuropsychology, 14, 588 –598. doi:
10.1037/0894-4105.14.4.588
Hultsch, D. F., Strauss, E., Hunter, M. A., & MacDonald, S. W. S. (2008).
Intraindividual variability, cognition, and aging. In F. I. M. Craik &
T. A. Salthouse (Eds.), The handbook of aging and cognition (3rd ed.,
pp. 491–556). New York, NY: Psychology Press.
Iverson, G. L., Lovell, M. R., & Collins, M. W. (2005). Validity of
ImPACT for measuring processing speed following sports-related con-
cussion. Journal of Clinical and Experimental Neuropsychology, 27,
683– 689. doi:10.1081/13803390490918435
Kelly, A. M., Uddin, L. Q., Biswal, B. B., Castellanos, F. X., & Milham,
M. P. (2008). Competition between functional brain networks mediates
behavioral variability. NeuroImage, 39, 527–537. doi:10.1016/j
.neuroimage.2007.08.008
Kliegel, M., & Sliwinski, M. (2004). MMSE cross-domain variability
predicts cognitive decline in centenarians. Gerontology, 50, 39 – 43.
doi:10.1159/000074388
Lövdén, M., Li, S. C., Shing, Y. L., & Lindenberger, U. (2007). Within-
person trial-to-trial variability precedes and predicts cognitive decline in
old and very old age: Longitudinal data from the Berlin Aging Study.
Neuropsychologia, 45, 2827–2838. doi:10.1016/j.neuropsychologia
.2007.05.005
Lovell, M. R., Collins, M. W., Podell, K., Powell, J., & Maroon, J. (2000).
ImPACT: Immediate Post-Concussion Assessment and Cognitive Test-
ing. Pittsburgh, PA: NeuroHealth Systems.
MacDonald, S. W. S., Li, S. C., & Bäckman, L. (2009). Intraindividual
variability and aging. Psychology and Aging, 24, 792– 808. doi:10.1037/
a0017798
Ponsford, J., & Kinsella, G. (1992). Attentional deficits following closed-
head injury. Journal of Clinical and Experimental Neuropsychology, 14,
822– 838. doi:10.1080/01688639208402865
Ragan, B. G., & Kang, M. (2007). Measurement issues in concussion
testing. Athletic Therapy Today, 12, 2– 6.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
489
COGNITIVE VARIABILITY AND CONCUSSION
Ram, N., Rabbitt, P., Stollery, B., & Nesselroade, J. R. (2005). Cognitive
performance inconsistency: Intraindividual change and variability. Psy-
chology and Aging, 20, 623– 633. doi:10.1037/0882-7974.20.4.623
Randolph, C., McCrea, M., & Barr, W. B. (2005). Is neuropsychological
testing useful in the management of sport-related concussion? Journal of
Athletic Training, 40, 139 –154.
Reynolds, C. R. (2002). Comprehensive Trail Making Test (CTMT). Aus-
tin, TX: Pro-Ed.
Ruff, R. M., Light, R. H., & Parker, S. B. (1996). Benton Controlled Oral
Word Association Test: Reliability and updated norms. Archives of
Clinical Neuropsychology, 11, 329 –338.
Schatz, P., Pardini, J. E., Lovell, M. R., Collins, M. W., & Podell, K.
(2006). Sensitivity and specificity of the ImPACT test battery for con-
cussion in athletes. Archives of Clinical Neuropsychology, 21, 91–99.
doi:10.1016/j.acn.2005.08.001
Schretlen, D. J., Munro, C. A., Anthony, J. C., & Pearlson, G. D. (2003).
Examining the range of normal intraindividual variability in neuropsy-
chological test performance. Journal of the International Neuropsycho-
logical Society, 9, 864 – 870. doi:10.1017/S1355617703960061
Smith, A. (1982). Symbol Digit Modalities Test (SDMT) manual (revised).
Los Angeles, CA: Western Psychological Services.
Sowell, E. R., Peterson, B. S., Thompson, P. M., Welcome, S. E., Henke-
nius, A. L., & Toga, A. W. (2003). Mapping cortical change across the
human life span. Nature Neuroscience, 6, 309 –315. doi:10.1038/nn1008
Stuss, D. T., Murphy, K. J., Binns, M. A., & Alexander, M. P. (2003).
Staying on the job: The frontal lobes control individual performance
variability. Brain, 126(Pt. 11), 2363–2380. doi:10.1093/brain/awg237
Trenerry, M. R., Crosson, B., DeBoe, J., & Leber, W. R. (1989). Stroop
Neuropsychological Screening Test. Odessa, FL: Psychological Assess-
ment Resources.
Vanderploeg, R. D., Curtiss, G., & Belanger, H. G. (2005). Long-term
neuropsychological outcomes following mild traumatic brain injury.
Journal of the International Neuropsychological Society, 11, 228 –236.
doi:10.1017/S1355617705050289
Walhovd, K. B., & Fjell, A. M. (2007). White matter volume predicts
reaction time instability. Neuropsychologia, 45, 2277–2284. doi:
10.1016/j.neuropsychologia.2007.02.022
Wechsler, D. (1997). Wechsler Adult Intelligence Scale—Third Edition
(WAIS–III). New York, NY: Psychological Corporation.
Wechsler, D. (2001). The Wechsler Test of Adult Reading (WTAR): Test
manual. San Antonio, TX: Psychological Corporation.
West, R., Murphy, K. J., Armilio, M. L., Craik, F. I. M., & Stuss, D. T.
(2002). Lapses of intention and performance variability reveal age-
related increases in fluctuations of executive control. Brain and Cogni-
tion, 49, 402– 419. doi:10.1006/brcg.2001.1507
Received August 17, 2012
Revision received January 4, 2013
Accepted April 9, 2013 䡲
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
490
RABINOWITZ AND ARNETT
