ArticlePDF Available

Abstract and Figures

The ImPACT (Immediate Postconcussion Assessment and Cognitive Testing) is a computerized neurocognitive test used to assist in the management of sport concussion management. A number of studies have documented the reliability and sensitivity of the ImPACT, but no studies have examined the equivalence of the ImPACT's alternate forms. The objective of our study was to determine the equivalence of the ImPACT's five alternate forms. Participants were administered alternate forms of ImPACT based on clinically relevant time frame derived from an extensive sports concussion database. Participants completed a baseline assessment followed by various combinations of the remaining alternate forms at 45 and 50 days. Inferential Confidence Intervals were calculated for each composite score for all alternate forms. We found non-equivalence between ImPACT form 1 and forms 2, 3, and 4 on the Verbal Memory and between forms 2 and 4. ImPACT forms 1 and 3 were not equivalent on the Visual Memory Composite. Finally, ImPACT forms 3 and 4 were not equivalent on the Visual Motor Speed and Reaction Time Composites. Alternate form equivalence is necessary to minimize measurement error and optimize clinical decision making. Clinicians using the ImPACT should consider non-equivalence of some forms on certain Composites when interpreting ImPACT following sports concussion.
Content may be subject to copyright.
This article was downloaded by: [Jacob Resch]
On: 09 October 2013, At: 06:34
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The Clinical Neuropsychologist
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ntcn20
Preliminary Evidence of Equivalence of
Alternate Forms of the ImPACT
Jacob E. Resch
a
, Stephen Macciocchi
b
& Michael S. Ferrara
c
a
Brain Injury Laboratory, Department of Kinesiology, University of
Texas Arlington, TX, USA.
b
Shepherd Center, Atlanta, GA, USA.
c
The University of New Hampshire, Durham, NH, USA.
Published online: 08 Oct 2013.
To cite this article: Jacob E. Resch, Stephen Macciocchi & Michael S. Ferrara , The Clinical
Neuropsychologist (2013): Preliminary Evidence of Equivalence of Alternate Forms of the ImPACT,
The Clinical Neuropsychologist, DOI: 10.1080/13854046.2013.845247
To link to this article: http://dx.doi.org/10.1080/13854046.2013.845247
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Preliminary Evidence of Equivalence of Alternate Forms of
the ImPACT
Jacob E. Resch
1
, Stephen Macciocchi
2
, and Michael S. Ferrara
3
1
Brain Injury Laboratory, Department of Kinesiology, University of Texas Arlington, TX, USA
2
Shepherd Center, Atlanta, GA, USA
3
The University of New Hampshire, Durham, NH, USA
The ImPACT (Immediate Postconcussion Assessment and Cognitive Testing) is a computerized
neurocognitive test used to assist in the management of sport concussion management. A
number of studies have documented the reliability and sensitivity of the ImPACT, but no
studies have examined the equivalence of the ImPACTs alternate forms. The objective of our
study was to determine the equivalence of the ImPACTs ve alternate forms. Participants were
administered alternate forms of ImPACT based on clinically relevant time frame derived from
an extensive sports concussion database. Participants completed a baseline assessment followed
by various combinations of the remaining alternate forms at 45 and 50 days. Inferential
Condence Intervals were calculated for each composite score for all alternate forms. We
found non-equivalence between ImPACT form 1 and forms 2, 3, and 4 on the Verbal Memory
and between forms 2 and 4. ImPACT forms 1 and 3 were not equivalent on the Visual
Memory Composite. Finally, ImPACT forms 3 and 4 were not equivalent on the Visual Motor
Speed and Reaction Time Composites. Alternate form equivalence is necessary to minimize
measurement error and optimize clinical decision making. Clinicians using the ImPACT should
consider non-equivalence of some forms on certain Composites when interpreting ImPACT
following sports concussion.
Keywords: Equivalence; Alternate forms; Concussion; Computerized neuropsychological tests; Psychometric;
Cognitive.
INTRODUCTION
During the past 10 years, research and consensus panels have recommended a
multi-faceted clinical approach for managing sports concussions (Aubry et al., 2002;
Broglio, Macciocchi, & Ferrara, 2007; Guskiewicz et al., 2004; Harmon et al., 2013;
McCrory et al., 2005, 2009). Neurocognitive testing, balance assessment, and
self-reported symptom scales are typically utilized to assist healthcare professionals in
making return-to-play (RTP) decisions for athletes following the diagnosis of sport
concussion (McCrory et al., 2009). Despite controversy, computerized neurocognitive
testing (CNT) has been commonly accepted as a replacement for traditional neurocog-
nitive testing at all levels of sport. Rationale for this shift in clinical practice includes
the ability to assess large numbers of athletes, standardized delivery, and the availabil-
ity of alternate forms (Covassin, Elbin, Stiller-Ostrowski, & Kontos, 2009; Guskiewicz
et al., 2004; Meehan, dHem ecourt, Collins, Taylor, & Comstock, 2012). The ImPACT
Address correspondence to: Jacob E. Resch, Ph.D., ATC, Box 19259, 113 Maverick Activities Center,
Arlington, TX 76019-0259, USA. Email: resch@uta.edu
(Received 28 October 2012; accepted 11 September 2013)
The Clinical Neuropsychologist, 2013
http://dx.doi.org/10.1080/13854046.2013.845247
Ó 2013 Taylor & Francis
Downloaded by [Jacob Resch] at 06:34 09 October 2013
is a popular CNT and is used in many sports settings (Broglio, Ferrara, Macciocchi,
Baumgartner, & Elliott, 2007; Meehan et al., 2012; Randolph, McCrea, & Barr, 2005).
Psychometric studies conducted over the past 10 years examining the reliability
of the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) have
found variable testretest reliability coefcients (Broglio, Ferrara et al., 2007; Resch
et al., 2013; Schatz, 2009). Broglio, Ferrara et al. (2007) and Resch et al. (2013)
reported sub-optimal ImPACT testretest reliability coefcients ranging from .15 to .76
over a 50-day period. In contrast, intraclass correlation coefcients (ICC) ranged from
.42 to .85 in two studies employing 1- and 2-year testretest intervals (Elbin, Schatz,
& Covassin, 2011; Schatz, 2009).
One possible explanation for variability in ImPACT testretest reliability may be
the different testretest time intervals used in existing research. Another plausible expla-
nation for the observed differences in test retest reliability coefcients may be variability
in ImPACT forms used in testretest reliability research (Broglio, Ferrara et al., 2007;
Elbin, 2012; Elbin et al., 2011; Resch et al. 2013; Schatz, 2009, 2012). The ImPACT
has ve alternate or parallel forms designed to minimize practice effects associated with
repeat testing while simultaneously maintaining test sensitivity. While alternate forms
are a strength of any instrument, tests using multiple forms must show some evidence of
form equivalence (Crocker & Algina, 2008). For example, the Hopkins Verbal Learning
Test (HVLT) and Hopkins Verbal Learning Test-Revised have six alternate forms. Each
version of the HLVT has been demonstrated to possess equivalent alternate forms using
various statistical methods over varying periods of time (Brandt, 1991; ONeil-Pirozzi,
Goldstein, Strangman, & Glenn, 2012). Although the ImPACTs alternate forms are
presumed to be equivalent, no evidence exists to support or refute this claim.
The objective of the current study was to inves tigate form equivalence of the ve
alternate forms of ImPACT using a clinically relevant post-concussive testing
paradigm. Evidence of alternate form equivalence or non-equivalence may help
identify and explain the reported variability in testretest reliability. Our ndings also
have the potential to contribute to the ImPACTs development by examining form
equivalence and to determine which forms may be most appropriate in repeated
examinations of concussed athletes.
METHOD
Participants
The current study is based on a separate, but related, investigation which
addressed the testretest reliability of forms 1, 2, and 3 of the ImPACT using clinically
relevant time points. (Resch et al., 2013), in which 152 collegiate-aged participants
were recruited from the general student body of a large metropolitan university to take
part. Participants were recruited via face-to-face classroom recruitment, word-of-mouth,
and yers. Participants were excluded if they had a history of concussion diagnosed
by a physician or certied athletic trainer within 6 months prior to or during the study;
if English was not their primary language; if they were diagnosed with a learning
disability or attention decit disorder; and if they consumed alcohol within 24 hours
of their testing session. Furthermore, participants were removed from analysis if their
performance on the rst administration of ImPACT was considered invalid based on
2 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
the manufacturers criteria, or if he or she fell below acceptable performance (85%) on
Greens Word Memory Test (WMT), a measure of effort (Green, 2003; M. Lovell,
Collins, & Maroon, 2009).
In order to assess the equivalence of alternate forms participants were randomly
assigned into one of six groups via systematic sampling. Each group was tested at
three time points (day 1, day 45, and day 50) with three form s of ImPACT. All groups
were administered form 1 at day 1. Groups randomly and systematically differed on
which form of ImPACT they received at day 45 and day 50. The ordering of forms
for each of the six groups was as follo ws: (Group 1) (forms 1,2,3), Group 2 (forms
1,2,4), Group 3 (forms 1,2,5), Group 4 (forms 1,3,4), Group 5 (forms 1,3,5), and
Group 6 (forms 1,4,5). While utilization of forms 1, 2, and 3 is presumably the most
frequently used combination of alternate forms, comparisons were designed to directly
compare each of the ImPACTs alternate forms.
Tests
Greens Word Memory Test (Greens Publishing Inc, Edmonton, Alberta) was
used as a measure of participant effort. The Word Memory Test (WMT) provides
scores for immediate and delayed recall and consistency of responses, which is the
percent agreement between delayed recall sections of the test. The remaining compos-
ite scores include multiple choice, paired associates, and free recall. A score P 85%
for each composite score is needed to be considered a valid effort (Green, 2003). The
WMT has an extensive research background and is routinely used in clinical practice
for detecting suboptimal effort (Green, 2005).
The ImPACT (version 6.7.723, ImPACT applications, Pittsburgh, PA) assesses
attention, memory, reaction time, and information processing speed (M. Lovell,
2007b). The ImPACT is commonly used at the seconda ry, post-secondary, collegiate,
and professional levels of sport (M. Collins, Lovell, Iverson, Ide, & Maroon, 2006;
M. W. Collins et al., 1999; M. R. Lovell & Collins, 1998; M. R. Lovell et al., 2003;
Pellman, Lovell, Viano, & Casson, 2006). The ImPACT consists of eight tests
including immediate and delayed word recall, immediate and delayed design recall, a
symbol match test, a three letter recall, the Xs and Os test, and the color match test.
Combinations of two or more of the aforementioned tests are used to calculate ve
composite scores including Visual and Verbal Memory, Reaction Time, Visual Motor
Speed, and Impulse Control.
Valid performance on the ImPACT is determined by several indicators which are
automated during the baseline (form 1) assessment. The validity indicators include
Impulse Control which is the total number of incorrect responses for the Xs and Os
interference task and the number of commissions during the Color Match Test
(M. Lovell, 2007a, 2007b). Impulse Control scores > 20 on a baseline test indicate an
invalid effort. Additional validity criteria include scores > 30 for the number of Xs
and Os incorrect, < 69% correct for word memory, < 50% correct for design
memory, and correctly recalling < 8 letters for the Three Letters test (M. Lovell,
2007a). In addition to reviewing each participants performance based on the
ImPACTs automated criteria, the same criteria from each participant were reviewed
manually to ensure a valid baseline test. The invalidity criteria for ImPACT were cou-
pled with Greens WMT and exclusion criteria (<85% for the immediate and delayed
ALTERNATE FORMS OF ImPACT 3
Downloaded by [Jacob Resch] at 06:34 09 October 2013
recall, consistency of responses, multiple choice, and paired associate composite
scores) found on the health questionnaire to determine the validity of each participants
data and subsequent inclusion for data analysis at each time point. Greens WMT was
used during the second and third time points to ensure a valid effort.
Testing sessions
Time point 1. Recruited participants reported to the Athletic Training Research
Laboratory. Each participant was provided a separate work-station/ofce in order to
avoid any environmental distractions and no more than four participa nts were tested at
each session. All participants reviewed and signed an Institutional Research Board con-
sent form prior to beginning the study. Consented participants then completed a healt h
questionnaire which was reviewed to determine inclusion/exclusion from the study. Par-
ticipants then completed the initial portion of Greens WMT followed by the completion
of the initial demographic, concussion history, and symptom portions of the ImPACT
test battery. Participants then completed the form 1 (baseline) of ImPACT. Following the
completion of ImPACT form 1, participants were asked to remain, seated quietly, until
the remainder of the 30-minute delay imposed by Greens WMT expired. To complete
the baseline session, participa nts completed the remainder of Greens WMT.
Time points 2 and 3. Participants returned to the Athletic Training Research
Laboratory on time point 2 (46.9 ± 3.4 days) and time point 3 (54 ± 2.5 days) at the
same time of day as time point 1. Participants completed the health questionnaire at
each session in order to determine continued eligibility for study inclusion. After
determining eligibility, participants completed the initial portion of Greens WMT.
Dependent on random group assignment, participants completed the assigned alternate
form of ImPACT, followed by the remainder of Greens WMT.
Statistical procedures
For this study, form equivalence was determined by using inferential condence
intervals (ICIs). ICIs utilize a reduction factor to ensure statistically different values
also possess non-overlapping condence intervals (Tryon, 2001). The ICIs for this
study employ the revised version of Tryons formula which takes into account varying
sample sizes leading to different critical t values utilized to determine statistical
equivalence (Tryon & Lewis, 2008). The formulas utilized for this study are as follows
for two hypothetical groups:
Group 1:
Lower ICI:
Y
1
Et
v1
a=2
S
Y1
Upper ICI:
Y
1
þ Et
v1
a=2
S
Y1
Group 2:
Lower ICI:
Y
2
Et
v2
a=2
S
Y2
Upper ICI:
Y
2
þ Et
v2
a=2
S
Y2
where
Y is equal to the mean test score, E represents the reduction factor, t
v1
α/2
repre-
sents the t value for the respective degrees of freedom (n 1), and S
Y1
the standard
error of the mean. E and S
Y1
are calculated as follows:
4 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
E ¼
t
v12
a=2
S
Y2
Y1
t
v1
a=2
S
Y1
þ t
v2
a=2
S
Y2
S
Y 1
¼
S
ffiffi
n
p
where t
v12
α/2
is the t value determined by summing the sample size for group 1 and
group 2 with degrees of freedom dened by (n 1), and S
Y2
Y1
which represents esti-
mated standard error of difference between sample means for two independent samples
which is =
ffiffi
ð
p
S
2
Y1
þ S
2
Y2
Þ (Tryon & Lewis, 2008).
When determining equivalence, the difference between the lower limit of the ICI
for the group with the higher test mean and the upper limit of the ICI for the group
possessing the lower test mean must be less than or equal to zero. If the difference is
greater than zero this means the ICIs do not overlap, suggesting non-equivalence.
For analysis of equivalence, alternate forms of ImPACT were compared only if
the tests preceding the form were the same. For example, for the comparison of forms
1, 2, and 3 all six groups (n = 108) were administered form 1 at time point 1. Only
groups 1, 2, and 3 (n = 54) were administered form 2 at day 45 and only group 1
(n = 18) was administered form 3 at day 50. Thus, form 1 (n = 1 08) was compared to
form 2 (n = 54). Form 2 (n = 54) was compared to form 3 (n = 18). All form
comparisons were performed employing this grouping methodology. To further analyze
equivalence between forms and demonstrate the advantages of using ICIs, Pearson
correlation coefcients and independent t tests were employed. As previously
discussed, these statistical tests may suggest equivalence (non-signicant differences or
high correlation values) of alternate forms while ICIs may suggest differently. Our
results include all three statistics to further illustrate this point.
Repeated measures analysis of variance (ANOVA) was utilized to assess test
differences across time for the WMT. Greenhouse-Geisser corrections were imple-
mented when sphericity violations occurred. A Bonferroni adjustment was made for
multiple pair-wise comparisons used during post-hoc analysis. All data analyses were
performed utilizing SPSS version 18.0 (SPSS, Chicago, IL) and statistical equivalence
was set at α < .05 (Hinton-Bayre & Geffen, 2005).
RESULTS
Of the original sample, 44 participants did not meet inclusion criteria or did not
complete all testing sessions. The nal sample consisted of 108 participants composed
of 33 male (31%) and 75 female (69%) healthy, college-aged students who met inclu-
sion criteria (age: 20.6 ± 1.5 years; height: 171.6 ± 9.7 cm; weight: 66.9 ± 11.9 kg;
and self-reported SAT scores were 1124.5 ± 117.66. Descriptive statistics for each of
the six groups can be found in Table 1. No statistical differences existed between
groups in terms of height, weight, and/or SAT scores. All participants scored above
85% on the WMT subscores suggesting a valid effort was provided at all time points.
Sample size, ImPACT means, and standard deviations for each group at each
time point are presented in Table 2. The results of our ICI analyses are depicted in
ALTERNATE FORMS OF ImPACT 5
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Figure 1. Table 3 depicts which neurocognitive indices were equivalent (+) and
non-equivalent () to one another. Of a 74 comparisons, approximately 19% were
observed to be non-equivalent. Form 1 accounted for the majority (36%) of
non-equivalent comparisons. Specically, Form 1 Verbal Memory, Visual Memory, and
Visual Motor Speed were found to be non-equivalent to the same neurocognitive index
on one or more alternate forms.
Weak correlation coefcients (r = .304 to .263) were observed for the ImPACT
composite scores deemed non-equivalent by the ICI methods. A summary of
correlation coefcients between each alternate form and their composite scores may be
found in Table 4.
The results of our independent t-tests revealed signicant differences between
form 1 and 2, t
(160)
= 2.445, p = .016, and between forms 1 and 3, t
(142)
= 3.421,
p = .001, on the Verbal Memory Composite. The remaining analyses revealed in no
additional signicant differences. The conicting results between the indepe ndent t-test
and the comparisons ICIs are due to the more stringent nature of the ICI method.
Tryon (2001) suggests the ICI method is approximately 41% more stringent than the
standard t-test. Results for these analyses are described in Table 5.
DISCUSSION
The purpose of the current study was to investigate the equivalence of the
ImPACTs alternate forms. Several of the ImPACTs alternate form composite scores
(Verbal and Visual Memory, Visual Motor Speed, and Reaction Time Composite
scores) were observed to be non-equivalent in a healthy, college-aged sample who
provided good effort as evidenced by validity criteria set by the ImPACTs manufac-
turer and the WMT (Green 2005; M. Lovell, 2007b; M. R. Lovell & Colli ns, 1998).
The observed non-equivalence of some composites on some alternate forms may
partially explain the variability of reliability coefcients reported in earlier research
(Broglio, Ferrara et al., 2007; Elbin et al., 2011; Resch et al., 2013; Schatz, 2009)
Table 1. Means and standard deviations (SD) for study participant demographics
Group n
Age
Years of Education SAT(years)
1 18 20.2 13.4 1183.1
(1.54) (1.14) (125.84)
2 18 20.9 13.9 1242.5
(1.68) (1.63) (140.40)
3 18 20.5 13.8 1209.9
(1.50) (1.22) (69.76)
4 18 21.00 14.1 1223.1
(1.28) (1.16) (130.47)
5 18 21.1 14.1 1207.9
(1.30) (24.40) (117.75)
6 18 20.3 13.6 1262.1
(1.74) (1.5) (117)
Total 108 20.6 13.8 1224.5
(1.52) (1.30) (117.66)
6 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Table 2. Means and standard deviations (SD) for ImPACT composite scores at each time point for
comparisons exhibiting non-equivalence between forms
Form n Baseline Time Point 2 Time Point 3
Form 1 108
Verbal Memory 90.7 (7.3)
Visual Memory 81.0 (11.2)
Visual Motor Speed 42.09(6.26)
Reaction Time (ms) .54(.06)
Form 2 54
Verbal Memory 93.7 (7.2)
Visual Memory 81.7 (10.0)
Visual Motor Speed 42.34(5.22)
Reaction Time (ms) .53(.06)
Form 3 36
Verbal Memory 95.2(5.0)
Visual Memory 84.2(9.8)
Visual Motor Speed 44.11(5.64)
Reaction Time (ms) .54(.06)
Form 4 18
(Form 2 vs. Form 4)
Verbal Memory 90.5 (10.7)
Visual Memory 81.4 (8.9)
Visual Motor Speed 43.05(6.59)
Reaction Time (ms) .51(.04)
Form 4 18
(Form 3 vs. Form 4)
Verbal Memory 95.7 (4.3)
Visual Memory 84.5(8.2)
Visual Motor Speed 43.82(5.62)
Reaction Time (ms) .58(.06)
Form 5 18
(Form 2 vs. Form 5)
Verbal Memory 94.9 (5.7)
Visual Memory 81.8 (9.4)
Visual Motor Speed 42.99 (7.06)
Reaction Time (ms) .54 (.08)
Form 5
(Form 3 vs. Form 5) 18
Verbal Memory 94.2 (5.4)
Visual Memory 85.9 (11.1)
Visual Motor Speed 46.78 (5.41)
Reaction Time (ms) .52 (.06)
Form 5
(Form 4 vs. Form 5) 18
Verbal Memory 92.9 (8.4) 94.3 (6.7)
Visual Memory 78.6 (12.5) 80.5 (15.8)
Visual Motor Speed 42.78 (6.55) 43.22 (7.35)
Reaction Time (ms) .55 (.05) .57 (.08)
ALTERNATE FORMS OF ImPACT 7
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Additionally, our results provi de information for clinicians who must determine if
changes between baseline and post-injury assessments are the result of the sport
concussion or psychometric characteristics of the ImPACTs alternate forms.
Regardless of study design, the weakest reliability coefcients have been
routinely observed for the ImPACTs Verbal (.37 to .62) and Visual (.26 to .70)
Memory Composite scores (Broglio, Ferrara et al., 2007; Elbin et al., 2011; Resch
Figure 1.
8 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
et al., 2013; Schatz, 2009). Using ICIs we observed non-equivalence between form 1
and forms 2, 3, and 4 for Verbal Memory Composite scores. We also observed
non-equivalence for the Visual Memory Composite score between forms 1 and 3.
The methodology employed by Broglio and Resch incorporated Forms 1, 2, and 3
which were delivered over a 50-day period. When reviewing a correlation matrix
specically for these alternate forms (Table 6), correlations coefcients ranged from
.07 to .22 for Composite Verbal Memory, .15 to .42 for Composite Visual Memory,
Table 3. ImPACT alternate form composite scores which were observed to be equivalent (+) and
non-equivalent () to one another
Composite Score Form 1 Form 2 Form 3 Form 4 Form 5
Form 1 Verbal Memory + –––NA
Visual Memory + + +NA
VMS + + +NA
Reaction Time (ms) + + + + NA
Form 2 Verbal Memory ++ +
Visual Memory + ++++
VMS +++++
Reaction Time (ms) + ++++
Form 3 Verbal Memory ++++
Visual Memory ++++
VMS ++++
Reaction Time (ms) + + + +
Form 4 Verbal Memory ––+++
Visual Memory + ++++
VMS +++++
Reaction Time (ms) + + ++
Form 5 Verbal Memory NA ++++
Visual Memory NA ++++
VMS NA++++
Reaction Time (ms) NA ++++
Table 4. Correlation coefcient ranges for ImPACT composite scores for each form (Forms used for
comparison)
Form Verbal Memory Visual Memory Visual Motor Speed Reaction Time
Form 1 .03 .40 .11 .45 .01 .74 .00 .43
Form 2 .03 .31 .04 .52 .03 .74 .05 .44
Form 3 .03 .66 .13 .49 .02 .76 .06 .71
Form 4 (1) .03 .55 .04 .73 .03 .74 .18 .74
Form 4 (2) .08 .33 .02 .52 .04 .41 .03 .44
Form 4 (3) .03 .66 .02 .49 .08 .76 .02 .71
Form 5 (2) .06 .33 .05 .51 .03 .48 .00 .44
Form 5 (3) .03 .31 .05 .46 .01 .41 .01 .32
Form 5 (4) .03 .55 .02 .73 .02 .74 .03 .74
ALTERNATE FORMS OF ImPACT 9
Downloaded by [Jacob Resch] at 06:34 09 October 2013
.08 to .75 for Visual Motor Speed, and .08 to .43 for Reaction Time. These weak to
moderate correlation coef cients may further explain the variable reliability results
reported by Broglio and Resch (Broglio, Ferrara et al., 2007; Resch et al., 2013).
Non-equivalence of the ImPACTs Verbal and Visual Memory Composites may
be due to word and designs lists that vary in difculty/complexity. For example, one
component of the ImPACTs Verbal Memory Composite score is immedi ate and
delayed word memory. Athletes completing the ImPACT are presented with a series of
12 words twice during the rst testing module. They are then asked to recall if a word
was or was not on the original list of 12 words. At the completion of the ImPACT
(approximately 15 minutes later), the individuals are asked to complete the same task
without a nother presentation of the original word set. The ImPACT administers differ-
ent words sets between a lternate forms, so non-equivalence for the Verbal Memory
Composite score may be due to varying difculty of word sets on different forms.
Word composition has been shown to impact an individuals ability to recall a
series of words over a period of time. Baddeley and Buchanan demonstrated that
participants are able to recall signicantly fewer words when monosyllabic words (can,
van, dog) were delivered compared to polysyllabic (water, dinner, supper) words,
spoken duration increased (i.e., morphine versus décor), and the number of stimuli
was increased from two to eight administered over short periods of time (Baddeley,
Thompson, & Buchanan, 1975). A review of the ImPACTs ve alternate word lists
suggests varying word complexity based on the numbe r of syllables and duration. The
ImPACT form 1 word list consists of 10 monosyllabic words and 2 polysyllable
words. The form 2 word list consi sts of seven monosyllabic words and ve two-sylla-
ble words. The words lists for forms 3, 4, and 5 consist of 11 monosyllabic words and
1 two-syllable word. This discrepancy in the number of mono- and two-syllable words
between alternate forms coupled with word list length of 12 words may partially
explain form non-equivalence. Additionally, the discrepancies in word composition
may also contribute to the weak to moderate reliability coefcients reported by Broglio
and Resch between the ImPACT alternate forms 1, 2 and 3 for Verbal Memory
Table 5. Results of alternate form comparison analyses
Comparison t (p value) Pearsons r eRG
Form 1 vs. Form 2 2.445
.263 .010
(Verbal Memory) (.016)
Form 1 vs. Form 3 3.421
.218 .030
(Verbal Memory) (.001)
Form 1 vs. Form 4 1.188 .028 .002
(Verbal Memory) (.237)
Form 1 vs. Form 3 1.543 .147 .002
(Visual Memory) (.125)
Form 1 vs. Form 3 1.719 .117 .333
(Visual Motor Speed) (.757)
Form 3 vs. Form 4 1.742 .712
.002
(Reaction Time) (.087)
Form 2 vs. Form 4 1.441 .087 .005
(Verbal memory) (.154)
Equivalence Range is denoted by (eRG). In order for two forms to be deemed non-equivalent, eRG must
satisfy x > 0.
represents signicance at p .05.
10 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Table 6. Correlation matrix to demonstrate discriminant validity
Verbal Memory Visual Memory Visual Motor Speed Reaction Time
Form 1 Form 2 Form 3 Form 1 Form 2 Form 3 Form 1 Form 2 Form 3 Form 1 Form 2 Form 3
Verbal Memory Form 1 1 .26 .22 .22 .27 .01 .07 .21 .25 .05 .05 .05
(n = 108)
Form 2 1 .07 .22 .37‡–.19 .18 .15 .03 .32 .14 .09
(n = 54)
Form 3 1 .14 .09 .33†–.08 .04 .20 .05 .05 .17
(n = 36)
Visual Memory Form 1 1 .42‡–.15 .03 .09 .10 .29‡–.07 .07
(n = 108)
Form 2 1 .36 .28 .21 .01 .28†–.09 .00
(n = 54)
Form 3 1 .18 .15 .18 .11 .03 .07
(n = 36)
VMS Form 1 1 .75‡–.12 .33‡–.35 .07
(n = 108)
Form 2 1 .08 .21 .40‡–.02
(n = 54)
Form 3 1 .32 .07 .56
(n = 36)
Reaction Time Form 1 1 .43 .25
(n = 108)
Form 2 1 .08
(n = 54)
Form 3 1
(n = 36)
In order to possess discriminant validity, composite scores should not be signicantly correlated to each other. p .01, p .001.
ALTERNATE FORMS OF ImPACT 11
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Composite (Broglio, Ferrara et al., 2007; Resch et al., 2013). Using alternate word lists
composed of words possessing the same number of syllables may result in alternate
form equivalence and increase testretest reliability.
Word frequency may also contribute to the non-equivalence associated with the
verbal memory composite score. Connine and colleagues reported the presentation of
high-frequency words in a recogniti on task resulted in signicantly decreased participant
reaction time (ms) and an increase in the number of words correctly identied (Connine,
Mullenix, Shernoff, & Yelen, 1990). Additionally when participa nts were presented with
low-frequency words that they were familiar with, they responded similarly to the high-
frequency words (Connine et al., 1990). Future research should address the inuence of
word frequency on computerized neurocognitive tests such as the ImPACT.
As with the ImPACTs immediate and delayed word memory test, the visual
memory test involves presentation of with 12 stimuli (designs) consi sting of free form
lines. Following the second showing of the same 12 designs, athletes completing the
ImPACT are asked to recall if a series of the designs share the same orientation as
those previously shown. At the conclusion of the ImPACT (approximately 20 minutes
later), athletes are once again asked to recall if the orientation of a series of presented
designs matched the orientation of the original design list. As with word memory, the
ImPACT provides an alternate design list for each alternate form.
The non-equivalence between forms 1 and 3 of the ImPACT for the Visual
Memory Composite score may be due to several factors. First, research addressing
visual short-term memory (VSTM) has shown the ability to correctly recall complex
polygons signicantly decreases after the presentation of four items (Luck & Vogel,
1997; Luria, Sessa, Gotler, Jolicoeur, & DellAcqua, 2010) An increased set of designs
(> 4 items) leads to an increased number of decisions to be made (i.e., the correct
orientation of each design) which results in decreased accuracy and an increased
number of errors during delayed design recall (Luck & Vogel, 1997). Variable design
complexity may also inuence form equivalence. Design features such as shape, color,
and/or orientation have been shown to signicantly reduce an individua ls VSTM
capacity. Polygons, like those used in the ImPACT consist of several features.
Specically, the ImPACTs designs are dened by a conjunction of shape, size, and
orientation. The ImPACTs alternate designs lists also vary by color but this feature is
consistent across shapes in a given design list. VSTM capacity decreases consequent
to design complexity (i.e. more features) (Luria et al., 2010).
Two ways to potent ially correct for form non-equivalence include decreas ing the
number of designs presented (< 4) while maintaining design complexity. The second
way to achieve form equivalence is to decrease the number of design features while
maintaining the same number of stimuli (i.e., 12). For instance, a series of 12 po lygons
consisting of the same number of straight lines (i.e., 4) and that are all the same color
would potentially allow healthy participants to increase VSTM resolution and increase
recall accuracy while still challenging those with limited VSTM capacity (i.e.,
concussed athletes).
Clinical implications
A recent study reported 93% of certied athletic trainers use computerized
neurocognitive testing use the ImPACT (Meehan et al., 2012). With this in mind, it is
12 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
important for healt h professionals, specically athletic trainers, who use the ImPACT
to understand which composite scores are equivalent across alternate forms. When
using two alternate forms of the ImPACT, Table 3 may be used to assist clinicians in
determining which composite scores were observed to be equivalent or non-equivalent.
For example, when conducting the baseline assessment, the ImPACT automatically
delivers form 1. The delivery of form 1 also provides identication of impaired effort
and the automated calculation of reliable change indices upon repeated administration.
In the event of a concussion, typically forms 2, 3, 4, and/or 5 are administered
by clinicians. If a clinician chooses to deliver form 2 following a concussion, he/she
must be aware that the Verbal Memory Composite score may be non-equivalent when
compared to form 1. In this scenario the clinician should rely more on the remaining
composite scores such as Visual Memory, Visual Motor Speed, and Reaction Time
Composite scores which were observed to be equivalent. Likewise if a clinician
chooses to administer form 3 to compare to form 1, caution is warranted when
interpreting Verbal Memory, Visual Memory, and Visual Motor Speed composite
scores due to non-equivalence. Alternatively, since form 1 possessed the majority of
non-equivalent scores when compared to the remaining ImPACT alternate forms, clini-
cians may choose to deliver forms 2, 3, 4, or 5 as a baseline assessment. The major
disadvantages to this approach are clinicians would have to manually review each
baseline assessment to ensure test validity and calculate reliable change indexes
following a concussion. These calculations would be necess ary since ImPACTs
validity criteria and reliable change index are calculated using form 1.
In terms of study limitations, unequal sample sizes were used to determine form
equivalence. Due to our study design, a varying number of participants had to com-
plete each alternate form of ImPACT. Additionally, the elaps ed time between test
administrations may have inuenced our results. Though not typical of a study
addressing alternate form equivalence, our time points are consistent with the clinical
use of the ImPACT (Broglio, Ferrara et al., 2007). Typically when determining equiva-
lence of alternate forms participants complete both forms with minimal time between
administrations (Crocker & Algina, 2008). Alternate form s, as used by the ImPACT,
are delivered over relatively long periods of time. The current study employed clini-
cally relevant time points representative of the time between baseline, post-concussion,
and asymptomatic computerized neuropsychological testing at the collegiate level
(Broglio, Ferrara et al., 2007). Lastly, our study design did not allow for the calcula-
tion of practice effects. Though it appears that group scores improved across time for
each ImPACT composite score, the varying sample sizes make it difcult to determine
if improved performance is the results of repeated exposure.
CONCLUSION
The ImPACT has been commonly accepted as a replacement for traditional
neurocognitive testing at all levels of sport. Studies addres sing psychometric properties
of the ImPACT have shown high levels of sensitivity (Schatz, Pardini, Lovell, Collins,
& Podell, 2006) but variable levels of reliability with repeat testing (Broglio, Maccioc-
chi et al., 2007; Erlanger et al., 2003; Randolph et al., 2005; Schatz, 2009). Our results
suggest that the variable reliability reported in previous literature may partially be the
ALTERNATE FORMS OF ImPACT 13
Downloaded by [Jacob Resch] at 06:34 09 October 2013
result of non-equivalent composite scores across alternate forms. Ultimately, a
clinicians primary concern when using computerized neuroco gnitive measures such as
the ImPACT is that any uctuation in an athletes performance is due to their injury
rather than error (such as non-equivalent alternate forms) associated with the test. Our
results suggest that performance uctuations following the baseline assessment may be
due to psychometric characteristics of the ImPACT. Many of the composite scores of
ImPACTs alternate forms are equivalent, but several are not. Our primary concern is
the ImPACT form 1 compared to the remaining alternate forms. In order to enhance
form equivalence, test developers may want to address the psychometric properties of
form 1. Test manufacturers can also address form equiva lence in future iterations of
their software, which wi ll inherently improve the testretest reliability coefcients and
ultimately make a more sensitive measure of sport concussion.
ACKNOWLEDGMENTS
The authors would like to thank Dr. Deborah Bandalos and Dr. Stephen Olejnik
for their statistical expertise and review throughout the development of this
manuscript. The authors would also like to thank Mikhail Bondarew, Jess Sandlin,
Teddy Sanders, and Anastasia Bobilev for their time and efforts in assisting in data
collection for this project.
REFERENCES
Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K. M., Kelly, J., Schamasch, P.
(2002). Summary and agreement statement of the 1st International Symposium on Concus-
sion in Sport, Vienna 2001. Clinical Journal of Sports Medicine, 12(1), 611.
Baddeley, A. D., Thomson, N., & Buchanan, M. (1975). Word length and structure of short-term
memory. Journal of Verbal Learning and Verbal Behavior, 14, 575589.
Brandt, J. (1991). The Hopkins Verbal Learning Test: Development of a new memory test with
six equivalent forms. The Clinicial Neuropsychologist, 5(2), 125142.
Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007). Test
retest reliability of computerized concussion assessment programs. Journal of Athletic
Training, 42(4), 509514.
Broglio, S. P., Macciocchi, S. N., & Ferrara, M. S. (2007). Sensitivity of the concussion assess-
ment battery. Neurosurgery, 60(6), 10501057; discussion 10571058.
Collins, M., Lovell, M. R., Iverson, G. L., Ide, T., & Maroon, J. (2006). Examining concussion
rates and return to play in high school football players wearing newer helmet technology:
A three-year prospective cohort study. Neurosurgery, 58(2), 275286; discussion 275286.
Collins, M. W., Grindel, S. H., Lovell, M. R., Dede, D. E., Moser, D. J., Phalin, B. R., & McK-
eag, D. B. (1999). Relationship between concussion and neuropsychological performance in
college football players. JAMA, 282(10), 964970.
Connine, C. M., Mullennix, J., Shernoff, E., & Yelen, J. (1990). Word familiarity and frequency
in visual and auditory word recognition. Journal of Experimental Psychology, Learning,
Memory, & Cognition, 16(6), 10841096.
Covassin, T., Elbin, R. J. III, Stiller-Ostrowski, J. L., & Kontos, A. P. (2009). Immediate post-
concussion assessment and cognitive testing (ImPACT) practices of sports medicine profes-
sionals. Journal of Athletic Training, 44(6), 639644.
14 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Mason, OH:
Cengage Learning.
Elbin, R. (2012, October 3). [Personal communication.].
Elbin, R. J., Schatz, P., & Covassin, T. (2011). One-year testretest reliability of the online ver-
sion of ImPACT in high school athletes. The American Journal of Sports Medicine, 39(11),
23192324.
Erlanger, D., Feldman, D., Kutner, K., Kaushik, T., Kroger, H., Festa, J., Broshek, D. (2003).
Development and validation of a web-based neuropsychological test protocol for sports-
related return-to-play decision making. Archives of Clinical Neuropsychology, 18, 293316.
Green, P. (2003). Word Memory Test for Windows: Users manual and program. Edmonton,
Canada: Greens Publishing.
Green, P. (2005). Greens Word Memory Test for Windows: Users manual and program. Edmon-
ton, Canada: Greens Publishing Inc.
Guskiewicz, K. M., Bruce, S. L., Cantu, R. C., Ferrara, M. S., Kelly, J. P., McCrea, M., Valo-
vich McLeod, T. C. (2004). National Athletic Trainers Association position statement:
Management of sport-related concussion. Journal of Athletic Training, 39(3), 280297.
Harmon, K. G., Drezner, J. A., Gammons, M., Guskiewicz, K. M., Halstead, M., Herring, S. A.,
& Roberts, W. O. (2013). American Medical Society for Sports Medicine position state-
ment: Concussion in sport. British Journal of Sports Medicine, 47(1), 1526.
Hinton-Bayre, A., & Geffen, G. (2005). Comparability, reliability, and practice effects on
alternate forms of the Digit Symbol Substitution and Symbol Digit Modalities tests.
Psychological Assessment, 17(2), 237241.
Lovell, M. (2007a). ImPACT 2007 (6.0) clinical interpretation manual. Pittsburgh, PA: ImPACT
Applications Inc.
Lovell, M. (2007b). ImPACT 2007 (6.0) software users manual. Pittsburgh, PA: ImPACT Appli-
cations Inc.
Lovell, M., Collins, M. W., & Maroon, J. C. (2009). ImPACTtest.com Retrieved 12/01/2009,
2009, from http://www.impacttest.com/.
Lovell, M. R., & Collins, M. W. (1998). Neuropsychological assessment of the college football
player. Journal of Head Trauma Rehabilitation, 13(2), 926.
Lovell, M. R., Collins, M. W., Iverson, G. L., Field, M., Maroon, J. C., Cantu, R., Fu, F. H.
(2003). Recovery from mild concussion in high school athletes. Journal of Neurosurgery,
98, 296301.
Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and
conjunctions. Nature, 390(6657), 279281.
Luria, R., Sessa, P., Gotler, A., Jolicoeur, P., & Dell
Acqua, R. (2010). Visual short-term mem-
ory capacity for simple and complex objects. Journal of Cognitive Neuroscience, 22(3),
496512.
McCrory, P., Johnston, K., Meeuwisse, W., Aubry, M., Cantu, R., Dvorak, J., Schamasch, P.
(2005). Summary and agreement statement of the 2nd International Conference on Concus-
sion in Sport, Prague 2004. British Journal of Sports Medicine, 39(4), 196204.
McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M., Molloy, M., & Cantu, R.
(2009). Consensus statement on concussion in sport: The 3rd International Conference on
Concussion in Sport, held in Zurich, November 2008. British Journal of Sports Medicine,
43, i7690.
Meehan, W. P.III, dHemecourt, P., Collins, C. L., Taylor, A. M., & Comstock, R. D. (2012).
Computerized neurocognitive testing for the management of sport-related concussions. Pedi-
atrics, 129(1), 3844. doi: 10.1542/peds.2011-1972
ONeil-Pirozzi, T. M., Goldstein, R., Strangman, G. E., & Glenn, M. B. (2012). Testre-test reli-
ability of the Hopkins Verbal Learning Test-Revised in individuals with traumatic brain
injury. Brain injury, 26(12), 14251430.
ALTERNATE FORMS OF ImPACT 15
Downloaded by [Jacob Resch] at 06:34 09 October 2013
Pellman, E. J., Lovell, M. R., Viano, D. C., & Casson, I. R. (2006). Concussion in professional
football: Recovery of NFL and high school athletes assessed by computerized neuropsycho-
logical testing Part 12. Neurosurgery, 58(2), 263274; discussion 263274.
Randolph, C., McCrea, M., & Barr, W. B. (2005). Is neuropsychological testing useful in the
management of sport-related concussion? Journal of Athletic Training, 40(3), 139152.
Resch, J., Driscoll, A., McCaffrey, N., Brown, C. N., Macchiocchi, S., Baumgartner, T. A.,
Ferrara, M. S. (2013). ImPACT test retest reliability: Reliably unreliable? Journal of
Athletic Training, 48(4), 506511.
Schatz, P. (2009). Long-term testretest reliability of baseline cognitive assessments using
ImPACT. American Journal of Sports Medicine, 38(1), 4753.
Schatz, P. (2012, October 3). [Personal communication.].
Schatz, P., Pardini, J. E., Lovell, M. R., Collins, M. W., & Podell, K. (2006). Sensitivity and
specicity of the ImPACT Test Battery for concussion in athletes. Archives of Clinical Neu-
ropsychology, 21(1), 9199.
Tryon, W. (2001). Evaluating statistical difference, equivalence, and indeterminancy using infer-
ential condence intervals: An integrated alternative method of conducting null hypothesis
statistical tests. Psychological Methods, 6(4), 371386.
Tryon, W., & Lewis, C. (2008). An inferential condence interval method of establishing statisti-
cal equivalence that corrects Tryons (2001) reduction factor. Psychological Methods, 13(3),
272277.
16 JACOB E. RESCH ET AL.
Downloaded by [Jacob Resch] at 06:34 09 October 2013
... The sum of the rated symptoms was calculated as the Total Symptom Score (TSS) which ranged from 0 to 132 with higher scores indicative of worse symptom burden. ImPACT provides five alternate forms (form 1 [baseline] and forms 2-5 [post-injury forms [1][2][3][4]) that were created to reduce known practice effects (23). ImPACT has been reported to have variable test-retest reliability with intraclass correlation coefficient [ICC] values ranging from 0.19 to 0.88) and variable sensitivity (67.8%-97.5%), ...
... Though repeated exposure to ImPACT may have contributed to this finding, approximately eight to 13 months elapsed between time points reducing the influence of practice effects (22,31). An alternate form of ImPACT was administered during the PI assessment which likely reduced the influence of repeated exposure to the computerized neurocognitive assessment (23). A separate study of collegiate athletes reported improvements in psychomotor speed, cognitive flexibility, processing speed, executive functioning, reasoning and balance, as compared to baseline, approximately five and half months following a post-injury assessment after reporting asymptomatic (32). ...
Article
Primary objective: The objective of this study was to examine neurocognition, postural control, and symptomology at multiple timepoints following concussion. We hypothesized that collegiate athletes would perform similar to or better than their baseline in terms of each outcome at both timepoints. Research design: This was a retrospective study of 71 collegiate athletes (18.3 ± 0.89 years old; 182.2 ± 10.05 cm; 84.2 ± 20.07 kg) to observe changes in outcomes from a previously established clinical protocol. Methods and procedures: Participants were administered ImPACT™, the Sensory Organization Test (SOT), and the revised head injury scale (HIS-r) prior to their seasons (baseline); upon reporting symptom-free following concussion (post-injury); and approximately 8-months after return-to-play to establish a new baseline. Main outcomes and results: There were no changes in ImPACT scores or HIS-r reporting over time. ImPACT total symptom score (TSS) decreased over time (p = .002, ηp2 = 0.08). Significant main effects occurred for the SOT equilibrium score (p < .01, ηp2 = 0.34) and Vestibular sensory ratio (p < .001, ηp2 = 0.22). Conclusions: Our data suggest no decline in neurocognition, balance, or symptom burden approximately eight months post-injury. As clinicians continue to explore "best practices" for concussion management and potential long-term implications of these injuries it is important to monitor outcome measures longitudinally.
... With such compromised test-retest reliability, it is not surprising that ImPACT also struggles with respect to validity. For example, ImPACT has high misclassification rates, misclassifying 25% to almost 50% of healthy individuals as concussed (Resch et al. 2013), fails to predict scores on well-validated instruments assessing the same constructs (Schatz and Putz 2006), and scores do not relate to concussion history (Broglio, Ferrara, Piland, and Anderson 2006). In addition, ImPACT only assesses neurocognitive functioning, and in so doing, explicitly ignores (neurobehavior) one of the two areas identified in the best practice guidelines as common sequelae of concussion. ...
Article
Machine learning (ML) techniques can help harness insights from data that complement and extend those that can be attained by traditional statistical methods. The current article introduces clinicians to concepts underlying ML and explores how it can be applied within the domain of neuropsychology. Specifically, we illustrate an application of ML to a dataset that includes a battery of standardized measures designed to provide diagnostic support for concussions, including standardized neurocognitive (CPT 3) and neurobehavioral (BESS, NIH 4 meter gait) measures, gait sensor data, and a CDC concussion symptom checklist. These variables were used to predict the decision-making of a pediatric neurologist evaluating a group of child/adolescent patients. With a sample of 111 cases, ML (using a general linear model and deep learning as illustrations) achieved accuracies of 91% and 84.8% and AUCs of 1.0 and .947, respectively, when predicting the neurologist’s binomial decision-making (safe/remove). In presenting the data and various considerations for interpretation, we attempt to balance both the promise and perils of ML.
... With such compromised test-retest reliability, it is not surprising that ImPACT also struggles with respect to validity. For example, ImPACT has high misclassification rates, misclassifying 25% to almost 50% of healthy individuals as concussed (Resch et al. 2013), fails to predict scores on well-validated instruments assessing the same constructs (Schatz and Putz 2006), and scores do not relate to concussion history (Broglio, Ferrara, Piland, and Anderson 2006). In addition, ImPACT only assesses neurocognitive functioning, and in so doing, explicitly ignores (neurobehavior) one of the two areas identified in the best practice guidelines as common sequelae of concussion. ...
Article
Objective Sports related concussions, or mild traumatic brain injuries, have been steadily increasing over the past two decades. Effective screening and identification of concussions play a critical role in the diagnosis and rehabilitation process. Although side line assessment tools are available, there are few well validated tests available to assist medical providers in this decision-making process. This study aims to determine whether previously validated tools which assess neurocognitive and neurophysiological abilities can predict concussion symptom endorsement in a sample of child and adolescent athletes. Background Participants were recruited from two settings: The office of a pediatric neurologist (seen 3 to 109 days post incident) and from preseason baseline assessments. Design/Methods Method: Participants were 113 individuals, aged 6 to 17, representing 84 consecutive cases of individuals completing standardized baseline assessments with no recent history of concussion, and 29 consecutive cases undergoing a post-concussion evaluation by a pediatric neurologist. Participants completed a standardized battery of tests comprised of the Connors’ Continuous Performance Test (CPT-3), the Balance Error Scoring System (BESS) and the NIH 4-meter Gait Test and completed a checklist of CDC concussion symptoms. Results The screening battery explained 33% of the variance (d = 1.4) in concussion symptom endorsement, after controlling for age. The neurocognitive test alone (CPT-3) accounts for 21.5% of the variance (d = 1.05) in symptoms after controlling for age, and the neurobehavioral measures (BESS and NIH 4m Gait) then account for an additional 11.5% variance (they account for 18.6% variance, d = 0.96, when entered first). These effect sizes are considered large to very large and reflect a marked increase in predictive validity relative to existing measures used in concussion assessments. Conclusions An easy to administer and relatively brief screening test can be used in medical settings to identify concussions and predict significant and substantial variability in CDC concussion symptoms.
... With such compromised test-retest reliability, it is not surprising that ImPACT also struggles with respect to validity. For example, ImPACT has high misclassification rates, misclassifying 25% to almost 50% of healthy individuals as concussed (Resch et al. 2013), fails to predict scores on well-validated instruments assessing the same constructs (Schatz and Putz 2006), and scores do not relate to concussion history (Broglio, Ferrara, Piland, and Anderson 2006). In addition, ImPACT only assesses neurocognitive functioning, and in so doing, explicitly ignores (neurobehavior) one of the two areas identified in the best practice guidelines as common sequelae of concussion. ...
Article
Objective: Effective screening for concussion is increasingly important, and medical professionals play a critical role in diagnostic and return-to-play decisions. However, few well-validated measures are available to assist in those decisions. This study aims to determine whether previously validated measures assessing neurocognitive and neurobehavioral abilities can predict Centers for Disease Control (CDC) concussion symptom endorsement in a sample of child or youth athletes. Method: Participants were 113 individuals, aged 6-17, representing 29 consecutive cases undergoing a post-concussion evaluation by a pediatric neurologist and 84 consecutive cases completing standardized baseline assessments (i.e., not being evaluated as a follow-up to a concussion). All participants completed the same standardized battery of tests comprised of the Connors' Continuous Performance Test (CPT 3), the Balance Error Scoring System (BESS), and the NIH 4-Meter Gait Test as well as completing a checklist of CDC concussion symptoms. Results: Regression analyses indicate that the screening battery explained 33% of the variance (d = 1.4) in concussion symptom endorsement, after controlling for age. The neurocognitive test alone (CPT 3) accounts for 21.5% of the variance (d = 1.05) in symptoms after controlling for age, and the neurobehavioral measures (BESS and NIH 4-Meter Gait) then account for an additional 11.5% variance (accounting for 18.6% variance, d = .96, when entered first). These effect sizes are considered large to very large and reflect a marked increase in predictive validity relative to existing measures commonly used in concussion assessments. Conclusions: A relatively brief screening battery can function in medical settings to predict significant and substantial variability in CDC concussion symptoms in a pediatric sample.
... Identical baseline forms were used for both testing sessions in order to examine the test-retest reliability of each ImPACT scale using the same form. In addition, prior research has demonstrated that not all ImPACT forms are psychometrically equivalent (Resch, Macciocchi, & Ferrara, 2013). Although it is the case that scores from the baseline form are compared to post-injury (alternate) forms when ImPACT is used clinically, it is important to examine the test-retest reliability of the same form for psychometric purposes, much like what is done with other clinical tests, such as the Wechsler Intelligence Scales. ...
Article
Full-text available
The present study examined the short-term test-retest reliability of the Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) variables with healthy 11- to 14-year-old athletes. 53 young athletes (Mage = 12.4 years, 9 female) were administered the ImPACT on two separate occasions two weeks apart. Participants were instructed to complete the Post-Concussion Symptom Scale (PCSS) and the baseline computerized neurocognitive test during both the baseline and retest phases. Intraclass correlation (ICC), standard error of measurement (SEM), and reliable change index (RCI) were used as reliability metrics. PCSS Total Symptoms and Visual-Motor Speed were the only scores to reach clinical reliability standards (i.e., R > 0.7). None of the scores exceeded RCI cut-offs. Results indicate that the composite scores of the ImPACT are differentially reliable in a preadolescent sample across a two-week retest period, with only motor processing speed and self-reported symptoms exceeding clinical reliability standards. The findings support the view that neurocognitive testing should not be the sole determining factor in concussion assessment. This study highlights the importance of continuing research with younger athletes to assess the reliability of neurocognitive measures in concussion management programs. Future research should focus on a larger, heterogeneous sample, including children with learning disabilities and ADHD.
Article
Recent research has yielded multivariate base rates (MBRs) of low scores in healthy populations using a widely adopted concussion screening measure, Immediate Postconcussion and Cognitive Testing (ImPACT). However, the extent to which individuals with concussion obtain reliable changes at divergent frequencies relative to healthy individuals is largely unknown. The present study examined whether MBRs of reliable change accurately discriminated between those with and without concussion. This archival review consisted of 129 healthy individuals and 81 individuals with concussion. MBRs of reliable change scores were examined at varying cutoffs and frequencies between those with and without concussion. Composites showed small to medium effect sizes in differentiating between those with and without concussion. MBRs of reliable change scores on ImPACT provided limited discriminative utility in isolation. Computations of posttest probabilities using Bayes’ Theorem yielded evidence for incremental gains when utilizing MBRs of reliable change under certain constraints.
Article
Full-text available
Introduction Sport-related concussion management remains a diagnostic dilemma to clinicians in all strata of care, coaching staff and players alike. The lack of objective diagnostic and prognostic biomarkers and over-reliance on subjective clinical assessments carries a significant health risk of undiagnosed concussive episodes and early return to play before full recovery increasing the risk of sustaining additional concussion, and leading to long-term sequelae and/or unfavourable outcome. Objective To identify a set of parameters (neuroimaging with neurophysiological, biological and neuropsychological tests) that may support pitch-side and outpatient clinical decision-making in order to objectively diagnose concussion, determine the severity of injury, guide a safe return to play and identify the potential predictors of the long-term sequelae of concussion. Methods and analysis An exploratory, observational, prospective, cohort study recruiting between 2017 and 2020. The participants will have a baseline preseason screening (brain imaging, neuropsychological assessments, serum, urine and saliva sampling). If a screened player later suffers a concussion and/or multiple concussions then he/she will be assessed again with the same protocol within 72 hours, and their baseline data will be used as internal control as well as normative data. Inferential statistical analysis will be performed to determine correlations between biological, imaging techniques and neuropsychological assessments. Ethics and dissemination This study was approved by the East of England—Essex Research Ethics Committee on 22 September 2017—REC 17/EE/0275; IRAS 216703. The results of this study will be presented at national and international conferences and submitted for publication in peer reviewed journals. Trial registration number ISRCTN16974791 ; Pre-results.
Article
Objective: The purpose of this review was to examine sex differences in concussion, or mild traumatic brain injury (mTBI) outcome, updating previous critical reviews of the literature. Method: Within adult human studies, we reviewed a wide range of concussion outcome variables: prevalence of concussion, injury characteristics, postconcussion symptom trajectories and psychiatric distress, neuropsychological performance, and neuroimaging findings. Sports-related concussion, civilian, and military samples were included in the review. Results: Given the robust concussion literature, there is a relative paucity of research addressing sex differences following concussion. The majority of available studies focused on sports-related concussion, with fewer studies targeting other civilian causes of concussion or military-related concussion in females. Prevalence of concussion was generally reported to be higher in females than males. Although symptom reporting largely showed a pattern for females to report greater overall symptoms than males, examining individual symptoms or symptom clusters resulted in mixed findings between the sexes. Neuropsychological studies generally showed females performing more poorly than males on measures of visual memory following concussion, though this finding was not consistently reported. Conclusion: Research examining sex differences in humans following concussion, in general, is in its infancy, and exploration of sex differences in studies outside of the sports concussion domain is particularly nascent. Given the increased prevalence of concussion and potential higher symptom reporting among women, ongoing research is necessary to better understand the role of biological sex on outcome following concussion. Understanding sex differences has important implications for assessment, management, and treatment of concussion.
Article
Context: Meta-analyses examining construct-specific cognitive impairment concurrently with self-reported symptoms postconcussion are sparse. Objective: To review the literature on the effects of concussion on construct-specific neurocognitive declines and to compare them with self-reported symptoms before 1 week and between 1 and 3 weeks postconcussion. Data sources: Relevant studies in PubMed, CINAHL, and PsycINFO published from January 1, 1999 through November 30, 2015. Study selection: Studies were included if participants completed the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) before and after concussion and if test performance and Postconcussion Symptom Scale (PCSS) scores were reported at both times. Data extraction: After reviewing the full texts, we extracted data from 17 studies consisting of 29 independent samples; therefore, this meta-analysis consisted of 1777 unique participants. Data synthesis: The Hedges g effect size (ES) was estimated. A random-effects or fixed-effects model was used based on heterogeneity findings. When heterogeneity was present, we used meta-regression to assess unexplained between-studies variance. Within the first week of injury, the ESs were small to moderate for cognitive declines, ranging from -0.43 (95% confidence interval [CI] = -0.52, -0.35) to -0.67 (95% CI = -0.77, -0.58), and large for the PCSS score (Hedges g = -0.81; 95% CI = -0.91, -0.71). After 1 week, the ESs for cognitive declines (Hedges g range = -0.25 [95% CI = -0.35, -0.15] to -0.37 [95% CI = -0.55, -0.19]) and PCSS score (Hedges g = -0.38; 95% CI = -0.53, -0.22) were also small. Within 2 weeks of injury, PCSS score and time since injury weakly moderated the cognitive ES. Conclusions: When a neurocognitive test was administered within 1 week of injury, the ES was larger for self-reported symptoms than for ImPACT scores generated at the same session. After 1 week of injury, the ESs for ImPACT and PCSS scores were comparable. If the athlete reports symptoms within 1 week of injury, administering a cognitive test does not appear to offer additional information to the clinician. However, if the athlete does not report symptoms postconcussion, cognitive testing may inform the clinical management of the injury.
Article
Background: Clinicians typically rely on neuropsychological and balance tests to track concussion recovery. The aforementioned tests imply impairments that are based on performance, but these tests do not directly measure brain physiology throughout concussion recovery. Because of these issues, an objective biomarker that can index severity and the recovery timeline is needed. Moreover, the number of concussions occurring at a recreational level requires the biomarker to be easily administered in a cost effective manner, and the results easily interpreted. Methods: To address these issues, near-infrared spectroscopy was used to assess the relative changes in oxy (HbO2 )- and deoxyhaemoglobin and the associated standard deviations (SD) in the prefrontal cortex. Resting haemoglobin, and haemoglobin changes in response to hypercapnia (five repeated 20s breath holds), was measured in all participants. Data were aggregated into healthy baselines (n = 115), and concussed participants on days 1-3 (n = 14), 4-6 (n = 8), and 7-14 (n = 11). The data were statistically compared using a 1 × 4 ANOVA. Results: Resting HbO2 values progressively lowered from days 1-3 to 7-14 (with no differences compared to controls). The second major finding showed that hypercapnic HbO2 SD was significantly lower than resting values on days 1-3 and 4-6, but reversed back towards the healthy control group on day 7-14. Conclusion: Monitoring cerebral oxygenation changes is a viable biomarker to assess the physiological state of the brain following concussion.
Article
Full-text available
A new test of verbal learning and memory, the Hopkins Verbal Learning Test, was developed. The test consists of three trials of free-recall of a 12-item, semantically categorized list, followed by yes/no recognition. Six parallel forms yielded equivalent results in normals. The performance of patients with Alzheimer's disease and chronic amnesia is described. The test is likely to be useful in patients too impaired for more comprehensive memory assessments and where repeated testing is necessary.
Article
Full-text available
Context: Computerized neuropsychological testing is commonly used in the assessment and management of sport-related concussion. Even though computerized testing is widespread, psychometric evidence for test-retest reliability is somewhat limited. Additional evidence for test-retest reliability is needed to optimize clinical decision making after concussion. Objective: To document test-retest reliability for a commercially available computerized neuropsychological test battery (ImPACT) using 2 different clinically relevant time intervals. Design: Cross-sectional study. Setting: Two research laboratories. Patients or other participants: Group 1 (n = 46) consisted of 25 men and 21 women (age = 22.4 ± 1.89 years). Group 2 (n = 45) consisted of 17 men and 28 women (age = 20.9 ± 1.72 years). Intervention(s): Both groups completed ImPACT forms 1, 2, and 3, which were delivered sequentially either at 1-week intervals (group 1) or at baseline, day 45, and day 50 (group 2). Group 2 also completed the Green Word Memory Test (WMT) as a measure of effort. Main outcome measures: Intraclass correlation coefficients (ICCs) were calculated for the composite scores of ImPACT between time points. Repeated-measures analysis of variance was used to evaluate changes in ImPACT and WMT results over time. Results: The ICC values for group 1 ranged from 0.26 to 0.88 for the 4 ImPACT composite scores. The ICC values for group 2 ranged from 0.37 to 0.76. In group 1, ImPACT classified 37.0% and 46.0% of healthy participants as impaired at time points 2 and 3, respectively. In group 2, ImPACT classified 22.2% and 28.9% of healthy participants as impaired at time points 2 and 3, respectively. Conclusions: We found variable test-retest reliability for ImPACT metrics. Visual motor speed and reaction time demonstrated greater reliability than verbal and visual memory. Our current data support a multifaceted approach to concussion assessment using clinical examinations, symptom reports, cognitive testing, and balance assessment.
Article
Full-text available
Unlabelled: PURPOSE OF THE STATEMENT: ▸ To provide an evidence-based, best practises summary to assist physicians with the evaluation and management of sports concussion. ▸ To establish the level of evidence, knowledge gaps and areas requiring additional research. Importance of an amssm statement: ▸ Sports medicine physicians are frequently involved in the care of patients with sports concussion. ▸ Sports medicine physicians are specifically trained to provide care along the continuum of sports concussion from the acute injury to return-to-play (RTP) decisions. ▸ The care of athletes with sports concussion is ideally performed by healthcare professionals with specific training and experience in the assessment and management of concussion. Competence should be determined by training and experience, not dictated by specialty. ▸ While this statement is directed towards sports medicine physicians, it may also assist other physicians and healthcare professionals in the care of patients with sports concussion. Definition: ▸ Concussion is defined as a traumatically induced transient disturbance of brain function and involves a complex pathophysiological process. Concussion is a subset of mild traumatic brain injury (MTBI) which is generally self-limited and at the less-severe end of the brain injury spectrum. Pathophysiology: ▸ Animal and human studies support the concept of postconcussive vulnerability, showing that a second blow before the brain has recovered results in worsening metabolic changes within the cell. ▸ Experimental evidence suggests the concussed brain is less responsive to usual neural activation and when premature cognitive or physical activity occurs before complete recovery the brain may be vulnerable to prolonged dysfunction. Incidence: ▸ It is estimated that as many as 3.8 million concussions occur in the USA per year during competitive sports and recreational activities; however, as many as 50% of the concussions may go unreported. ▸ Concussions occur in all sports with the highest incidence in football, hockey, rugby, soccer and basketball. RISK FACTORS FOR SPORT-RELATED CONCUSSION: ▸ A history of concussion is associated with a higher risk of sustaining another concussion. ▸ A greater number, severity and duration of symptoms after a concussion are predictors of a prolonged recovery. ▸ In sports with similar playing rules, the reported incidence of concussion is higher in female athletes than in male athletes. ▸ Certain sports, positions and individual playing styles have a greater risk of concussion. ▸ Youth athletes may have a more prolonged recovery and are more susceptible to a concussion accompanied by a catastrophic injury. ▸ Preinjury mood disorders, learning disorders, attention-deficit disorders (ADD/ADHD) and migraine headaches complicate diagnosis and management of a concussion. Diagnosis of concussion: ▸ Concussion remains a clinical diagnosis ideally made by a healthcare provider familiar with the athlete and knowledgeable in the recognition and evaluation of concussion. ▸ Graded symptom checklists provide an objective tool for assessing a variety of symptoms related to concussions, while also tracking the severity of those symptoms over serial evaluations. ▸ Standardised assessment tools provide a helpful structure for the evaluation of concussion, although limited validation of these assessment tools is available. Sideline evaluation and management: ▸ Any athlete suspected of having a concussion should be stopped from playing and assessed by a licenced healthcare provider trained in the evaluation and management of concussions. ▸ Recognition and initial assessment of a concussion should be guided by a symptoms checklist, cognitive evaluation (including orientation, past and immediate memory, new learning and concentration), balance tests and further neurological physical examination. ▸ While standardised sideline tests are a useful framework for examination, the sensitivity, specificity, validity and reliability of these tests among different age groups, cultural groups and settings is largely undefined. Their practical usefulness with or without an individual baseline test is also largely unknown. ▸ Balance disturbance is a specific indicator of a concussion, but not very sensitive. Balance testing on the sideline may be substantially different than baseline tests because of differences in shoe/cleat-type or surface, use of ankle tape or braces, or the presence of other lower extremity injury. ▸ Imaging is reserved for athletes where intracerebral bleeding is suspected. ▸ There is no same day RTP for an athlete diagnosed with a concussion. ▸ Athletes suspected or diagnosed with a concussion should be monitored for deteriorating physical or mental status. Neuropsychological testing: ▸ Neuropsychological (NP) tests are an objective measure of brain-behaviour relationships and are more sensitive for subtle cognitive impairment than clinical exam. ▸ Most concussions can be managed appropriately without the use of NP testing. ▸ Computerised neuropsychological (CNP) testing should be interpreted by healthcare professionals trained and familiar with the type of test and the individual test limitations, including a knowledgeable assessment of the reliable change index, baseline variability and false-positive and false-negative rates. ▸ Paper and pencil NP tests can be more comprehensive, test different domains and assess for other conditions which may masquerade as or complicate assessment of concussion. ▸ NP testing should be used only as part of a comprehensive concussion management strategy and should not be used in isolation. ▸ The ideal timing, frequency and type of NP testing have not been determined. ▸ In some cases, properly administered and interpreted NP testing provides an added value to assess cognitive function and recovery in the management of sports concussions. ▸ It is unknown if use of NP testing in the management of sports concussion helps prevent recurrent concussion, catastrophic injury or long-term complications. ▸ Comprehensive NP evaluation is helpful in the post-concussion management of athletes with persistent symptoms or complicated courses. Return to class: ▸ Students will require cognitive rest and may require academic accommodations such as reduced workload and extended time for tests while recovering from a concussion. Return to play: ▸ Concussion symptoms should be resolved before returning to exercise. ▸ A RTP progression involves a gradual, step-wise increase in physical demands, sports-specific activities and the risk for contact. ▸ If symptoms occur with activity, the progression should be halted and restarted at the preceding symptom-free step. ▸ RTP after concussion should occur only with medical clearance from a licenced healthcare provider trained in the evaluation and management of concussions. SHORT-TERM RISKS OF PREMATURE RTP: ▸ The primary concern with early RTP is decreased reaction time leading to an increased risk of a repeat concussion or other injury and prolongation of symptoms. LONG-TERM EFFECTS: ▸ There is an increasing concern that head impact exposure and recurrent concussions contribute to long-term neurological sequelae. ▸ Some studies have suggested an association between prior concussions and chronic cognitive dysfunction. Large-scale epidemiological studies are needed to more clearly define risk factors and causation of any long-term neurological impairment. Disqualification from sport: ▸ There are no evidence-based guidelines for disqualifying/retiring an athlete from a sport after a concussion. Each case should be carefully deliberated and an individualised approach to determining disqualification taken. Education: ▸ Greater efforts are needed to educate involved parties, including athletes, parents, coaches, officials, school administrators and healthcare providers to improve concussion recognition, management and prevention. ▸ Physicians should be prepared to provide counselling regarding potential long-term consequences of a concussion and recurrent concussions. Prevention: ▸ Primary prevention of some injuries may be possible with modification and enforcement of the rules and fair play. ▸ Helmets, both hard (football, lacrosse and hockey) and soft (soccer, rugby) are best suited to prevent impact injuries (fracture, bleeding, laceration, etc.) but have not been shown to reduce the incidence and severity of concussions. ▸ There is no current evidence that mouth guards can reduce the severity of or prevent concussions. ▸ Secondary prevention may be possible by appropriate RTP management. Legislation: ▸ Legislative efforts provide a uniform standard for scholastic and non-scholastic sports organisations regarding concussion safety and management. Future directions: ▸ Additional research is needed to validate current assessment tools, delineate the role of NP testing and improve identification of those at risk of prolonged post-concussive symptoms or other long-term complications. ▸ Evolving technologies for the diagnosis of concussion, such as newer neuroimaging techniques or biological markers, may provide new insights into the evaluation and management of sports concussion.
Article
Null hypothesis statistical testing (NHST) has been debated extensively but always successfully defended. The technical merits of NHST are not disputed in this article. The widespread misuse of NHST has created a human factors problem that this article intends to ameliorate. This article describes an integrated, alternative inferential confidence interval approach to testing for statistical difference, equivalence, and indeterminacy that is algebraically equivalent to standard NHST procedures and therefore exacts the same evidential standard. The combined numeric and graphic tests of statistical difference, equivalence, and indeterminacy are designed to avoid common interpretive problems associated with NHST procedures. Multiple comparisons, power, sample size, test reliability, effect size, and cause-effect ratio are discussed. A section on the proper interpretation of confidence intervals is followed by a decision rule summary and caveats.
Article
The Concussion Resolution Index (CRI) is an online assessment tool designed to track resolution of symptoms following sports-related concussion. The CRI is composed of six subtests measuring reaction time, visual recognition, and speed of information processing. Three factors are derived from the subtests: Simple Reaction Time (SRT), Complex Reaction Time (CRT), and Processing Speed (PS). Multiple alternate forms within subtests afford simple, reliable, assessment of change, relative to a baseline test completed by an athlete. The test also assesses self-reported neurophysiological symptoms at the time of injury and tracks resolution of these symptoms. The data demonstrate the CRI is a valid and reliable measure of cognitive performance in a relatively heterogeneous group of athletes aged 13–35. Two methods of statistical analysis for assessing change from baseline were compared to establish a psychometric basis for return-to-play decision-making: the Reliable Change Index (RCI) and multiple regression. Multiple regression was more accurate than the RCI in determining a decline in performance relative to the baseline.
Article
Primary objective: To determine test-re-test reliability of the Hopkins Verbal Learning Test-Revised (HVLT-R) in a group of individuals with traumatic brain injury (TBI). Research design: Single-group repeated measures design. Methods and procedures: Seventy-five individuals with TBI were administered the HVLT-R twice, with 6-8 weeks between the two test sessions. Main outcomes and results: Test-re-test reliability on HVLT-R scoring parameters ranged from 0.537-0.818, with seven of the eight scoring parameters exhibiting r > 0.6. At re-test, scores did not significantly change on any of the eight HVLT-R scoring parameters. Conclusions: HVLT-R use with individuals with TBI is supported. Test-re-test reliability of total recall and delayed recall sub-scores was particularly high.
Article
A number of experiments explored the hypothesis that immediate memory span is not constant, but varies with the length of the words to be recalled. Results showed: (1) Memory span is inversely related to word length across a wide range of materials; (2) When number of syllables and number of phonemes are held constant, words of short temporal duration are better recalled than words of long duration; (3) Span could be predicted on the basis of the number of words which the subject can read in approximately 2 sec; (4) When articulation is suppressed by requiring the subject to articulate an irrelevant sound, the word length effect disappears with visual presentation, but remains when presentation is auditory. The results are interpreted in terms of a phonemically-based store of limited temporal capacity, which may function as an output buffer for speech production, and as a supplement to a more central working memory system.