Validity of ImPACT for Measuring Processing Speed Following Sports-Related Concussion

Abstract

The purpose of this study was to examine the validity of ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing), a computerized neuropsychological test battery, for measuring attention and processing speed in athletes with concussions. This was accomplished by comparing the computerized testing to a traditional neuropsychological measure, the Symbol Digit Modalities Test (SDMT). Participants were 72 amateur athletes who were seen within 21 days of sustaining a sports-related concussion (Mean = 9.4, SD = 5.4 days). As predicted, the SDMT correlated more highly with the Processing Speed and Reaction Time composites than the Verbal Memory and Visual Memory Composites from ImPACT. The composite scores from ImPACT and the SDMT were subjected to exploratory factor analysis, revealing a two-factor solution interpreted as Speed/Reaction Time and Memory. It appears as if the Processing Speed Composite, Reaction Time Composite, and SDMT are measuring a similar underlying construct in this sample of concussed amateur athletes.

Full text

Validity of ImPACT 1
Running Head: VALIDITY OF ImPACT
Validity of ImPACT for Measuring Attention & Processing Speed
Following Sports-Related Concussion
Grant L. Iverson, Ph.D.
University of British Columbia & Riverview Hospital
Mark R. Lovell, Ph.D.
University of Pittsburgh Medical Center
Michael W. Collins, Ph.D.
University of Pittsburgh Medical Center
Author Notes: The authors thank Jennifer Bernardo for assistance with manuscript
preparation. Please address correspondence to Grant Iverson, Ph.D., Department of
Psychiatry, 2255 Wesbrook Mall, University of British Columbia, Vancouver, B.C.
Canada, V6T 2A1.
Draft #4: April 29, 2004

Validity of ImPACT 2
Abstract
The purpose of this study was to examine the validity of ImPACT (Immediate Post-
Concussion Assessment and Cognitive Testing), a computerized neuropsychological test
battery, for measuring attention and processing speed in athletes with concussions. This
was accomplished by comparing the computerized testing to a traditional
neuropsychological measure, the Symbol Digit Modalities Test (SDMT). Participants
were 72 amateur athletes who were seen within 21 days of sustaining a sports-related
concussion (Mean = 9.4, SD = 5.4 days). As predicted, the SDMT correlated most highly
with the Processing Speed and Reaction Time composites from ImPACT. The composite
scores from ImPACT and the SDMT were subjected to exploratory factor analysis,
revealing a two-factor solution interpreted as Speed/Reaction Time and Memory. It
appears as if the Processing Speed Composite, Reaction Time Composite, and SDMT are
measuring a similar underlying construct in this sample of concussed amateur athletes.
Key Words: Concussion, Sports, ImPACT, Validity, Computerized Testing

Validity of ImPACT 3
Validity of ImPACT for Measuring Attention & Processing Speed
Following Sports-Related Concussion
Neuropsychological tests are sensitive to the subtleties of cognitive decrements
associated with concussions in sports (Barr & McCrea, 2001; Collins et al., 1999;
Echemendia, Putukian, Mackin, Julian, & Shoss, 2001; Kelly, 2001; Macciocchi et al.,
1996; McCrea, Kelly, Randolph, Cisler, & Berger, 2002; Randolph, 2001). The general
course begins with disruption of cognitive functioning immediately following injury,
particularly in the areas of orientation, attention and concentration, mental set shifting,
memory, information processing, and executive functioning (Collins et al., 1999;
Delaney, Lacroix, Gagne, & Antoniou, 2001; Erlanger et al., 1999; Guskiewicz, Ross, &
Marshall, 2001; Johnston et al., 2001; Matser, Kessels, Lezak, & Troost, 2001; Randolph,
2001). Gradual recovery occurs within several hours and full recovery to baseline
cognitive and functional status typically is reached within a few days, regardless of
concussion severity (Barr & McCrea, 2001; Guskiewicz, Marshall, Broglio, Cantu, &
Kirkendall, 2002; Johnston et al., 2001; Lovell et al., 2003; Macciocchi et al., 1996;
McCrea et al., 2002; McCrea et al., 2003; Powell & Barber-Foss, 1999).
Individualized approaches to concussion management have recently been
implemented by many sports organizations across the United States. Increasingly,
organizations are using baseline and post-injury neuropsychological testing. This
approach is currently being used clinically with the National Football League (Lovell,
1999), Major League Baseball (Collins, 2001), and it is mandated within the National
Hockey League (Lovell & Burke, 2000). With the availability of computerized
neuropsychological testing (e.g., Maroon, Lovell, Norwig, Podell, Powell, & Hartl, 2000;

Validity of ImPACT 4
Erlanger et al., 2001), many high schools and colleges are implementing this approach to
injury management.
Athletes with concussions show performance decrements on computerized
neuropsychological tests (Erlanger et al., 2001; Erlanger et al., 2003; Makdissi et al.,
2001; Warden et al., 2001). In essence, demonstrating the sensitivity of computerized
testing to the acute effects of concussion is the most important aspect of the test
validation process. The following are some recent examples of the sensitivity of
computerized neuropsychological testing to concussions in athletes: (a) high school
athletes with Grade I (“ding”) concussions showed a decline in memory between 1-3
days post injury followed by a return to baseline at 5-10 days post injury (Lovell, Collins,
Iverson, Johnston, & Bradley, 2004), (b) some concussed athletes showed a clear increase
in simple and choice reaction times at approximately two days post injury with
improvement at approximately 6 days post injury (Erlanger et al., 2003), (c) concussed
athletes reporting headaches at one week post injury had slower reaction times and lower
memory scores than concussed athletes who did not report headaches (Collins et al., in
press), and (d) concussed athletes reporting perceived “fogginess” at one week post
injury had slower reaction times, reduced processing speed, and lower memory scores
than concussed athletes who did not report fogginess (Iverson, Gaetz, Lovell, & Collins,
in press).
The purpose of this study was to examine the construct validity of ImPACT
(Immediate Post-Concussion Assessment and Cognitive Testing; (Maroon et al., 2000), a
computerized neuropsychological test battery, for measuring attention and processing
speed in athletes with concussions. This was accomplished by comparing the

Validity of ImPACT 5
computerized testing to a traditional neuropsychological measure, the Symbol Digit
Modalities Test (SDMT; Smith, 1982). The SDMT has been routinely used in sport
concussion research (e.g., Collins et al., 1999; Erlanger et al., 2003; Hinton-Bayre,
Geffen, McFarland, 1997; Guskiewicz, Marshall, Broglio, Cantu, & Kirkendall, 2002;
Macciocchi, Barth, Littlefield, & Cantu, 2001; McRea et al., 2003; Mrazik et al., 2000;
Zillmer, 2003). It is believed to measure scanning and tracking aspects of attention and
speed of processing (Spreen & Strauss, 1998). It was hypothesized that the SDMT would
be more highly related to the Processing Speed and Reaction Time composites on
ImPACT than the two memory composites.
Method
Participants & Procedures
Participants were 72 amateur athletes who were seen within 21 days of sustaining
a sports-related concussion (Mean = 9.4, Median = 9, SD = 5.4 days). The breakdown of
athletes by concussion severity, based on the American Academy of Neurology
guidelines, was as follows: Grade 1 = 33%, Grade 2 = 49%, and Grade 3 = 18%. Their
average age was 17.1 years (SD = 1.9), and their average education was 10.5 years (SD =
1.8). The majority of athletes were male (83.8%). The breakdown of athletes by sport was
as follows: football = 58.3%, hockey = 11.2%, soccer = 9.7%, basketball = 6.9%,
wrestling = 5.6%, and other sports = 8%. This was the first documented concussion for
59% of the sample. Twenty-one percent reported one previous concussion, 14% reported
two previous concussions, and 6% reported three or more previous concussions.

Validity of ImPACT 6
Measures
Version 2.0 of ImPACT is a computer administered neuropsychological test
battery that consists of six individual test modules that measure aspects of cognitive
functioning including attention, memory, reaction time, and processing speed. Version
1.0 of the battery has been used in multiple studies relating to outcome from concussion
(Collins et al., in press; Iverson, Gaetz, Lovell, & Collins, 2002; Iverson, Gaetz, Lovell,
Collins, & Maroon, 2002; Lovell et al., 2003; Lovell, Collins, Iverson, Johnston, &
Bradley, in press). Five composite scores were used for this study. In general, the test
battery is designed to yield multiple types of information within a brief period of time.
Each test module may contribute scores to multiple composite scores. The
Verbal
Memory composite score represents the average percent correct for a word recognition
paradigm, a symbol number match task, and a letter memory task with an accompanying
interference task. The
Visual Memory
composite score is comprised of the average
percent correct scores for two tasks; a recognition memory task that requires the
discrimination of a series of abstract line drawings, and a memory task that requires the
identification of a series of illuminated X’s or O’s after an intervening task (mouse
clicking a number sequence from 25 to 1). The
Reaction Time
composite score represents
the average response time (in milliseconds) on a choice reaction time, a go/no-go task,
and the previously mentioned symbol match task. The
Processing Speed
composite
represents the weighted average of three tasks that are done as interference tasks for the
memory paradigms. The
Impulse Control
composite score represents the total number of
errors of omission or commission on the go/no-go test and the choice reaction time test.
This composite is used to identify athletes who are not putting forth maximum effort or

Validity of ImPACT 7
who are seriously confused about test instructions. This composite was not one of the
dependent measures for this study. In addition to the cognitive measures, ImPACT also
contains a
Post-Concussion Symptom Scale
that consists of 22 commonly reported
symptoms (e.g. headache, dizziness, “fogginess”) that is utilized throughout organized
sports (Lovell & Collins, 1998; Aubry, 2001). The dependent measure is the total score
derived from this 22-Item scale.
Most research to date has used version 1.0 of the program. ImPACT 2.0 is very
similar to the original version. However, there are some significant changes. Version 2.0
includes an additional test module (design memory). In addition, one of the working
memory tasks (X's and O's) was expanded and modified, making it more difficult than the
previous version. Version 2.0 also yields two memory composite scores (Verbal Memory
and Visual memory) while Version 1.0 contains only one memory composite score.
Version 2.0 has been shown to be sensitive to the acute effects of concussion (Iverson,
Lovell, & Collins, 2004).
The Symbol Digit Modalities Test (Smith, 1982) was developed in the 1960s. The
manual was first published in 1973, and was last revised in 1982 (although it is now in its
8
th
printing, as of March of 2000). The SDMT requires the examinee to quickly substitute
a number for a randomized series of geometric figures. The target number is presented at
the top of the page with each corresponding geometric figure. The test items present the
geometric figures only, and the examinee must quickly write in the target number that
goes with each figure. The total number of correctly completed numbers in 90 seconds is
the score derived from this test. The SDMT has been used extensively, over several
decades, in dozens of studies with diverse clinical groups.

Validity of ImPACT 8
Results
Descriptive statistics for the measures are provided in Table 1. The Pearson
correlations between the ImPACT composite scores and the SDMT also are presented in
Table 1. The SDMT correlated more highly with the Processing Speed and the Reaction
Time composites than with the memory composites or total symptoms score.
________________________
Insert Table 1 About Here
________________________
Exploratory principal components factor analysis was conducted using the five
ImPACT composite scores and the SDMT. The subject to variable ratio was 12:1. The
communalities for the seven variables were high, ranging from .70 to .95. With the
eigenvalues set at .5, three components were extracted accounting for 81.9% of the total
variance. The first component accounted for approximately 55.1%, the second
component for 15.6%, and the third component for 11.3% of the variance. A varimax
rotation with Kaiser normalization was applied to the components. The rotated
components were interpreted as follows: (a)
Speed/Reaction Time
: SDMT .87,
Processing Speed Composite .85, and Reaction Time Composite -.76; (b) Memory
:
Verbal Memory Composite .87 and Visual Memory Composite .80; and (c) Total
symptoms .93.
Discussion
The validation of neuropsychological tests is a gradual process, involving
numerous studies over extended periods of time. One aspect of validity is to correlate
computerized test scores with traditional test scores to better understand the presumed

Validity of ImPACT 9
underlying constructs being measured by the computerized tests. As predicted, the SDMT
correlated most highly with the Processing Speed and Reaction Time composites from
ImPACT. Exploratory factor analysis revealed a two-factor solution interpreted as
Speed/Reaction Time and Memory. The total symptoms score was extracted as a unique
component, but as a single variable it should not be considered a factor. It appears as if
the Processing Speed Composite, Reaction Time Composite, and SDMT are measuring a
similar underlying construct in this sample of concussed amateur athletes.
Similar results were obtained by Erlanger and colleagues (2003), who compared
the Concussion Resolution Index to the Symbol Digit Modalities Test and other
traditional neuropsychological measures. The Concussion Resolution Index is a web-
based, online, computerized neuropsychological assessment comprised of six cognitive
subtests. They found that the Processing Speed Index correlated .66 with the SDMT. The
correlations between the SDMT and the Simple Reaction Time (r = .31) and the Complex
Reaction Time (r = .39) Indexes were smaller.
The ongoing validation of a test relates to its clinical use, not to the test itself. Our
goal should be to validate the clinical inferences we derive from tests (Franzen, 1989,
2000). By focusing on the validity of inferences, we focus on the decision-making
process of the clinician. Toward that end, ImPACT has been used in multiple studies of
athletes with concussions. The composite scores, especially the memory and symptoms
composites from Version 1 (Collins et al., 2003; Iverson et al., in press; Lovell et al.,
2003; Lovell et al., in press) and the Verbal Memory, Reaction Time, and symptoms
composites from Version 2 (Iverson et al., 2004), are very sensitive to the initial effects
of concussion in amateur athletes.

Validity of ImPACT 10
Current ongoing research at the University of Pittsburgh Medical Center
employing ImPACT and functional magnetic resonance imaging (fMRI) can also be
conceptualized as validity work. The physiological changes following concussion at the
cellular level have been clarified over the past 15 years, through animal and in vitro
modeling (e.g., see Gaetz, 2002; Giza & Hovda, 2004; ADD 2 others for reviews).
Essentially, it is believed that cerebral concussion triggers a multilayered neurometabolic
cascade of physiological changes at the cellular level (Giza & Hovda, 2004). It is these
physiological changes that are believed to underlie the symptoms reported by athletes in
the initial hours and days post injury, as well as the decrements in neuropsychological
test performance. Conducting baseline and serial postconcussion evaluations with
ImPACT, in tandem with fMRI, might (a) help us better understand the underlying
physiological constructs being measured by the test, and (b) facilitate decision making
regarding the clinical management of athletes with concussions. Initial results from this
work will be forthcoming in the near future.

Validity of ImPACT 11
References
Barr, W.B. & McCrea, M. (2001). Sensitivity and specificity of standardized
neurocognitive testing immediately following sports concussion. Journal of the
International neuropsychological Society, 7, 693-702.
Collins, M.W., Field, M., Lovell, M.R., Iverson, G.L., Johnston, K.M., Maroon, J., & Fu,
F.H. (2003). Relationship between post-concussion headache and
neuropsychological test performance in high school athletes. American Journal of
Sports Medicine, 31, 168-173.
Collins, M.W., Lovell, M.R., Iverson, G.L., Cantu, R., Maroon, J. Field, M. (2002).
Cumulative effects of concussion in high school athletes. Neurosurgery, 51, 1175-
1179.
Collins, M.W. (2001). Neuropsychological testing in sports. A paper presented at the
Annual Winter Meeting of Major League Baseball, Boston, MA.
Collins, M.W., Grindel, S.H., Lovell, M.R., Dede, D.E., Moser, D.J., Phalin, B.R., Nogle,
S., Wasik, M., Cordry, D., Klotz Daugherty, M., Sears, S.F., Nicolette, G.,
Indelicato, P., & McKeag, D.B. (1999). Relationship between concussion and
neuropsychological performance in college football players. The Journal of the
American Medical Association, 282, 964-970.
Collins, M.W., Iverson, G.L., Lovell, M.R., McKeag, D.B., Norwig, J., & Maroon, J. (in
press). On-field predictors of neuropsychological and symptom deficit following
sports-related concussion. Clinical Journal of Sports Medicine.

Validity of ImPACT 12
Delaney, J.S., Lacroix, V.J., Gagne, C., & Antoniou, J. (2001). Concussions among
university football and soccer players: a pilot study. Clincial Journal of Sport
Medicine, 11, 234-240.
Erlanger, D, Saliba E, Barth J, Almquist J, Webright W, Freeman J (2001). Monitoring
resolution of postconcussion symptoms in athletes: Preliminary results of a web-
based neuropsychological test protocol. Journal of Athletic Training, 36, 280-287.
Erlanger, D.M., Kutner, K.C., Barth, J.T., & Barnes, R. (1999). Neuropsychology of
sports-related head injury: Dementia Pugilistica to Post-Concussion Syndrome. The
Clinical Neuropsychologist, 13, 193-209.
Franzen, M.D. (1989). Reliability and validity in neuropsychological assessment
. New
York: Plenum Press.
Franzen, M.D. (2000). Reliability and validity in neuropsychological assessment
. (2
nd
Edition) New York: Kluwer Academic/Plenum Press.
Guskiewicz, K.M., Marshall, S.W., Broglio, S.P., Cantu, R.C., & Kirkendall, D.T.
(2002). No evidence of impaired neurocognitive performance in collegiate soccer
players. The American Journal of Sports Medicine, 30, 157-162.
Guskiewicz, K.M., Ross, S.E., & Marshall, S.W. (2001). Postural stability and
neuropsychological deficits after concussion in collegiate athletes. Journal of
Athletic Training, 36, 263-273.
Hinton-Bayre, A. D., Geffen, G. M., Geffen, L. B., McFarland, K. A., & Friis, P. (1999).
Concussion in contact sports: Reliable change indices of impairment and recovery.
Journal of Clinical and Experimental Neuropsychology, 21
, 70-86.
Iverson, G.L., Gaetz, M., Lovell, M.R., & Collins, M.W. (2002a). Relation between

Validity of ImPACT 13
fogginess and outcome following concussion. Archives of Clinical
Neuropsychology, 17, 769-770.
Iverson, G. L., Gaetz, M., Lovell, M.R., Collins, M., & Maroon, J. (2002b). Cumulative
effects of concussion in amateur athletes.
Archives of Clinical Neuropsychology,
17, 769.
Johnston, K.M., McCrory, P., Mohtadi, N.G., & Meeuwisse, W. (2001). Evidence-based
review of sport-related concussion: clinical science. Clinical Journal of Sports
Medicine, 11, 150-159.
Kelly J.P., & Rosenberg, J.H. (1998). The development of guidelines for the management
of concussion in sports.
Journal of Head Trauma Rehabilitation, 13
, 53-65.
Kelly, J.P. (1999). Traumatic brain injury and concussion in sports. The Journal of the
American Medical Association, 282, 989-991.
Kelly, J.P. (2001). Loss of consciousness: Pathophysiology and implications in grading
and safe return to play. Journal of Athletic Training, 36, 249-252.
Kushner, D.S. (2001). Concussion in sports: Minimizing the risk for complications.
American Family Physician, 64, 1007-1014.
Lovell, M.R. (1999). Evaluation of the professional athlete. In J.E. Bailes, M.R. Lovell,
and J.C. Maroon (eds). Sports-Related Concussion. St. Louis, Quality Medical
Publishing.
Lovell, M.R., Collins, M.W., Iverson, G.L., Field, M., Maroon, J., Cantu, R., Podell, K.,
Powell, J., & Fu, F.H. (2003). Recovery from mild concussion in high school
athletes. Journal of Neurosurgery, 98, 296-301.

Validity of ImPACT 14
Lovell, M.R. & Burke C.J. (2000). Neuropsychological testing in ice hockey: The NHL
program. In R. Cantu (ed). Neurological Athletic Head and Spine Injuries.
Philadelphia: Sanders.
Lovell, M.R. & Collins, M.W. (1998). Neuropsychological assessment of the college
football player. Journal of Head Trauma Rehabilitation, 13, 9-26.
Lovell, M.R., Collins, M.W., Iverson, G.L., Johnston, K.M., & Bradley, J.P. (in press).
Grade 1 or “Ding” concussions in high school athletes. American Journal of Sports
Medicine.
Macciocchi, S.N., Barth, J.T., Alves, W., Rimel, W.R., & Jane, J.A. (1996).
Neuropsychological functioning and recovery after mild head injury in collegiate
athletes. Neurosurgery, 39, 510-514.
Maroon, J.C., Lovell, M.R., Norwig, J., Podell, K., Powell, J.W. & Hartl, R. (2000).
Cerebral concussion in athletes: evaluation and neuropsychological testing.
Neurosurgery, 47, 659-672.
Matser, J.T., Kessels, A.G.H., Lezak, M.D., & Troost, J. (2001). A dose-response relation
of headers and concussions with cognitive impairment in professional soccer
players. Journal of Clinical and Experimental Neuropsychology, 23, 770-774.
McCrea, M. (2001). Standardized mental status testing on the sideline after sport-related
concussion. Journal of Athletic Training, 36, 274-279.
McCrea, M., Kelly, J.P., Randolph, C., Cisler, R., & Berger, L. (2002). Immediate
neurocognitive effects of concussion. Neurosurgery, 50, 1032-1042.

Validity of ImPACT 15
McCrory, P., Johnston, K.M., Mohtadi, N.G., & Meeuwisse, W. (2001) Evidence-based
review of sport-related concussion: Basic science. Clinical Journal of Sport
Medicine, 11, 160-165.
Powell, J.W., & Barber-Foss, K.D. (1999). Traumatic brain injury in high school athletes.
The Journal of the American Medical Association, 282, 958-963.
Quality Standards Subcommittee. (1997). Practice parameter: The management of
concussion in sports (summary statement). Neurology, 48
, 581-585.
Randolph, C. (2001). Implementation of neuropsychological testing models for the high
school, collegiate, and professional sport settings. Journal of Athletic Training, 36,
288-296.
References have not been checked.

Validity of ImPACT 16
Table 1. Descriptive statistics and correlations.
Measure Mean SD Range r
Verbal Memory 81.2 12.4 46 – 100 .46**
Visual Memory 72.3 14.9 29 – 95 .37**
Procession Speed 35.6 8.3 16.1 – 53.8 .70**
Reaction Time .58 .12 .41 – 1.04 -.60**
Total Symptoms 17.4 16.3 0 – 60 -.29*
SDMT 58.0 10.0 39 – 83 ---
p < .05, **p < .01