Content uploaded by Ira Casson
Author content
All content in this area was uploaded by Ira Casson on Mar 17, 2021
Content may be subject to copyright.
CLINICAL STUDIES
CONCUSSION IN PROFESSIONAL FOOTBALL:
NEUROPSYCHOLOGICAL TESTING—PART 6
Elliot J. Pellman, M.D.
Mild Traumatic Brain Injury
Committee,
National Football League,
New York, and ProHEALTH Care
Associates, LLP,
Lake Success, New York
Mark R. Lovell, Ph.D.
Mild Traumatic Brain Injury
Committee,
National Football League,
New York, New York, and
Department of Orthopaedics and
Sports Medicine, University of
Pittsburgh Medical Center,
Pittsburgh, Pennsylvania
David C. Viano, Dr. med.,
Ph.D.
Mild Traumatic Brain Injury
Committee,
National Football League,
New York, New York, and
ProBiomechanics LLC,
Bloomfield Hills, Michigan
Ira R. Casson, M.D.
Mild Traumatic Brain Injury
Committee,
National Football League,
New York; Department of
Neurology, Long Island Jewish
Medical Center, New Hyde Park;
Department of Neurology,
New York University Medical
Center, New York; and
Department of Neurology,
Albert Einstein College of
Medicine, Bronx, New York
Andrew M. Tucker, M.D.
Mild Traumatic Brain Injury
Committee,
National Football League,
New York, New York, and
University of Maryland School of
Medicine, Timonium, Maryland
Reprint requests:
David C. Viano, Dr. med., Ph.D.,
ProBiomechanics LLC,
265 Warrington Road,
Bloomfield Hills, MI 48304-2952.
Email: dviano@comcast.net
Received, May 21, 2004.
Accepted, October 21, 2004.
OBJECTIVE: The National Football League (NFL) neuropsychological testing program
is reviewed, and neuropsychological test data are presented on various samples of NFL
athletes who sustained concussion (mild traumatic brain injury, MTBI).
METHODS: This study evaluated post-MTBI neuropsychological testing of NFL players
from 1996 to 2001. All athletes completed a standardized battery of neuropsycholog-
ical tests and underwent postinjury neuropsychological testing within a few days after
concussion. Test scores were compared with baselines using analysis of variance for
athletes having on-field memory dysfunction, three or more concussions, or 7⫹days
out from practice and play.
RESULTS: The MTBI group did not display significant neuropsychological dysfunction
relative to baseline scores within a few days of injury. However, a subsample of the
injured athletes who displayed on-field memory dysfunction performed significantly
more poorly on two of the memory tests. The neuropsychological test results of a group
of athletes with a history of three or more MTBIs did not differ significantly compared
with a group who had fewer than three concussions or compared with league-wide
normative data. The neuropsychological performance of athletes who were out from
full participation 7⫹days was not significantly different from the group who returned
to play within 7 days or the norms.
CONCLUSION: Neuropsychological testing is used within the overall medical eval-
uation and care of NFL athletes. Players who experience MTBI generally demonstrate
rapid recovery of neuropsychological performance, although poorer neuropsycholog-
ical test results were related to on-field memory dysfunction. NFL players did not
demonstrate evidence of neurocognitive decline after multiple (three or more) MTBIs
or in those players out 7⫹days. The data show that MTBI in this population is
characterized by a rapid return of neuropsychological function in the days after injury.
KEY WORDS: Concussion, Neuropsychological testing, Sport injury, Traumatic brain injury
Neurosurgery 55:1290-1305, 2004 DOI: 10.1227/01.NEU.0000149244.97560.91 www.neurosurgery-online.com
Over the past decade, mild traumatic
brain injury (MTBI) has become a ma-
jor area of interest in sports medicine.
Interest in sports-related MTBI has led to pro-
grams aimed at improving both the diagnosis
and clinical management of athletes. As part of
this overall effort, neuropsychological testing is
being used to aid in the evaluation and manage-
ment of players with MTBI in the National Foot-
ball League (NFL). The development and use of
neuropsychological testing in the NFL has been
rapid and has contributed to the implementa-
tion of neuropsychological testing in other
professional sports, including ice hockey (25),
automobile racing (31), and Australian Rules
football (17). When used in concert with other
medical information, neuropsychological test
data contribute quantitative information regard-
ing neurocognitive processes, such as atten-
tional, memory, and cognitive processing speed.
Neuropsychological testing provides informa-
tion about changes in cognitive processes that
may be missed through sideline or on-field ex-
aminations of the athlete and also provides ob-
jective information regarding the recovery pro-
cess during the acute recovery period (15). The
testing process also compares the athlete’s per-
formance against established normative data
and the individual athlete’s preinjury level of
performance.
1290 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
The program developed for the NFL has influenced the
development of neuropsychological testing at the high school
(3, 14, 27) and college levels (10, 26, 30). In published studies
that have used the NFL test battery, Collins et al. (10) found
that mild cognitive impairment in a group of collegiate ath-
letes generally resolved within a week. More recently, McCrea
et al. (30) found neuropsychological deficits in college athletes
that were evident at 2 days after injury but dissipated within
7 days. Echemendia et al. (12) documented a decline in neu-
ropsychological functioning 2 hours after MTBI and 1 week
after injury. Working with high school athletes and using a
computer-based test battery, Lovell et al. (27, 28) have found
impairments in memory processes and elevated symptoms in
concussed athletes relative to baseline test results. These def-
icits resolved within 1 week of injury.
Recent neuropsychological studies of nonprofessional ath-
letes have demonstrated a strong relationship between on-
field markers of MTBI, such as amnesia and specific neuro-
psychological tests of memory administered days after injury.
Collins et al. (11) found that the presence of anterograde and
retrograde amnesia was linked to poor outcome as deter-
mined by neuropsychological testing and symptoms (using
the ImPACT computer-based neuropsychological test bat-
tery). Lovell et al. (27, 28) linked poor neurocognitive outcome
as measured by the ImPACT battery to on-field memory im-
pairment. Finally, McCrea et al. (30) found that on-field dis-
ruption of memory was related to severity of injury. The
association of on-field markers of memory dysfunction with
formal neuropsychological test results is predictable, because
MTBI is known to disrupt the memory system (40, 43, 45).
These initial studies emphasized the importance of conducting
a careful evaluation of the neurocognitive status of athletes
immediately after injury. However, no such study has been
completed with professional football players. This is an im-
portant issue, because NFL athletes represent a cohort dis-
tinctly different from high school and collegiate athletes in age
and skill level. Generalization from the results of studies with
younger athletes may lead to inaccurate and misleading ex-
pectations regarding the severity of injury and expected pat-
tern of recovery after MTBI in professional athletes.
In a series of additional studies of collegiate athletes, Lovell
and Collins (26) and Collins et al. (10) documented memory
impairment and a decline in speed of information processing
on the Symbol Digit Modalities and Trail Making Tests, which
resolved within 5 to 7 days after injury. Working with profes-
sional rugby athletes in Australia, Hinton-Bayre et al. (17)
found deficits in speed of information processing, with a
return to baseline levels of performance within 1 to 2 weeks of
injury. More recently, Field et al. (14) have documented im-
pairments in memory processes in high school and collegiate
athletes with MTBIs using the Hopkins Verbal Learning Test
(HVLT) and the Brief Visuospatial Memory Test–Revised
(BVMT-R).
The issue of whether or not the effects of sports-related
MTBIs are cumulative has also been an important and highly
controversial area of study. Some authors have suggested that
multiple MTBIs result in discernible decrements on neuropsy-
chological testing in high school (11) and collegiate athletes
(10, 16). Other studies have not supported this finding (12).
This issue has yet to be investigated with neuropsychological
testing in professional football athletes.
This article describes the neuropsychology program in the
NFL from 1996 to 2001. It also investigates neuropsychological
test performance during the initial follow-up evaluation
within several days of concussion. The article addresses the
following questions. 1) Do professional football players dis-
play neuropsychological dysfunction in the days after their
injury? 2) Is neuropsychological dysfunction related to on-
field diagnosis of injury by team medical staff? 3) Do athletes
with three or more MTBIs in the study period display greater
neuropsychological dysfunction than professional football
athletes who have not had three or more MTBIs? and 4) Do
athletes who miss 7 or more (7⫹) days after MTBI perform
more poorly on neuropsychological testing than those who
return to play earlier?
THE USE OF NEUROPSYCHOLOGICAL
TESTING IN PROFESSIONAL FOOTBALL:
HISTORICAL ROOTS
The use of neuropsychological assessment procedures in
sport is a recent phenomenon. The first large-scale study of
MTBI in football players was performed at the college level
and involved the cooperative efforts of the University of Vir-
ginia, the Ivy League schools, and the University of Pittsburgh
(4). The University of Virginia study was conceptualized pri-
marily as a research study, and data were not initially used to
make clinical return-to-play decisions (4). However, these ef-
forts helped to establish a model for neuropsychological as-
sessment that was later adapted for clinical use.
In an effort to improve the overall care of professional
football athletes, a neuropsychological evaluation program
was instituted by the Pittsburgh Steelers in the early 1990s
with the active participation of the Steelers management and
medical/athletic training staff (23, 24, 29). This represented the
first clinically oriented project in a professional sport to use
neuropsychological testing to assist team medical personnel in
making return-to-play decisions after concussion. The Steel-
ers’ program involved the baseline evaluation of each athlete
before the season. This provided a basis for comparisons when
an injury occurred during the season. Testing was then re-
peated within 24 to 48 hours after head injury and again
before the player’s return to contact.
During the 1993 to 1994 seasons, the Steelers project in-
volved the baseline evaluation of approximately 60 NFL ath-
letes who volunteered to undergo preseason testing. The ini-
tial two seasons used neuropsychological testing to assist in
determining player readiness to return to play after concus-
sion. Postinjury neuropsychological testing results were com-
pared with preseason baseline results, and this information, in
addition to player symptoms, was used by the team medical
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1291
staff to make return-to-play decisions. The project expanded
to other athletes on the Steelers’ roster after the 1994 season,
and testing was used effectively to evaluate a number of
injured athletes during that season.
In 1994, the NFL Committee on MTBI was created with the
support of the Commissioner, Paul Tagliabue. The committee
was formed and chaired by Dr. Elliot Pellman of the New York
Jets (32). The scientific activities of the NFL MTBI committee
have been described in publications by Pellman et al. (33,
35–37), and the committee has overseen multiple projects in
the NFL designed to better understand and manage concus-
sion. In addition to supporting the neuropsychology program
discussed in this article, the committee has spearheaded re-
search on the epidemiology of MTBI, the biomechanical forces
involved in concussion, and studies designed to evaluate cur-
rent helmet safety testing standards (36, 37). The most recent
studies involve the epidemiology of players with three or
more concussions and those out from play 7⫹days (33–35).
GOALS OF THE NFL
NEUROPSYCHOLOGY PROGRAM
The NFL neuropsychological testing program was estab-
lished as a clinical research program with the goal of investi-
gating the use of neuropsychological testing in assisting team
physicians in the return-to-play decision. The program was
designed to assist in the gathering of neuropsychological out-
come data in the broader context of the MTBI committee’s
epidemiological and biomechanics projects. Initial funding for
the project was provided through a grant from NFL Charities
in 1995. Participation in the project has not been mandatory
but has been strongly encouraged throughout the league by
the MTBI committee, the NFL Injury Panel, and the Commis-
sioner’s office. Since the inception of the neuropsychological
testing program, participation has grown to include all NFL
teams.
PATIENTS AND METHODS
Data Collection and Analyses
All participating subjects in this study were NFL athletes
who underwent neuropsychological testing during the 1996 to
2001 seasons. Preseason normative data were collected on 655
NFL athletes. Although many participants in this study had
undergone preseason baseline testing before their injury, ath-
letes were also included who had completed only postinjury
evaluations. The overall sample of injured players who under-
went neuropsychological testing consisted of 143 athletes. This
sample represents 22% of the 650 NFL athletes who experi-
enced 887 concussions in the study period from 1996 to 2001
(33–35). Because participation in the neuropsychological study
was voluntary, not all athletes with MTBI completed neuro-
psychological testing.
Within the group of 143 athletes with completed neuropsy-
chological testing, four subsamples were subjected to statisti-
cal analyses depending on the specific research question being
addressed and the availability of completed physician and
neuropsychological test data, as follows.
The postconcussion performance of NFL players was com-
pared with preseason baseline assessments in 95 athletes.
The neuropsychological test performance of a subsample of
athletes with and without documented on-field amnesia was
contrasted.
The potential role of multiple MTBIs on neuropsychological
test performance was examined through a preliminary study
in which a sample of injured NFL players, some of whom had
experienced more than three concussions, were compared
with a sample who had not experienced three concussions.
The neuropsychological test performance of athletes who
lost 7⫹days of playing time was compared with that of a
sample of players who had returned to play within the week.
Neuropsychological testing was completed by trained neu-
ropsychologists or by closely supervised psychology techni-
cians or interns. The administration and test procedures were
standardized and uniform from team to team. After data
collection by individual team neuropsychologists, the data
were stripped of identifying information in compliance with
Health Insurance Portability and Accountability Act standards
and stored at the University of Pittsburgh.
Procedures
The NFL neuropsychology program involves the participa-
tion of a network of neuropsychological consultants in each
NFL city. The consultants complete preseason baseline testing
and postinjury follow-up testing. Neuropsychological consult-
ants were selected for this program on the basis of their
expertise in dealing with MTBI. All neuropsychological con-
sultants held Ph.D. degrees and were licensed in the state of
their practice.
Before each testing session, the athletes were given a brief
explanation of the purpose of neuropsychological testing. All
athletes volunteered for the project, and no player was re-
quired to participate. Participating players represented a rel-
atively homogeneous group. All were men 21 to 35 years old
involved in all offensive and defensive positions.
After neuropsychological testing of the player with a re-
ported concussion, the team physician discussed the results
with the club’s consulting neuropsychologist. The information
from the testing was used in conjunction with the player’s
clinical status to determine return to play. The team physician
has the ultimate responsibility for this decision. As mentioned
in previous articles by Pellman et al. (33–35), the Committee
on MTBI did not impose guidelines or recommendations on
the club’s medical staffs regarding concussion evaluation, test-
ing, treatment, or return-to-play criteria.
The NFL Test Battery
Table 1 lists the neuropsychological tests that were used for
this study. The test battery was based on those used with the
Pittsburgh Steelers (23) and by a limited number of teams
PELLMAN ET AL.
1292 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
from 1993 to 1995. The battery was selected to assess the brain
processes most often affected by MTBI. Some of these tests
have been used with collegiate (4) and professional athletes
(23). More recently, the test battery has undergone revision to
include additional test instruments.
The NFL test battery evaluates multiple aspects of cognitive
functioning, while being relatively brief. The entire battery
takes approximately 30 minutes to administer. The battery is
heavily oriented to the evaluation of attentional processes,
visual scanning, and information processing, although it also
evaluates visual memory, verbal memory, visual-motor coor-
dination, and speech fluency. Past research in neuropsychol-
ogy has identified these as the cognitive functions most likely
to be affected by MTBI. For example, Barth et al. (4) described
mild deficits in psychomotor speed in a large sample of col-
legiate athletes, with the majority of these difficulties resolving
within 5 days of injury.
Each test in the NFL battery has been validated with brain-
injured patients and has been administered according to stan-
dardized instructions to avoid variation in test results across
testing sessions and teams. More details can be found in the
individual test manuals and the review by Lezak (22), which
includes supporting psychometric data. The tests are de-
scribed briefly as follows.
The HVLT (7) consists of a 12-word list that is presented to
the athlete on three consecutive trials. In its revised version,
the athlete is assessed for recall after each presentation and
again after a 20-minute delay period (maximum of 12 words).
The HVLT total score represents the total words produced
over the three trials (maximum of 36 words).
The BVMT-R (5) evaluates visual memory (e.g., memory for
designs) and involves the presentation of six abstract spatial
designs on three consecutive trials. As in the HVLT, the ath-
lete’s recall after each trial and his delayed recall are evalu-
ated. The total score represents the total number of design
elements recalled (maximum of 36), and the delayed recall
condition represents the number of elements recalled after
approximately 20 minutes (maximum of 12). Both the HVLT
and the BVMT-R have six equivalent forms, which minimize
practice effects and makes them ideal for use with athletes
who are likely to undergo evaluation on multiple occasions
throughout the course of their careers. The BVMT-R was
added to the protocol in 1998 after its publication.
The Trail Making Test (38) consists of two parts and re-
quires the athlete to use spatial scanning, speed, and cognitive
flexibility skills. For this protocol, the time to complete Parts A
and B were analyzed, and a lower score reflects better (faster)
performance.
The Controlled Oral Word Association Test (COWAT) (6)
requires the athlete to recall as many words as possible that
begin with a given letter of the alphabet within a 60-second
time period. This is completed for three separate letters and
provides a measure of verbal fluency. A higher score reflects
better performance.
The Digit Span (44) subtest represents a test of attention
span in which the subject is required to repeat increasingly
longer strings of numbers in both forward and backward
order. A higher score represents better performance.
The Symbol Digit Modalities Test (SDMT) (41) requires the
player to use visual scanning and processing speed to match a
series of numbers and symbols while under time pressure. A
higher score reflects better performance.
Wherever possible, tests were used that had multiple forms
to avoid the improvement in test performance secondary to
practice effects. Practice effects represent improvements in test
performance as a result of previous exposure to the test and
may cloud the interpretation of the recovery process. Past
studies of athletes using tests that make up the NFL battery
have demonstrated that the Trail Making Test, the SDMT, and
the COWAT demonstrate significant practice effects. For in-
stance, Lovell and Collins (26) reported mean improvements
on the Trail Making Test, Part B, of 2 seconds in a sample of 40
TABLE 1. National Football League neuropsychological test battery (1996–2001)
a
Series (ref. no.) Ability evaluated Test
Brandt, 1991 (7) Memory for words (verbal memory), immediate recall HVLT
Brandt, 1991 (7) Delayed memory for words (20-min delay from presentation) Delayed recall from HVLT
Benedict, 1997 (5) Memory for designs (visual memory) BVMT-R
Benedict, 1997 (5) Delayed memory for designs (20-min delay from presentation) Delayed recall from BVMT-R
Reitan, 1958 (38) Visual scanning, mental flexibility Trail Making Test
Smith, 1982 (41) Symbol Digit Modalities
Benton and Hamsher, 1978 (6) Word fluency, word retrieval Controlled Oral Word Fluency
Wechsler, 1997 (44) Attention span, forward and backward conditions Digit Span
a
HVLT, Hopkins Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–Revised.
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1293
nonconcussed college football players tested at the beginning
and end of the season. Nearly identical scores were found for
the HVLT at preseason and postseason. Working with unin-
jured high school athletes, Barr (3) reported an improvement
of approximately 4 seconds on the Trail Making Test, Part B.
No significant change was found on the HVLT. Both of these
studies referenced above reported improvements of two
words across two administrations for the COWAT test. To
date, practice effects have not been studied in an NFL sample.
Unfortunately, acceptable equivalent forms of these tests did
not exist at the time of initiation of the study.
Timeline of the Neuropsychological Evaluations
After a reported MTBI, the initial neuropsychological eval-
uation of the athlete usually took place within 24 to 48 hours,
although this varied somewhat depending on the readiness of
the athlete to engage in cognitive exertion after injury. Even
when athletes denied initial symptoms, a neuropsychological
evaluation was recommended to evaluate subtle aspects of
cognitive functioning, such as information processing speed
and memory. If the athlete displayed cognitive deficits on
testing or continued to exhibit postconcussive symptoms, a
follow-up neuropsychological evaluation was recommended 5
to 7 days after injury. This time interval represents a useful
and practical time span and is consistent with brain metabo-
lism studies, which have demonstrated metabolic changes that
persist several days after injury in animals (19) and humans
(27).
Determination of abnormal test performance was made by
the individual team neuropsychologist and was based on a
comparison of postinjury test results with established norma-
tive data, which were provided to each team consultant by the
program director (MRL). Abnormal performance was defined
as performance that deviated negatively from the athlete’s
baseline performance. If baseline testing was not completed,
league-wide normative values were used to aid in interpreta-
tion. Clinical interpretations were made by team neuropsy-
chologists and not by the program director. If baseline testing
had been completed, the player’s preinjury performance was
used as the basis for comparison. Ultimately, return-to-play
decisions were made by team physicians after consultation
with the team’s athletic trainer, consulting neuropsychologist,
and other personnel.
Statistics
Because of limitations in sample sizes, analysis of variance
(ANOVA) models were performed using Statistica software
(42). No adjustments were made for multiple comparisons.
RESULTS
Normative Values for NFL Players Test Performance
Baseline neuropsychological test data were gathered to de-
termine average performance, or “norms,” for NFL athletes.
The data establish the normal range for performance in each
test score in the battery and also provide the basis for com-
parisons if an athlete is injured during the season. Baseline
evaluations were conducted at preseason, before contact.
Table 2 summarizes the baseline performance of NFL ath-
letes. These results fall within expected limits for the general
population and are similar to data on collegiate athletes (10).
Data were collected on up to 655 active NFL athletes, depend-
ing on the neuropsychological test. The NFL players have a
mean age of 25.4 years (range, 20–44 yr; median age, 24 yr).
The college data represent a sample of 386 collegiate athletes
with a mean age of 20.4 years (10).
Acute Effects of Concussion: Do Concussed Athletes
Display Abnormal Neuropsychological Test Results
Compared with Preseason Baseline Results?
The neuropsychological performance of a large sample of
NFL athletes who had experienced MTBI was compared with
the group’s baseline performance by use of a series of
repeated-measures ANOVAs. This addresses the question of
whether professional football players experienced measurable
neuropsychological decline after documented MTBI. Athletes
completed neuropsychological testing an average of 1.4 days
after injury (range, 1–10 d; standard deviation, 1.29 d). The
average time from baseline to postinjury testing was 531 days
(range, 4–2190 d). This wide range in time between baseline
testing and postconcussion is characteristic of athletes and
reflects the uncertainty of when each injury will occur. Al-
though it is conceivable that an athlete tested days after injury
TABLE 2. Normative neuropsychological test data for
National Football League athletes
a
Test No. Mean (SD) College sample
Hopkins VLT (Total) 653 26.0 (3.9) 24.6 (4.0)
Hopkins VLT (Delay) 630 9.0 (2.1) 7.8 (2.0)
BVMT-R (Total)
b
479 27.4 (5.5)
BVMT-R (Delay)
b
478 10.4 (2.0)
Trail Making Test, Part
A
651 21.4 (7.4) 21.0 (5.9)
Trail Making Test, Part
B
654 55.6 (17.1) 55.4 (17.3)
COWAT 655 41.9 (11.4) 37.5 (9.3)
Symbol Digit Modalities
Test
155 56.6 (8.4) 56.8 (8.9)
Digit Span-Total 646 14.9 (4.0) 15.8 (3.9)
a
VLT, Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–
Revised; COWAT, Controlled Oral Word Association Test; SD, standard
deviation.
b
The BVMT-R was not used in the study by Collins et al. (10).
PELLMAN ET AL.
1294 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
might demonstrate a larger practice effect, the number of
athletes who were retested within 2 weeks of baseline testing
was quite small (n ⫽4) and therefore is not likely to have had
a significant impact on the overall group data. Each neuro-
psychological test score was treated as a within-subjects factor.
This allowed analysis of changes on each score over time. Table
2shows that most athletes had completed the HVLT, Trail
Making Test, and COWAT, whereas fewer had completed the
BVMT-R and SDMT because of changes in the test battery
during the course of the study.
Table 3 shows that athletes with MTBI did not perform
significantly more poorly on any of the individual tests that
make up the NFL battery. In fact, the only significant differ-
ences between baseline and postinjury performance were on
the Trail Making Test, Part A (F⫽4.97, P⬍0.03), the Digit
Span Forward test (F⫽10.53, P⬍0.001), and the SDMT (F⫽
8.13, P⬍0.006). For these tests, the postinjury group per-
formed significantly better than the overall performance of the
group at baseline. However, these differences are in the “nor-
mal range,” on the basis of the NFL norms presented in Table
2. The differences are likely to reflect the influence of practice
effects on these tests, as have been documented by previous
studies using the NFL or similar test batteries (3, 10, 18, 26).
Practice effects refer to improvements in performance on neu-
ropsychological testing based on repeated exposure to a given
test. Practice effects vary widely, depending on the nature of
the test and whether or not alternative equivalent forms of the
test are used. For this study, alternative forms were used for
the memory tests (HVLT and BVMT-R), whereas no alterna-
tive forms were used for the Trail Making Test, SDMT, or
COWAT. Therefore, larger practice effects were expected for
the Trail Making test, the SDMT, and the COWAT compared
with the HVLT and the BVMT.
Does Postinjury Neuropsychological Dysfunction Relate
to On-field Memory Dysfunction?
Next, a series of analyses was completed to evaluate the
relationship between neuropsychological testing within days
of injury and on-field memory dysfunction. Athletes were
evaluated by the team medical staff after injury, and the
results were reported on a standard NFL physician form (33).
Athletes were selected according to the following criteria: 1)
completion of postinjury neuropsychological testing before
return to play and 2) a completed physician evaluation detail-
ing the on-field sequelae of the concussion. Athletes who did
not have completed physician data were excluded from the
analysis. A series of one-way ANOVAs was completed, with
the individual neuropsychological tests representing the de-
pendent variables and the absence or presence of on-field
memory dysfunction as the independent variable. Follow-up
evaluations were completed an average of 1.3 days after injury
(range, 1–9 d), with 90% of the athletes being evaluated 1 day
after injury. The diagnosis of on-field memory dysfunction
was determined by team physicians (33, 35). There was no
significant difference between the memory dysfunction and
no memory dysfunction groups with regard to whether or not
they were rendered unconscious (
2
⫽0.008, P⫽0.926).
Table 4 shows that athletes who were diagnosed as having
memory dysfunction after injury perform more poorly on
memory tests than injured athletes who did not display on-
field memory dysfunction. For example, the on-field memory
dysfunction group performed significantly more poorly on
immediate memory (F⫽6.1, P⬍0.02) and delayed memory
(F⫽5.4, P⬍0.03) aspects of the BVMT-R. In addition, the
on-field memory dysfunction group also displayed relatively
poorer performance on immediate memory and delayed
TABLE 3. Baseline and postinjury test performance for National Football League concussions in 1996–2001
a
Test N Baseline Follow-up F,P
HVLT (Total) 95 25.8 (4.4) 26.6 (4.6) F⫽2.97 (P⬍0.10)
HVLT (Delay) 95 8.8 (2.2) 8.7 (2.4) F⫽0.02 (P⬍0.89)
BVMT-R (Total) 47 29.0 (4.3) 28.7 (6.8) F⫽0.14 (P⬍0.71)
BVMT-R (Delay) 47 11.1 (1.3) 10.8 (1.7) F⫽0.70 (P⬍0.41)
Trail Making Test, Part A 93 21.2 (8.4) 18.9 (6.2) F⫽4.97 (P⬍0.03)
Trail Making Test, Part B 89 52.8 (14.3) 51.9 (30.3) F⫽0.09 (P⬍0.80)
COWAT 90 39.9 (9.7) 41.1 (10.1) F⫽1.42 (P⬍0.23)
SDMT-Total 59 59.2 (9.1) 62.2 (11.4) F⫽8.1 (P⬍0.006)
Digit Span-Forward 87 9.8 (2.3) 10.6 (2.4) F⫽10.5 (P⬍0.001)
Digit Span-Backward 87 7.9 (2.8) 8.0 (2.5) F⫽0.71 (P⬍0.40)
a
HVLT, Hopkins Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–Revised; COWAT, Controlled Oral Word Association Test; SDMT, Symbol Digit
Modalities Test.
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1295
memory components of the HVLT, although this did not reach
statistical significance. Differences in test performance seem to
be specific to memory and not, more generally, to cognitive
dysfunction, because test performance on speed tests such as
the SDMT and the Trail Making Test were nearly identical for
both groups.
Effect of Multiple MTBIs on Neuropsychological
Testing: Are There Cumulative Effects of Injury in
Professional Football Athletes?
Two subject groups were formed to investigate the potential
cumulative effect of multiple MTBIs on neuropsychological
test performance. One group consisted of athletes with a his-
tory of three or more MTBIs during the course of the study,
and the other was made up of athletes who had fewer than
three concussions. All athletes underwent neuropsychological
testing within 2 days of their third injury (mean, 1.04 d; range,
1–2 d). A series of ANOVAs was completed with individual
neuropsychological testing scores as dependent variables and
multiple concussion status as independent variables. Multiple
ANOVAs were completed because not all of the athletes had
completed all of the tests that made up the test battery because
of changes in test procedures over the course of the neuropsy-
chological testing program.
Table 5 shows that the two groups did not differ on any of
the neuropsychological tests in the battery. Players with three
or more concussions did not display statistically significant
differences on the 10 individual scores that made up the
neuropsychological test battery. It should be noted that the
completion of multiple ANOVAs created an increased poten-
tial for finding significant differences between groups. How-
ever, despite the use of this very liberal research design, no
such differences were found between groups.
Do Athletes Who Miss 7ⴙDays after MTBI Perform
More Poorly on Neuropsychological Testing?
Next, the neuropsychological test performance of a group of
athletes who missed 7⫹days after MTBI was compared with
that of an MTBI group who returned to play within a week of
injury. This particular group had undergone previous study
(34, 36, 37). The 7⫹days out group was evaluated neuropsy-
chologically relatively soon after injury (an average of 2.2 d
after injury; range, 1–9 d). Group performance was analyzed
statistically via multiple ANOVAs, with the individual neu-
ropsychological measures gathered after the injury represent-
ing the dependent variables. Neuropsychological test results
from the BVMT delayed memory data were not sufficient to
allow meaningful statistical analysis. Consistent with our
analysis of the cumulative effects of injury, the use of multiple
ANOVAs served to increase the risk of finding significant
differences in test performance between groups.
Table 6 shows that the 7⫹days out group did not perform
more poorly on any of the neuropsychological test measures
compared with athletes who had been returned to play with-
out missing 7 days. None of the group differences reached
statistical significance. In addition, the mean values for all of
the scores were comparable to NFL normative values pre-
sented in Table 2 and to age expectations for a large sample of
collegiate athletes who had been tested by the NFL test
battery.
Figure 1 shows the means and standard deviations in
BVMT-R (Total) neuropsychological test scores for various
TABLE 4. Neuropsychological test performance at follow-up for athletes with and without on-field memory dysfunction
a
Test Memory
dysfunction NNo memory
dysfunction NF,P
HVLT (Total) 25.7 (5.0) 35 27.2 (4.4) 25 F⫽1.52 (P⬍0.25)
HVLT (Delay) 8.5 (2.6) 35 9.4 (2.1) 24 F⫽1.70 (P⬍0.20)
BVMT-R (Total) 24.6 (6.8) 27 29.5 (5.7) 17 F⫽6.1 (P⬍0.02)
BVMT-R (Delay) 9.4 (2.2) 24 10.9 (1.6) 14 F⫽5.4 (P⬍0.03)
Trail Making Test, Part A 19.3 (7.1) 35 18.9 (6.6) 24 F⫽0.04 (P⬍0.85)
Trail Making Test, Part B 47.6 (13.3) 35 44.6 (21.4) 24 F⫽0.48 (P⬍0.50)
COWAT 43.0 (10.7) 35 43.0 (10.6) 24 F⫽0.0001 (P⬍0.99)
SDMT-Total 60.9 (9.5) 31 60.8 (6.9) 23 F⫽0.52 (P⬍0.49)
Digit Span-Forward 10.6 (2.5) 33 10.0 (3.2) 24 F⫽0.49 (P⬍0.50)
Digit Span-Backward 8.4 (2.5) 33 7.7 (3.1) 24 F⫽0.77 (P⬍0.39)
a
HVLT, Hopkins Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–Revised; COWAT, Controlled Oral Word Association Test; SDMT, Symbol Digit
Modalities Test.
PELLMAN ET AL.
1296 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
groups of NFL players for the purposes of comparison. The
baseline scores are compared with those of players with mem-
ory problems, three or more concussions, and 7⫹days out
from play. Only the memory problem group showed a signif-
icant difference from the comparison group; however, the
differences are subtle when the standard deviations are con-
sidered. Also shown are data from a sample of high school
players (14) and college players at Penn State University (un-
published data; 12) showing scores similar to those in the
professional players.
DISCUSSION
This study represents the first on neuropsychological function
of NFL athletes in the United States. Numerous studies have
been published on the neurocognitive recovery of high school (3,
TABLE 5. Neuropsychological test performance at follow-up for athletes with and without a history of three or more concussions from
1996 to 2001
a
Test Baseline for
concussed group
Three or fewer
concussions NThree or more
concussions NF,P
HVLT (Total) 25.8 (4.4) 95 26.9 (4.7) 49 27.2 (5.0) 20 F⫽0.74 (P⬍0.40)
HVLT (Delay) 8.8 (2.2) 95 9.0 (2.4) 49 9.6 (2.5) 21 F⫽0.093 (P⬍0.80)
BVMT-R (Total) 29.0 (4.3) 47 26.7 (7.2) 36 27.9 (5.9) 17 F⫽0.29 (P⬍0.59)
BVMT-R (Delay) 11.1 (1.3) 47 10.1 (2.1) 30 10.3 (1.9) 15 F⫽0.040 (P⬍0.84)
Trail Making Test, Part A 21.2 (8.4) 93 19.5 (6.7) 49 17.6 (7.8) 20 F⫽1.03 (P⬍0.32)
Trail Making Test, Part B 52.8 (14.3) 89 46.4 (12.8) 49 44.7 (23.0) 20 F⫽0.16 (P⬍0.70)
COWAT 39.9 (9.7) 90 41.9 (8.7) 49 46.2 (13.5) 20 F⫽2.5 (P⬍0.13)
SDMT-Total 59.2 (9.1) 59 61.5 (11.5) 28 65.5 (8.2) 12 F⫽1.21 (P⬍0.28)
Digit Span-Forward 9.8 (2.3) 87 10.6 (2.1) 49 10.6 (3.6) 20 F⫽0.0001 (P⬍0.98)
Digit Span-Backward 7.9 (2.8) 87 8.2 (2.5) 47 8.5 (3.2) 20 F⫽0.136 (P⬍0.72)
a
HVLT, Hopkins Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–Revised; COWAT, Controlled Oral Word Association Test; SDMT, Symbol Digit
Modalities Test.
TABLE 6. Neuropsychological test performance at follow-up for athletes who missed 7 or more days of practice/playing
after concussion
a
Test <7 Days Out N 7ⴙDays Out N F,P
HVLT (Total) 26.3 (4.8) 74 28.3 (4.7) 17 F⫽2.6 (P⬍0.11)
HVLT (Delay) 8.8 (2.4) 73 9.6 (2.0) 17 F⫽1.9 (P⬍0.18)
BVMT-R (Total) 27.4 (6.6) 54 26.3 (6.4) 15 F⫽0.32 (P⬍0.60)
Trail Making Test, Part A 19.3 (6.9) 73 17.0 (4.2) 17 F⫽1.3 (P⬍0.26)
Trail Making Test, Part B 48.2 (16.5) 73 49.6 (20.7) 17 F⫽0.09 (P⬍0.76)
COWAT 41.2 (10.9) 72 44.3 (8.4) 18 F⫽1.1 (P⬍0.33)
SDMT-Total 61.5 (11.5) 28 65.5 (8.2) 12 F⫽1.21 (P⬍0.28)
Digit Span-Forward 10.2 (2.6) 71 11.4 (2.5) 17 F⫽2.5 (P⬍0.12)
Digit Span-Backward 7.7 (2.6) 71 8.8 (2.6) 17 F⫽2.3 (P⬍0.14)
a
HVLT, Hopkins Verbal Learning Test; BVMT-R, Brief Visuospatial Memory Test–Revised; COWAT, Controlled Oral Word Association Test; SDMT, Symbol Digit
Modalities Test.
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1297
27, 28) and collegiate (10, 26, 30) athletes. The NFL neuropsycho-
logical test battery has been used extensively over the past de-
cade both in and outside of the NFL. In addition to gaining
general acceptance by NFL team medical staff and athletic train-
ers, neuropsychological testing has also been well accepted by
the athletes. There is currently general agreement that neuropsy-
chological testing provides quantitative information regarding
recovery from injury, thereby improving the evaluation and
management of MTBIs in athletes (2). More specifically, neuro-
psychological testing may provide diagnostic information re-
garding subtle disruptions of cognitive processes such as atten-
tion, memory, and speed that may not be detected by a cursory
sideline evaluation.
On-field Cognitive Impairment
This study supports previous research that has shown that
on-field signs of cognitive impairment, such as amnesia, are
useful in determining the severity of brain injury (13, 27, 28).
Consistent with studies of high school and collegiate athletes,
professional athletes who were diagnosed by team physicians
as having impaired memory function after concussion per-
formed significantly more poorly on memory testing, while
displaying generally intact performance on other measures
requiring cognitive speed. These findings suggest that neuro-
psychological testing in combination with other diagnostic
information helps track recovery from MTBI in professional
football players. The results of this study also suggest that
team physician diagnosis of “memory impairment” on the
sideline has predictive value. Athletes who display on-field
memory impairment have measurable memory impairment
within 1 to 2 days after injury. Neuropsychological testing
evaluates disruptions in cognitive processes such as attention,
memory, and cognitive speed and adds quantitative informa-
tion for the evaluation and management of concussion.
This suggests that only those players identified as having
cognitive and memory disturbances are likely to have neuropsy-
chological test impairments on follow-up testing. Athletes with
no clinical cognitive and memory impairments on physician
examination did not, as a group, have more subtle changes in
cognitive processes that were missed on sideline clinical exami-
nation. This suggests that the on-field evaluation used by NFL
team physicians is effective with regard to the identification of
cognitive and memory impairments immediately after injury.
The authors attribute this success to the training and knowledge
of the NFL team physicians and the thorough nature of the
league’s injury documentation process. Furthermore, it must be
pointed out that even the players with on-field memory impair-
ments and neuropsychological test memory impairments 1 to 2
days later went on to make rapid, complete recoveries.
NFL Athletes Recover Quickly from Concussion
The finding of no overall significant neuropsychological
testing differences between a group of injured NFL athletes
who had previously undergone baseline neuropsychological
testing suggests that NFL athletes with MTBIs recover quickly
after injury. In contrast to previous studies that have sug-
gested cognitive difficulties lasting 1 week or more, NFL ath-
letes demonstrated generally intact performance within sev-
eral days relative to baseline performance levels. There are a
number of potential hypotheses for the apparent difference
between professional and amateur athletes.
First, athletes selected for inclusion in this study were diag-
nosed as having experienced concussion based on multiple di-
agnostic criteria that were not limited to on-field neurocognitive
dysfunction. In other words, a number of athletes in the sample
did not necessarily demonstrate amnesia, confusion, or disorien-
tation but rather noncognitive difficulties such as headache, nau-
sea, or balance dysfunction. Within the population of NFL player
injuries from which this sample was drawn (n ⫽887), approxi-
mately 50% did not have memory or other cognitive findings on
the field (33). Therefore, it is not surprising that a significant
number of injured athletes would not display signs of cognitive
impairment days after concussion. Studies on high school and
collegiate athletes have relied heavily on on-field amnesia
and/or confusion, which is more predictive of neuropsycholog-
ical sequelae. Lovell et al. (28) found group differences between
baseline and postinjury performance in high school athletes who
had a large proportion of on-field amnesia or confusion.
Second, recently published evidence suggests that there
may be a recovery gradient, with younger athletes (high
school age and below) displaying more pronounced injury
and longer recovery times than older athletes (college age and
above). Several lines of evidence support this hypothesis. Chil-
FIGURE 1. Graph showing average and standard deviation in BVMT-R
(Total) neuropsychological test scores for NFL baselines, players with
memory problems, three or more concussions, and out 7⫹days from play.
Also shown are data in the first two columns from a sample of high school
players (from, Field M, Collins MW, Lovell MR, Maroon J: Does age
play a role in recovery from sports-related concussion? A comparison of
high school and college athletes. J Pediatr 142:546–553, 2003 [14]) and
college players at Pennsylvania State University (unpublished data; and
from, Echemendia R, Putukian M, Mackin S, Julian L, Shoss N: Neuro-
psychological test performance prior to and following sports related mild
traumatic brain injury. Clin J Sports Med 11:23–31, 2001 [12]).
PELLMAN ET AL.
1298 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
dren are known to exhibit more diffuse and prolonged cere-
bral swelling than adults (1, 21), and the majority of fatal brain
injuries in contact sports have occurred in children below the
age of 18 years (8). There is also evidence from animal research
suggesting that the immature brain may be more sensitive to
the negative effects of glutamate-mediated N-methyl
d-aspartate than the brains of older animals. This has been
hypothesized to play a role in the detrimental effects of exci-
tatory amino acids after brain injury. Finally, recent clinical
data have documented neuropsychological deficits lasting
days after injury and a differing rate of recovery depending on
the age of the athlete (14). A sample of collegiate athletes
improved more rapidly on standardized tests of memory com-
pared with a group of high school athletes with MTBI. The
authors suggest that neurodevelopmental differences between
older and younger subjects play a role in the recovery differ-
ences between the groups.
Finally, there may be a different tolerance for concussion
between professional and nonprofessional athletes. The level
of conditioning and skill necessary for success in the NFL may
result in an overall sample pool of athletes who are less prone
to injury than younger and less talented or well-conditioned
individuals. There may be a “natural selection” process in
professional football whereby athletes who are easily injured
do not successfully rise through the ranks of high school to
collegiate football to the NFL without resilience to cerebral
concussion. This hypothesis is supported, albeit indirectly, by
current NFL data suggesting that a relative minority of ath-
letes develop postconcussion syndrome (34, 35).
Relevance to Previous NFL Concussion Studies
In previous articles, the authors have presented the results of
clinical evaluations in professional football players after MTBI
(33–35). Those studies demonstrated that NFL players, as a
group, made relatively quick recoveries after cerebral concus-
sion. As a group, players who sustained multiple MTBIs fared no
differently from those with single injuries. Analysis of the group
of players with the most prolonged recoveries after MTBI re-
vealed no evidence of permanent brain dysfunction or other
long-term effects on the basis of clinical examination. One pos-
sible limitation of those earlier studies was the absence of any
neuropsychological test results from the database and its subse-
quent analysis. This article presents the neuropsychological test-
ing results that were not included in those earlier articles.
The issue of the potential cumulative effects of sports-
related MTBI has been a particularly controversial one, and
many studies have offered different views on the significance
of multiple injuries. This study did not find a pattern of poorer
neuropsychological test scores in a group of professional ath-
letes who were followed up closely for 6 years. The interpre-
tation of these findings is complicated and may relate to a
variety of factors, including potential differences in response
to MTBI between professional and younger athletes as well as
the general resiliency of highly conditioned athletes who have
largely escaped serious consequences of brain injury through-
out their high school, college, and professional careers.
As a group, NFL athletes who were held from play 7⫹days did
not display poorer neuropsychological performance than a group
who returned within a week. Although based on a somewhat
limited sample size, the results suggest that these athletes did not
display significant cognitive impairment at the time of their postin-
jury follow-up evaluation within days of injury. Athletes are held
from participation by a number of factors, which include both
medical and nonmedical considerations. As suggested by Pellman
et al. (34), it is possible that athletes who made up this group were
treated more conservatively relative to other athletes. The over-
representation of “skill position” players such as quarterbacks may
also help explain why these athletes were held out of competition
for more than one game. However, the 7⫹days out group repre-
sented the most severely injured players on the basis of signs and
symptoms (34). It is noteworthy that they had normal neuropsycho-
logical test performance and medical evaluation of cognitive and
memory function within several days of injury.
The neuropsychological testing results presented here corrob-
orate earlier purely clinical reports. As noted previously, the
athletes who were held out of play for 7⫹days displayed neu-
ropsychological test performances equivalent to those of the
group who returned to play in less than 1 week. This supports
the findings of Pellman et al. (34) that even those players in the
more seriously injured group in terms of number of signs and
symptoms made good recoveries, apparently without discernible
residual neurocognitive effects. Furthermore, the neuropsycho-
logical test performances did not differ between the group with
three or more concussions and the group with fewer than three
concussions. The strong correlation between the results of clinical
and neuropsychological evaluations also provides supportive
evidence for the position that there is no evidence in this study of
widespread permanent or cumulative effects of single or multi-
ple MTBIs in professional football players.
In other words, the results of this present study support the
authors’ previous work, which indicated that there was no evidence
of worsening injury or chronic cumulative effects of multiple MTBIs
in NFL players. Critics might contend that the specific neuropsy-
chological test battery used in this study was not sensitive enough to
detect chronic brain injury. However, the evidence suggests other-
wise. Similar (frequently the same) neuropsychological test batteries
were used to detect chronic brain dysfunction in retired boxers (9,
39). In both studies, neuropsychological tests uncovered evidence of
cognitive and memory impairments, which were much more wide-
spread than was suspected from purely clinical evaluations. Thus,
the failure to find abnormalities in NFL athletes even after multiple
and/or more serious MTBIs is not suggestive of a cumulative effect
of injuries in NFL players.
Critics might also suggest that the paper-and-pencil neuro-
psychological test battery used in this present study was not
sensitive enough to detect subtle cognitive and memory im-
pairments. However, similar test batteries have been used in
numerous studies of high school and college players and have
been found to be sensitive to cognitive dysfunction (3, 10–12,
14, 16, 26, 30). The authors’ interpretation of the present re-
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1299
sults, which found a strong relationship between on-field
signs and symptoms and neuropsychological test results,
would be that a thorough clinical, mental status examination
of the players after MTBI by a well-trained physician is per-
haps more sensitive than had previously been thought. There-
fore, the thorough on-field evaluation of signs and symptoms
is encouraged in all concussed athletes.
Importance of Clinical Evaluation by Team Physicians
and Role of Neuropsychological Testing
The standard practice of NFL team physicians is to perform
thorough, not cursory, sideline evaluations. This explains the
strong correlation between on-field clinical findings and neu-
ropsychological test results reported in this article. Given the
established league-wide neuropsychological database, the test
performance of the injured athlete can be compared with
appropriate league standards, or ideally, the athlete’s perfor-
mance can be compared with his preinjury baseline levels of
performance. In addition, given the tendency for some athletes
to underreport symptoms in an effort to hasten their return to
play (27), neuropsychological testing provides performance-
based indicators of recovery that help ensure resolution of
cognitive difficulties before return to play.
This study points out the overriding importance of the clinical
evaluation by the team physician in determining the appropriate
management of athletes who sustain MTBI. Had the team physi-
cians relied solely on the neuropsychological test results in making
their decisions, many of these more severely injured players would
have returned to play sooner than was warranted by the clinical
picture. Furthermore, of the 32 signs and symptoms of MTBI that
were evaluated by NFL team physicians (33), Table 7 shows that at
most 13 (most likely only 8) are evaluated by neuropsychological
testing. Therefore, if one relies solely on neuropsychological testing
when managing MTBI, then one will almost certainly overlook
more than half of the signs and symptoms that may occur. It clearly
follows from the above that neuropsychological testing alone cannot
be relied on, in and of itself, to make return-to-play decisions. The
authors strongly caution against the growing trend of substituting
neuropsychological testing for decision making based on physician
evaluations in the treatment of sports-related MTBI.
As has been noted, all NFL teams use neuropsychological
testing in the evaluation of concussion. The extent of testing
varies greatly among teams. Likewise, the role of testing in the
management of MTBI may differ. The data from earlier reports
(33–35) reinforce that the vast majority of concussions in NFL
players are mild, with little time lost from play. The neuropsy-
chological data corroborate the rapid clinical recovery made by
the players in routine cases. In these instances, the testing would
seem to play a minor role, serving to confirm neuropsychological
test recovery that correlates with the rapid clinical recovery.
In general, team physicians find neuropsychological testing
most helpful in the management of more severe concussions. In
cases of prolonged postconcussion symptoms, neuropsycholog-
ical testing may help to “objectify” the neurocognitive changes
after injury, allow monitoring of the recovery process, and assist
the medical team and player regarding return-to-play decisions.
NFL team physicians are relying on neuropsychological testing
as a corroborative diagnostic modality when trying to quantify
whether injury results in cognitive dysfunction. As refinements
in testing are made, the role of testing in the evaluation and
TABLE 7. Signs and symptoms of concussion
a
and whether
neuropsychological testing can evaluate changes
Signs and symptoms Evaluated by
neuropsychological testing
General symptoms
Headaches No
Neck pain No
Nausea No
Syncope No
Vomiting No
Back pain No
Seizures No
Cranial nerve symptoms
Dizziness No
Blurred vision No
Vertigo No
Photophobia No
Tinnitus No
Diplopia No
Nystagamus No
Pupil response No
Pupil size No
Hearing loss No
Memory problems
Retrograde amnesia, delayed Yes
Information processing problems Yes
Attention problems Yes
Anterograde amnesia, delayed Yes
Cognition problems
Immediate recall Yes
Not oriented to time Yes
Not oriented to place Yes
Not oriented to persons Yes
Somatic complaints
Fatigue Possibly
Anxiety Possibly
Personality change Possibly
Irritability Possibly
Sleep disturbance No
Loss of appetite No
Depression Possibly
Loss of libido No
Loss of consciousness No
a
From, Pellman et al. (33).
PELLMAN ET AL.
1300 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
management of concussions in NFL players is likely to evolve
and remain a diagnostic tool for team physicians.
Neuropsychological tests represent one tool that can be used to
assist treating physicians when diagnosing and managing cere-
bral concussion. It should be used in conjunction with the clinical
evaluation, brain imaging, and other diagnostic studies, where
appropriate. However, neuropsychological testing should not be
used alone to determine diagnosis or make management deci-
sions such as return to play. In addition to having diagnostic
value, the quantitative nature of the results of neuropsychologi-
cal testing is often reassuring to injured players and their treating
physicians and offers an objective means of tracking recovery.
The present study has clearly indicated that the results of neu-
ropsychological testing are strongly correlated with the results of
clinical mental status testing on the sideline by the team physi-
cian. This does not, however, minimize the usefulness of such
testing. The usefulness of neuropsychological testing can be ex-
plained best in parallel to the clinical neurological examination
(excluding mental status). The clinical neurological examination
is almost always normal after MTBI. The examining physician
expects the neurological examination to be normal in these ath-
letes. The normal neurological examination is reassuring to the
physician that their diagnosis is correct.
If the neurological examination is abnormal, this raises a red
flag for the treating physician and alerts him or her to the pos-
sibility that some other process may be going on. Even though
the neurological examination is almost always normal in the
setting of MTBI, no one is suggesting that the neurological ex-
amination is unnecessary or should be discarded. Similarly, the
neuropsychological test results most often correlate with the
clinical evaluation findings and almost always confirm the clin-
ical evaluation findings after MTBI in the NFL. When the neu-
ropsychological test results confirm the clinical examination, this
is reassuring to the treating physician that his or her diagnosis is
correct. In the rare instances in which the neuropsychological test
results do not correlate with and/or confirm the clinical findings,
this should also serve as a red flag warning to the treating
physician to reevaluate the situation to be certain that no other
abnormal processes are occurring. Thus, neuropsychological test-
ing should be a part of the evaluation of athletes after MTBI.
On the basis of the results of this study, the MTBI Committee
makes the following recommendations regarding the proper role of
neuropsychological testing in the NFL. Neuropsychological testing
is a tool that can assist the physician in evaluating and managing
MTBI. It is definitely not to be used in isolation and cannot and
should not be used to replace the clinical judgment of the treating
physician in the diagnosis and management of MTBI. The main
value of neuropsychological testing in this setting is its ability to
confirm and corroborate the results of the clinical and mental status
evaluation. Just as a normal clinical neurological examination (ex-
cluding mental status) helps confirm the diagnosis of MTBI, so
neuropsychological test results consistent with the clinical findings
help confirm the clinical impression regarding the presence or ab-
sence of cognitive and/or memory impairments.
Similarly, just as the occasional focal abnormality on the neu-
rological examination alerts the physician to the possibility of a
more serious underlying intracranial pathological condition, so
the occasional neuropsychological test result that is incongruous
with the clinical findings should alert the physician to investigate
further. Neuropsychological testing is also useful because it pro-
vides quantifiable results regarding neurocognitive processes.
These “objective” data can help both physicians and athletes
realize the validity and the organic basis of cognitive and mem-
ory dysfunction after MTBI. It is also sometimes easier to track
recovery using such quantifiable results as those of purely clin-
ical examinations. The Committee believes that neuropsycholog-
ical testing can be of value for the treating physician when used
in the appropriate manner outlined above.
The Neuropsychological Test Battery
It is apparent that some of the tests in the NFL test battery
are more effective in discriminating NFL concussions than
others. This is similar to what has been found in other studies
using the NFL test battery on collegiate and high school player
samples (10, 26, 30). Future studies in the NFL will continue to
implement tests that reflect the “state of the art” and have high
sensitivity in detecting subtle brain injuries. As this refinement
occurs, there will be attention to using tests with more reli-
ability and fewer practice effects.
Although regarded as being sensitive to MTBI, tests such as the
Trail Making and the Symbol Digit Modalities may eventually be
replaced by tests that are less contaminated by practice effects when
used on multiple occasions. Specifically, tests should be used that
have high sensitivity to subtle deviations in neurocognitive pro-
cesses, while also being reliable across multiple evaluation sessions.
As Figure 1 demonstrates, efforts are needed to reduce the normal
deviation in responses for baseline evaluations and injured players
to better differentiate the effects of injury. The goal of developing
increasingly reliable and sensitive neuropsychological tests has been
the impetus for the development of computer-based testing proto-
cols over the past 5 years.
Transition to Computer-based
Neuropsychological Testing
Although the program described in this article represents
the standard for neuropsychological testing within the NFL
during 1996 to 2001, many professional football teams are
currently transitioning to a computer-based neuropsycholog-
ical testing protocol. Approximately two-thirds of NFL teams
are now using computer-based testing. Computer-based neu-
ropsychological testing has become increasingly popular, for
several reasons. First, computer-based testing evaluates many
athletes with limited manpower (29). This has increased the
participation in baseline testing throughout the league. Be-
cause the number of athletes evaluated depends only on the
number of computers available, teams can potentially com-
plete baseline testing during a 1-day session. Second, com-
puter testing provides a more effective evaluation of certain
neurocognitive areas, such as reaction time, which can be
measured to within 0.01 second. In addition, the computer
ensures uniformity of administration across teams and elimi-
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1301
nates differences in administration because of human error or
differences in administrator style. There is also some evidence
that computer testing may be less subject to the vagaries of
practice effects than standard pen-and-paper testing (20).
Limitations of the Study
This research is an observational study of NFL players within
the context of six seasons. The neuropsychology program was
designed to operate with minimum disruption to the NFL teams
and athletes. The study relied on a convenience sample of injured
athletes without assurance that the sample is representative of
the overall population of injured NFL athletes from 1996 to 2001.
Furthermore, because this study was initiated with only a few
participating teams in 1996 and the use of neuropsychological
testing became more accepted toward the end of the study pe-
riod, the overall number of participating athletes is relatively
small compared with the 650 injured athletes that made up the
MTBI sample (33). This somewhat limited sample size led to the
use of completion of multiple individual ANOVAs rather than a
more elegant multiple ANOVA design that is usually preferable
in studies with multiple dependent variables.
Currently, all NFL teams are using neuropsychological testing,
but not all use it in the same manner. Some teams perform baseline
testing on all players, whereas others select baseline tests to be
performed on players in higher-risk positions or players who have
a history of previous concussion in high school and/or college. As
a result, the overall number of NFL players participating in the
neuropsychological testing program is much higher at present than
it was during the early years of this study. The lack of completed
baseline assessments in all athletes resulted in smaller sample sizes,
which limits the statistical power of some of the analyses completed
in this study. The authors expect that future studies on this much
larger database will yield more information on the neurocognitive
recovery process after concussion.
All neuropsychological testing was conducted by trained neuro-
psychology personnel using standardized administration protocols
and test scoring rules. Test results are thought to accurately reflect
the athlete’s level of performance on any given test. However, as is
true of any study involving multiple personnel, it is possible that
scores may have differed somewhat from team to team, thereby
resulting in a variation in test scores. The future use of computer-
based neuropsychological protocols should minimize this factor.
Along similar lines, the diagnosis of injury on the field was based on
the clinical judgment of the team medical staff and did not neces-
sarily involve the use of standardized on-field mental status exam-
inations. Therefore, the diagnosis of memory impairment or other
cognitive difficulties may have varied from team to team.
Another potential limitation of this study is the lack of
established exact standards for the determination of abnormal
test performance. If preseason baseline testing had been com-
pleted on a particular player, abnormal performance was de-
termined by comparison of the athlete’s postinjury score to
this baseline. If the score was poorer than baseline levels, the
score was determined to be “abnormal.” If a preinjury baseline
test battery had not been completed, the athlete’s score was
compared with existing norms. Although this represented the
standard for analysis of neuropsychological test scores in the
NFL in the 1990s, more recently, Reliable Change Index (RCI)
scores have evolved to provide a more objective determination
of abnormal performance (3, 20). RCIs are based on the test-
retest reliability of the particular test and adjust for practice
effects. RCIs provide confidence intervals for determining
whether or not a test score is reliably different from baseline.
RCIs have not been calculated for professional football ath-
letes, although they do exist for high school (3, 20) and pro-
fessional rugby athletes (18). As the NFL neuropsychology
program develops, the implementation of RCIs of other sta-
tistical standards will aid in future clinical interpretations.
DISCLOSURE
None of the Committee members have a financial or business
relationship posing a conflict of interest to the research con-
ducted on concussion in professional football. MRL has no finan-
cial interest in the neuropsychological tests used in this study but
does have a financial interest in the ImPACT computer-based
neuropsychological test battery used by many NFL teams.
REFERENCES
1. Aldrich EF, Eisenberg HM, Saydjaric C, Luerssen TA, Foulkes MA, Jane J,
Marshall LF, Marmarous A, Young HF: Diffuse brain swelling in severely
head injured children: A report from the NIH traumatic coma bank.
J Neurosurg 76:450–454, 1992.
2. Aubry M, Cantu RC, Dvorak J, Graf-Baumann T, Johnston K, Kelly J, Lovell
MR, McCrory P, Meuwisse W, Schamasch P: Summary and agreement
statement of the 1st International Conference on Concussion in Sports,
Vienna, 2001. Clin J Sports Med 12:6–11, 2002.
3. Barr WB: Neuropsychological testing of high school athletes: Preliminary
norms and test-retest indices. Arch Clin Neuropsychol 18:91–101, 2003.
4. Barth J, Alves W, Ryan TV, Macciocchi SN, Rimel RW, Jane JA, Nelson WE:
Mild head injury in sports: Neuropsychological sequelae and recovery of
function, in Levin H, Eisenberg H, Benton A (eds): Mild Head Injury. New
York, Oxford University Press, 1989.
5. Benedict RH: Brief Visuospatial Memory Test: Revised. Odessa, Psychological
Assessment Resources, Inc., 1997.
6. Benton A, Hamsher K: Multilingual Aphasia Examination. Iowa City, Univer-
sity of Iowa Press, 1978.
7. Brandt J: The Hopkins Verbal Learning Test: Development of a new memory
test with six equivalent forms. Clin Neuropsychol 5:125–142, 1991.
8. Cantu RC: Second impact syndrome. Clin Sports Med 17:37–44, 1998.
9. Casson IR, Siegel O, Sham R, Cambell E, Tarlau M, DiDomenico A: Brain
damage in modern boxers. JAMA 251:2663–2667, 1984.
10. Collins MW, Grindel S, Lovell MR, Dede DE, Moser DJ, Phalin BR, Nogle S,
Wasik M, Cordry D, Klotz-Daugherty M, Sears SF, Nicolette G, Indelicato P,
McKeag D: Relationship between MTBI and neuropsychological perfor-
mance in college football players. JAMA 282:964–970, 1999.
11. Collins MW, Iverson GL, Lovell MR, McKeag DB, Norwig J, Maroon J:
On-field predictors of neuropsychological and symptom, deficit following
sports-related concussion. Clin J Sports Med 13:222–229, 2003.
12. Echemendia R, Putukian M, Mackin S, Julian L, Shoss N: Neuropsycholog-
ical test performance prior to and following sports related mild traumatic
brain injury. Clin J Sports Med 11:23–31, 2001.
13. Erlanger D, Kaushik T, Cantu RC, Barth JT, Broshek DK, Freeman JR, Webbe
FA: Symptom-based assessment of the severity of concussion. J Neurosurg
98:477–484, 2003.
PELLMAN ET AL.
1302 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
14. Field M, Collins MW, Lovell MR, Maroon J: Does age play a role in recovery
from sports-related concussion? A comparison of high school and college
athletes. J Pediatr 142:546–553, 2003.
15. Grindel SH, Lovell MR, Collins MW: The assessment of sports-related
concussion: The evidence behind neuropsychological testing and manage-
ment. Clin J Sports Med 11:134–143, 2001.
16. Guskiewicz KM, McCrea M, Marshall SW, Cantu RC, Randolf C, Barr W, Onate
JA, Kelly JP: Cumulative effects associated with concussion in collegiate football
players: The NCAA concussion study. JAMA 290:2549–2555, 2003.
17. Hinton-Bayre AD, Geffen GM, Geffen LB, McFarland KA, Friis P: Concus-
sion in contact sports: reliable change indices of impairment and recovery.
J Clin Exp Neuropsychol 21:70–86, 1999.
18. Hinton-Bayre AD, Geffen GM, McFarland KA: Mild head injury and speed
of information processing: A prospective study of professional rugby play-
ers. J Clin Exp Neuropsychol 19:275–289, 1997.
19. Hovda DA, Prins M, Becker DP, Lee S, Bergsneider M, Martin N: Neurobi-
ology of MTBI, in Bailes J, Lovell MR, Maroon JC (eds): Sports-related MTBI.
St. Louis, Quality Medical Publishers, 1998, pp 12–51.
20. Iverson GL, Lovell MR, Collins MW: Interpreting change on ImPACT fol-
lowing sports concussion. Clin Neuropsychol 17:460–467, 2003.
21. Lang DA, Teasdale GM, Macpherson P, Lawrence A: Diffuse brain swelling
after head injury more often malignant in adults than children? J Neurosurg
80:675–680, 1994.
22. Lezak MD: Neuropsychological Assessment. New York, Oxford University
Press, 1995, ed 3.
23. Lovell MR: Evaluation of the professional athlete. Presented at the First
Annual Sports Related Mild Traumatic Brain Injury (MTBI) Conference,
Pittsburgh, Pennsylvania, 1996.
24. Lovell MR: Evaluation of the professional athlete, in Bailes JE, Lovell MR,
Maroon JC (eds): Sports-related MTBI. St. Louis, Quality Medical Publishers,
1999, pp 200–214.
25. Lovell MR, Burke CJ: Neuropsychological testing in ice hockey, in Cantu R
(ed): Neurological Disorders of the Head and Spine. Philadelphia, W.B. Saunders
Co., 2000, pp 109–116.
26. Lovell MR, Collins MW: Neuropsychological assessment of the college
football player. J Head Trauma Rehabil 13:9–26, 1998.
27. Lovell MR, Collins MW, Iverson GL, Field M, Maroon JC, Cantu R, Podell K,
Powell JW, Belza M, Fu FH: Recovery from mild MTBI in high school
athletes. J Neurosurg 98:296–301, 2003.
28. Lovell MR, Collins MW, Iverson GL, Johnston KM, Bradley J: Grade 1 or
“ding” concussions in high school athletes. Am J Sports Med 32:1–8, 2004.
29. Maroon JC, Lovell MR, Norwig J, Podell K, Powell JW, Hartl R: Cerebral
concussion in athletes: Evaluation and neuropsychological testing. Neuro-
surgery 47:659–672, 2000.
30. McCrea M, Guskiewicz KM, Marshall SW, Barr W, Randolf C, Cantu RC,
Onate JA, Young J, Kelly JP: Acute effects and recovery time following
concussion in collegiate football players: The NCAA Concussion Study.
JAMA 290:2556–2563, 2003.
31. Olvey S: Concussion in motor sports. Presented at the New Developments in
Sports-related Concussion Conference, Pittsburgh, Pennsylvania, July, 2002.
32. Pellman EJ: Commentary: Background on the National Football League’s re-
search on concussion in professional football. Neurosurgery 53:797–798, 2003.
33. Pellman EJ, Powell JW, Viano DC, Casson IR, Tucker AM, Feuer H, Lovell
M, Waeckerle JF, Robertson DW: Concussion in professional football: Epi-
demiological features of game injuries and review of the literature—Part 3.
Neurosurgery 54:81–97, 2004.
34. Pellman EJ, Viano DC, Casson IR, Arfken C, Powell J: Concussion in pro-
fessional football: Injuries involving 7 or more days out—Part 5. Neuro-
surgery 55:1100–1119, 2004.
35. Pellman EJ, Viano DC, Casson IR, Tucker AM, Waeckerle JF, Powell JW,
Feuer H: Concussion in professional football: Repeat injuries—Part 4. Neu-
rosurgery 55:860–876, 2004.
36. Pellman EJ, Viano DC, Tucker AM, Casson IR, Waeckerle JF: Concussion in
professional football: Reconstruction of game impacts and injuries. Neuro-
surgery 53:799–814, 2003.
37. Pellman EJ, Viano DC, Tucker AM, Casson IR, Waeckerle JF: Concussion in
professional football: Location and direction of helmet impacts—Part 2.
Neurosurgery 53:1328–1341, 2003.
38. Reitan R: Validity of the Trail Making Test as an indicator of organic brain
damage. Perceptual Motor Skills 8:271–276, 1958.
39. Ross RJ, Cole M, Thompson JS, Kim KH: Boxers: Computed tomography,
EEG and neurological evaluation. JAMA 249:211–213, 1983.
40. Russell WR, Smith A: Post-traumatic amnesia after closed head injury. Arch
Neurol 5:16–29, 1961.
41. Smith A: The Symbol Digit Modalities Test: Revised Version. Los Angeles,
Western Psychological Services, 1982.
42. Statistica. Tulsa, Statsoft, Inc., 2001.
43. Symonds CP: Concussion and its sequelae. Lancet 1:1–5, 1962.
44. Wechsler D: Wechsler Adult Intelligence Scale-III. San Antonio, The Psycho-
logical Corporation, 1997.
45. Yarnell PR, Lynch S: Retrograde memory immediately after concussion.
Lancet 1:863–864, 1970.
Acknowledgments
The NFL has a Committee on MTBI. It is chaired by Dr. Elliot Pellman and
includes representatives from the NFL Team Physicians Society, NFL Athletic
Trainers Society, NFL equipment managers, and scientific experts in the area of
traumatic brain injury, biomechanics, basic science research, and epidemiology.
The authors of this article are members of the Committee. The efforts of other
Committee members are gratefully acknowledged, including John Powell,
Ph.D., Henry Feuer, M.D., Douglas Robertson, M.D., Joseph Waeckerle, M.D.,
Ronnie Barnes, ATC, and Jay Brunetti.
The MTBI Committee gratefully acknowledges the insights of the Commissioner,
Paul Tagliabue, for forming the Committee and issuing a charge to scientifically inves-
tigate concussion and means to reduce injury risks in football. The encouragement and
support from the NFL’s Jeff Pash and Peter Hadhazy is also appreciated. The Com-
mittee appreciates the contributions of all of the NFL team neuropsychologists, includ-
ing William Barr, Don Gerber, Thomas Hardey, Gary Solomon, Thomas Sullivan, C.
Munro Cullum, Ralph Benedict, David Coppel, Dan Blackwell, Robert Fucetola,
Kenneth Podell, William Hitch, Steven Morgan, William Black, Robert Heaton, Kathy
Knee, William Stier, John Woodard, Richard Naugle, Mark Kelly, and Phillip Fastenau.
The Committee also commends the NFL players who consented to participate in the
study through a blinded identification in the database. The Committee appreciates the
assistance of Cynthia Arfken, Ph.D., from the Wayne State University Department of
Psychiatry and Behavioral Neurosciences, Center for Healthcare Effectiveness Research,
in the statistical analysis of the neuropsychological data. Funding for this research was
provided by the NFL and NFL Charities. The Charities is funded by the NFL Players’
Association and League. Their support and encouragement to conduct research on
concussion is greatly appreciated.
COMMENTS
This study by Pellman et al. analyzes the neuropsychological
testing profile of 143 National Football League (NFL) players
who experienced concussions, out of a total of 650 athletes. An
additional analysis was performed on four subgroups consisting of
the postconcussion performance compared with preseason baseline
assessments in 95 athletes. They assessed the neuropsychological
test performance of 70 athletes in relation to the presence of on-field
amnesia, the potential role of multiple concussions on neuropsycho-
logical test performance in 70 athletes, and the neuropsychological
testing profile of 18 players who lost 1 week or more of playing time
compared with 71 who returned to play within the same week.
Their findings showed that the presence of cognitive and memory
abnormalities correlate with neuropsychological test impairment,
although they point out the limitations of neuropsychological test-
ing. This includes particularly the lack of established standards, the
use of neuropsychological testing being uniform among all teams
and neuropsychologists, and the occurrence of practice effects. Neu-
ropsychological testing has been established as a very valuable
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1303
ancillary test to help objectify the extent of injury and recovery in
athletic mild traumatic brain injury (MTBI). The authors postulate
an interesting hypothesis that the younger athletes sustain more
serious MTBIs; however, I do not believe that this has yet been
proved. In addition, I do not believe that this study, with correlation
between clinical and neuropsychological evaluation, proves that
there are no widespread permanent or cumulative effects of single
or multiple MTBI in NFL players. I think that it is premature to
conclude that there are no long-term consequences of MTBI in
football while players are still active, for many reasons. Further
studies using neuropsychological testing, balance testing, neuroim-
aging, metabolic studies, functional magnetic resonance imaging, or
other ancillary objective measures will be welcome as we attempt to
understand this phenomenon further.
Julian E. Bailes
Morgantown, West Virginia
In this sixth in a series of articles on MTBI in NFL players,
neuropsychological data from concussed players were ana-
lyzed. The sample included 143 athletes, representing 22% of
650 players who experienced 887 concussions during six sea-
sons. Although the majority of players showed rapid recovery
after concussion, a subset with on-field memory disturbance
showed persistent memory problems within 2 days of the
concussive event. The authors conclude that MTBI is associ-
ated with a rapid return to normal functioning in the great
majority of players. The findings are somewhat surprising, but
at first glance, they seem encouraging for the athletes in terms
of short-term neuropsychological risk from concussion. How-
ever, these results should be interpreted with caution. Further
follow-up of players sustaining MTBI is needed to better de-
termine the cumulative effect of multiple concussions. In ad-
dition, it would be important to better identify specific players
at risk or particular injury mechanisms that result in more
severe concussions with on-field memory loss. Finally, as the
authors acknowledge, a major shortcoming of this study is
that a relatively small sample size of concussed athletes, only
22%, actually underwent neuropsychological testing, and par-
ticipation was voluntary. Why did 78% of concussed players
choose not to participate? This study design creates potential
for significant sampling bias. The authors are encouraged to
continue and expand this important work to better character-
ize the short- and long-term consequences of MTBI in profes-
sional athletes.
Daniel F. Kelly
Los Angeles, California
This study addresses important questions and uses a highly
interesting group of subjects; in short, it is an irresistible
read. It also is ambitious, because studies based on an absence
of findings (“absence of proof is proof of absence”) are inher-
ently difficult and controversial.
An article’s experimental sample defines the outer bound-
aries of what that article can find, conclude, and recommend.
It is perplexing that the authors chose to include athletes 1 to
10 days after injury in their MTBI group. As the authors note
in the introduction of the article, previous studies with the
same test battery have found recovery “within a week.” The
authors’ choice to include subjects more than 7 days after
injury, therefore, increases the likelihood that their experimen-
tal group will contain subjects who have already recovered.
This would dilute the overall cognitive symptoms of the ex-
perimental group and increase the variability within it, both of
which make it more difficult to obtain statistically significant
findings.
Most of the conclusions and recommendations of this article
are based on statistical analyses in which no differences were
found. Therefore, methodological choices that decrease the
likelihood of obtaining statistically significant differences are
worrisome. In the present study, they also are perplexing. As
the authors note, the MTBI group’s range of days after injury
was 1 to 10, with a mean of 1.40 days and a standard deviation
of 1.29 days. This indicates that, even if the experimental
sample had been defined to include subjects as far as 2 stan-
dard deviations from the mean, most subjects would have
been 4 days or less after injury, and that subjects at the 10-day
end of the range almost certainly are outliers. The authors’
data thus demonstrate that they had a large corpus of subjects
within 4 days after injury. Why not use this group? Or why
not apply their own understanding of the literature and use a
group 7 days or less after injury? The authors’ choice to dilute
their experimental sample by including outliers weakens the
impact of their findings, conclusions, and recommendations.
The authors possess a remarkable data set. My strongest
impression after reading the article was that the data set was
so important that it deserved additional analysis and that a
good place to start would be to remove the outliers and see the
results.
Joseph Bleiberg
Neuropsychologist
Washington, District of Columbia
This article provides a review of the use of neuropsycholog-
ical testing in the NFL from the inception of the program in
the mid 1990s to the present time. This is an important article,
because the NFL’s neuropsychological testing program has
become an important model for the neuropsychological as-
sessment of athletes in sports.
Overall, the article is very well written, provides an excellent
review of research in the area, and makes an important contri-
bution to the literature. In addition, the article provides norma-
tive values that have never before been published. This will
provide useful data for the interpretation of neuropsychological
test results in professional football players and may also help in
the interpretation of emerging data in other professional athletes
(e.g., ice hockey, rugby, and automobile racing). The article also
presents new data from subsamples of injured NFL athletes to
investigate the following research questions. 1) Do concussed
NFL athletes display abnormal test results when tested days after
injury? 2) Does postinjury neuropsychological dysfunction relate
PELLMAN ET AL.
1304 | VOLUME 55 | NUMBER 6 | DECEMBER 2004 www.neurosurgery-online.com
to on-field memory dysfunction? 3) Is there evidence of the
cumulative effect of injury in professional athletes? and 4) Do
athletes who miss 7⫹days after MTBI perform more poorly on
testing? These are all extremely interesting questions that have
been addressed to some extent by previous articles in younger
athletes. However, this is the first article to attempt to address
these issues in a group of professional athletes.
The strength of this article is its ambitious attempt to address
multiple aspects of the concussion issue. To answer these re-
search questions, the article provides analyses of several different
subsamples of athletes. Although some of these groups are quite
large (e.g., the baseline group of more than 600 athletes) and
provide a very sound statistical foundation for analysis of the
data, the analyses of the 3⫹concussions and 7⫹days concussion
groups are based on relatively small sample sizes. Therefore,
although extremely interesting and worthy of discussion, the
conclusions based on these analyses should be tempered some-
what, particularly in the Discussion section. For example, the
authors seem to imply that the data reported in this article
unequivocally indicate that there is diminished risk of long-term
injury in professional athletes relative to younger players. Al-
though this may turn out to be true in the long run, this study, in
and of itself, does not allow a strong conclusion regarding this
issue. Clearly, further research is needed in this area. It is specif-
ically recommended that the statement that there are no wide-
spread permanent or cumulative effects of single or multiple
MTBIs in professional football players be softened somewhat.
The article makes the important point that neuropsychological
testing, at least in its present form, does not detect all postconcussive
symptoms, particularly those that may be noncerebral or vestibular
in nature. This point is often overlooked. The best approach to
concussion diagnosis and treatment clearly should involve attention
to both neuropsychological and non-neuropsychological signs and
symptoms. However, the authors seem to suggest that the role of
neuropsychological testing is “minor.” Such a strong statement does
not seem to be justified. Regarding this point, it is important to note
that this present study uses “traditional” testing procedures rather
than computer-based tests that have demonstrated increased sensi-
tivity to concussion. Therefore, any broad generalizations regarding
the limitations of this particular neuropsychological test battery do
not seem to be warranted. This point should be clearly made within
the article.
The comparison of test results on the basis of different samples
of athletes (e.g., high school, college, professional) is particularly
interesting. This represents the first “cross-sample” comparison
of test results across professional and amateur ranks. However,
Figure 1 could be more clearly labeled so that it is clear that
Columns 4 to 8 represent NFL athletes.
Overall, this article presents very interesting new data that
will enhance the overall understanding of concussion. The
authors have drafted an ambitious article.
Joseph C. Maroon
Pittsburgh, Pennsylvania
The NFL’s MTBI Committee began the neuropsychological
testing program several years ago, and we can only hope
that the league and the players’ union will find a way to
mandate (sooner, rather than later) the program for all players
in the league. My understanding is that while nearly all teams
“participate” in the program, not all players on every team
choose to participate. This is one of the obvious problems with
the most recent article by Pellman et al. It is unfortunate that
only 22% of the concussed players (16% of the overall concus-
sions sustained) are represented because of incomplete data.
In fairness to the authors, it is challenging to capture data on
these injured players at consistent time points. Still, the study
is based on only a small subsample of players who experi-
enced MTBI and agreed to participate in the neuropsycholog-
ical testing program. The reader is left wondering about those
concussed players who chose not to participate in the neuro-
psychological testing program. Could there be players with
three or more concussions who were experiencing cognitive
difficulties and whose outcomes would have changed the
findings if included in the data analyses? Thus, a major limi-
tation of the study is that it could involve a biased sample.
In addition to the sampling issues, the authors have overinter-
preted some of their findings. Given the methods and statistical
design used, it is difficult to understand how they can comment
that “the strong correlation between the results of clinical and
neuropsychological evaluations also provides supportive evi-
dence for the position that there is no evidence in this study of
widespread permanent or cumulative effects of single or multi-
ple MTBIs in professional football players.” They only studied
the acute neuropsychological effects of single and repeat cuncus-
sion, and the data presented tell us nothing about potential
“permanent” or long-term complications. The authors cannot
assume that there could not be chronic effects, especially since
they have only looked at a brief window of time.
The authors use of the word “preliminary” at one point
within the text of the article was appropriate, because—at the
very best—it is a preliminary study. “Preliminary” should
have been used in the title and abstract as well. I think that
future studies on this topic will capture a clearer picture of
what is really occurring, given that there is now more wide-
spread use of neuropsychological testing in the NFL. Neuro-
psychological testing is very important in helping physicians
and athletic trainers to manage sport-related concussion.
Hopefully, it can soon be mandated in the NFL so that players
participate in the program; in time, such participation will
lead to more complete data. I hope that the findings will be
consistent with these preliminary findings.
Kevin M. Guskiewicz
Director, Sports Medicine Research Laboratory
University of North Carolina
Chapel Hill, North Carolina
NEUROPSYCHOLOGICAL TESTING AFTER CONCUSSION
NEUROSURGERY VOLUME 55 | NUMBER 6 | DECEMBER 2004 | 1305
