Standard regression-based methods for measuring
recovery after sport-related concussion
MICHAEL McCREA,1,2WILLIAM B. BARR,3KEVIN GUSKIEWICZ,4,5,6
CHRISTOPHER RANDOLPH,7STEPHEN W. MARSHALL,6,8ROBERT CANTU,4,9
JAMES A. ONATE,10and JAMES P. KELLY11
1Neuroscience Center, Waukesha Memorial Hospital, Waukesha
2Department of Neurology, Medical College of Wisconsin, Milwaukee
3Departments of Neurology and Psychiatry, New York University School of Medicine, New York
4Department of Exercise and Sport Science, University of North Carolina at Chapel Hill, Chapel Hill
5Department of Orthopedics, University of North Carolina at Chapel Hill, Chapel Hill
6Injury Prevention Research Center, University of North Carolina at Chapel Hill, Chapel Hill
7Department of Neurology, Loyola University Medical School, Maywood
8Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill
9Neurosurgery Service, Emerson Hospital, Concord
10Department of Rehabilitation Sciences, Athletic Training Program, Sargent College of Health and Rehabilitation Sciences,
Boston University, Boston
11Department of Neurosurgery, University of Colorado Health Sciences Center, Denver
(Received April 12, 2004; Revised July 29, 2004; Accepted August 30, 2004)
Clinical decision making about an athlete’s return to competition after concussion is hampered by a lack of systematic
methods to measure recovery. We applied standard regression-based methods to statistically measure individual rates
of impairment at several time points after concussion in college football players. Postconcussive symptoms, cognitive
functioning, and balance were assessed in 94 players with concussion (based onAmericanAcademy of Neurology
Criteria) and 56 noninjured controls during preseason baseline testing, and immediately, 3 hr, and 1, 2, 3, 5, and 7 days
postinjury. Ninety-five percent of injured players exhibited acute concussion symptoms and impairment on cognitive
or balance testing immediately after injury, which diminished to 4% who reported elevated symptoms on postinjury
day 7. In addition, a small but clinically significant percentage of players who reported being symptom free by day 2
continued to be classified as impaired on the basis of objective balance and cognitive testing. These data suggest that
neuropsychological testing may be of incremental utility to subjective symptom checklists in identifying the residual
effects of sport-related concussion. The implementation of neuropsychological testing to detect subtle cognitive
impairment is most useful once postconcussive symptoms have resolved. This management model is also supported
by practical and other methodological considerations. (JINS, 2005, 11, 58–69.)
Keywords: Brain injury, Brain concussion, Athletic injuries
public health concern in the United States and worldwide
(Aubry et al., 2002; Collins et al., 1999; Kelly, 1999). The
Centers for Disease Control and Prevention report that
approximately 300,000 sport-related concussions occur
annually in the United States (Thurman et al., 1998), and
recent data from the National Collegiate Athletic Associa-
tion (NCAA) Injury Surveillance System reveal that con-
cussion is amongst the most frequently observed injuries in
collegiate ice hockey, football, and soccer (Dick, 2003).
Epidemiological and prospective clinical studies estimate
that approximately 3% to 8% of high school and collegiate
football players sustain a concussion each season (Collins
et al., 1999; Echemendia et al., 2001; Guskiewicz et al.,
2000, 2003; Macciocchi et al., 1996; McCrea et al., 1997,
1998, 2002; Powell & Barber-Foss, 1999), and the rate of
documented concussion in collegiate football has been on
the rise in recent years (Covassin et al., 2003; Dick, 2003).
tor, Neuroscience Center, Waukesha Memorial Hospital, 721 American
Journal of the International Neuropsychological Society (2005), 11, 58–69.
Copyright © 2005 INS. Published by Cambridge University Press. Printed in the USA.
Sports medicine professionals consider diagnosing con-
cussion and determining postinjury recovery among their
intense pressure to formulate a rapid assessment of injury
and prompt decision about how soon an athlete can safely
return to competition (Cantu, 1986; Kelly, 1999; Vastag,
2002). Despite a growing body of research, there remains
little evidence-based guidance on how long it takes for an
athlete to recover and deciding when it is safe for the ath-
lete to return to play after a concussion. Existing guidelines
largely stem from expert consensus, with limited scientific
support from prospective studies (Aubry et al., 2002; Kelly,
1999; Vastag, 2002).As a result, sports medicine clinicians
have historically relied on their experience and subjective
observations to track postinjury recovery and guide clinical
decision making about return-to-play.
Neuropsychologists have made a significant contribu-
tion to advancing the scientific understanding of the effects
and expected recovery course after sport-related concus-
sion (Echemendia & Julian, 2001). Several neuropsycho-
logical studies have reported estimates of recovery in
symptoms and cognitive dysfunction ranging from several
hours to several days (Collins et al., 1999, 2003; Echemen-
dia et al., 2001; Erlanger et al., 2003; Hinton-Bayre et al.,
1999; Lovell et al., 2003; Macciocchi et al., 1996; Mad-
docks & Saling, 1996; McCrea, 2001; McCrea et al., 1998,
2002). Interpretation of recovery data across studies has
been hampered by a number of methodological factors,
however, including small sample sizes, varied definitions
of concussion, absence of an immediate injury assessment
to measure the most acute effects, limited follow-up assess-
ment of injured players, failure to establish a recovery
endpoint, and lack of an appropriate control group. The
findings from a recent 3-year prospective study demon-
strated that collegiate football players, on average, fol-
lowed a gradual course of recovery in symptoms, cognitive
functioning, and postural stability over the first 5–7 days
after concussion, but that 10% of injured players required
more than 7 days to reach a full recovery (McCrea et al.,
2003). The study focused on analysis of group effects,
limiting the application of the findings to actual clinical
decision making in individual cases.
There has been a growth of recent interest in defining the
sis has been to move beyond analysis of group statistics
toward the detection of change in individual participants.
Much attention has focused on the use of the reliable change
index (RCI) and standardized regression-based (SRB) meth-
ods, which are techniques initially developed and refined in
studies of outcome from psychotherapy and surgical treat-
ment. Conclusions drawn from recent reviews indicate that,
while RCI methods may be easier to use in a clinical set-
ting, SRB methods are demonstrated to be more accurate
for detecting meaningful change as a result of their ability
to correct for practice effects, regression to the mean, and
the impact of baseline test performance . The SRB methods
are thus recommended for use in research settings examin-
ing test–retest changes (Barr, 2002; Temkin et al., 1999).
Techniques to more precisely measure and characterize
meaningful and reliable change in neurocognitive test per-
formance in individual patients over time are especially
intriguing in the case of sports concussion assessment. The
prototypic sports concussion assessment model imple-
mented in many professional, collegiate, and high school
athletic programs incorporates a preseason baseline evalu-
ation of all players that includes a clinical history, base rate
symptom index, neurocognitive battery, and postural stabil-
ity testing. Any player who sustains a concussion over the
course of the season is then reevaluated with these mea-
sures at several time points, initially to determine the sever-
ity of their injury and then track their postinjury recovery.
Clinical research programs also include matched, non-
injured controls in the identical testing protocol. Despite
the methodological advantages of this model, serial testing
of injured athletes presents the clinician with the challenge
of distinguishing between “real” change in individual test
performance indicative of true recovery versus perfor-
mance variability due to practice effects, measurement error,
or random influence.
The current study sponsored by the NCAAapplied a serial
testing paradigm and regression-based methods to statisti-
ment at several time points after concussion in collegiate
A total of 1631 football players from 15 National Colle-
giate Athletic Association (NCAA) Division I, II, and III
member institutions were enrolled in one arm of a larger
cohort study of the effects of sports-related concussion from
1999 through 2001. In total, 2410 player seasons were ana-
lyzed, as 779 players were enrolled for more than 1 year of
the study.Acase series of 94 injured players who sustained
a concussion (5.76% of players, 3.90% of player seasons)
were enrolled in an extensive injury assessment protocol.
No player who sustained a concussion refused to partici-
pate or was excluded from the study protocol, but informa-
tion on unidentified or unreported concussions was not
Fifty-six noninjured controls matched to injured partici-
pants on the basis of age, years of education, team, and
baseline performance on concussion assessment measures
were administered the identical protocol under the same
conditions and retest intervals as injured players during the
ment of additional control participants in years 2 or 3 of the
study, which had a minimal effect on matching characteris-
tics for the complete study sample. Table 1 provides a com-
parison of demographic and medical history data for the
injured and control groups.
Measuring concussion recovery
(IRB) for protection of human research participants at the
host institutions of the principal investigators. All partici-
pants granted written informed consent prior to enrollment
in the study.
All players underwent a preseason baseline evaluation on a
battery of concussion assessment measures and extensive
health questionnaire prior to their first year of participation
in the study protocol by a team physician or certified ath-
letic trainer present on the sideline during an athletic con-
from a blow to the head causing an alteration in mental
status and one or more of the following symptoms pre-
scribed by the American Academy of Neurology Guideline
for Management of Sports Concussion (Practice Parameter,
1997): headache, nausea, vomiting, dizziness0balance prob-
lems, fatigue, trouble sleeping, drowsiness, sensitivity to
light or noise, blurred vision, difficulty remembering, or
difficulty concentrating (Kelly & Rosenberg, 1997). Loss
inability to recall exiting the field, aspects of the examina-
tion, etc.), and retrograde amnesia (RGA) (e.g., in ability to
recall aspects of the play, events prior to injury, score of the
game, etc.) were documented immediately after injury.
All players identified by the team physician or certified
athletic trainer as having sustained a concussion according
tom Checklist (GSC) (Lovell & Collins, 1998), the Stan-
dardized Assessment of Concussion (SAC) (McCrea et al.,
2000), and the Balance Error Scoring System (BESS)
(Guskiewicz et al., 2001) on the sideline immediately fol-
lowing injury. Follow-up testing on these measures was
then conducted postgame0postpractice (2–3 hr after injury),
and again on postinjury days 1, 2, 3, 5, and 7. The brief
neuropsychological test battery (see Table 2) was adminis-
tered to assess neurocognitive functioning at baseline and
on days 2 and 7 postinjury. A day-90 assessment point was
also included in the original design, but significant attrition
for both injured and control participants did not allow appli-
cation of the standard regression-based methods employed
in this study for this assessment point. Assessments were
conducted by certified athletic trainers who were trained by
the researchers and required to watch a training video on
administration and scoring of all outcome measures used in
Main Outcome Measures
to assess postconcussive symptoms, cognitive functioning,
and postural stability is provided in Table 2. Several studies
on the effects of sport-related concussion have demon-
2003), SAC (Barr & McCrea, 2001; McCrea, 2001), BESS
(Guskiewicz et al., 2001; Riemann et al., 1999; Riemann &
ical test battery (Collins et al., 1999) in correctly classifying
injured and noninjured participants after concussion.
There was limited missing data, with 93% of data cells
complete across all time points for all participants.To exam-
ine the potential effect of missing data on the results, we
compared the baseline scores for the missing and nonmiss-
ing participants at every time point for all outcomes. The
baseline scores did not differ between missing and nonmiss-
ing, suggesting that the data was missing at random (Diggle
et al., 1994). As part of our previous analysis (McCrea
et al., 2003), we also estimated the missing data using a
based on time, participants status (injured vs. control), and
Table 1. Concussion and control group characteristics
(n 5 94)
(n 5 56)
Academic year (collegiate)
Self-reported history of:
No. of previous concussions (past 7 years)
Any concussion (lifetime) (%)
Learning disability (%)
.58 .78.39.68 .191.47.145
Notes. *Statistically significant. ADHD 5Attention Deficit Hyperactivity Disorder. LD 5 Learning Disability.
M. McCrea et al.
Table 2. Assessment measure characteristics
MeasureFunctional domain Description Score rangeTime to administer
Graded Symptom Checklist (GSC)Postconcussive symptoms Subject rates presence and
severity of 17 symptoms (e.g.,
headache, dizziness, etc.)
0 (no symptoms)–6 (severe)
Likert scale per item; total
score range: 0–102; higher
score indicates more severe
Total score range: 0–30; lower
score indicates more severe
Standardized Assessment of Concussion (SAC)* Cognitive functioning:
- immediate & delayed memory
Brief neurocognitive assessment
and neurologic screening;
amnesia, retrograde amnesia
Balance Error Scoring System (BESS)Noninstrumented, clinical
assessment of postural stability
in double leg, single leg,
tandem stances on firm and
Hopkins Verbal Learning Test
Trail Making Test Part B
Symbol Digit Modalities Test
Stroop Color-Word Test
Controlled Oral Word
No defined range; test score
equals total number of errors
committed by the participant;
higher score indicates more
severe postural instability
Total score range based on
individual measures; lower
score indicates more severe
impairment except for Trail
Making Test (total time to
Neuropsychological Test Battery* Cognitive functioning:
- processing speed
- mental flexibility
- anterograde memory
*Alternate forms utilized to minimize practice effects practice effects from repeat testing on SAC and neuropsychological test battery.
Measuring concussion recovery
baseline test performance, and obtained essentially identi-
cal results on separate analysis of the nonimputed and
Group recovery data analyses have been published pre-
viously (McCrea et al., 2003), including illustrations of the
natural course of recovery in symptoms, cognitive dysfunc-
tion, and balance problems following concussion. Those
analyses utilized Generalized Estimating Equations (GEE)
to examine the adjusted group mean differences on each
assessment measure at each assessment point, while con-
to influence performance on specific measures (e.g., years
of education on cognitive testing).
The current data analysis focused on individual rates of
impairment, rather than generating group recovery curves
on each of the study’s main assessment measures. Classifi-
cation of impairment at the individual case level was based
on an empirical method using SRB indices for detection of
significant change in test scores (McSweeny et al., 1993;
Temkin et al., 1999). This method uses linear regression on
baseline scores from the healthy control group to generate a
formula for predicting follow-up scores at various time
points. The resulting regression coefficient and the inter-
cept of the regression line were used with the baseline score
to compute a predicted score for each participant at Time 2
and at subsequent testing points. This approach provides an
empirical method for detecting meaningful change while
also providing correction for practice effects and regression
to the mean.
Participants were considered to have undergone a mean-
ingful change in test performance if the difference between
the obtained and predicted score, divided by the standard
error of prediction, was larger than a specific criterion value,
translated to a 90% confidence interval (two-tailed, 5%
chance of Type I error). Predictions for the GSC, BESS,
and SAC were computed for all time points from the time
of injury through day 7. Predicted scores for each of the
neuropsychological tests were computed for days 2 and 7.
Based on the conventional standard within neuropsychol-
ogy and to minimize the rate of false positives, impairment
on the neuropsychological test battery was defined as hav-
ing significantly decreased scores on two or more tests.
come measures, which is not likely the case due to shared
group variance across measures, the expected rates of false
positives (identifying a normal participant as impaired)
ranged from 5% for a single measure to 15% for impair-
ment on any one of three measures (e.g., impairment on the
brief battery of the GSC, BESS, and SAC), and ranged up
to 27.3% when adding the criterion of impairment on two
of the seven measures in the neuropsychological test battery.
Frequencies were also generated for the percentage of
participants who had reached a full symptom recovery based
on GSC score, but continued to show impairment on the
BESS, SAC, or the neuropsychological test battery on post-
injury days 2 and 7, based on the respective SRB indices
for each test.
Measures of sensitivity and specificity were computed
for establishing each test’s ability to distinguish between
the injured and control groups. In this context, sensitivity
(Se) refers to the probability that an injured participant
will be identified as “abnormal” by a change in test per-
formance. At time points subsequent to time of injury,
sensitivity values indicate the probability that a player orig-
inally injured continued to be classified as “abnormal”
according to at least one of the test measures. Specificity
(Sp) refers to the probability that a control participant will
be correctly classified as “normal” using the same method.
Data were analyzed with SPSS 11.0 statistical software
Ninety-four players who sustained a concussion during foot-
ball practice (56.8% of injuries) or games (43.2%) were
studied. Most injuries were classified as either Grade 1 or 2
concussions according to the Cantu (Cantu, 1998) (98.6%),
Colorado (Colorado Medical Society, 1991) (93.3%), and
hoc review of injury characteristics. A small number of
injured participants sustained LOC (6.4%; median duration
30 s).Additionally, a small percentage of participants exhib-
ited PTA (19.1%; median duration 90 min) or RGA (7.4%;
median duration 120 min). There was no LOC, PTA, or
RGAassociated with most injuries (77.8%). Ninety injured
participants (96%) completed the assessment protocol
through the day 7 assessment point.
Results from group (concussion vs. control) data analy-
ses on the GSC, BESS, SAC, and neuropsychological test
battery have been reported previously, including illustra-
tions of the pattern of recovery in symptoms, cognitive func-
tioning, and postural stability (McCrea et al., 2003). As
context for interpretation of the current results from the
SRB analysis, raw means for the concussion and control
groups on the GSC, BESS, and SAC at baseline and each
postinjury assessment point are provided in Table 3, and
group data from the neuropsychological test battery are pro-
vided in Table 4.
Results from the current analyses focus on the rate of
recovery by individual players with concussion. The per-
centages of injured participants and controls impaired at
each assessment point on the GSC, SAC, BESS, and neuro-
psychological test battery, as defined for each test accord-
ing to the SRB calculation, are presented in Figure 1. During
the acute postinjury period, the highest rate of abnormality
was observed on the GSC, with 89% of the sample report-
ing symptoms 1 day later. There was a gradual decline in
the percentage of participants with elevated GSC scores on
ensuing days, with only 4% reporting significant symptoms
1 week after concussion. The base rate of common concus-
sive symptoms in the control group was zero across all
M. McCrea et al.
Acute cognitive dysfunction, as measured by impairment
on the SAC, was evident in 80% of the injured sample at the
on the SAC in 31% of the injured sample on day 1, 23% on
tically defined abnormality on the SAC ranged from only
more measures on postinjury day 2 and 17% were impaired
indicate that the largest percentage of injured participants
obtained abnormal test scores on measures of delayed recall
nition), cognitive processing speed (Trails B, Symbol Digit
Modalities Test), and verbal fluency (COWAT).
Impairments in postural stability, as defined by poorer
performance on the BESS, were seen in 36% of the injured
participants immediately following concussion, compared
to 5% of the control group. Twenty-four percent of injured
participants remained impaired on the BESS on day 2, com-
pared to 9% by day 7 postinjury. Table 5 presents the rates
of impairment in concussion and control groups for the
composite battery of brief measures (GSC, BESS, SAC) at
all time points, and the addition of neuropsychological test-
ing at postinjury days 2 and 7.
Sensitivity and specificity values are provided for each
measure in Table 6. The results indicate that the GSC pro-
vided the most sensitive (Se5.89) and specific (Sp51.00)
measure of abnormality at the time of injury. The specific-
ity values remained at 1.00 at each time point thereafter,
indicating that none of the controls exhibited a significant
increase in self-reported symptoms at any time point. Sen-
sitivity values for the BESS were highest at the time of
injury (Se 5 .34). Specificity values for this instrument
ranged from .91 to .97 across the various time points. A
similar pattern of data was obtained with the SAC, with a
peak sensitivity value of .80 at the time of injury and spec-
ificity values ranging from .89 to .98 through day 7. The
neuropsychological test battery classified injured partici-
pants with sensitivity values of .23 and .19 at days 2 and 7,
respectively. Specificity values were .93 and .91.
Table 3. GSC, SAC, and BESS data for concussion and control groups at baseline and postinjury assessment points
Concussion ControlConcussion Control ConcussionControl
Mean SDMean SD MeanSD MeanSD MeanSDMean SD
Time of concussion
GSC 5 Graded Symptom Checklist (Lovell & Collins, 1998); SAC 5 Standardized Assessment of Concussion (McCrea et al., 2000); BESS 5 Balance
Error Scoring System (Guskiewicz et al., 2001).
Table 4. Neuropsychological test data for concussion and control groups at baseline and postinjury days 2 and 7
BaselineDay 2Day 7
Concussion ControlConcussion ControlConcussion Control
Mean SDMean SDMean SDMean SDMean SDMean SD
HVLT Immediate Memory
HVLT Delayed Recall
Trail Making Test, Part B
Stroop CW Trial
HVLT 5 Hopkins Verbal Learning Test (Shapiro et al., 1999); Trail Making Test (Reitan & Wolfson, 1985); SDMT 5 Symbol Digit Modalities Test
(Smith, 1991); Stroop CW Trial 5 Color Word Trial (Golden, 1978); COWAT 5 Controlled Oral Word Association Test (Benton et al., 1983).
Measuring concussion recovery
An examination was made of sensitivity and specificity
values for the entire battery of brief measures, defined as an
abnormal score identified on either the GCS, BESS, or SAC.
Again, sensitivity was highest at the time of injury, with
94% accuracy in classifying injured participants on the bat-
tery of brief measures. Specificity values ranged from .84
to .93 across the various time points. Inclusion of the neuro-
psychological test data barely increased the sensitivity of
the battery from .51 to .56 at day 2, with a decrease in
specificity from .84 to .79. However, the neuropsychologi-
cal test data more than doubled the sensitivity of the battery
from .14 to .30 at day 7, accompanied by a modest increase
in specificity from .79 to .86.
A small but clinically significant percentage of asymp-
tomatic players continued to show impairment on the BESS,
SAC, and neuropsychological testing on postinjury days 2
Fig. 1. Percentage of concussion and control participants classified as “impaired” from time of injury through day 7 on
GSC, BESS, SAC and Neuropsychological Test Battery. GSC 5 Graded Symptom Checklist; BESS 5 Balance Error
Scoring System; SAC5Standardized Assessment of Concussion; and NP Battery5Neuropsychological Test Battery.
Assessment points: CC5time of concussion; PG5post-game0post-practice; D15postinjury day 1, D25postinjury
day 2, etc.
M. McCrea et al.
ing to be completely symptom free were impaired on the
BESS on day 2, with fewer impaired on cognitive mea-
sures. On day 7, the highest percentage of asymptomatic
players were impaired on neuropsychological testing, while
rates of impairment on the BESS and SAC for asymptom-
atic players were comparable to the control group.
ods combined with a baseline and serial testing paradigm to
statistically measure individual rates of cognitive and func-
tional impairment in collegiate football players after sus-
reported an increase from their baseline rate of common
from baseline cognitive performance during the acute post-
injury period. A significant percentage of concussed play-
ers also exhibited acute impairment on postural stability
testing. The percentage of injured athletes impaired on cog-
nitive and balance testing gradually declined over the first
several days postinjury, but still remained considerably
higher than the comparative rate of statistically defined
impairment in matched control participants. Less than 5%
of injured athletes reported higher-than-base rate postcon-
cussive symptoms by day 7 postinjury, and rates of impair-
ment on brief screening measures of cognitive functioning
and balance were comparable for injured and noninjured
participants 1 week postinjury. Seventeen percent of con-
cussed players continued to show impairment on neuropsy-
chological testing 1 week after injury, compared to 9% of
It is also important to note that functional recovery as
measured by objective testing often lagged behind the res-
olution of postconcussive symptoms. Most concerning was
our finding that 26% of concussed players who reported
being symptom free, and who presumably were eager to
return to competition 1 week after their injury, continued to
show measurable impairment on standardized cognitive and
Table 5. Rates of impairment on battery of brief measures (GSC, BESS, SAC) and brief battery with
GSC, BESS, SAC*(GSC, BESS, SAC) 1 NP testing#
Concussion Abnormal Control AbnormalConcussion AbnormalControl Abnormal
Time of Injury
*“Abnormal” refers to impairment on any measure in brief battery (GSC, BESS, or SAC).
#“Abnormal” refers to impairment on any measure in brief battery or impairment on at least two neuropsychological measures.
GSC 5 Graded Symptom Checklist (Lovell & Collins, 1998); BESS 5 Balance Error Scoring System (Guskiewicz et al., 2001);
SAC 5 Standardized Assessment of Concussion (McCrea et al., 2000); NP Testing 5 Neuropsychological Test Battery.
Table 6. Sensitivity (Sn) and specificity (Sp) for detecting impairment at postinjury time points
Time of injuryPostgameDay 1 Day 2 Day 3 Day 5 Day 7
SeSp SeSpSe SpSeSp SeSp SeSp Se Sp
Brief battery without NP testing
Full battery with NP testing
Notes. Sensitivity values indicate the probability that a player originally injured continued to be correctly classified as “abnormal”. Specificity (Sp) refers
to the probability that a control participant will be correctly classified as “normal” using the same method.
Brief battery refers to GSC, BESS, and SAC. Full battery refers to brief battery plus neuropsychological testing.
GSC 5 Graded Symptom Checklist (Lovell & Collins, 1998); BESS 5 Balance Error Scoring System (Guskiewicz et al., 2001); SAC 5 Standardized
Assessment of Concussion (McCrea et al., 2000); NP testing 5 neuropsychological test battery
Measuring concussion recovery
balance testing, compared to 14% of controls. Neuropsy-
chological testing was most sensitive to detecting cognitive
impairment in otherwise asymptomatic players 1 week post-
injury, with 16% impaired on at least two of the seven mea-
sures in the battery, compared to 9% of controls. These
findings suggest that a period of cerebral dysfunction and
vulnerability persists beyond resolution of subjective symp-
toms, which suggests the need for a thorough evaluation of
all elements of recovery in making decisions about an
athlete’s readiness for a safe return to competition.
Advanced Methods for Measuring Recovery
This study used an alternative method of analysis to deter-
mally low scores on measures of symptoms, balance, and
cognitive functioning, during the first week following con-
offers several advantages in the research setting by taking
into account differences in test scores at baseline, as well as
controlling for other factors, including practice effects and
tial to more closely investigate the influence of additional
demographic variables on test–retest outcomes, which then
allows the inclusion of an empirically derived demographic
correction in the standard regression.
Our use of the two-tailed 90% confidence interval for
detection of impairment is rather conservative, providing a
5% chance of false positives (i.e., misclassifying a normal
participant as impaired) for any single measure.When look-
ing at the full battery of measures (e.g., GSC, SAC, BESS,
and NeuropsychologicalTesting Battery), the expected false
positive rate increased to 27.3%, conservatively assuming
mine impairment in the concussion group fell within the ex-
pected range of false positives in nearly all instances. In a
clinical setting, a less conservative confidence interval may
be preferred to maximize the sensitivity of injury detection
Implications for Sports Concussion
Prospective, empirical data on the rate and trajectory of
individual recovery not only advance our understanding of
the natural history of concussion, but also have more direct
clinical implications specific to the management of sport-
related head injuries. Defining “recovery” following sports
concussion on the basis of findings from serial neuropsy-
chological and other standardized testing has been the sub-
ject of great debate. Our previously reported group data
analyses delineated a typical slope of recovery from con-
cussion (McCrea et al., 2003), but did not provide a meth-
odology for clinical decision making on an individual case
basis. The more advanced SRB methods employed in this
study revealed a higher percentage of players with concus-
sion to still be impaired 1 week after injury than that pre-
viously suggested by group recovery data analyses. These
data from the current study indicate that on most measures
“return to baseline” does not necessarily reflect full recov-
ery. The use of statistical models that empirically identify
meaningful change while controlling for baseline test per-
formance and practice effects on serial testing is essential
to classifying impairment due to any condition. Further
refinement and simplification of this approach is likely nec-
essary to provide the sports medicine clinician with a user-
friendly method to measure a player’s level of recovery and
readiness to return to play after concussion.
Relying primarily on a player’s self-reported symptom
recovery to guide injury management and return to play
decision making after sport-related concussion is problem-
atic. The sensitivity of symptom assessment based on a
ishes after a few days. It is clear from the data on days 2 and
7, that even when players have experienced the resolution
of symptoms as measured by the GSC, they may continue
to exhibit deficits on objective measures of balance and
cognitive functioning. A player’s potential motivation to
under-report symptoms in hopes of a more rapid return to
Table 7. Percentage of asymptomatic participants after concussion (Sx-) and control participants classified as
“Impaired” on postinjury days 2 and 7
Day 2Day 7
(n 5 68)
Control impaired (%)
(n 5 56)
(n 5 85)
Control impaired (%)
(n 5 56)
BESS 5 Balance Error Scoring System (Guskiewicz et al., 2001); SAC 5 Standardized Assessment of Concussion (McCrea et al.,
2000); NP battery 5 neuropsychological test battery
M. McCrea et al.
play also complicates matters. Those injured players who
reported being symptom free but continued to exhibit mild
impairment on standardized testing and formal neuropsy-
chological testing 1 week postinjury may be at increased
risk of recurrent or more severe injury (Guskiewicz et al.,
2003) if returned to play based solely on their reported
symptoms without objective recovery data to guide the
clinician’s decision making. Future studies may consider
including a formal measure of symptom minimization
response bias, and this forum may provide an interesting
laboratory for neuropsychologists to compare this form of
response bias to that of symptom exaggeration or malinger-
ing often encountered in other clinical or forensic settings.
A multidimensional model of sports concussion assess-
ment is supported by our findings. Postconcussive symp-
toms, cognitive dysfunction, and postural instability are all
common sequelae of concussion, but may manifest differ-
ently across individuals. Multidimensional assessment of
all domains affected by concussion yields the greatest sen-
sitivity in detecting injury and the best method for assess-
ing postinjury recovery. The combined battery of brief
screening instruments (total administration time , 15 min)
that measured postconcussive symptoms, cognitive func-
tioning, and balance provided a sensitive and specific means
to detect and characterize concussion during the acute post-
injury period, particularly on the sports sideline immedi-
ately after concussion. The SAC and neuropsychological
test battery yielded similar rates of impairment on day 2,
but neuropsychological testing provided a more sensitive
measure of subtle cognitive dysfunction further out from
injury, characterized by mild residual deficits in delayed
recall memory, cognitive processing speed, and verbal flu-
ing instruments appropriate for emergent use on the sideline
and more extensive neuropsychological testing to assess
recovery may be most appropriate for clinical use.
Given the frequency of impaired postural stability in the
of any assessment model to identify neurological dysfunc-
tion following concussion in athletes. Balance deficits, like
postconcussion symptoms and cognitive impairment, may
with cognitive testing, obtaining baseline testing on clinical
Inclusion of the neuropsychological test data only mini-
mally increased the sensitivity and reduced the specificity
of the battery of brief instruments within the first 2 days
after concussion, but the addition of neuropsychological
testing more than doubled the sensitivity and slightly
increased the specificity of the brief battery at day 7. The
relative value of neuropsychological testing is demon-
strated by its superior sensitivity in detecting subtle neuro-
cognitive impairment further out from injury, especially in
players otherwise reporting to be completely symptom free,
although there are some methodological considerations
which may complicate the interpretation of these data.These
data support the recommendation that neuropsychological
being symptom free, which is also supported by practical
and methodological considerations. Protocols that include
neuropsychological testing at fixed intervals are appropri-
ate for research intended to empirically track recovery, but
are inappropriate in a clinical setting when a player is still
symptomatic and should be withheld from competition
regardless of the neuropsychological test results. Unneces-
sary serial testing, in addition to being burdensome to the
athlete and medical staff, also introduces practice effects
that may confound the interpretation of performance on
subsequent testing (i.e., when a player reports being symp-
tom free and is otherwise ready to return to play).
It is possible that some players who sustained a concussion
while enrolled in this study did not report their injury and
were not identified by the team physician or certified ath-
letic trainer. Recent reports (McCrea et al., 2004) suggest
that the rate of concussion, even in well-controlled studies,
is higher than that documented in the literature, due to a
combination of players not recognizing the signs of injury
or not reporting a concussion in order to continued uninter-
rupted sports participation.
The SRB method is subject to the reliability and validity
of the main outcome measures used in this study, which
have been supported by earlier studies on the effects of
sports concussion. The availability of baseline data for
injured and control participants on all concussion assess-
ment measures also adds strength to the model by provid-
ing the most reliable means for detecting meaningful change
in individual athletes. Although we incorporated most of
in sports concussion research, it is conceivable that a dif-
ferent test battery may have proven more or less sensitive
to the lingering effects of concussion. There have as yet,
pencil-and-paper or computer batteries being proposed for
this purpose, and no conventional or computerized test bat-
tery has been established as the “gold standard”.
The serial testing model employed in this study differs
from previous SRB paradigms in epilepsy and cardiac sur-
gery in the number of postevent testing sessions, particu-
consecutive days after concussion. Further investigation is
required to fully appreciate the effects of repeated neuro-
psychological testing, and the SRB approach at least pro-
vides one methodology do so. It is possible, for instance,
that the control group is more readily able to benefit from
practice on some neurocognitive measures, and this prac-
tice effect is likely to be attenuated in concussed players
who are still encephalopathic, especially during the acute
postinjury phase. Therefore, the normative group may not
really be an appropriate control group to use for this pur-
pose in predicting performance for any time point beyond
Measuring concussion recovery
to benefit from practice effects to a significantly greater
extent than players who are suffering from the acute effects
of concussion at the time of testing. This is likely to result
in overclassifying injured players as being “impaired” even
though they may have completely recovered. This issue
cannot be overlooked, particularly when interpreting the
comparative rates of impairment for injured and control
participants beyond the acute phase on postinjury day 7.
Future studies should consider utilizing a controlled, ran-
domized design to compare cognitive and balance recovery
patterns for concussed symptomatic athletes with con-
cussed asymptomatic athletes to validate the current find-
ings, and to substantiate our recommendation that testing
be conducted once an athlete is reporting to be symptom
free. Unfortunately, participant attrition disallowed appli-
cation of our analysis to look at the lengthier course of
recovery 90 days after concussion. Future studies should
players beyond the completion of the sports season to allow
analysis of longer term outcomes.
In summary, sport-related concussion is characterized by a
combination of measurable symptoms, cognitive dysfunc-
tion, and postural instability that follows a gradual course
of recovery over several days in most cases. Relying solely
on a player’s self-reported recovery is not recommended,
as a significant percentage of participants who report being
completely symptom free after concussion may continue to
show significant impairments on more sensitive standard-
ized testing. Brief screening instruments are most sensitive
and specific in accurately classifying injured and non-
injured participants during the acute postinjury phase, while
neuropsychological testing may be more sensitive to the
especially when a player otherwise reports being com-
pletely symptom free. Deferring neuropsychological test-
ing until the injured player reports a full symptom recovery
is recommended in a clinical setting, and for future studies
investigating the incremental utility of neuropsychological
testing in identifying the residual effects of sport-related
concussion. Statistical models that empirically measure
meaningful change on serial testing may provide a reliable
benchmark for determining recovery and clinical decision
making about an individual athlete’s readiness to return to
competition after sport-related concussion. These data pro-
vide an empirical base to be considered by sports governing
bodies and expert panels responsible for developing prac-
tice guidelines for clinical decision making on return to
competition after sport-related concussion.
This research was funded in part by the National Collegiate Ath-
letic Association (NCAA) and the National Operating Committee
on Standards for Athletic Equipment (NOCSAE). The National
Centers for Injury Prevention and Control and the University of
North Carolina Injury Prevention Research Center also contrib-
uted to the success of this project. We thank the certified athletic
trainers, team physicians, and football players from the following
NCAA institutions for their participation in the study: Auburn
University, Carnegie-Melon University, Colorado College, Flor-
ida State University, Georgia Tech Institute, Kutztown University,
Lehigh University, Louisiana State University, Slippery Rock Uni-
versity, Syracuse University, University of Arizona, University of
Oklahoma, University of South Carolina, University of Toledo,
and Wake Forest University.
Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K.,
Kelly, J.P., Lovell, M., McCrory, P., Meeuwisse, W., & Scha-
masch, P. (2002). Concussion in sport group. Summary and
agreement statement of the first International Conference on
concussion in sport, Vienna 2001. Physician and Sports Med-
icine, 30, 57–63.
Barr, W.B. (2002). Neuropsychological testing for assessment of
treatment effects: Methodological issues. CNS Spectrums, 7,
Barr, W.B. (2003). Neuropsychological testing of high school ath-
letes. Preliminary norms and test–retest indices. Archives of
Clinical Neuropsychology, 18, 91–101.
Barr, W.B. & McCrea, M. (2001). Sensitivity and specificity of
standardized neurocognitive testing immediately following
sports concussion. Journal of the International Neuropsycho-
logical Society, 7, 693–702.
Benton, A.L., Hamsher, K., & Sivian, A.B. (1983). Multilingual
Aphasia Examination (3rd ed.). Iowa City, IA:AJAAssociates.
Cantu, R.C. (1986). Guidelines for return to contact sports after a
Cantu, R.C. (1998). Return to play guidelines after a head injury.
Clinics in Sports Medicine, 17, 45–60.
Relationship between concussion and neuropsychological per-
formance in college athletes. Journal of the American Medical
Association, 282, 964–970.
Collins, M.W., Lovell, M.R., & McKeag, D.B. (1999). Current
issues in managing sports-related concussion. Journal of the
American Medical Association, 282, 2283–2285.
Collins, M.W., Field, M., Lovell, M.R., Iverson, G., Johnston,
K.M., Maroon, J., & Fu, F.H. (2003). Relationship between
postconcussion headache and neuropsychological test perfor-
mance in high school athletes. American Journal of Sport Med-
icine, 31, 168–173.
Colorado Medical Society (1991). Report of the sports medicine
committee: Guidelines for the management of concussion in
sports. Denver, CO: Colorado Medical Society.
Covassin, T., Swanik, C.B., & Sachs, M.L. (2003). Epidemiolog-
ical considerations of concussions among intercollegiate ath-
letes. Applied Neuropsychology, 10, 12–22.
Dick, R. (2003). National Collegiate Athletic Association (NCAA)
Injury Survellience System 2002–2003. Indianapolis, IN:
Diggle, P.J., Liang, K.Y., & Zeger, S.L. (1994). Analysis of longi-
tudinal data. Oxford, England: Oxford University Press.
M. McCrea et al.
Echemendia, R.J. & Julian, L.J. (2001). Mild traumatic brain injury
in sports: Neuropsychology’s contribution to a developing field.
Neuropsychological Reviews, 11, 69–88.
Echemendia, R.J., Putukian, M., Mackin, R.S., Julian, L., & Shoss,
N. (2001). Neuropsychological test performance prior to and
following sports-related mild traumatic brain injury. Clinical
Journal of Sport Medicine, 11, 23–31.
Erlanger, D., Kaushik, T., Cantu, R., Barth, J.T., Broshek, D.K.,
Freeman, J.R., & Webbe, F.M. (2003). Symptom-based assess-
ment of the severity of a concussion. Journal of Neurosurgery,
Golden, J.C. (1978). Stroop Color and Word Test. Chicago, Illi-
nois: Stoelting Co.
Guskiewicz, K.M., McCrea, M., Marshall, S.W., Cantu, R.C., Ran-
dolph, C., Barr, W., Onate, J.A., & Kelly, J.P. (2003). Cumula-
tive effects associated with recurrent concussion in collegiate
football players: The NCAA Concussion Study. Journal of the
American Medical Association, 290, 2549–2555.
Guskiewicz, K.M., Ross, S.E., & Marshall, S.W. (2001). Postural
stability and neuropsychological deficits after concussion in
collegiate athletes. Journal of Athletic Training, 36, 263–273.
Guskiewicz, K.M., Weaver, N.L., Padua, D.A., & Garrett, W.E.
(2000). Epidemiology of concussion in collegiate and high
school football players. American Journal of Sport Medicine,
Hinton-Bayre,A.D., Geffen, G.M., Geffen, L.B., McFarland, K.A.,
indices of impairment and recovery. Journal of Clinical and
Experimental Neuropsychology, 21, 70–86.
Kelly, J. (1999). Traumatic brain injury and concussion in sports.
Journal of the American Medical Association, 282, 989–991.
Kelly, J.P. & Rosenberg, J.H. (1997). Diagnosis and management
of concussion in sports. Neurology, 48, 575–580.
Lovell, M.R. & Collins, M.W. (1998). Neuropsychological assess-
ment of the college football player. Journal of Head Trauma
Rehabilitation, 13, 9–26.
Lovell, M.R., Collins, M.W., Iverson, G.L., Field, M., Maroon,
J.C., Cantu, R., Podell, K., Powell, J.W., Belza, M., & Fu, F.H.
(2003). Recovery from mild concussion in high school ath-
letes. Journal of Neurosurgery, 98, 296–301.
Macciocchi, S.N., Barth, J.T., Alves, W., Rimel, R.W., & Jane,
J.A. (1996). Neuropsychological functioning and recovery after
mild head injury in collegiate athletes. Neurosurgery, 39,
Maddocks, D. & Saling, M. (1996). Neuropsychological deficits
following concussion. Brain Injury, 10, 99–103.
McCrea, M. (2001). Standardized mental status testing on the side-
line after sport-related concussion. Journal of Athletic Train-
ing, 36, 274–279.
McCrea, M., Guskiewicz, K.M., Marshall, S., Barr, W.B., Ran-
dolph, C., Cantu, R.C, Onate, J.A.,Yang, J., & Kelly, J.P. (2003).
Acute effects and recovery time following concussion in col-
legiate football players: The NCAA Concussion Study. Jour-
nal of the American Medical Association, 290, 2556–2563.
McCrea, M., Hammeke, T., Olsen, G., Leo, P., & Guskiewicz K.
(2004). Unreported concussion in high school football players:
Implications for prevention. Clinical Journal of Sport Medi-
cine, 14, 13–17
McCrea, M., Kelly, J.P, Kluge, J., Ackley, B., & Randolph, C.
(1997). Standardized assessment of concussion in football play-
ers. Neurology, 48, 586–588.
McCrea, M., Kelly, J.P., Randolph, C., Cisler, R.A., & Berger, L.
surgery, 50, 1032–1042.
McCrea, M., Kelly, J.P., Randolph, C., Kluge, J., Bartolic, E., Finn,
G., & Baxter, B. (1998). Standardized assessment of concus-
sion: On-site mental status evaluation of the athlete. Journal of
Head Trauma Rehabilitation, 13, 27–35.
McCrea, M., Randolph, C., & Kelly, J.P. (2000). The Standardized
Assessment of Concussion (SAC): Manual for Administration,
Scoring and Interpretation (2nd ed.). Waukesha, WI.
McSweeny, A.J., Naugle, R.I., & Chelune, G.J. (1993). T-scores
for change: An illustration of a regression approach to depict-
ing change in clinical neuropsychology. The Clinical Neuro-
psychologist, 7, 300–312.
Powell, J.W. & Barber-Foss, K.D. (1999). Traumatic brain injury
in high school athletes. Journal of the American Medical Asso-
ciation, 282, 958–963.
(summary statement). Report of the quality standards commit-
tee. Neurology, 48, 581–585.
Reitan, R.M. & Wolfson, D. (1985). The Halstead-Reitan Neuro-
Riemann, B.L. & Guskiewicz, K.M. (2000). Effects of mild head
injury on postural stability as measured through clinical bal-
ance testing. Journal Athletic Training, 25, 19–25.
Riemann, B.L., Guskiewicz, K.M., & Shields, E.W. (1999). Rela-
tionship between clinical and forceplate measures of postural
stability. Journal of Sports Rehabilitation, 8, 71–82.
Rubin, D.B. (1976). Inference and missing data. Biometrika, 63,
Rubin, D.B. (1996). Multiple imputation after 181 years. Journal
of the American Statistical Association, 91, 473–489.
Schafer, J.L. (1997). Analysis of incomplete multivariate data. New
York, NY: Chapman and Hall.
Shapiro,A.M., Benedict, R.H., Schretlen, D., & Brandt, J. (1999).
Construct and concurrent validity of the HopkinsVerbal Learn-
ing Test–Revised. Clinical Neuropsychologist, 13, 348–358.
Smith,A. (1991). Symbol Digit Modalities Test. LosAngeles, CA:
Western Psychological Services.
SPSS (1999). SPSS Advanced Models 10.0. SPSS, Inc.
Temkin, N.R., Heaton, R.K., Grant, I., & Dikmen, S.S. (1999).
Detecting significant change in neuropsychological test perfor-
mance: A comparison of four models. Journal of the Inter-
national Neuropsychological Society, 5, 357–369.
Thurman, D., Branche, C., & Sniezek, J. (1998). The epidemiol-
ogy of sports-related traumatic brain injuries in the United
States: recent developments. Journal of Head Trauma Reha-
bilitation, 13, 1–8.
Vastag, B. (2002). Football brain injuries draw increased scrutiny.
Journal of the American Medical Association, 287, 437–439.
Measuring concussion recovery