ArticlePDF AvailableLiterature Review

Computerized Neurocognitive Testing within 1 Week of Sport-Related Concussion: Meta-analytic Review and Analysis of Moderating Factors


Abstract and Figures

The purpose of this study is to perform a meta-analysis assessing the effects of sport-related concussion as measured by computerized neurocognitive tests (NCT) 1-week post injury. Thirty-seven studies involving 3960 participants between 2000 and 2011 were included. Hedge's g provides an adjusted effect size for smaller sample sizes and was calculated for overall and cognitive task effects, and subgroup analyses were conducted for age, type of NCT, and sport. Concussions had a low negative effect (g = -0.16; p < .001) across all groups, outcomes, and time points. Code substitution (g = -0.27; p < .05), visual memory (g = -0.25; p < .05), processing speed (g = -0.18; p < .05), and memory (g = -0.21; p < .05) tasks demonstrated negative effects for concussion. Younger adolescents had lower (g = -0.29; p < .05) NCT performance than older adolescents (g = -0.01) and college aged athletes (g = -0.11). ImPACT studies (g = -0.19; p < .05) demonstrated a negative effect for concussion as did those involving contact sports (g = -0.20; p < .05). A low to moderate overall effect size of concussion on neurocognitive performance was supported. Subgroup analyses revealed different effect sizes for specific cognitive tasks, types of NCTs, age, and type of sport. (JINS, 2014, 20, 1-9).
Content may be subject to copyright.
Journal of the International Neuropsychological Society (2014), 20, 324–332.
Copyright EINS. Published by Cambridge University Press, 2014.
Computerized Neurocognitive Testing within 1 Week of
Sport-Related Concussion: Meta-analytic Review and
Analysis of Moderating Factors
Anthony P. Kontos,
Rock Braithwaite,
Scott Dakan,
AND R.J. Elbin
Department of Orthopaedic Surgery/UPMC Sports Medicine Concussion Program, University of Pittsburgh, Pittsburgh, Pennsylvania
Department of Kinesiology and Recreation Administration, Humboldt State University, Arcata, California
Department of Health, Human Performance and Recreation, University of Arkansas, Fayetteville, Arkansas
(RECEIVED June 13, 2013; FINAL REVISION December 19, 2013; ACCEPTED December 19, 2013; FIRST PUBLISHED ONLINE February 13, 2014)
The purpose of this study is to perform a meta-analysis assessing the effects of sport-related concussion as measured by
computerized neurocognitive tests (NCT) 1-week post injury. Thirty-seven studies involving 3960 participants between
2000 and 2011 were included. Hedge’s gprovides an adjusted effect size for smaller sample sizes and was calculated for
overall and cognitive task effects, and subgroup analyses were conducted for age, type of NCT, and sport. Concussions
had a low negative effect (g520.16; p,.001) across all groups, outcomes, and time points. Code substitution (g520.27;
p,.05), visual memory (g520.25; p,.05), processing speed (g520.18; p,.05), and memory (g520.21; p,.05)
tasks demonstrated negative effects for concussion. Younger adolescents had lower (g520.29; p,.05) NCT performance
than older adolescents (g520.01) and college aged athletes (g520.11). ImPACT studies (g520.19; p,.05)
demonstrated a negative effect for concussion as did those involving contact sports (g520.20; p,.05). A low to moderate
overall effect size of concussion on neurocognitive performance was supported. Subgroup analyses revealed different effect
sizes for specific cognitive tasks, types of NCTs, age, and type of sport. (JINS, 2014, 20, 324–332)
Keywords: ANAM, CogSport, Headminder, ImPACT, Neuropsychology, Mild traumatic brain injury
An estimated 1.6 to 3.8 million concussions occur annually
among the 208 million participants in organized sport in the
United States (Langlois, Rutland-Brown, & Wald, 2006;
‘Report on Trends and Participation in Youth Sports,’’
2001). Concussions involve significant variability in the
presentation of clinical signs and symptoms, including physical
signs (e.g., loss of consciousness), somatic symptoms (e.g.,
headache), cognitive impairment (e.g., reduced memory,
delayed reaction time), vestibular-ocular (e.g., dizziness,
convergence), as well as behavioral and emotional changes
(e.g., depression, irritability) (Aubry et al., 2002). Due to this
variability, the assessment of concussion involves a com-
prehensive interdisciplinary approach that uses different tools
including clinical exams and interviews, symptom reports,
balance assessments, vestibular-ocular exams, and neuro-
cognitive assessments (Johnson, Kegel, & Collins, 2011).
During the past decade, the use of computerized neuro-
cognitive testing (NCT) as one tool in a comprehensive
assessment and management approach to concussion has
become more and more common (Johnson et al., 2011;
McCrory et al., 2009). Recently, the reliability and validity of
computerized NCTs for use in assessing and managing con-
cussion have been questioned in several review papers
(Mayers & Redick, 2012; Randolph, 2011). However, as
Schatz, Kontos, and Elbin (2012) pointed out, these reviews
have been characterized by a flawed research methodology
(Schatz et al., 2012). Specifically, these review papers have
tended to include incomplete sampling of literature with
no regard to the quality or differences in studies including
samples, measures, and other factors that may impact the
outcomes of the research. As such, a more objective exam-
ination of the efficacy of computerized NCT to identify the
subtle effects of concussion using meta-analytic techniques is
warranted. Such an examination should include all empirical
Correspondence and reprint requests to: Anthony P. Kontos, UPMC
Sports Medicine Concussion Program, Department of Orthopedic Surgery,
University of Pittsburgh, UPMC Center for Sports Medicine, 3200 South
Water Street, Pittsburgh, Pennsylvania 15203. E-mail:
research that meets a priori and accepted criteria for inclusion
into a meta-analytic study rather than including only a
selective sample of studies.
Although there have been several published meta-analytic
reviews on the neurocognitive effects of sport-related con-
cussion (Belanger, Spiegel, & Vanderploeg, 2010; Broglio &
Puetz, 2008; Dougan, Horswill, & Geffen, 2013), none have
focused exclusively on the more commonly used computer-
ized versions of NCTs. In so doing, these previous studies
also included studies in their meta-analyses that reflect defi-
nitions of concussion [e.g., based on loss of consciousness
(LOC), post-traumatic amnesia (PTA)] and inclusion/
exclusion criteria that biased them toward more severely
concussed samples. Given that we now know that many
(up to 90% if based on LOC alone) concussions were not
properly identified using these criteria (Guskiewicz, Weaver,
Padua, & Garrett, 2000), inclusion of these studies that used
paper and pencil NCTs likely biased results toward larger
effect sizes. Moreover, previous meta-analyses of NCT have
included paper and pencil NCTs and failed to consider
known moderating factors such as age (Belanger et al., 2010;
Broglio & Puetz, 2008). Dougan et al. (2013) reported initial
support for an age effect with adolescent athletes demon-
strating a larger effect than athletes older than 24 years in
their meta-analysis. This effect is supported in the literature
(Field, Collins, Lovell, & Maroon, 2003). In addition,
researchers have argued that future meta-analyses should
examine effects sizes for specific cognitive test modules
such as reaction time, verbal memory, processing speed
(Dougan et al., 2013). There has been speculation among
clinicians that test administration personnel (e.g., physi-
cian, certified athletic trainer [ATC], neuropsychologist)
may affect results obtained using computerized NCTs.
Finally, and as reported by Broglio and Puetz (2008) and
Dougan et al. (2013), time since injury is a factor known
to affect post-concussion neurocognitive performance.
Therefore, the primary purpose of the current study was
to use meta-analytic techniques to determine the effects of
concussion as measured by current computerized NCTs
administered within the first week of injury across multiple
studies. A secondary purpose of this study was to examine
the subgroup analyses for variables including NCT type,
sport, and age.
Search Strategy
A literature search strategy was developed using key words to
locate and identify relevant research for the current study.
Combinations of the following key terms: concussion, mild
traumatic brain injury, mTBI, sport(s), athlete, cognitive
impairment, computerized (computer), neurocognitive test/
performance, symptoms, Automated Neuropsychological
Assessment Metrics (ANAM), CNS Vital Signs, CogSport
(i.e., AXON), Headminder, and ImPACT were entered into
the following electronic database search engines: Cochran
Libraries, Medline/PubMed, Proquest (Dissertations & Theses),
PsychINFO, SportDiscus, Science Direct, and Web of Science.
Literature search findings from each set of key words were
recorded and screened to determine inclusion in or exclusion
from the current investigation. In addition to electronic
database searches, personal files were reviewed and manual
searches of reference lists from relevant literature facilitated
the process. Articles were screened by title and abstract for
information relevant to the current investigation. During
the screening process, if relevant information was insufficient
the article was retrieved to complete the review. Studies
included in the analysis had to meet each of the following
inclusion criteria: (a) participants sustained a concussion
during sports participation; (b) use of a desktop-based com-
puterized neurocognitive test (NCT); (c) concussed partici-
pants received a baseline computerized NCT and at least one
post injury computerized NCT within 1 week of injury or
there was a concussed and healthy control comparison group,
also within 1 week of injury; (d) sufficient descriptive and/or
inferential statistical data were reported to allow for calcula-
tion of effect sizes; and (e) published in the English language
between December 2000 (time corresponding to advent of
computerized NCT) and December 2012. When articles
contained insufficient information or data, we contacted the
authors via email requesting specific information. If there was
no response, a follow-up email request was made approxi-
mately 2 weeks after the initial contact. Studies were
removed from analysis after one month if authors did not
respond to these inquiries. All data included in this manuscript
were obtained in compliance with regulations of the University
of Pittsburgh Institutional Review Board.
Coding Procedures and Data Extraction
Standard coding forms were developed and information was
extracted from each article and divided into three categories
representative of methodological characteristics, sample
characteristics, or study characteristics. Methodological
characteristics included how the study was conducted and
information on the Study Design (case control, cohort, or
cross sectional), Concussion Test (ANAM, CogSport/
AXON, Headminder, or ImPACT), Personnel Training
administering the computerized concussion test (Athletic
Trainer, Neuropsychologist, Physician, or Researcher), and
Sport Type (contact/collision OR all types of sports). Sample
characteristics provided information concerning the partici-
pants being studied and included Sex (Females and Males OR
Males Only), and Age (younger adolescents 12–15 years,
older adolescents 16–18 years, or college age 19 years or
older). Study characteristics were categorized as Status
(Published OR Unpublished), and Grant Funded (Funded or
Unfunded). Table 1 provides the codes associated with each
category for the study.
Two researchers independently reviewed, coded, and rated
each study according to the methodological, sample, and
study characteristics identified. After all studies meeting
inclusion criteria had been coded, the independent results
Meta-analysis of computerized neurocognitive testing 325
were compared for agreement. Disagreements were analyzed
to determine the type of error associated with individual
discrepancies and classified as either factual disagreements or
interpretative disagreements. Factual disagreements were
transcription errors in which the correct information was
present in a study but was recorded incorrectly; whereas,
interpretative disagreements occurred when information
reported in a study was vague or imprecise allowing for
different conclusions. Factual disagreements were simply
corrected. A third researcher, who coded the study, reviewed
interpretative disagreements and the decision was based on a
simple majority of agreement.
Data Analysis
Outlier and publication bias
Data were screened to determine the presence of outliers and
influence of publication bias on overall results. Outliers were
identified by reviewing and identifying residual values
(similar to Z-scores) approximately two standard deviations
(61.96) above or below the study’s mean effect size. Criteria
for including studies were based on overall results remaining
within the 95th percent confidence interval and a significant
summary effect size. Publication bias has the potential to
influence meta-analytic results when relevant studies are
Table 1. Coding characteristics for studies meeting inclusion criteria
Methodological characteristics Sample characteristics Study characteristics
Study Design Test Training Type Gender NAge Funding Sample
Broglio et al., 2009 Cr I A CC B 32 C U B
Chen et al., 2007 Ca C R CC M 28 Adult mean age 26.9, 30.8, 21.9 C? F B
Collie et al., 2006 Ca C P CC M 145 C U B
Collins et al., 2003 Co I A All B 78 H & C U W
Colvin et al., 2009 Co I R CC B 234 H U W
Covassin et al., 2007 Co I R CC B 79 C U B
Covassin et al., 2008 Co I R All B 57 C U W
Covassin et al., 2012 Co I A NR B 296 H & C U W
Eckner et al., 2011 Ca C R CC B 95 H U W
Fazio et al., 2007 Ca I R All B 192 H & C U B
Hutchison et al., 2011 Ca A R All B 72 C F B
Iverson et al., 2003 Ca I O CC B 97 Y U B
Iverson et al., 2006 NR I NR NR M 30 H & 10% C U W
Iverson, 2007 Ca I R CC M 114 H F B
Kontos et al., 2010 Ca I R All B 96 H & C U W
Kontos et al., 2012 Ca I R NR B 75 C F W
Lau et al., 2009 Ca I A CC M 108 H U B
Lau et al., 2011 Co I R CC M 107 H U B
Lovell et al., 2003 Ca I A CC B 88 H U W
Lovell et al., 2004 Cr I P CC B 43 H U B
Lovell et al., 2007 Ca I R NR B 41 H F B
Majerske et al., 2008 Co I P CC B 95 H 15.88 61.35 F W
Makdissi et al., 2001 Ca C O CC M 13 C U B
McClincy et al., 2006 Cr I P CC B 104 H & C U W
McCrea et al., 2010 Ca A A CC M 56 H & C F B
Mihalik et al., 2005 Ca I P CC B 261 H & C U W
Parsons et al., 2009 Cr A A CC B 40 C U W
Pellman et al., 2006 Cr I P CC M 85 H & NFL F B
Rabinowitz et al., 2011 Ca I R CC B 574 C U B
Register-Mihalik et al., 2012 Ca A R All B 40 H & C U W
Sim et al., 2008 Ca A R CC B 419 H F B
Sosnoff et al., 2007 Ca H O NR B 44 C U B
Sosnoff et al., 2008a Cr H P All B 36 C U W
Sosnoff et al., 2008b Cr H P All B 36 C U W
Sosnoff et al., 2008c Cr H P All B 36 C U W
Warden et al., 2001 Ca A A CC M 14 C F W
Note. Design (Study Design): Ca 5Case Control; Co 5Cohort; Cr 5Cross-sectional. Test (Concussion Test): A 5ANAM; C 5CogSport;
H5Headminder; I 5ImPACT. PCT (Number of Post-Concussion Tests): 1 5baseline 11 PCT, 2 5baseline 12 PCTs; 3 5baseline 13 PCTs;
45baseline 1more than 3 PCTs. Training (Personnel Training): A 5Certified Athletic Trainer; P 5Physician; R 5Researcher; O 5Other. Type (Type of
Sport): Col 5Collision; Con 5Contact; N 5Non-contact. Gender (Sample Composition): B 5Female & Male; M 5Male Only. Age (Sample age):
C5College/University; H 5High School; Y 5Youth. Funding (Grant Funded): F 5Funded; U 5Unfunded. Status (Publication Status): P 5Published;
U5Unpublished. NR 5Not Reported.
326 A.P. Kontos et al.
overlooked during the literature search process (Rosenthal,
1979; Rothstein, Sutton, & Borenstein, 2005). Three proce-
dures were used to identify and control for publication bias
including review of the funnel plot (Egger, Davey Smith,
Schneider, & Minder, 1997), a Fail-Safe Ncalculation
(Rosenthal, 1979), and a ‘‘Trim & Fill’’ method (Duval &
Tweedie, 2000). A funnel plot graphs studies using effect size
(x-axis) and standard error (y-axis) and when publication bias
is present an asymmetrical distribution of studies will be
randomly clustered away from the mean effect size (Borenstein,
Hedges, Higgins, & Rothstein, 2009; Light & Pillemer, 1984).
Rosenthal’s (1979) Fail Safe Nprovides an additional
measure of certainty regarding publication bias as there is
an estimation of the number of studies needed to nullify a
significant effect. Duval and Tweedie’s (2000) ‘‘Trim and
Fill’’ estimate is an iterative procedure that uses and algo-
rithm on a funnel plot to calculate an estimate of symmetry
by imputing missing studies and adjusting the effect size
calculations (if publication bias is present).
Effect size calculations
All analyses were performed using Comprehensive Meta-
Analysis version-2 software (Borenstein, Hedges, Higgins, &
Rothstein, 2005). Interpretation of the effect size calculations
were based on Cohen’s (1988) determination of small (d r.20),
medium (d r.50), and large (d Z.80) effect sizes. For the
purposes of the current investigation, the study was considered
to be the unit of analysis and when studies contained multiple
measures (outcomes) the standard procedure averages the dif-
ferent metrics into a summary effect. An inverse weighting
method was applied to improve precision of data analyses and is
considered to be an appropriate method when several metrics
are used to compute a summary effect (Borenstein, Hedges,
Higgins, & Rothstein, 2010). Hedges gwas the effect size
metric selected and is used to provide a correction in calcula-
tions when small sample sizes (k ,20) are used (Field, 2001;
Hedges, 1981; Hedges & Olkin, 1985). Whenever possible,
baseline and post-test means and standard deviations were used
to calculate the study effect size. When unavailable, post-test
means and standard deviations or mean change in each group
were used to calculate the study effect size. There were 37 stu-
dies included in the current analysis and the rational for using
Hedges gwas based on the use of additional analyses (outcomes
and moderator) that contained fewer than the recommended
number of studies and consistency in reporting methods.
Hedges gwas calculated using the following formula:
The two statistical approaches used to model error include
a ‘‘fixed’’ effects model that assumes error is connected to
sampling procedures as compared to a ‘‘random’’ effects
model that assumes an additional source of between study
variance contributes to error (Borenstein et al., 2010; Hunter
& Schmidt, 2000). Evidence suggests that the assumption of
‘fixed’’ effects models of error are not applicable to real
world data (Field, 2001, 2003; Hunter & Schmidt, 2000),
therefore, a random effects model was selected for the current
investigation due to the variability between studies.
Neurocognitive test module analyses
The neurocognitive test modules reported across studies
included several similar variables with different terminology
associated with similar outcomes. For example, reaction time
included simple reaction time, cue reaction time, and com-
plex reaction time. To be consistent in reporting the summary
effect for different neurocognitive test modules, the authors
reviewed studies for consistent definitions of measures used
during data collection and grouped like tests across studies.
Neurocognitive test modules were defined, grouped, and
sorted according to the purpose of the measure and what was
recorded in the literature (see the Results section for specific
modules). It is important to note that these test module names
includes measures across multiple test types (i.e., manu-
facturers). The final groupings for outcome measures were
agreed upon by all authors before analyses were completed.
Subgroup analyses
When using a random effects model, data (i.e., studies
included) are assumed to be heterogeneous due to sampling
and between study variance. Subgroup (i.e., moderator)
analyses in meta-analysis provide an understanding of the
strength and/or direction of relationships between indepen-
dent and dependent variables (Shaddish & Sweeney, 1991).
In the current investigation, we were interested in differences
between several levels of independent variables (e.g., age,
NCT test, sport) on neurocognitive outcomes in athletes
following concussion. The three statistics used to evaluate
heterogeneity included the Q
)value which is based
on a w-square (w
) distribution, the tau-square (t
) value, and
I-square (I
) value provide a comprehensive approach to
interpreting results. When the Q
statistic is significant then
variance is categorized into Q
) and Q
values with significant Q
values (p,.05) requiring statis-
tical techniques (i.e., ttest or analysis of variance) to deter-
mine subgroup differences (Borenstein et al., 2009; Hedges
& Olkin, 1985). The t
statistic provides an estimate of total
variance between studies. Small subgroup sample sizes
(kr5) may influence the precision of t
; therefore, a pooled
estimate of variance was used for all calculations (Borenstein
et al., 2009). The I
statistic reflects the overlap of confidence
intervals and can be interpreted as low (25%), moderate
(50%), or high (75%) values of the total variance attributed to
covariates (Higgins, Thompson, Deeks, & Altman, 2003).
Literature Search and Coding
The literature search process identified 1126 potential studies
of which 147 (13%) studies included the appropriate variables.
Meta-analysis of computerized neurocognitive testing 327
After screening for inclusion criteria, 51 of the 147 (35%)
papers met each of the inclusion requirements. Further
review identified 15 studies that failed to report the necessary
data for an effect size to be calculated and authors were
emailed on two separate occasions requesting additional
information. There was no response from primary authors
following the two separate contacts, therefore, all 15 (29%)
studies were excluded from the analysis. Analyses were
completed for the remaining 37 (71%) studies with the same
number of independent samples for a total of 3960 subjects.
A total of 31 disagreements were identified during coding
that included 20 factual and 11 interpretative disagreements.
Examination of coding and data extraction determined that
there was high inter-rater reliability agreement (k50.938).
Table 1 provides a summary of the final coding categories for
studies that met the inclusion criteria.
Overall Model Results
As a caveat to reporting results the authors have selected to
use Cohen’s (1988) interpretation of effect sizes that defines
NCT outcomes as low/small, medium/moderate, high/large.
The terms used to interpret results are descriptive and do not
differentiate between statistical (group) and clinical/practical
(individual) significance. In many cases several papers
included in the current synthesis did not report the necessary
information to calculate a reliability change index (RCI)
score preventing further analyses from being conducted and
were beyond the scope of the current investigation. Recom-
mendations concerning future studies have been included in
the discussion.
Results from the primary analysis determined that con-
cussions had a low negative effect (g520.16; SE 50.04;
95% CI 520.24; 20.08; p,.001) across all groups,
outcomes, and time points. The summary effect can be
interpreted as individuals or groups having a concussion
performed approximately one sixth of a standard deviation
lower than baseline levels (for within subjects designs) or
individuals or groups without a concussion (for between
subjects designs) across all outcomes. Review of homo-
geneity statistics determined a significant heterogeneous
5123.20; p,.001; I
570.78) study distribution
and further analyses of covariates (subgroups) may explain
large portions of variance between studies. Collins et al.
(2003) and Fazio, Lovell, Pardini, and Collins (2007) clas-
sified as outliers (Z522.10, 22.60) making it necessary to
perform a ‘‘one study removed’’ (sensitivity analysis) pro-
cedure. Both studies were retained as the influence on overall
effect size was marginal (g510.03) and results remained
within the 95 percent confidence interval. The overall influ-
ence of publication bias on the analyses was determined to be
acceptable based on three criteria: (1) Fail-Safe Nof 330
studies were needed to nullify significant results (i.e.,
p..05), (2) a symmetrical funnel plot was observed, and
(3) the ‘‘Trim & Fill’’ procedure added 9 studies to the right
of the mean effect and decreased the summary effect size
Analysis of Neurocognitive Test Modules
The results supported both positive and negative outcomes
that ranged from 0.17 to 20.27 (see Table 2). Neurocognitive
test modules with positive effect sizes (e.g., complex reaction
time) were indicative of performance that was equivalent to
or greater than baseline concussion scores or controls. As
expected, NCT modules including code substitution (k56;
g520.27; Z522.22; p,.05), visual memory (k520;
g520.25; Z523.45; p,.05), processing speed (k518;
g520.18; Z522.68; p,.05), and composite memory
(k525; g520.21; Z524.07; p,.05) demonstrated
negative effects for concussion. Surprisingly, complex reac-
tion time (k56; g50.17; Z52.99; p,.05) demonstrated
a positive effect for concussion. The matching (k57;
g520.03; Z520.18; p..05) and mathematical processing
Table 2. Outcome analyses
Effect size statistics Null test Heterogeneity statistics Publication bias
k g SE s
95% CI ZQt
Fail safe N
Random effects model
37 20.16 0.04 0.002 (20.24, 20.08) 23.82* 123.2* 0.04 70.78 330
Complex reaction time 6 0.17 0.06 0.003 (0.06, 0.28) 2.99* 7.07 0.01 29.28 15
Simple reaction time 34 20.11 0.05 0.002 (20.21, 20.02) 22.34* 147.4* 0.05 77.61 106
Code substitution 6 20.27 0.12 0.015 (20.51, 20.03) 22.22* 10.9 0.05 54.04 15
Matching 7 20.03 0.16 0.025 (20.34, 0.28) 20.18 29.0 0.13 79.30 0
Mathematical processing 5 0.12 0.10 0.010 (20.08, 0.31) 1.17 5.75 0.02 30.37 0
Memory composite 25 20.21 0.05 0.003 (20.31, 20.11) 24.07* 103.9* 0.05 76.90 337
Verbal memory 19 20.17 0.07 0.004 (20.30, 20.04) 22.62* 106.9* 0.06 83.18 134
Visual memory 20 20.25 0.07 0.005 (20.39, 20.11) 23.45* 141.6* 0.04 86.58 269
Processing speed 18 20.18 0.07 0.004 (20.30, 20.03) 22.68* 88.5* 0.03 80.78 78
Symptoms 24 20.23 0.07 0.005 (20.37, 20.08) 23.08* 165.6* 0.10 86.11 300
Note.k5number of effect sizes. g5effect size (Hedges g). SE 5standard error. s
5variance. 95% CI 5confidence intervals (lower limit, upper limit).
Z5test of null hypothesis. t
5between study variance in random effects model. I
5total variance explained by moderator. *indicates p,.05. a 5Total
Q-value used to determine heterogeneity.
328 A.P. Kontos et al.
(k55; g50.12; Z521.17; p..05) modules demonstrated
no significant effects. Large Q
-values (p,.05) were indica-
tive of heterogeneous distributions and corresponding
-values greater than 70 supported the need to conduct sub-
group analyses. Smaller sample sizes prevented moderator
analyses within NCT modules. Fail-Safe Ncalculations
suggested that there was the potential for publication bias as
missing studies would have nullified significant results.
In summary, the results support mostly negative, though
small effect sizes combined with large variations between
studies for the individual NCT modules.
Subgroup Analyses
The results of the moderator analyses supported only one
significant (Q
513.75; p,.05) between groups effect for
age (see Table 3). Specifically, neurocognitive performance
in younger adolescents was more impaired (k517;
g520.29) than both older adolescents (k514; g520.01)
and college aged athletes (k56; g520.11). Although the
results for the other moderators were not significant, several
statistical trends- where the overall model was not significant,
but the individual effects were significant- were supported.
Specifically, among NCT tests only studies using the
ImPACT NCT (k523; g520.19; Z523.74; p,.05)
demonstrated a negative effect for concussion. With regard to
sport type, a negative effect for concussion was evident only
for collision/contact based sports (k522; g520.20;
Z523.67; p,.05). Additionally, there was an effect for
test personnel background with both neuropsychologists
(k53; g520.37; Z523.74; p,.05) and researchers
Between study variance for all categories was relatively
Table 3. Subgroup Analyses
Effect size statistics Null test Heterogeneity statistics Publication bias
k g SE s
95% CI ZQt
Fail safe N
Random effects model
37 20.16 0.04 0.002 (20.24, 20.08) 23.82* 123.40* 0.04 70.83 330
Methodological characteristics
Research design 5.39
Case control 21 20.24 0.06 0.003 (20.35, 20.13) 24.23* 0.05 70.60
Cohort 7 20.12 0.08 0.007 (20.28, 0.04) 21.45 0.03 69.52
Cross-sectional 8 20.04 0.08 0.007 (20.20, 0.12) 20.54 0.01 47.70
Concussion Test 1.36
ANAM 6 20.10 0.12 0.014 (20.33, 0.13) 20.87 0.03 44.22
CogSport/AXON 4 20.04 0.17 0.029 (20.38, 0.29) 20.25 0.08 48.35
Headminder 4 20.10 0.12 0.013 (20.33, 0.12) 20.88 0.02 74.62
ImPACT 23 20.19 0.05 0.003 (20.29, 20.09) 23.74* 0.04 74.23
Personnel background 3.46
Athletic trainer 9 20.15 0.10 0.009 (20.33, 0.04) 21.55 0.06 64.89
Neuropsychologist 3 20.37 0.18 0.034 (20.72, 20.01) 21.99* 0.00 00.00
Physician 9 20.09 0.08 0.006 (20.24, 0.06) 21.20 0.04 81.06
Researcher 15 20.21 0.07 0.004 (20.34, 20.08) 23.09* 0.04 72.39
Sport type 2.11
All 10 20.13 0.07 0.005 (20.26, 0.01) 21.91 0.04 80.51
Contact/collision 22 20.21 0.06 0.003 (20.32, 20.10) 23.67* 0.04 56.56
Not-reported 5 20.04 0.11 0.012 (20.26, 0.17) 20.37 0.02 53.07
Sample characteristics
Sex 1.22
Females & males 24 20.15 0.05 0.002 (20.24, 20.05) 23.01* 0.03 70.54
Males only 12 20.17 0.08 0.006 (20.33, 20.02) 22.18* 0.07 70.17
Age 13.75*
College 6 20.11 0.08 0.007 (20.27, 0.050) 21.35 0.03 69.21
Older adolescents 14 20.01 0.06 0.003 (20.12, 0.103) 20.12 0.01 24.89
Younger adolescents 17 20.29 0.05 0.003 (20.40, 20.185) 25.42* 0.04 64.08
Study characteristics
Granted funded 0.086
Funded 11 20.14 0.08 0.007 (20.30, 0.03) 21.64 0.043 63.46
Unfunded 26 20.17 0.05 0.002 (20.26, 20.07) 23.42* 0.036 73.51
Sample 1.195
Between subjects 18 20.22 0.07 0.004 (20.34, 20.09) 23.29* 0.071 72.85
Within subjects 19 20.12 0.06 0.003 (20.23, 20.01) 22.17* 0.025 69.27
Note.k5number of effect sizes. g5effect size (Hedges g). SE 5standard error. s
5variance. 95% CI 5confidence intervals (lower limit, upper limit).
Z5test of null hypothesis. t
5between study variance in random effects model. I
5total variance explained by moderator. * indicates p,.05. a 5Total
Q-value used to determine heterogeneity. b 5Between Q-value used to determine significance (a,0.05).
Meta-analysis of computerized neurocognitive testing 329
small, however, moderate to large I
-values for most mod-
erators reflects variability between studies. Overall, there
were significant negative effects for age and trends for NCT
test and sport type.
The current study represents the largest meta-analysis
(k537) to date focusing on the neurocognitive effects of
concussion as measured by computerized NCTs, which are
commonly used by high schools, colleges, and non-scholastic
sport organizations. This study also included subgroup ana-
lyses of age, sport, NCT test modules, and computerized
NCT test type, which have been collectively absent from
previous meta-analyses. Overall, the results supported a low
to moderate- based on Cohen’s (1988) criteria that establishes
the magnitude for effect sizes- overall effect size for con-
cussion as measured by computerized NCTs (g52.17).
Previous studies have reported effects sizes of g520.81
(Broglio & Puetz, 2008) and g520.54 (Dougan et al.,
2013). These previously reported effect sizes may differ from
the current study due to dissimilar (i.e., less stringent)
exclusion criteria, definitions of concussion, and inclusion
of moderators across studies. Typically, if more stringent
criteria for study inclusion are used, previously reported
effect sizes for a relationship may decrease. Moreover, the
current and previous research did not account for individual
risk factors (e.g., ADHD/LD, migraine history), which may
influence the effects of concussion. Although other
researchers have considered concussion history (e.g., Dougan
et al., 2013) in their meta-analysis, the inclusion of self-reported
recall of injury history is susceptible to recall bias and is
varied in definition and criteria (e.g., suspected concussion
vs. diagnosed concussion) from study to study. Additionally,
it is important to note that because the current study included
only studies using computerized NCTs, the data were
more current. Consequently, the samples from the studies
included in the current study may more closely reflect current
definitions of concussion than those in previous studies,
which included studies with samples of concussed athletes
that were determined using injury criteria (e.g., loss of
consciousness, post-traumatic amnesia) that excluded milder
forms of concussion.
The low to moderate magnitude of the effect size in the
current study is more likely a reflection of the complex and
individualized nature of concussion and its effects than it is
a reflection of decreased utility of computerized NCTs.
Although there is consistent support in the literature for
cognitive deficits following concussion, some individuals
may experience limited cognitive deficits and instead
experience other effects such as migraine-like, vestibular,
ocular-motor, affective, sleep-related. We believe that this
individualized nature of the injury will result in mostly low to
moderate effect sizes in subsequent meta-analytic analyses of
the various effects of concussion. As a result, researchers and
clinicians may need to re-conceptualize concussion into
specific clinical trajectories that require targeted therapies
and treatments. In other words, as indicated in expert con-
sensus (e.g., McCrory et al., 2013), no single tool can or
should be used to measure the effect of concussion. Instead,
clinicians and researchers should adopt a comprehensive
approach to assessing this injury.
Among computerized NCTs, studies using the ImPACT
test detected the highest effect size (g52.19). This finding
may reflect the fact that this test was the most frequently
(k523) used by researchers of studies included in the current
analysis. However, this finding may also be reflective of the
types of cognitive tasks that comprise this NCT. In support of
this notion, the ImPACT NCT includes a balanced set of
component tasks that incorporated the four strongest effect
sizes reported in this study: visual memory (g52.27), code
substitution (g52.25), processing speed (g52.21), and
composite memory (g52.17). Other tasks such as matching
and mathematical processing did not show any significant
effect for concussion. Surprisingly, the direction of the effect
size for complex RT (g5.17) was in the opposite direction
(i.e., performance improved with concussion) of what was
anticipated, suggesting that this task and NCTs that rely on it
may have limited utility to detect the effects of concussion.
As expected, the adolescent age group experienced the
greatest effect size (g52.29) for concussion. This finding
lends further support to the notion that younger athletes are at
greatest risk from the effects of concussion (Field et al.,
2003), and corroborates initial findings recently reported by
Dougan et al. (2013). With regard to sport type, not surpris-
ingly, the effect size for collision/contact sports (g52.21)
such as football, rugby, and soccer was the only significant
effect size for concussion. However, most samples in the
literature include sport types that are aggregated such that
there is no way to compare specific sport types, let alone
individual sports, within or across studies. The findings also
supported an overall effect for computerized NCT perfor-
mance and test personnel background, with significant effects
reported in studies where neuropsychologists (g520.37) or
researchers (g520.21) administered the tests. In studies
where tests were administered by ATCs or physicians
the effects associated with concussion on NCTs were not
significant. This disparity may be due to differences in
training and familiarity with the tests between these different
personnel. However, these findings may also reflect the
environment in which the testing occurred. For example,
ATCs and physicians may have been testing in less than ideal
conditions such as on site at sport facilities, where distrac-
tions may have influenced the test results at both baseline
and post-injury.
The current study was limited by several factors that
should be considered when interpreting results. The defini-
tion of concussion varied across studies, which potentially
limited the effect sizes for concussion reported in the study.
Due to the lack of reported information in studies we were
unable to include demographic variables that are known to
influence concussion effects including concussion history
and sex, in our subgroup analyses. This exclusion was due to
incomplete reporting or combining of these variables by
330 A.P. Kontos et al.
researchers for the included studies. For example, male and
female athletes were often combined into one group with no
additional direct sex comparisons. Often the sample size for
the females in these studies was very small, which did not
allow the researchers to conduct further analyses comparing
males and females. In so doing, we could not infer an effect
for sex on NCT performance following concussion per se.
Previously, researchers (Dougan et al., 2013) reported higher
NCT effects for concussion in females than males, suggesting
that future research should examine sex differences. We also
did not account for differences in time since injury across
studies. The studies in the current paper were limited to those
conducted within the first week of injury, thereby limiting
our generalizability beyond this acute injury time period.
Most studies included in the sample were conducted 2–7 days
post-injury. Finally with regard to the meta-analytic techni-
ques, publication bias is always a concern when potentially
relevant studies are not retrieved during the literature search
process or excluded from the analysis for variety of reasons
(Rothstein et al., 2005). While established techniques (see the
Methods section) were used to control for publication bias at
least 15 potential studies were eliminated from the analysis.
The influence of missing studies has been approximated and
our strategy was to provide a conservation interpretation and
suggestions for future research.
One indirect conclusion that can be drawn from the results
of this study is that researchers and publishers alike should
include complete statistics in papers. The current study
resulted in the exclusion of 110/147 (75%) studies involving
concussion the met inclusion criteria for the study failed to
report sufficient statistical data to be included in the analysis.
Moreover, researchers need to delineate in their studies
among sex, age, sports, and concussion history subgroups to
allow for assessments of the role of these factors on the
effects of concussion. As the concussion literature and med-
ical field in general moves toward more outcomes-oriented
research involving systematic and meta-analytic reviews,
providing complete statistical data and more information
about subgroups will become even more important.
To our knowledge, the current study serves as the largest
and sole meta-analysis to date on the effect of concussion as
measured using computerized NCTs. This study also incor-
porated moderators or subgroup analyses that were not
considered collectively in one study in previous research. The
low to moderate overall effect sizes reported here may reflect
the complex and individualized nature of concussion and its
effects on athletes. The subgroup analyses revealed different
effect sizes for specific neurocognitive tasks, brands of
NCTs, age, and type of sport that need to be considered by
researchers and clinicians alike. The individualized nature of
concussion and myriad risk and moderating factors together
with the relatively small overall ES reported here suggest that
other impairments in addition to neurocognitive, such ves-
tibular and ocular-motor may occur following concussion.
As such, more comprehensive assessments and approaches to
treatment may be warranted following concussion.
This study was not funded. The information in this manuscript, and
the manuscript itself, has never been published in any format. None
of the authors have any conflict of interest to declare and do not have
any affiliation with any software companies or products mentioned
in this article. There is no financial or other benefit to be gained by
any of these companies due to the publishing of the present results.
Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K.,
Kelly, J., ySchamasch, P. (2002). Summary and agreement
statement of the first international conference on concussion in
sport, vienna 2001. Physician and Sportsmedicine,30, 57–63.
Belanger, H.G., Spiegel, E., & Vanderploeg, R.D. (2010).
Neuropsychological performance following a history of multiple
self-reported concussions: A meta-analysis. Journal of the
International Neuropsychological Society,16, 262–267.
Borenstein, M., Hedges, L., Higgins, J.P., & Rothstein, H.R. (2005).
Comprehensive meta-analysis version 2. Engelwood, NJ: Biostat.
Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R.
(2009). Introduction to meta-analysis. Chichester, UK: Wiley.
Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R.
(2010). A basic introduction to fixed effect and random effects
models for meta-analysis. Research Synthesis Methods,1, 97–111.
Broglio, S.P., & Puetz, T.W. (2008). The effect of sport concussion
on neurocognitive function, self-report symptoms and postural
control: A meta-analysis. Sports Medicine,38, 53–67.
Cohen, J. (1988). Statistical power analysis for the behavioral
sciences (2nd ed.). Hillsdale, NJ: Lawerence Erlbaum Associates.
Collins, M.W., Iverson, G.L., Lovell, M.R., McKeag, D.B.,
Norwig, J., & Maroon, J. (2003). On-field predictors of
neuropsychological and symptom deficit following sports-related
concussion. Clinical Journal of Sport Medicine,13, 222–229.
Dougan, B.K., Horswill, M.S., & Geffen, G.M. (2013). Athletes’
age, sex, and years of education moderate the acute neuropsy-
chological impact of sports-related concussion: A meta-analysis.
Journal of the International Neuropsychological Society, 1–17.
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-
plot-based method of testing and adjusting for publication bias in
meta-analysis. Biometrics,56, 455–463.
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997).
Bias in meta-analysis detected by a simple, graphical test. British
Medical Journal,315, 629–634.
Fazio, V.C., Lovell, M.R., Pardini, J.E., & Collins, M.W. (2007).
The relation between post concussion symptoms and neurocog-
nitive performance in concussed athletes. Neurorehabilitation,
22, 207–216.
Field, A.P. (2001). Meta-analysis of correlation coefficients: A
Monte Carlo comparison of fixed- and random-effects methods.
Psychological Methods,6, 161–180.
Field, A.P. (2003). Can meta-analysis be trusted? The Psychologist,
16, 642–645.
Field, M., Collins, M.W., Lovell, M.R., & Maroon, J. (2003). Does
age play a role in recovery from sports-related concussion?
A comparison of high school and collegiate athletes. Journal of
Pediatrics,142, 546–553.
Meta-analysis of computerized neurocognitive testing 331
Guskiewicz, K., Weaver, N.L., Padua, D.A., & Garrett, W.E. (2000).
Epidemiology of concussion in collegiate and high school football
players. American Journal of Sports Medicine,28, 643–650.
Hedges, L.V. (1981). Distribution theory for glass’s estimator of
effect size and related estimators. Journal of Educational
Statistics,6, 107–128.
Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-
analysis. Orlando, FL: Academic Press.
Higgins, J.P., Thompson, S.G., Deeks, J.J., & Altman, D.G. (2003).
Measuring inconsistency in meta-analyses. British Medical
Journal,327, 557–560.
Hunter, J.E., & Schmidt, F.L. (2000). Fixed effects vs. random
effects meta-analysis models: Implications for cumulative knowl-
edge in psychology. International Journal of Selection and
Assessment,8, 275–292.
Johnson, E.W., Kegel, N.E., & Collins, M.W. (2011). Neuropsy-
chological assessment of sport-related concussion. Clinics in
Sports Medicine,30, 73–88.
Langlois, J.A., Rutland-Brown, W., & Wald, M.M. (2006). The
epidemiology and impact of traumatic brain injury: A brief
overview. Journal of Head Trauma Rehabilitation,21, 375–378.
Light, R.J., & Pillemer, D.B. (1984). Summing up: The science of
reviewing research. Cambridge, MA: Harvard University Press.
Mayers, L.B., & Redick, T.S. (2012). Clinical utility of ImPACT
assessment for postconcussion return-to-play counseling:
Psychometric issues. Journal of Clinical and Experimental
Neuropsychology,34, 235–242.
McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M.,
Molloy, M., & Cantu, R. (2009). Consensus statement on
Concussion in Sport 3rd International Conference on Concussion
in Sport held in Zurich, November 2008. Clinical Journal of
Sport Medicine,19, 185–200.
Randolph, C. (2011). Baseline neuropsychological testing in
managing sport-related concussion: Does it modify risk? Current
Sports Medicine Reports,10, 21–26.
Report on Trends and Participation in Youth Sports. (2001). Stuart,
FL: National Council of Youth Sports.
Rosenthal, R. (1979). The ‘‘File Drawer Problem’’ and tolerance for
null results. Psychological Bulletin,86, 638–641.
Rothstein, H.R., Sutton, A.J., & Borenstein, M. (2005). Publication
bias in meta-analysis: Prevention assessment and adjustments.
Chichester, UK: John Wiley.
Schatz, P., Kontos, A., & Elbin, R. (2012). Response to Mayers and
Redick: ‘‘Clinical utility of ImPACT assessment for postconcus-
sion return-to-play counseling: Psychometric issues’’. Journal of
Clinical and Experimental Neuropsychology,34, 428–434.
Shaddish, W.R., & Sweeney, R.B. (1991). Mediators and
moderators in meta-analysis: There’s a reason we don’t let dodo
birds tell us which psychotherapies should have prizes. Journal of
Consulting and Clinical Psychology,59, 883–893.
332 A.P. Kontos et al.
... This allows for grading of initial symptomology in the immediate post-concussion phase alongside tracking of recovery over the following days. [29][30][31] Neurocognitive tests often follow a model of baseline and post-concussion scoring, whereby a pre-injury baseline is established, often preseason, with post-injury testing directly compared with this facilitating tracking of recovery to the athletes own baseline over time. 31 It has been established that neurocognitive deficits remain following symptomatic recovery, 32,33 indicating that recurrent neurocognitive testing, rather than symptomatic improvement in isolation, should be used to guide return to activity. ...
... When directly comparing multiple mechanisms within one week of injury, a meta-analysis demonstrated a low to moderate effect size of concussion on neurocognitive functioning. 30 Of the testing mechanisms studied, the ImPACT test, a computerized battery of neurocognitive testing allowing for baseline and post-injury testing, demonstrated the highest effect size in direct comparison with other testing batteries. 30 This approach of pre-injury baseline testing, with neurocognitive testing in the acute and subacute phase post injury, has been widely accepted in the care of cricket concussions by the major cricket boards, including the England and Wales Cricket board (ECB) and Cricket Australia (CA), 35,36 and is in widespread use around the professional game. ...
... 30 Of the testing mechanisms studied, the ImPACT test, a computerized battery of neurocognitive testing allowing for baseline and post-injury testing, demonstrated the highest effect size in direct comparison with other testing batteries. 30 This approach of pre-injury baseline testing, with neurocognitive testing in the acute and subacute phase post injury, has been widely accepted in the care of cricket concussions by the major cricket boards, including the England and Wales Cricket board (ECB) and Cricket Australia (CA), 35,36 and is in widespread use around the professional game. ...
Full-text available
Concussion following head injury remains a significant cause of injury in cricket, with recent high profile events highlighting the ongoing risk of head injury in the sport and the emerging use of concussion assessments for elite players following injury. Sport‐related concussion has long been recognized to present significant concerns for athletes in both the short and long term, with an increasing evidence base surrounding diagnostic and prognostic approaches to the issue. Though cricket represents one of the most participated sports worldwide, there is a lack of evidence relating to the acute management and prognosis of head injury in cricket players. In this review, we searched the literature for terms related to head injury, concussion, and cricket; summarizing the main findings regarding the incidence, mortality, acute management, diagnostic approach and outcome of cricket related head injury. Further, this review places these findings in the wider context of sports‐related concussion literature to ascertain both the current evidence base surrounding current cricket related concussion guidelines, and the direction of research for future approaches to sports concussion management.
... Because cognitive dysfunction has been demonstrated to take longer to resolve than the self-reported symptoms (Carman et al., 2015), computerized neurocognitive tests have become the most commonly used objective clinical measure of mTBI. The Immediate Post-Concussion Assessment and Cognitive Test (ImPACT; Kontos et al., 2014;Higgins et al., 2017) is widely used in both youth and NCAA sports for concussion management in the United States. ImPACT was found to have 81.9% sensitivity and 89.4% specificity to mTBI (Schatz et al., 2006) and generates composite scores for four domains: Verbal Memory, Visual Memory, Processing Speed, and Reaction Time, of which visual memory, and reaction time are the most sensitive domains to the cognitive changes following concussion (Covassin et al., 2007;Majerske et al., 2008). ...
... ImPACT was found to have 81.9% sensitivity and 89.4% specificity to mTBI (Schatz et al., 2006) and generates composite scores for four domains: Verbal Memory, Visual Memory, Processing Speed, and Reaction Time, of which visual memory, and reaction time are the most sensitive domains to the cognitive changes following concussion (Covassin et al., 2007;Majerske et al., 2008). Other computerized neurocognitive assessment tools such as the ANAM TM Core Battery (Kane et al., 2007) can measure changes in attention, working memory, and cognitive efficiency, and may therefore be even more sensitive to the long-term effects of mTBI on brain health (Covassin et al., 2007;Kane et al., 2007;Majerske et al., 2008;Kontos et al., 2014;Martini et al., 2017). Preliminary data from our cohort (Gorgens et al., unpublished observations) strongly suggest that executive function, reaction time, and processing speed should be included in the routine assessment of PCS, since these cognitive changes appear to linger after most other cognitive functions have returned to baseline. ...
Full-text available
Concussion or mild traumatic brain injury (mTBI) in athletes can cause persistent symptoms, known as post-concussion syndrome (PCS), and repeated injuries may increase the long-term risk for an athlete to develop neurodegenerative diseases such as chronic traumatic encephalopathy (CTE), and Alzheimer’s disease (AD). The Center for Disease Control estimates that up to 3.8 million sport-related mTBI are reported each year in the United States. Despite the magnitude of the phenomenon, there is a current lack of comprehensive prognostic indicators and research has shown that available monitoring tools are moderately sensitive to short-term concussion effects but less sensitive to long-term consequences. The overall aim of this review is to discuss novel, quantitative, and objective measurements that can predict long-term outcomes following repeated sports-related mTBIs. The specific objectives were (1) to provide an overview of the current clinical and biomechanical tools available to health practitioners to ensure recovery after mTBIs, (2) to synthesize potential biological mechanisms in animal models underlying the long-term adverse consequences of mTBIs, (3) to discuss the possible link between repeated mTBI and neurodegenerative diseases, and (4) to discuss the current knowledge about fluid biomarkers for mTBIs with a focus on novel exosomal biomarkers. The conclusions from this review are that current post-concussion clinical tests are not sufficiently sensitive to injury and do not accurately quantify post-concussion alterations associated with repeated mTBIs. In the current review, it is proposed that current practices should be amended to include a repeated symptom inventory, a cognitive assessment of executive function and impulse control, an instrumented assessment of balance, vestibulo-ocular assessments, and an improved panel of blood or exosome biomarkers.
... Though we observed fairly consistent RT deficits acutely, our mean effects differ from a previous meta-analysis examining only computerized neurocognitive testing outcomes 0-7 days post-injury [76]. Kontos et al. analyzed 37 RT effects from 37 studies using any computerized neurocognitive testing platform following concussion and observed mixed RT cumulative effects for simple and complex RT measures. ...
... Kontos et al. analyzed 37 RT effects from 37 studies using any computerized neurocognitive testing platform following concussion and observed mixed RT cumulative effects for simple and complex RT measures. In our moderator analyses only examining computerized neurocognitive testing (D+ range − 0.3735, − 1.4799) and RT measure types (D+ range − 0.6409, − 0.9826), we observed medium-to-large negative effects which still suggest larger magnitude RT deficits post-concussion than the previous work [76]. Differences between their observed mean effect sizes and ours are likely driven by different methods employed. ...
Full-text available
Background Reaction time (RT) deficits are reported following concussion, but it is unknown when these deficits normalize to pre-injury status. It is also unclear how factors such as RT measurement technique and participant characteristics influence post-concussion RT.Objective The purpose of this systematic review and meta-analysis was to (1) characterize acute post-concussion (0–3 days) RT impairments, (2) examine RT recovery over time, and (3) explore moderating factors related to acute RT impairment following concussion.Methods Database searches (PubMed, CINAHL, EBSCOhost) were conducted according to PRISMA guidelines for articles published in English from January 2002 to March 2019. Studies compared baseline-to-post-injury RT within individuals (within-subject) and/or RT in concussed individuals to non-concussed controls (between-subject). Sixty studies met inclusion criteria, reporting on a total of 9688 participants with 214 discrete RT effects (Hedges’ d; between-subject: N = 29, k = 129; within-subject: N = 42, k = 85). Of the 214 effects, 93 occurred in the acute (0–3 days) post-injury timeframe (k = 47 between-subject). Numerous demographic [sex, age, concussion history, population type (athlete, military, and general population), athlete level (high school, college), and sport], and method-based (RT test and measure type, computerized neurocognitive testing platform, concussion definition, and time post-injury) moderators were examined for mean effect influence. Mixed-effects multi-level modeling with restricted-maximum-likelihood estimation was used to account for nested effects and high heterogeneity for the pooled effect size (D+).ResultsSignificant medium-magnitude RT deficits were observed acutely for between- (D+ = − 0.7279, 95% CI − 0.9919, − 0.4639, I2 = 88.66, p < 0.0001) and within-subject (D+ = − 0.7472, 95% CI − 0.9089, − 0.5855, I2 = 89.21, p < 0.0001) effect models. RT deficits were present at the sub-acute and intermediate-term timeframes for between-subject effects (sub-acute: D+ = − 0.5655, 95% CI − 0.6958, − 0.4352, p < 0.0001; intermediate-term: D+ = − 0.3219, 95% CI − 0.5988, − 0.0450, p = 0.0245). No significant RT mean effect was observed for the between-subject model at the long-term timeframe, indicating RT recovery among concussed participants relative to controls (D+ = 0.3505, 95% CI − 0.4787, 1.1797, p = 0.3639). Sex was a significant moderator for between-subject effects, with every 1% male sample size increase demonstrating − 0.0171 (95% CI − 0.0312, − 0.0029, p = 0.0193) larger RT deficits. Within-subject effect models resulted in RT measure type (simple: [D+ = − 0.9826] vs. mixed: [D+ = − 0.6557], p = 0.0438) and computerized neurocognitive testing platforms (ANAM: [D+ = − 0.3735] vs. HeadMinder CRI: [D+ = − 1.4799] vs. ImPACT: [D+ = − 0.6749], p = 0.0004) having significantly different RT-deficit magnitudes. No other moderators produced significantly different RT-deficit magnitudes (between-subject: [p ≥ 0.0763], within-subject: [p ≥ 0.1723]).Conclusions Robust RT deficits were observed acutely following concussion. Minimal magnitude differences were noted when comparing between- and within-subject effects, suggesting that pre-injury baselines may not add clinical value in determining post-injury RT impairment. RT deficits persisted up till the intermediate-term (21–59 days post-injury) timeframe and indicate lingering deficits exist. Mean effect size differences were observed between RT measure types and computerized neurocognitive testing platforms; however, all categories displayed negative effects consistent with impaired RT following concussion. Clinical interpretation suggests that measuring RT post-concussion is more important than considering the RT method employed so long as reliable and valid tools are used. PROSPERO Registration #CRD42019119323.
... Although the value of these assessments has been heavily debated in the literature, 5,6 there is strong empirical evidence in support of cognitive testing to assist athletic trainers with concussionrelated clinical decisions. [7][8][9] More specifically, evidence has suggested that many of the instruments currently in use by athletic trainers and other health care professionals suffer from poor reliability, which could lead to potential misdiagnoses. 9 The BrainFx 360 performance assessment (BrainFx, Toronto, Ontario, Canada) was designed in 2013 as a "comprehensive assessment that measures the cognitive, physical, and psychosocial areas of neurofunction through engaging and interactive activities delivered via tablet by a certified BrainFx administrator (CBA) that can be influenced by brain disorders" (CEO of BrainFx, personal communication, 2015). ...
... Visual Perception Skills Patient is provided a series of tasks that include the following: [1] Scanning-a two digit number is highlighted to match on screen with same numbers in a different order; [2] Visual closure-a dotted drawing is shown and needs to choose which of four solid drawings match it; [3] Spatial-a shape is cut out of a solid rectangle and needs to choose with of the four shapes was the one cut out; [4] A word is presented where the patient needs to select from four options that the word fits inside, [5] Rotation-a drawing is presented and it needs to be drawn to a specified rotation; [6] Mirror Image-a drawing is presented that needs to be completed via a mirror image; [7] A drawing is presented that needs to be copied; [8] Draw a Clock-instruction is provided to draw the face of a clock with it set to a specific time. ...
Full-text available
Purpose: To examine the reliability of the BrainFx 360 (BrainFx, Toronto, Ontario, Canada) digital assessment tool for detection of mild-to-moderate symptoms following concussion. Methods: Fifteen healthy adults were administered the BrainFx 360 at two time points. Reliability was assessed using two-way random effects intraclass correlation coefficients (ICCs) for average measures. Results: Reliability was high for the overall performance score (ICC = .85). Reliability of individual tests ranged from low to high within each domain of cognitive function. Conclusion: The BrainFx 360 is a promising new instrument. Reliability of the scores for overall performance and performance categories were relatively high; however, reliability was much lower for the individual cognitive outcomes associated with each task. Application of advanced measurement models may be useful in identifying specific tasks with poor measurement properties.
... Neuropsychologische Testung: Verschiedene Untersuchungen zeigen, dass nach Gehirnerschütterung mit relevanten neurokognitiven Störungen zu rechnen ist. Mittels computerbasierten neuropsychologischen Testungen kann die Sensitivität und Spezifität der Erholung im Vergleich zur alleinigen Symptombeurteilung verbessert werden (29,30). ...
... Athletes with an SRC may present with neurocognitive impairments, balance deficits, and various symptoms associated with the injury. 1 Despite these health impairments, significant recovery has been reported to occur in as little as 2 weeks. 2,3 However, up to 15% of athletes with an SRC may experience a protracted recovery lasting longer than 30 days. 4 In an effort to combat this protracted recovery, cognitive and physical rest have often been utilized when managing athletes with an SRC. ...
Context: Cognitive and physical rest are commonly utilized when managing a sport-related concussion (SRC); however, emerging research now suggests that excessive rest may negatively impact recovery. Despite current research recommendations, athletic trainers (ATs) may be behind in implementing this emerging research into clinical practice. Objective: To assess college ATs' perceptions and implementation of an emerging SRC management approach (cognitive and physical rest and activity). Design: Cross-sectional study. Setting: Survey. Participants: A total of 122 (11.8%) ATs (53.3% female; 10.8 [9.8] y experience; 8.7 [6.9] SRCs managed annually) responded to the survey, which was randomly distributed to 1000 members of the National Athletic Trainers' Association, as well as 31 additional ATs from varying universities. Main outcome measures: A 5-point Likert scale assessed the ATs' perceptions and clinical practices as they relate to specific athlete behaviors (ie, texting, sleeping). The ATs were asked about their willingness to incorporate physical activity into clinical practice. Results: Playing video games (95.9%) and practicing (93.4%) were the activities most perceived to extend SRC recovery. However, sleeping more than usual (7.4%) and increased time in a dark environment (11.5%) were viewed as less likely to extend recovery. ATs restricted practicing (98.4%) and working out (91.8%) for athletes with SRC, while sleeping more than usual (6.6%) and increased time in a dark environment (13.1%) were less restricted. About 71% of the ATs would implement light physical activity for athletes with a symptom score of 1 to 5, 31% with scores of 6 to 10, and 15% with scores of 11 to 20. About 43%, 74%, and 97% believe that light, moderate, and vigorous physical activity, while symptomatic, will extend recovery, respectively. Conclusions: The ATs were receptive to including light physical activity into their SRC management, although only in certain situations. However, most ATs' beliefs and clinical practices did not completely align with emerging research recommendations for the management of SRCs.
... One alternative explanation for this finding pertains to the limitations in sensitivity of neurocognitive testing identifying deficits beyond 3 days after injury. 33 Previous research suggests neurocognitive testing is most sensitive within the first 3 days after injury, 30 and our study included participants 1 to 10 days after injury. Only the concussed athletes with SD had low neurocognitive test scores in addition to higher symptom scores compared with all other groups, potentially suggesting worse injuries. ...
Background: Sleep dysfunction (SD) is associated with a high symptom burden and lower neurocognitive performance after concussion and on baseline testing without injury. However, few studies have compared concussed athletes and controls with and without SD on clinical outcomes. Objective: To evaluate differences in clinical outcomes among both concussed athletes and matched controls with and without SD. Design: Retrospective cross-sectional study. Participants: Participants aged 12 to 20 years were recruited from a concussion clinic (n = 50 patients) and research registry/flyers (n = 50 healthy age-/sex-matched controls). Participants were categorized by self-reported SD into one of 4 groups: sport-related concussion (SRC) + SD, SRC only, SD only, and controls. Main outcome measures: Post-Concussion Symptom Scale (PCSS), Vestibular/Oculomotor Screening (VOMS), and neurocognitive testing (Immediate Postconcussion Assessment Cognitive Test). Results: Compared with the SRC only group, the SRC + SD group performed worse on all neurocognitive domains, had a higher total symptom score, and endorsed more symptoms on most VOMS items. In addition, the SRC + SD group was at an increased likelihood of having at least 1 abnormal VOMS item compared with SRC only group. The SRC only group had neurocognitive test scores and symptom reports statistically similar to the SD only group. Conclusion: Sleep dysfunction after concussion is related to worse neurocognitive performance and higher concussion symptom reporting. This study extended findings to suggest vestibular symptomology is worse among athletes with SD after injury compared to injured athletes without SD. Similar performances on concussion assessments for the SRC only and SD only groups suggest SD may appear similar to clinical presentation of concussion, even at baseline in the absence of SRC.
... 8 It is well established that many adolescent patients with sport-related concussion experience transient cognitive decline within the first days to weeks post injury, 33,73,97,125 with some studies suggesting that deficits can persist for a few months following concussion in a small subset of patients. 88,147 Although deficits are often mild 72 and some patients do not notice overt changes in mental capacity in day-to-day functioning, they exhibit impairments in attention, processing speed, and memory processes when evaluated on objective neurocognitive testing 48,127,143 (FIGURE 8). ...
Synopsis: Concussion is an ongoing concern for health care providers, as incidence rates continue to be high and the rate of recovery is variable due to potential risk factors. With no valid biomarkers at present, the diagnosis and assessment of concussion remains a clinical challenge. The heterogeneity in presentation following injury provides an additional level of complexity, as it requires that clinicians are capable of screening and evaluating diverse body systems including oculomotor, vestibular, autonomic, psychiatric, cervical, and cognitive. While a few tools such as the VOMS and BESS have been developed specifically for use with concussion, the vast majority of tests are adapted from other conditions. Further complicating the process is the overlapping and interactive nature of the multiple domains of post-concussion presentation. This commentary will illustrate how clinicians may conceptualize the multiple profiles that present following concussion and tools that are available to assist with screening and evaluation of each area. The multi-faceted nature of concussion warrants broad clinical screening skills and an interdisciplinary approach to management. J Orthop Sports Phys Ther, Epub 9 Oct 2019. doi:10.2519/jospt.2019.8855.
Full-text available
Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.
IntroductionIndividual studiesThe summary effectHeterogeneity of effect sizesSummary points
Publication bias is the tendency to decide to publish a study based on the results of the study, rather than on the basis of its theoretical or methodological quality. It can arise from selective publication of favorable results, or of statistically significant results. This threatens the validity of conclusions drawn from reviews of published scientific research. Meta-analysis is now used in numerous scientific disciplines, summarizing quantitative evidence from multiple studies. If the literature being synthesised has been affected by publication bias, this in turn biases the meta-analytic results, potentially producing overstated conclusions. Publication Bias in Meta-Analysis examines the different types of publication bias, and presents the methods for estimating and reducing publication bias, or eliminating it altogether. Written by leading experts, adopting a practical and multidisciplinary approach. Provides comprehensive coverage of the topic including: • Different types of publication bias, • Mechanisms that may induce them, • Empirical evidence for their existence, • Statistical methods to address them, • Ways in which they can be avoided. • Features worked examples and common data sets throughout. • Explains and compares all available software used for analysing and reducing publication bias. • Accompanied by a website featuring software, data sets and further material. Publication Bias in Meta-Analysis adopts an inter-disciplinary approach and will make an excellent reference volume for any researchers and graduate students who conduct systematic reviews or meta-analyses. University and medical libraries, as well as pharmaceutical companies and government regulatory agencies, will also find this invaluable.
Assessment of concussion can be challenging for medical practitioners given the different factors associated with each individual injury. The use of neuropsychological testing provides an objective method in the evaluation and management of concussion. Over the last 20 years it has become increasingly useful in the realm of sports concussion and has been deemed a cornerstone of concussion management by the Concussion in Sport group at the International Symposia on Concussion in Sport. Neuropsychological assessment has evolved to using computer-based neurocognitive testing, which has become increasingly common over the last decade, especially in organized sports. Neuropsychological assessment has also proven to be effective in the detection of differences based on several individual factors, including age, gender, and history of prior concussion. Despite its documented value, neuropsychological assessment should be one of several tools used as part of the concussion assessment/management process.
Glass's estimator of effect size, the sample mean difference divided by the sample standard deviation, is studied in the context of an explicit statistical model. The exact distribution of Glass's estimator is obtained and the estimator is shown to have a small sample bias. The minimum variance unbiased estimator is obtained and shown to have uniformly smaller variance than Glass's (biased) estimator. Measurement error is shown to attenuate estimates of effect size and a correction is given. The effects of measurement invalidity are discussed. Expressions for weights that yield the most precise weighted estimate of effect size are also derived.