Accounting for expert performance: The devil is in the details

Full text

Accounting for expert performance: The devil is in the details
David Z. Hambrick
a,
⁎, Erik M. Altmann
a
, Frederick L. Oswald
b
, Elizabeth J. Meinz
c
,
Fernand Gobet
d
, Guillermo Campitelli
e
a
Department of Psychology, Michigan State University, United States
b
Department of Psychology, Rice University, United States
c
Department of Psychology, Southern Illinois University Edwardsville, United States
d
Institute of Psychology, Health, and Society, University of Liverpool, United Kingdom
e
School of Psychology and Social Science, Edith Cowan University, Australia
article info abstract
Article history:
Received 30 December 2013
Received in revised form 24 January 2014
Accepted 27 January 2014
Available online 24 February 2014
The deliberate practice view has generated a great deal of scientific and popular interest in
expert performance. At the same time, empirical evidence now indicates that deliberate
practice,while certainlyimportant, is not as important as Ericsson and colleagues have argued it is.
In particular, we (Hambrick, Oswald, Altmann, Meinz, Gobet, & Campitelli, 2014-this issue)found
that individual differences in accumulated amount of deliberate practice accounted for about
one-third of the reliable variance in performance in chess and music, leaving the majority of the
reliable variance unexplained and potentially explainable by other factors. Ericsson's (2014-this
issue) defense of the deliberate practice view, though vigorous, is undercut by contradictions,
oversights, and errors in his arguments and criticisms, several of which we describe here. We
reiterate that the task now is to develop and rigorously test falsifiable theories of expert
performance that take into account as many potentially relevant constructs as possible.
© 2014 Elsevier Inc. All rights reserved.
Keywords:
Expert performance
Deliberate practice
Talent
Ability
Intelligence
We credit Anders Ericsson for generating interest in expert
performance. Ericsson, Krampe, and Tesch-Römer's (1993)
study of musicians has been cited over 4000 times (Google
Scholar), and, as Ericsson (2013a) noted, was the “stimulus”for
what Malcolm Gladwell (2008) dubbed the “10,000 hour rule.”
Ericsson has made an important contribution to psychology.
The goal of our study (this issue) was to test Ericsson
et al.'s (1993) claim that “individual differences in ultimate
performance can largely be accounted for by differential amounts
of past and current levels of practice”(p. 392, emphasis added).
The claim was not supported: amount of deliberate practice
accounted for about a third of the reliable variance in perfor-
mance in music and chess, leaving the majority unexplained.
Ericsson (2014-this issue) claims we reject his view on a
“common sense basis”(p.18),butinfact,werejectitonthis
empirical basis.
Ericsson's (2014-this issue) defense of his view is, in our
view, unsuccessful for several reasons. First, he rejects evidence
that challenges his view even though he has used the same
type of evidence to support his view. Specifically, he criticizes
our analysis for ignoring “the effects of forgetting, injuries, and
accidents, along with the differential effects of different types
of practice at different ages and levels of expert performance”
(p. 4), but has never included all of these factors in his own
published analyses. Most notably, Ericsson et al. (1993) based
their conclusion about the great importance of deliberate
practice on the relationship between skill level in music and
a single variable: self-reported amount of practice alone. Our
reanalysis included studies that used Ericsson et al. as the
model for measuring and operationally defining deliberate
practice—indeed, our reanalysis included studies that Ericsson
has praised for rigor and cited as support for his view (e.g.,
Charness, Tuffiash, Krampe, Reingold, & Vasyukova, 2005). We
did no more than follow Ericsson's standards for evidence in
selecting studies for our reanalysis.
Second, Ericsson (2014-this issue) contradicts claims he
has made in the past. Most notably, Ericsson downplays the
Intelligence 45 (2014) 112–114
⁎Corresponding author at: Department of Psychology, Michigan State Univer-
sity, East Lansing, MI 48824, United States.
E-mail address: hambric3@msu.edu (D.Z. Hambrick).
0160-2896/$ –see front matter © 2014 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.intell.2014.01.007
Contents lists available at ScienceDirect
Intelligence

emphasis he has put on deliberate practice, now suggesting it
is just one of any number of factors other than innate talent
that could affect performance directly. That is, he states that
although we criticize him “for attributing too much emphasis
to the effects due to deliberate practice”(p. 3), he and his
colleagues were explicit that “there might be other types of
individual differences than those linked to innate talent”(p. 3).
However, in the past, Ericsson has argued that such non-talent
factors (e.g., motivation) affect performance indirectly through
deliberate practice. For example, he wrote, “The theoretical
framework of expert performance explains individual differ-
ences in attained performance by the factors that influence the
engagement in sustained extended deliberate practice, such
as motivation”(Ericsson, 2007, p. 4;seeDuckworth, Kirby,
Tsukayama, Berstein, & Ericsson, 2011). Hypothesizing that
certain factors affect individual differences in deliberate practice
is not the same as hypothesizing that these factors affect
individual differences in performance directly.
Third, Ericsson (2014-this issue) does not mention findings
that contradict his view. For example, he notes Horn and
Masunaga (2006) found no significant correlations between Go
ranking and scores on intelligence tests, but fails to note that
measures of Go-related performance did correlate significantly
with scores on intelligence tests (pb.01 for 50 of 56 rs;
see Masunaga & Horn, 2001). Similarly, he fails to note the
central result of Meinz and Hambrick's (2010) study of
sight-reading —the finding that there was no interaction
between deliberate practice and working memory capacity,
indicating that working memory capacity positively predicted
performance even at high levels of deliberate practice. This
finding is inconsistent with Ericsson’s hypothesis that mecha-
nisms acquired through deliberate practice enable circumven-
tion of basic cognitive capacities. Describing the results of
another study of sight-reading, Ericsson (2013b) claimed,
“Kopiez and Lee (2006) found that for musicians with lower
sight-reading skill there was a correlation with their working
memory. For musicians with a higher level of sight-reading skill
there was no significant relation between their performance
and their working memory”(p. 236). In this case, Ericsson's
error is one of commission: Kopiez and Lee reported no such
finding.
Finally, Ericsson (2014-this issue) criticizes others' research
based on what turn out to be material errors in his descriptions
of that research. For example, he writes, “It is surprising that
Hambrick et al. (2014-this issue) did not cite Howard's (2012)
data for evidence of an elite chess player, who had never studied
chess”(p. 14). If Howard had claimed he found evidence for an
elite chess player who had never studied chess, one might
wonder whether we did not report this finding because it seems
implausible and would cast doubt on the validity of Howard's
study, which we included in our re-analysis. However, Howard
(2012) made no such claim. Moreover, ratings of individual
players in Howard's sample cannot be determined from
Howard's published report. As another example, Ericsson
makes two errors in describing a study of chess by two of us.
First, he writes, “Data was collected from 104 respondents, but
Campitelli and Gobet's (2008) [sic] only analyzed 90 partici-
pants and did not describe the objective reasons for discarding
14 of the collected questionnaires”(p.14).Ifthisclaimwere
true, one would be well advised to dismiss the results of that
study. However, this claim is not true. Campitelli and Gobet did
not discard collected questionnaires; rather, as they clearly
explained in their article, there were missing data: “Not all
players answered all questions, with the result that the number
of data points varies across our measures”(p. 448). Second,
Ericsson writes, “ItwouldbenicetohaveGobet and Campitelli
(2007) conduct a re-analysis that would identify the amount of
practice required prior to first attaining the rating of master”(p.
14). If Gobet and Campitelli had not performed this analysis,
then they would have had no basis for their conclusion that,
contrary to Ericsson's view, there is a large amount of variability
in the amount of deliberate practice players need to achieve a
given level of skill in chess. However, as they report in a major
section of their article (pp. 165–166), Gobet and Campitelli
performed exactly this analysis and found that amount of
deliberate practice required to first attain the rating of master
ranged from 728 to 16,120 h. Thus, one player reached the
master level twenty-two times faster than another player.
Ericsson also calls attention to our reporting of Gobet and
Campitelli's (2007) results. He writes, “Surprisingly, Hambrick
et al. (2014-this issue) reports the lowest value for a chess
master as 832 h instead of the 728 h as reported by Gobet and
Campitelli (2007, p. 166) without providing an explanation
for the difference”(p. 14), and “Surprisingly, Hambrick et al.
(2014-this issue) reports the highest value for a chess master
as 24,284 h instead of the 16,120 h reported by Gobet and
Campitelli (2007, p. 166) without providing an explanation for
the difference”(p. 14). The explanation is that the different
numbers reflect different measures. The range of 832 to
24,284 h reported in Hambrick et al. (2014-this issue) is for
accumulated amount of deliberate practice, the focus of our
study. The range of 728 to 16,120 h reported in Gobet and
Campitelli (2007) is for hours to master status. So, in several
cases, what Ericsson characterizes as problems with studies
that contradict his view are not problems.
The deliberate practice view has had a major impact on
the trajectory of research on expert performance. But now
we have empirical evidence that deliberate practice, while
important, is not as important as Ericsson has argued it is—
evidence that it does not largely account for individual
differences in performance. The question now is what else
matters.
References
Campitelli, G., & Gobet, F. (2008). The role of practice in chess: A longitudinal
study. Learning and Individual Differences,18, 446–458. http://dx.doi.org/
10.1016/j.lindif.2007.11.006.
Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The
role of deliberate practice in chess expertise. Applied Cognitive Psychology,19,
151–165. http://dx.doi.org/10.1002/acp.1106.
Duckworth, A. L., Kirby, T. A., Tsukayama, E., Berstein, H., & Ericsson, K. A. (2011).
Deliberate practice spells success: Why grittier competitors triumph at the
National Spelling Bee. Social Psychological and Personality Science,2, 174–181.
http://dx.doi.org/10.1177/1948550610385872.
Ericsson, K. A. (2007). Deliberate practice and the modifiability of body and mind:
Toward science of the structure and acquisition of expert and elite
performance. International Journal of Sports Psychology,38,4–34.
Ericsson, K. A. (2013a). Psychotherapy and the science of human excellence.
Keynote presentation at the 2013. Washington, DC: Psychotherapy Net-
worker Symposium.
Ericsson, K. A. (2013b). My exploration of Gagné's “evidence”for innate
talent: It is Gagne who is omitting troublesome information so as to
present more convincing accusations. In S. B. Kaufman (Ed.), The
complexity of greatness: Beyond talent or practice (pp. 223–254). New
York: Oxford University Press.
113D.Z. Hambrick et al. / Intelligence 45 (2014) 112–114

Ericsson, K. A. (2014). Why expert performance is special and cannot be
extrapolated from studies of performance in the general population:
Aresponsetocriticisms.Intelligence,45,81–103 (this issue).
Ericsson, K. A., Krampe, R. Th., & Tesch-Römer, C. (1993). The role of deliberate
practice in the acquisition of expert performance. Psychological Review,100,
363–406. http://dx.doi.org/10.1037/0033295X.100.3.363.
Gladwell, M. (2008). Outliers: The story of success. New York: Little, Brown and
Company.
Gobet, F., & Campitelli, G. (2007). The role of domain-specific practice,
handedness, and starting age in chess. Developmental Psychology,43,
159–172. http://dx.doi.org/10.1037/00121649.43.1.159.
Hambrick,D.Z.,Oswald,F.L.,Altmann,E.M.,Meinz,E.J.,Gobet,F.,&Campitelli,G.
(2014). Deliberate practice: Is that all it takes to become an expert?
Intelligence,45,34–45 (this issue).
Horn, J., & Masunaga, H. (2006). A merging theory of expertise and intelligence. In
K. A. Ericsson, N. Charness, P. Feltovich, & R. R. Hoffman (Eds.), Cambridge
handbook of expertise and expert performance (pp. 587–611). Cambridge, UK:
Cambridge University Press.
Howard, R. W. (2012). Longitudinal effects of different types of practice on
the development of chess expertise. Applied Cognitive Psychology,26,
359–369. http://dx.doi.org/10.1002/acp.1834.
Kopiez, R., & Lee, J. I. (2006). Towards a dynamic model of skills involved
in sight reading music. Music Education Research,8,97–120. http://
dx.doi.org/10.1080/14613800600570785.
Masunaga, H., & Horn, J. (2001). Expertise and age-related changes in components
of intelligence. Psychology and Aging,16, 293–311. http://dx.doi.org/10.1037/
0882-7974.16.2.293.
Meinz,E.J.,&Hambrick,D.Z.(2010).Deliberatepracticeisnecessarybutnot
sufficient to explain individual differences in piano sight-reading skill:
The role of working memory capacity. Psychological Science,21,914–919.
http://dx.doi.org/10.1177/0956797610373933.
114 D.Z. Hambrick et al. / Intelligence 45 (2014) 112–114