ArticlePDF Available

Expert and novice problem solving strategies in chess: Sixty years of citing de Groot (1946)

Authors:

Abstract

In a famous study of expert problem solving, de Groot (1946/1978) examined how chess players found the best move. He reported that there was little difference in the way that the best players (Grand Masters) and very good players (Candidate Masters) searched the board. Although this result has been regularly cited in studies of expertise, it is frequently misquoted. It is often claimed that de Groot found no difference in the way that experts and novices investigate a problem. Comparison of expert and novice chess players on de Groot’s problem shows that there are clear differences in their search patterns. We discuss the troublesome theoretical and practical consequences of incorrectly reporting de Groot’s findings.
Expert and ‘‘novice’’ problem solving strategies in chess:
Sixty years of citing de Groot (1946)
Merim Bilalic
´and Peter McLeod
Oxford University, UK
Fernand Gobet
Brunel University, London, UK
In a famous study of expert problem solving, de Groot (1946/1978) examined
how chess players found the best move. He reported that there was little
difference in the way that the best players (Grand Masters) and very good
players (Candidate Masters) searched the board. Although this result has been
regularly cited in studies of expertise, it is frequently misquoted. It is often
claimed that de Groot found no difference in the way that experts and novices
investigate a problem. Comparison of expert and novice chess players on de
Groot’s problem shows that there are clear differences in their search patterns.
We discuss the troublesome theoretical and practical consequences of
incorrectly reporting de Groot’s findings.
Keywords: Chess; Experts vs novices; Expertise; Problem solving; Scientific
citation; Search; Thinking.
Understanding the changes that take place as expertise develops is a central
topic in cognitive psychology. Apart from a natural interest in what makes
an expert, there is also the belief that this would help to achieve the practical
goal of creating better training programmes. This is particularly true of
understanding how problem-solving strategies develop as these are thought
to play a central role in the acquisition of expertise (Anderson, 1993;
Correspondence should be address to Merim Bilalic
´, University of Tu
¨bingen, Experimental
MRI, Department of Neuroradiology, Hoppe-Seyler-Str. 3, 72076 Tu
¨bingen, Germany. E-mail:
Merim.Bilalic@med.uni-tuebingen.de
This research is based on a doctoral dissertation of the first author who was supported by
Oxford University Clarendon and ORS scholarships. The preparation of this paper was
supported by an ESRC postdoctoral fellowship to the first author.
THINKING & REASONING, 2008, 14 (4), 395 – 408
Ó2008 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
http://www.psypress.com/tar DOI: 10.1080/13546780802265547
Koedinger & Anderson, 1990; Newell, 1980; Patel & Groen, 1991; Smith,
2002; Williams, Papierno & Makel, 2004).
One approach has been to study the way that people with different
degrees of expertise tackle the same problem. An early example of this
method, which has since acquired classic status, was de Groot’s (1946/1965/
1978) analysis of problem solving by chess players. He showed the position
in Figure 1 to five Grand Masters, the most skilled players, and five
Candidate Masters, very good players.
1
He asked them to think aloud while
they tried to find the best move for White. On average, the Grand Masters
found better moves and found them more quickly, but analysis of the verbal
protocols suggested that the two groups differed little in the macrostructure
of their search. Both investigated a similar number of moves and searched
these to a similar depth. de Groot concluded that the superiority of the
Grand Masters lay somewhere other than their search processes (for an
Figure 1. de Groot’s position A (White to move). The winning move is 1. Bxd5 exd5 (1 . . . Nxd5
2. Nxd5 Bxd5 3. Bxe7; 1 . .. Bxd5 2. Bxf6 Bxf6 3.Nd7 with Nxf8) 2. Qf3 Qd8 (2. .. Kg7 3. Ng4
Nxg4 [3 . . . Qd8 4. Bh6þ] 4. Bxe7 Re8 5. Bc5 with 6. Qxg4) 3. Rce1 and the pin is decisive (e.g.,
3 . .. .Kg7 4. Ng4 Ng8 5. Bxe7 Nxe7 6. Qf6þKg8 7. Nh6 with checkmate).
1
Other players (three Masters, four Class C players, and two ‘‘lady players’’) also
participated in the experiment but the key analysis (de Groot, 1978, pp. 317–320) focuses on the
five Grand Masters and Candidate Masters.
396 BILALIC
´, MCLEOD, GOBET
explanation and historical review, see Charness, 1992; Gobet, de Voogt, &
Retschitzki, 2004).
One of the attractions of chess for studying expertise is that there is an
interval scale for measuring the skill levels of different players on the basis of
their results against other players of known rating. Thus it is possible to
compare the skill level of different groups of players precisely. The Elo scale
has a theoretical mean of 1500 and a theoretical standard deviation of 200
(see Elo, 1978). Grand Masters usually have a rating of over 2500. They are
at least 5 SD above the level of average chess players. Candidate Masters
2
have a rating between 2200 and 2000. They are about 3 SD above the level
of average players. By the standards of psychological research on expertise,
both groups of players tested by de Groot were experts. The Candidate
Masters might be called ‘‘ordinary’’ experts as opposed to the Grand
Masters who were ‘‘super’’ experts, but both groups were far superior to
average players.
3
Since the publication of the second English edition of de Groot’s book in
1978, his work has been frequently cited (see Figure 2). The ‘‘no difference’’
Figure 2. Frequency of citation for de Groot (1946/1965/1978) between 1970 and 2006. The data
were obtained using the Social Science Citation Index and Arts & Humanities Citation Index.
2
Players with a rating between 2000 and 2200 are sometimes called ‘‘Experts’’. In this paper
we use the alternative title ‘‘Candidate Masters’’ to avoid confusion with the stronger players
who are undeniably experts.
3
The Elo scale with its associated class levels was not available when de Groot collected his
data and published his thesis in 1946, but de Groot provided an explicit if rough correspondence
with these levels in the English translations of his book.
SCIENTIFIC CITATION OF DE GROOT (1946) 397
search result is mentioned in many textbooks but it is often misreported. It is
commonly stated that de Groot found no difference between the search
processes of experts and novices. For example: ‘‘Several other investigators
(de Groot, 1965; . . .) found no major strategic differences between experts
and novices’’ (VanLehn, 1989, p. 562); ‘‘de Groot (1965) . . . found that
masters . . . and novices did not differ in the number of possible moves they
considered, nor how far ahead they looked’’ (Smyth, Morris, Levy, & Ellis,
1994, p. 358); ‘‘de Groot (1978) . . . found no reliable differences in the depth
to which experts versus novices planned in advance’’ (Sternberg & Ben Zeev,
2001, p. 295).
4
Another common description of de Groot’s result in textbooks, less
extreme than claiming that he found no difference between experts and
novices but still misleading, is to report that he found no difference between
masters and weaker players: ‘‘de Groot found hardly any differences
between expert players and weaker players – except, of course, that the
expert players chose much better moves ...In fact, if anything, masters
consider fewer moves than chess duffers’’ (Anderson, 2000, p. 299); ‘‘. . . de
Groot (1965, 1966), who found that master players and weaker players
thought about the same number of moves, considered about the same
number of moves, and had a similar search patterns of moves’’ (Solso,
MacLin, & MacLin, 2005, p. 134); ‘‘Early research by de Groot (1965) failed
to uncover any evidence that expert players searched more moves, searched
farther ahead, or searched faster than ordinary players’’ (Stillings et al.,
1995, pp. 132–133). It is true that the Candidate Masters in de Groot’s study
were weaker players than the Grand Masters, but they were neither weak
players nor ordinary, and certainly not duffers. One should, however, point
out that some textbooks, such as Gilhooly (1996, p. 62), Eysenck and Keene
(2000, p. 414), Green (1996, p. 334), and Parkin (2000, p. 286) accurately
reproduced de Groot’s finding.
Textbooks are summaries of the primary literature. Is it possible that the
myth that de Groot found no differences between experts and novices stems
from erroneous reporting of de Groot’s result in research on expertise?
Using the Social Science Citation Index and Arts & Humanities Citation
Index we identified 651 journal articles from 1970 to the end of 2006 that
cited de Groot’s study (see Figure 2). We conducted the search for references
to all three available editions of the book: 1946 – Dutch first edition; 1965 –
English first edition; and 1978 – English second edition. Out of these 651
published articles we managed to examine 391 (61%). Problem-solving
4
Curiously enough, just three pages later (p. 298) Sternberg and Ben-Zeev (2001) got it right
in the context of position recall: ‘‘The assessment was further supported by de Groot’s asking
both the grand masters and the experts to recall a middle-game position shown to them for just
a short time.’’
398 BILALIC
´, MCLEOD, GOBET
aspects of the study were mentioned in 100 (26%) of which 49 referred to
details of the study. Of these, 24 (49%) used, at best, ambiguous descriptions,
making the mistake repeated by the writers of textbooks cited earlier. They
either stated that de Groot (a) found no differences in the problem-solving
strategies between experts and novices; (b) that he used experts and novices in
his study; or (c) that there were no differences between experts and weaker
players (e.g., less expert, less skilled, less able, or average players). The
Appendix presents the 49 articles that deal with the problem-solving aspect
of the study, together with the specific misquotations.
Since de Groot did not test novices, it is possible that there is no
difference between the search structure of experts and novices, and the
misquotations are reporting a correct conclusion even if misquoting
de Groot. This can be examined by comparing the performance of de
Groot’s Grand Masters with a group of novices who were tested on the
same problem with the same instructions by Gruber (1991; see also
Gobet, 1998). Gruber’s novices did not have Elo ratings because they had
not participated in official chess tournaments or been a member of a
chess club. However, they played chess on average once a month and
were not beginners. Table 1 compares the Grand Masters and Novices.
Although they spent a similar amount of time on the problem, t(27) ¼1,
ns,d¼0.40, Grand Masters were, unsurprisingly, much better at finding
the best move, t(27) ¼6.6, p5.01, d¼3.5. There were clear differences in
the macro-structure of search. Grand Masters reached greater maximal
depths than Novices, t(27) ¼4.2, p5.01, d¼3.0, and had more Episodes
(general investigations) t(27) ¼3.4, p5.01, d¼0.98). They did not
examine more Candidate moves (different solutions) t(27) ¼0.9, ns,
d¼0.44. The small difference in this measure is probably a consequence
of the position used by de Groot in which there are only a small number
TABLE 1
Average and standard deviation (in brackets) for Grand Masters (de Groot, 1946) and
Novices (Gruber, 1991) on de Groot’s problem
Elo Rating
Age
(yrs) Quality*
Time
(mins)
Max
Depth* Candidates Episodes* n
Grand
Masters
2658 (26) 32 (10) 4.8 (0.5) 9.4 (3.6) 7.4 (2.6) 3.6 (1.7) 6.4 (4.4) 5
Novices 30 (6) 1.6 (1.2) 7.8 (4.2) 3.3 (1.8) 2.9 (1.5) 3.1 (1.8) 24
Quality of solution on a scale 0 ¼mistake to 5 ¼best move (see Gobet, 1998). ‘‘Time’’ is time
spent trying to solve the problem. ‘‘Max Depth’’ is the number of half moves/ply in the deepest
solution searched. ‘‘Candidates’’ is the number of different solutions (i.e. first moves)
investigated. ‘‘Episodes’’ is the number of general investigations. *Difference between Grand
Masters and Novices significant p5.01.
SCIENTIFIC CITATION OF DE GROOT (1946) 399
of plausible first moves. There is also the fact that while novices are
selective due to the computational cost of considering alternative moves,
very strong players are highly selective in their search due to their
knowledge of plausible moves.
Vicente and Brewer (1993; see also Vicente & de Groot, 1990) showed
how a phenomenon from de Groot’s (1946) research, that of recall of
random positions, was often correctly reproduced (recall was not better in
stronger players),
5
but the credit for the discovery was not. In this case,
another seminal de Groot result has frequently been completely distorted.
Unlike in Vicente and Brewer’s case of misattribution, here we deal with
incorrect reporting of a finding of potentially great importance to the
understanding of expert–novice differences in problem solving. Equating the
search strategies of novices and experts could have serious theoretical and
practical consequences. The theories that emphasise analytical process and
the role of search in expertise (e.g., Holding, 1985, 1992) would be
completely discredited. Even the theories that highlight the role of pattern
recognition and chunks/templates in problem solving (e.g., Chase & Simon,
1973; Gobet, 1998; Gobet & Simon, 1996; Saariluoma, 1995) would turn out
to be inappropriate. Most of the mentioned theories are based not only on
the seminal findings of de Groot (1946) presented here, but also in numerous
others that show skill effects in search strategies (e.g., Campitelli & Gobet,
2004; Charness, 1981, 1989; Gobet, 1998; Gruber, 1991; Saariluoma, 1992).
Equally important are the practical consequences based on misreporting
of the differences between experts and novices. Training programmes based
on the misinterpreted results would completely abandon the emphasis on
analytical abilities, one of the crucial abilities in expertise. Indeed, the field of
nursing offers just such an example. S. E. Dreyfus and H. L. Dreyfus (1980)
have proposed an influential theory of expertise development whose
(incorrect) rejection of the role of analytical search with experts can be
traced back to the misconception that there are no differences in the extent to
which chess novices and masters look ahead when attempting to select a
move. For example, Dreyfus and Dreyfus (1986, pp. 36–37, our emphasis)
write:
. . . quality of move choice depends surprisingly little on anything but pure
intuitive response. . . . What does a masterful chess player think about when
time permits, even when an intuitively obvious move has already come
spontaneously to mind? Often he uses his time to follow out sequences of
moves. Players of all levels of skill have been shown to be equally good at this.
But strong intuitive players think about other things, too.
5
An expected twist to the story is that later research found that random positions are in fact
recalled slightly but reliably better by stronger players (Gobet & Simon, 1996).
400 BILALIC
´, MCLEOD, GOBET
Dreyfus and Dreyfus’s theory has been applied with little change to the
field of nursing expertise (Benner, 1984; Benner & Tanner, 1987), and has
had considerable impact on the development of curricula for nurse training.
Benner (1984) clearly underplays the role of teaching analytical methods
when forming expert nurses: while these methods should be taught to
novices and advanced beginners (the first two of five stages of expertise in
the Benner, Dreyfus, and Dreyfus model), they should not be taught in the
other stages (competence state, proficiency stage, and expertise stage), where
instruction should aim at developing intuitive skills. (See Gobet & Chassy,
2008, for a detailed discussion.)
Expert and novice chess players do differ in the macro-structure of search
when trying to find the solution to a problem. The numerous textbooks and
papers that cite de Groot as support for the idea that the search structure of
expert and novices is similar are not only misquoting de Groot, they are
putting forward an incorrect and dangerous claim. There may not be much
difference between the search patterns of super experts and ordinary experts
in chess positions as de Groot (1946) showed (see also Charness, 1981, 1989,
but see Campitelli & Gobet, 2004); there are, however, clear differences
between the search patterns of experts and novices.
Manuscript received 20 May 2008
Revised manuscript received 11 June 2008
First published online 18 August 2008
REFERENCES
Anderson, J. R. (1993). Problem solving and learning. American Psychologist,48, 35–44.
Anderson, J. R. (2000). Cognitive psychology and its implications (5th ed.). New York: W. H.
Freeman & Co.
Benner, P. (1984). From novice to expert: Excellence and power in clinical nursing practice.
Menlo Park, CA: Addison-Wesley.
Benner, P., & Tanner, C. (1987). Clinical judgement: How expert nurses use intuition. American
Journal of Nursing, January, 23–31.
Campitelli, G., & Gobet, F. (2004). Adaptive expert decision making: Skilled chessplayers search
more and deeper. Journal of the International Computer Games Association,27, 209–216.
Charness, N. (1981). Search in chess: Age and skill differences. Journal of Experimental
Psychology: Human Perception and Performance,2, 467–476.
Charness, N. (1989). Expertise in chess and bridge. In D. Klahr & K. Kotovsky (Eds.), Complex
information processing: The impact of Herbert A. Simon. Hillsdale, NJ: Lawrence Erlbaum
Associates Inc.
Charness, N. (1992). The impact of chess research on cognitive science. Psychological Research,
54, 4–9.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology,4, 55–81.
de Groot, A. D. (1946). Het denken van den schaker. Amsterdam: Noord Hollandsche.
de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton Publishers.
de Groot, A. D. (1978). Thought and choice in chess (2nd ed.). The Hague: Mouton Publishers.
SCIENTIFIC CITATION OF DE GROOT (1946) 401
Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in
directed skill acquisition (ORC 80-2). Berkeley, CA: University of California, Operations
Research Center.
Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine: The power of human intuition and
expertise in the era of the computer. New York: Free Press.
Elo, A. E. (1978). The rating of chess players, past and present. New York: Arco.
Eysenck, M. W., & Keane, M. T. (2000). Cognitive psychology: A student’s handbook (4th ed.).
Hove, UK: Psychology Press.
Gilhooly, K. J. (1996). Thinking: Directed, undirected and creative (3rd ed.). London: Academic Press.
Gobet, F. (1998). Chess players’ thinking revisited. Swiss Journal of Psychology,57, 18–32.
Gobet, F., & Chassy, P. (2008). Two theories of expert intuition in nursing: A discussion paper.
International Journal of Nursing Studies,45, 129–139.
Gobet, F., de Voogt, A., & Retschitzki, J. (2004). Moves in mind. Hove, UK: Psychology Press.
Gobet, F., & Simon, H. A. (1996). Recall of rapidly presented random chess positions is a
function of skill. Psychonomic Bulletin & Review,3, 159–163.
Green, D. W. (Ed.). (1996). Cognitive science: An introduction. Oxford, UK: Blackwell.
Gruber, H. (1991). Qualitative Aspekte von Expertise im Schach [Qualitative aspects of expertise
in chess]. Aachen: Feenschach.
Holding, D. H. (1985). The psychology of chess skill. Hillsdale, NJ: Erlbaum.
Holding, D. H. (1992). Theories of chess skill. Psychological Research,54, 10–16.
Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks:
Elements of expertise in geometry. Cognitive Science,14, 511–550.
Newell, A. (1980). Reasoning, problem-solving and decision processes: The problem space as a
fundamental category. In R. Nickerson (Ed.), Attention and performance, VIII (pp. 693–
718). Hillsdale, NJ: Lawrence Erlbaum Associates Inc.
Parkin, A. (2000). Essential cognitive psychology. Hove, UK: Psychology Press.
Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A
critical look. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise:
Prospects and limits (pp. 93–125). Cambridge, UK: Cambridge University Press.
Saariluoma, P. (1992). Error in chess: The apperception-restructuring view. Psychological
Research,54, 17–26.
Saariluoma, P. (1995). Chess player’s thinking. London: Routledge.
Smith, G. F. (2002). Thinking skills: The question of generality. Journal of Curriculum Studies,
34, 659–678.
Smyth M. M., Morris P.E., Levy P., & Ellis A. W. (1994). Cognition in action. Hove, UK:
Psychology Press.
Solso, R. L., MacLin, M. K., & MacLin, O. H. (2005). Cognitive psychology (7th ed.). Boston:
Allyn & Bacon.
Sternberg, R. J., & Ben-Zeev, T. (2001). Complex cognition: The psychology of human thought.
Oxford, UK: Oxford University Press.
Stillings, N. A., Weisler, S. E., Chase, C. H., Feinstein, M. H., Gareld, J. L., & Rissland, E. L.
(1995), Cognitive science: An introduction (2nd ed.). Cambridge, MA: MIT Press.
VanLehn, K. (1989). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.),
Foundations of cognitive science (pp. 527–579). Cambridge, MA: MIT Press.
Vicente, K. J., & Brewer, W. F. (1993). Reconstructive remembering of the scientific literature.
Cognition,46, 101–128.
Vicente, K. J., & de Groot, A. D. (1990). The memory recall paradigm: Straightening out the
historical record. American Psychologist,45, 285–287.
Williams, M. W., Papierno, B. P., & Makel, C. M. (2004). Thinking like a scientist about real-
world problems: The Cornell Institute For Research On Children science education
program. Journal of Applied Developmental Psychology,25, 107–126.
402 BILALIC
´, MCLEOD, GOBET
APPENDIX
List of 49 articles citing de Groot’s (1946/1965/1978) result on problem
solving with quotations from the articles that describe the study
inaccurately.
TABLE A1
Author/Year/Journal Vol. Page Quote
(A) Studies inaccurately reporting the differences between super experts and ordinary experts as
the difference between experts and novices (or unskilled/inexperienced players)
Gonzalez et al. (2003).
Cognitive Science
27 597 Experimental data from chess studies shows that
experts search very selectively using recognition
cues to guide their attention and achieving
greater computational efficiency. Novices,
however, must engage in a more thorough
search to determine the principles that are
applicable to the problem situation (Chase &
Simon, 1973; de Groot, 1978; Simon & Gobet,
1996).
Mitchell & Dacin
(1996). Journal of
Consumer Research
23 220 Given that most researchers initially believed that
expertise stemmed from the use of more efficient
and effective search strategies, most of the early
studies in this area focused on identifying
differences in search strategies between domain
experts and novices. Studies of chess experts,
however, revealed few identifiable differences in
search strategies (e.g., depth and breadth of
search; de Groot 1978).
Halpern & Bower
(1982). American
Journal of
Psychology
95 32 In his studies of chess players, de Groot (1965)
found no differences between chess masters and
novices in the basic reasoning associated with
playing, such as the number of moves considered
per play, or the depth of search for moves.
However, he did find difference between the
better and poorer player in a short-term memory
task.
Van der Maas &
Wagenmakers
(2005). American
Journal of
Psychology
38 Using spoken (i.e., think-aloud) protocols during
Choose-a-Move tasks, de Groot argued that
skill differences are instead associated with
performance in recall and recognition of
standard chess positions. Skilled players
presumably calculate as many moves as
unskilled players, but the recognition of familiar
chess patterns that drives the move selection
process allows skilled players to exclude bad
moves and focus their efforts only on promising
continuations.
(continued)
SCIENTIFIC CITATION OF DE GROOT (1946) 403
TABLE A1
(Continued)
Author/Year/Journal Vol. Page Quote
Bainbridge (1977).
Travail Humain
40 175 For example, de Groot (1965) found that
experienced chess players explore the same
number of possible moves as inexperienced
players, the difference is that all the moves
explored by experienced players are
potentionally good ones.
(B) Studies inaccurately reporting that de Groot (1946) used experts and novices
Arocha et al. (2005).
Journal of
Biomedical
Informatics
38 163 As is typical of experts, the network developed
from the physician’s explanation is fully
coherent in that all nodes are connected. This
contrasts with situations where a case
explanation is broken into components, which is
typical for novices. Similar results have been
reported in other domains (Charness, 1989;
Chase & Simon, 1973; de Groot, 1965).
Summers et al. (2004).
Journal of
Occupational and
Organizational
Psychology
77 293 Chess experts, for example, differ from novices
both in how they encode information on
positions and in their evaluation of
potential future moves (e.g. Chase & Simon,
1973a, 1973b; DeGroot, 1965, 1966; Holding &
Reynolds, 1982; Simon and Gilmartin, 1972)
Thompson et al.
(2003). Theory into
Practice
42 134 Key research identifying differences between
novices and experts has been summarized by
Glaser (1992) and Bransford et al. (2000). Very
briefly, experts’ proficiency is highly specific, and
allows them to (a) discern meaningful patterns in
information they encounter (e.g., the classic
chess master study by deGroot, 1965) . . .
Ben-Zeev & Star
(2001). Cognition
and Instruction
19 257 The literature on expertise shows that novices tend
to focus their attention on salient surface-
structural attributes of problems (attributes that
are irrelevant to the solution of a problem),
whereas experts perceive underlying deep-
structural principles (e.g., de Groot, 1965).
Jegede & Taplin
(2000). Educational
Research
42 286 Early studies of expertise include those of de Groot
(1965) and Chase and Simon (1973): what
research tells us about the novice-expert
phenomenon is that an expert differs from a
novice in three ways: level of tacit knowledge,
efficiency in solving problems and the
application of insight in creative problem-
solving.
Fletcher (1999).
Computers in
Human Behavior
15 394 One quite robust finding from empirical studies
contrasting the approaches of expert and novice
problem solvers has been that experts tend to
(continued)
404 BILALIC
´, MCLEOD, GOBET
TABLE A1
(Continued)
Author/Year/Journal Vol. Page Quote
spend much more time than novices in
representing the current state of the environment
or problem (Chase & Simon, 1973; deGroot,
1965; Lesgold, 1988).
Plsek (1999). Annals of
Internal Medicine
131 141 Research indicates that experts in a given area do
more effective thinking primarily because they
have better knowledge of heuristic principles
than novices do (de Groot, 1965).
Agarwal et al. (1996).
International
Journal of Human-
Computer Studies
45 643 The influence of experience on problem
solving is evident from research in expert
and novice problem solving in various technical
domains (de Groot, 1965; Chase & Simon, 1973;
Egan & Schwartz, 1979; Larkin, McDermott,
Simon & Simon, 1980). The studies indicate that
experts possess chunks representing functional
units in their domains, while novices do not.
Royalty (1995).
Psychological
Reports
156 478 The role of content-specific knowledge in critical
thinking has been addressed in studies that have
compared the performance of experts and novices
in a number of domains, including political
science (Voss, Tyler, & Yengo, 1983), chess
(Chase & Simon, 1973; de Groot, 1965) . . .
Stanislaw et al. (1994).
International
Journal of Human-
Computer Studies
41 351–2 Studies of skilled performance in a variety of
domains often distinguish between the
cognitive processes used by experts and
novices (e.g, de Groot, 1965; Ericsson & Smith,
1991) . . .
Sternberg (1981).
American
Psychologist
36 1185 This approach, which might be referred to as a
cognitive-contents approach, seeks to compare
the performances of experts and novices in
complex tasks such as the solution of physics
problems (Chi, Feltovich, & Glaser, 1981; Chi,
Glaser & Rees, in press; Larkin, McDermott,
Sinon, & Simon, 1980a, 1980b), the selection of
moves and strategies in chess and other games
(Chase & Simon, 1978; DeGroot, 1965;
Reitman, 1976), and the acquisition of domain-
related . . .
(C) Studies ambiguously reporting the differences between super expert and ordinary experts as
the difference between experts and lesser players
Billman & Shaman
(1990). American
Journal of
Psychology
103 146 Classic studies by Chase and Simon (1973) showed
that one dramatic difference between the more
and less skilled chess players was the ability to
chunk game patterns into meaningful,
memorable units. Furthermore, DeGroot (1965)
(continued)
SCIENTIFIC CITATION OF DE GROOT (1946) 405
TABLE A1
(Continued)
Author/Year/Journal Vol. Page Quote
found that experts do not search a larger number
of alternative moves.
Wolff et al. (1984).
Journal of
Psychology
118 7 Research on skill in the game of chess has
indicated that expert and average players are
similar in the way they reason about future
moves sequences (de Groot, 1965) but differ in
the way they perceive pieces groupings
(Charness, 1976; Chase & Simon, 1973; de
Groot, 1965; Frey & Adesman, 1976; Goldin,
1978).
Carter (1990). College
Composition and
Communication
3124 There have been many expert-novice studies, but
the seminal study, performed by deGroot on
chess masters and less experienced players, best
illustrates what has been learned. DeGroot
began his research with the assumption that
masters were better players because they could
think of more possible moves and could think
more moves ahead of the lesser chess players.
However, de Groot discovered that neither of
these hypotheses was correct: there was actually
little quantitative differences between the
masters and others.
Elstein et al. (1990).
Evaluation & the
Health Professions
36 The work of de Groot (1965) was especially
influential; he provided models for us in two
ways: the use of thinking aloud to study chess
masters planning their next move and
explorations of the difference between masters
and weaker players.
Holding & Reynolds
(1982). Memory &
Cognition
3237 Information processing models of chess skill have
vainly taken their departure from de Groot’s
(1965) finding that players of different strengths
were essentially alike in the number of moves
they considered, in the depth of their search for
move sequences and in other, similar measures
obtained from spoken protocols.
Frey & Adesman
(1976). Memory &
Cognition
3541 From an analysis of verbal protocols, de Groot
(1965) established that chess Masters and less
able players use similar thought processes in
analyzing a complex chess position. They
consider a similar number of moves (about 35),
calculate to similar depths (about 7 plies), make
the same number of fresh starts (about 7), and
analyze a similar number of moves per minute
(about 3). The major difference Groot noted was
that the masters invariably analyzed stronger
(continued)
406 BILALIC
´, MCLEOD, GOBET
TABLE A1
(Continued)
Author/Year/Journal Vol. Page Quote
moves than the weaker players.
Lawson (2004). Design
Studies
3448 de Groot’s work showed that a key distinguishing
factor between the chess master and the less
expert player was as much in perception as in
action. Chess masters, he found, rarely analysed
a board situation, rather they recognised it. He
showed that chess masters could remember mid-
game board situations much more reliably than
novices. However, their comparative expertise
vanished when asked to remember randomly
positioned pieces that did not relate to game
situations. Taken together these results suggest
something we are familiar with in design, the use
of known precedents that have been studied and
about which the expert has schemata. These
precedents linked problem to solution and such
chess masters could articulate this link. Thus, the
schema for the situation also includes one or
more known gambits for solving it.
Schultetus & Charness
(1999). American
Journal of
Psychology
3555 One the first attempts to explore the role of each
process in chess was a study by de Groot (1965)
in which he analyzed recall performance and
verbal protocols of expert and club-level chess
players. Although he found no skill-related
differences in depth of search, he reported that
skilled players were capable of reproducing
game positions with accuracy.
Gilhooly (1990).
Applied Cognitive
Psychology
4262–
263
de Groot found that while expert and less expert
chess players did not differ in the amount of
mental search in which they engaged before
choosing a move, they did differ in memory for
briefly presented chess positions.
SCIENTIFIC CITATION OF DE GROOT (1946) 407
TABLE A2
Articles correctly reproducing the finding
Name/Year/Journal Vol. Page
Shepherd et al. (2006). Psychology & Marketing 23 115
Fazey et al. (2006). Ecology and Society 10 2
Burns (2004). Psychological Science 15 443
Campitelli & Gobet (2004). ICGA journal 27 209
Gobet & Clarkson (2004). Memory 12 732
Sweller (2004). Instructional Science 32 11
Gobet (1998). Swiss Journal of Psychology 57 19
Sinclair (1998). Advances in Case-Based Reasoning 1488 127
Berliner & McConnell (1996). Artificial Intelligence 86 99
Ericsson & Lehmann (1996). Annual Review of Psychology 47 278
Gobet & Simon (1996). Psychological Science 7 52
Lindgaard (1995). Interacting with Computers 7 257
Burns (1994). Behavioral and Brain Sciences 17 535
Reynolds (1991). Bulletin of the Psychonomic Society 29 55
Klein & Peio (1989). American Journal of Psychology 102 322
Bransford et al. (1986). American Psychologist 41 1079
Holding & Pfau (1985). American Journal of Psychology 98 271
Boshuizen & Claessen (1982). Medical Education 16 84
Reynolds (1982). American Journal of Psychology 95 384
Charness (1981). Journals of Gerontology 7 467
Charness (1981). JEP: HP&P 36 618
Erickson & Jones (1978). Annual Review of Psychology 29 72
Simon & Chase (1973). American Scientist 62 396
Chase & Simon (1973). Cognitive Psychology 4 55
408 BILALIC
´, MCLEOD, GOBET
... Procedures, collections of rules to be applied in certain situations, in this particular context would be the common way of dealing with the water jug problem (e.g., the B -A -2 C method). As the procedure becomes stronger with more frequent use in the introductory problems, the participants become increasingly fixated on that particular way of dealing with the problem (for more details, see Bilali c, McLeod, & Gobet, 2008a). This explanation may indeed account for people's use of the old Emethod in the two critical (2-solution) problems. ...
... It is important to stress that the uncovered E-mechanism is not only limited to the water-jug paradigm. It has already been shown that the same mechanism, where activated memory biases intake of perceptual clues by driving the attentional resources towards elements related to the activated elements in memory, explains the rare mistakes of experts (Bilali c et al., 2008a(Bilali c et al., , 2008bSheridan & Reingold, 2013). The E-mechanism also bears resemblance to a number of seemingly unrelated phenomena (Bilali c et al., 2010;Bilali c & McLeod, 2014). ...
Article
Full-text available
Einstellung (mental set) effects designate the phenomenon where established routines can prevent people from finding other, possibly more efficient solutions. Here we investigate the mechanism behind this phenomenon by using Luchins’ classical water jug paradigm with concurrent verbalization. We find no difference in the extent of the Einstellung effect between the group which was instructed to think aloud during the problem solving and the group which was thinking silently. The think-aloud protocols indicate that the participants who exhibited the Einstellung effect repeatedly attempted to solve the water jug problem by using variations of the previously successful method which had been rendered inappropriate in the final problem. Our study underlines the usefulness of the think-aloud technique in tracking the cognitive processes. More importantly, it demonstrates how, once thought has been activated, it may bias subsequent dealings with new situations, even in the face of repeated failure that people experience in the Einstellung situations.
... There has been extensive research development into the contrasting nature of expertise and novice schemas. One notable inquiry that has been studied in detail, for example, relates to chess playing and how expert players differ from novice players (Gobet, 2006;Bilalic et al., 2008;Nokes et al., 2010;Lane and Chang, 2018). Expert players, in this case, are able to recognize and identify familiar patterns in chess positions, and they have knowledge and understanding of larger patterns. ...
Article
Full-text available
Possessing expert schemas is a positive feat that may yield different types of adaptive outcomes (e.g., informing procedural understanding that may result in a student skipping a few of the solution steps involved). Limited schemas, in contrast, may deter progress of a novice learner, limiting his/her capability to flourish. Taken as a whole, it may be concluded that expert schemas are more advantageous than novice schemas, differentiating learners in terms of expert and novice. Having said this, however, more recently, researchers have argued that possessing expert schemas could serve as deterrence. Recently, researchers have acknowledged a theoretical concept known as cognitive entrenchment , which is defined as a high level of stability in domain schemas. This description interestingly suggests that “entrenchment” or “situated fixation” of a course of action (e.g., a subject matter) could hinder the progress and learning experience of a person, namely—his/her inability and/or unwillingness to adapt to a new context, and/or his/her inflexibility and insistence to stay on course without any intent to change. One example of cognitive entrenchment is observed in professional football, wherein it has been argued that some football coaches are cognitively entrenched within their expert schemas, resulting in their demised game plans and strategic acumen. We advance the study of cognitive entrenchment by proposing an alternative viewpoint, which we term as the “perceived zone of certainty and uncertainty.” This proposition counters the perspective of cognitive entrenchment by arguing that it is cognitive appraisal, judgment, mental resolute, and determination of a person in cognitive certainty of his/her success or failure, or the cognitive uncertainty of success or failure, that would explain the notion of inflexibility and/or unwillingness to adapt, and/or insistence to stay on course without any attempt to deviate. Moreover, we rationalize that certainty of success or failure would closely associate with a feeling of comfort, whereas uncertainty would associate with his/her feeling discomfort. In this analysis, we strongly believe that willingness to change and adapt, reluctance and insistence to remain on course, and/or inclination to embrace flexibility may not necessarily relate to the concept of cognitive entrenchment; rather, inflexibility and/or reluctance to change for the purpose of adaptation has more to do with the desire of a person to seek a state of comfort. Finally, our conceptual analysis of cognitive entrenchment also considers an interesting theoretical concept, which we termed as “perceived optimal efficiency.” Perceived optimal efficiency, similar to cognitive relevance theory , is concerned with the relationship between minimum investment of time, effort, cognitive resources, etc., and an optimal best outcome. The issue for discussion, from our point of view, is related to the extent to which the certainty of success or failure would associate with perceived optimal efficiency.
... On the one hand, chess offers a complex and rich environment that requires a broad range of cognitive operations; on the other, it uses simple objects and rules that clearly specify and constrain the environment. Therefore, one can examine higher cognitive processes such as problem solving (Bilalić, McLeod, & Gobet, 2008cConnors, Burns, & Campitelli, 2011) and decision making (Bilalić & McLeod, 2014;Bilalić, McLeod, & Gobet, 2008a, 2008b as well as more basic perceptual processes such as object or pattern recognition (Gobet & Simon, 1996;Kiesel, Kunde, Pohl, Berner, & Hoffmann, 2009;Saariluoma, 1990). In this paper we will focus on the neural mechanisms behind skilled object and pattern recognition at the brain network level. ...
Article
Skilled visual object and pattern recognition form the basis of many everyday behaviours. The game of chess has often been used as a model case for studying how long-term experience aides in perceiving objects and their spatio-functional interrelations. Earlier research revealed two brain regions, posterior middle temporal gyrus (pMTG) and collateral sulcus (CoS), to be linked to chess experts’ superior object and pattern recognition, respectively. Here we elucidated the brain networks these two expertise-related regions are embedded in, employing resting-state functional connectivity analysis and meta-analytic connectivity modelling with the BrainMap database. pMTG was preferentially connected with dorsal visual stream areas and a parieto-prefrontal network for action planning, while CoS was preferentially connected with posterior medial cortex and hippocampus, linked to scene perception, perspective-taking and navigation. Functional profiling using BrainMap meta-data revealed that pMTG was linked to semantic processing as well as inhibition and attention, while CoS was linked to face and shape perception as well as passive viewing. Our findings suggest that pMTG subserves skilled object recognition by mediating the link between object identity and object affordances, while CoS subserves skilled pattern recognition by linking the position of individual objects with typical spatio-functional layouts of their environment stored in memory.
... Instead of asking chess experts to play a whole game in the laboratory, researchers choose to present them with an unfamiliar position from a normal game between two masters and ask them to find the best move. Once the laboratory task has been established, researchers can manipulate factors such as skill (Bilalić, McLeod, & Gobet, 2008a) and familiarity (Bilalić, McLeod, & Gobet, 2009) and see what processes mediate experts' outstanding performance. ...
... Behavioural studies in other fields of research have indeed demonstrated that domainspecific knowledge, acquired through prolonged and focused training (Ericsson et al. 1993), enables expertsin contrast to novicesto quickly grasp the essence of complex but highly familiar stimuli (Bilalic et al., 2008). Second, our findings reveal that, when asked what method they would opt for if they were to watch the whole film, the majority of viewers living in non-subtitling countries chose subtitling. ...
Article
After three decades of scholarship describing why and how students ought to be taught to think historically, this study asks what happens when they are. Ten high school students from a school that incorporated historical thinking into all history coursework repeated the think-aloud task from Wineburg’s 1991 study of the cognitive processes underlying the evaluation of historical evidence, reading eight documents with conflicting accounts of the Battle of Lexington. As a cohort, these contemporary students corroborated, sourced, and contextualized more frequently than their 1991 counterparts, despite representing a greater range of overall academic ability. The increase in historical reading did not, however, unambiguously demonstrate a change in their historical thinking. Students tended to source using a binary rating of either reliable or unreliable, corroborate pairs of documents rather than consider how all eight documents in the set created a narrative, and rely upon their ability to recall content information to contextualize. Their performance suggests that they have learned the process of historical thinking without taking up the underlying epistemology. These less sophisticated reading moves raise questions about how well the predominant model for historical thinking in the United States inspires and reflects students’ epistemological growth and suggests that there may be a need to revisit how available professional development, educative materials, and research help educators teach historical thinking.
Article
Early-stage science-based ventures require a wide range of intellectual resources and practical know-how to successfully commercialize their technologies. Often entrepreneurs actively gain this knowledge through advisory relationships providing commercial and technical guidance. We explore the effects of those dual advice domains – business and technology – and their overlaps and complementarities with the knowledge bases of entrepreneurs. To directly capture early science-based venture progress, we introduce the concept of application readiness to represent a technology's evolution from scientific discovery to commercial solution. Using hand-collected longitudinal data from 112 emerging science-based ventures, we find that business advice has a positive impact on application readiness; counter-intuitively, technology advice does not confer the same benefit. Moreover, advice shows the strongest effects when its domain is complementary to the entrepreneur’s experience. These insights help unpack the mechanisms through which advice – an often-used policy tool supporting entrepreneurship – is absorbed and implemented in emerging ventures.
Article
The study of anticipation in truly expert performers can provide insight into how they cope with extreme time constraints. The purpose of this dual-experiment paper was to investigate individual differences; in anticipation of the penalty corner drag-flick, its trainability, and transfer of improvement to field settings. Australian international and national male field-hockey goalkeepers participated. In experiment 1, international and national goalkeepers (n = 11) completed a penalty corner drag-flick temporal occlusion task that presented; defensive runner positioning at the penalty spot, drag-flicker kinematics, and ball flight. Results indicated seven goalkeepers integrated runner contextual and drag-flicker kinematic information to anticipate above chance. The cause of individual differences was independent pickup of run and kinematic cues that presented greater opportunity to integrate sources for anticipation. In experiment 2, a sub-sample of goalkeepers participated and received temporal occlusion training or no training. Results indicated individualized improvement in anticipation across video, field, and competition assessments for those that received the intervention, but not controls. Improvements on video test were retained for six months. An individual differences approach can identify deficiencies in anticipation, which can be improved through perceptual training that transfers to motor responses. This contributes to theoretical and practical knowledge to develop anticipation skill.
Article
Learning, memorizing and recalling knowledge are the basic functions of cognitive models. These models must prioritize which stimulants to respond to as well as package acquired knowledge in an easy to retrieve manner. The human brain is a cognitive model that derives information from sensor data such as vision, associates different patterns to create knowledge, and uses chunking mechanisms to package the acquired knowledge in manageable entities. The use of chunking mechanisms by the brain aids it to overcome its short-term memory (STM) capacity limitation. Through chunking, each entity held in the STM is a chunk containing more associations (knowledge) in it. By mimicking the human brain, this study proposes an associative memory and recall (AMR) model that stores associative knowledge from sensor data. Using chunking mechanisms, AMR can organize human activity knowledge in the manner that is efficient and effective to store and recall. The knowledge–information–data (KID) model is used for learning associative knowledge while the AMR continuously looks for associations among knowledge units and merges related units using merging mechanisms. The chunking mechanisms used in this study are inspired by the chunking mechanisms of the brain i.e. goal oriented chunking and automatic chunking.
Chapter
Full-text available
The purpose of the chapter is to present a diagnosis of the need for changes in education programs for healthcare professionals. The areas subject to analysis refer to the elements included in the concept of Healthcare Improvement Science. The fundamental idea behind the analyses conducted as part of the project was to identify gaps based on the analyses prepared by experts from partner countries. The presentation emphasises difficulties in comparing countries, which result from diversity in different countries at the level of the system and the education. The presented proposal includes the context of needs, and consequently changes in healthcare. It will be presented in the holistic thinking about risk factors and protective factors.
Article
Full-text available
A widely cited result asserts that experts' superiority over novices in recalling meaningful material from their domain of expertise vanishes when they are confronted with random material. A review of recent chess experiments in which random positions served as control material (presentation time between 3 and 10 sec) shows, however, that strong players generally maintain some superiority over weak players even with random positions, although the relative difference between skill levels is much smaller than with game positions. The implications of this finding for expertise in chess are discussed and the question of the recall of random material in other domains is raised.
Article
This paper develops a technique for isolating and studying the per- ceptual structures that chess players perceive. Three chess players of varying strength - from master to novice - were confronted with two tasks: ( 1) A perception task, where the player reproduces a chess position in plain view, and (2) de Groot's ( 1965) short-term recall task, where the player reproduces a chess position after viewing it for 5 sec. The successive glances at the position in the perceptual task and long pauses in tbe memory task were used to segment the structures in the reconstruction protocol. The size and nature of these structures were then analyzed as a function of chess skill. What does an experienced chess player "see" when he looks at a chess position? By analyzing an expert player's eye movements, it has been shown that, among other things, he is looking at how pieces attack and defend each other (Simon & Barenfeld, 1969). But we know from other considerations that he is seeing much more. Our work is concerned with just what ahe expert chess pIayer perceives.
Article
Five protocol-analysis experiments with tactical, endgame, and strategic positions were conducted to study cognitive errors in chess players' thinking. It will be argued that chess players' errors can be only partially explained in terms of unspecified working-memory overload, because the working-memory loads caused by the solution paths are usually small. It is therefore necessary to consider apperceptive mechanisms also, as these control information intake. Subjects fail either because they are not able to see the right prototypical problem space at all, or because they fail to close them as a result of missing some crucial task-relevant cue. This makes chess players lose their "belief in the idea" and restructure, after which the apperceptive information-selection mechanisms make the finding of the solution still more unlikely.