ArticlePDF Available

Abstract and Figures

Inductive learning -- that is, learning a new concept or category by observing exemplars -- happens constantly, for example, when a baby learns a new word or a doctor classifies x-rays. What influence does the spacing of exemplars have on induction? Compared with massing, spacing enhances long-term recall, but we expected spacing to hamper induction by making the commonalities that define a concept or category less apparent. We asked participants to study multiple paintings by different artists, with a given artist's paintings presented consecutively (massed) or interleaved with other artists' paintings (spaced). We then tested induction by asking participants to indicate which studied artist (Experiments 1a and 1b) or whether any studied artist (Experiment 2) painted each of a series of new paintings. Surprisingly, induction profited from spacing, even though massing apparently created a sense of fluent learning: Participants rated massing as more effective than spacing, even after their own test performance had demonstrated the opposite.
No caption available
… 
No caption available
… 
Content may be subject to copyright.
Research Article
Learning Concepts and
Categories
Is Spacing the ‘‘Enemy of Induction’’?
Nate Kornell and Robert A. Bjork
University of California, Los Angeles
ABSTRACT—Inductive learning—that is, learning a new
concept or category by observing exemplars—happens
constantly, for example, when a baby learns a new word or
a doctor classifies x-rays. What influence does the spacing
of exemplars have on induction? Compared with massing,
spacing enhances long-term recall, but we expected spac-
ing to hamper induction by making the commonalities that
define a concept or category less apparent. We asked
participants to study multiple paintings by different ar-
tists, with a given artist’s paintings presented consecutive-
ly (massed) or interleaved with other artists’ paintings
(spaced). We then tested induction by asking participants
to indicate which studied artist (Experiments 1a and 1b) or
whether any studied artist (Experiment 2) painted each of
a series of new paintings. Surprisingly, induction profited
from spacing, even though massing apparently created a
sense of fluent learning: Participants rated massing as
more effective than spacing, even after their own test
performance had demonstrated the opposite.
The spacing effect refers to the nearly ubiquitous finding that
items studied once and revisited after a delay are recalled better
in the long term than are items studied repeatedly with no in-
tervening delay (e.g., Cepeda, Pashler, Vul, Wixted, & Rohrer,
2006; Dempster, 1996; Glenberg, 1979; Hintzman, 1974;
Melton, 1970). The positive effects of spacing on long-term re-
call are large and robust, and have been demonstrated in a va-
riety of domains, such as conditioning (even in animals as simple
as Aplysia; see Carew, Pinsker, & Kandel, 1972), verbal learning
(e.g., Bahrick, Bahrick, Bahrick, & Bahrick, 1993; Ebbinghaus,
1885/1964), motor learning (e.g., Shea & Morgan, 1979), and
learning of educational materials (e.g., Bjork, 1979; Dempster,
1988).
In many everyday and educational contexts, however, what is
important to learn and remember transcends specific episodes,
instances, and examples. Instead, it is most important to learn
the principles, patterns, and concepts that can be abstracted
from related episodes or examples. In short, educators often
want to optimize the induction of concepts and patterns, and
there are reasons to think that such induction may be enhanced
by massing, rather than by spacing. As stated by E.Z. Rothkopf
(personal communication, September 1977), ‘‘spacing is the
friend of recall, but the enemy of induction.’’
SPACING AS THE ENEMY OF INDUCTION
There is a compelling logic behind Rothkopf’s assertion.
Massing allows one to notice the similarities between successive
episodes or exemplars, whereas spacing makes doing so more
difficult. Thus, for example, spacing presentations of individual
paintings by a given artist will make it more difficult to notice
any characteristics that define the artist’s style because spacing
increases the chances that those characteristics will be forgotten
between successive presentations.
MASSING AS THE FRIEND OF INDUCTION
The logic behind Rothkopf’s assertion is so compelling that, to
our knowledge, it has never been tested. Perhaps the most direct
evidence that massing facilitates induction comes from a study
by Kurtz and Hovland (1956), who asked participants to study
simple drawings that varied on four dimensions: size, shape,
position, and coloring. There were four categories of drawings
that participants learned to identify; for example, a ‘‘Kem’’ was
defined as a drawing containing a circle positioned near the top
of the display. Each category was presented eight times, with the
individual items either interleaved with items from the other
Address correspondence to Nate Kornell, Department of Psychology,
1285 Franz Hall, UCLA, Los Angeles, CA 90095, e-mail: nkornell@
ucla.edu.
PSYCHOLOGICAL SCIENCE
Volume 19—Number 6 585Copyright r2008 Association for Psychological Science
categories (i.e., spaced) or massed together. No item was ever
repeated exactly. On a memory test following the study phase,
participants’ performance was better for drawings in the massed
condition than for drawings in the spaced condition. Gagne
´
(1950) obtained a similar result using four categories of non-
sense-figure/nonsense-syllable pairs: Error rates were reduced
when the highly similar category members were grouped to-
gether, instead of being interleaved.
Less direct evidence comes from experiments that compared
exact and nonexact repetitions (i.e., verbatim repetitions vs.
paraphrased or gist repetitions). In such experiments, the spac-
ing effect appears to diminish or disappear altogether for non-
verbatim repetitions (e.g., Appleton-Knapp, Bjork, & Wickens,
2005; Dellarosa & Bourne, 1985; Glover & Corkill, 1987).
Similarly, Melton (1970) demonstrated that spacing effects do
not occur when participants fail to recognize that a repeated item
is a repetition. Given such findings, and given that inductive
learning involves exposure to a variety of different exemplars
and does not involve exact repetition, it seems possible that
spacing effects will disappear or turn into massing effects in
tasks requiring induction.
Finally, research in the domain of motor learning also pro-
vides indirect evidence that inductive learning may profit from
massing, rather than spacing. Learning a motor skill, such as a
tennis serve, involves induction in the sense that exposure to
one’s own proprioceptive feedback is an important component of
learning—and such repetitions are, by necessity, not exact,
especially for novices learning complex skills. Although spaced
practice is effective in many areas of motor learning (e.g., Shea &
Morgan, 1979), massed practice can be more effective for learn-
ing complex motor skills (Wulf & Shea, 2002).
GOALS OF THE PRESENT RESEARCH
Thus, there are both logical and empirical reasons to expect
massing, not spacing, to facilitate induction. One goal of the ex-
periments reported here was to investigate the size of the mass-
ing effect—that is, the advantage of massed study over spaced
study—in an inductive-learning context relevant to educational
practice.
Another goal was to investigate participants’ subjective as-
sessments of massed versus spaced study in the context of in-
duction. Prior research has demonstrated that people often rate
massing as more effective than spacing, even in contexts in
which spacing is actually superior (Baddeley & Longman, 1978;
Simon & Bjork, 2001; Zechmeister & Shaughnessy, 1980). Such
massing illusions may derive from the fact that metacognitive
judgments are often grounded in feelings of fluency (e.g., see
Benjamin, Bjork, & Schwartz, 1998). Presenting the same item
twice consecutively makes processing the second presentation
seem highly fluent, providing a (misleading) impression of learn-
ing, whereas spacing decreases the fluency of processing the
second presentation. In other words, massing provides a sense of
ease, which learners assume will translate to good memory on a
later test, whereas spacing is often a ‘‘desirable difficulty’’
(Bjork, 1994) in the sense that it enhances long-term retention.
If massing produces a sense of fluency of induction, participants
may prefer massing to spacing in the induction task we used in
these experiments.
EXPERIMENTS 1A AND 1B
In this experiment, participants were asked to learn the styles of
12 different artists by viewing six different paintings by each
artist. In Experiment 1a, spacing was manipulated within par-
ticipants: Paintings by each of 6 of the artists were presented
massed, and paintings by each of the other 6 artists were
presented spaced. In Experiment 1b, spacing was manipulated
between participants: For a given participant, the paintings
were presented either all massed by painter or all interleaved
(spaced). After the learning phase, participants were shown new
paintings by the same 12 artists and asked to select, from a list of
all the artists’ names, the artist who had painted each new
painting. After the test, participants in Experiment 1a were
asked what presentation condition, massing or spacing, they felt
had been more effective for learning a given artist’s style.
Method
Participants
The participants were University of California, Los Angeles,
undergraduates, who participated for course credit. There were
120 participants in Experiment 1a and 72 participants, 36 in
each condition, in Experiment 1b.
Materials
The materials were 10 paintings by each of 12 artists (Georges
Braque, Henri-Edmond Cross, Judy Hawkins, Philip Juras, Ryan
Lewis, Marilyn Mylrea, Bruno Pessani, Ron Schlorff, Georges
Seurat, Ciprian Stratulat, George Wexler, and YieMei). Six paint-
ings by each artist were presented during the study phase, and 4
more were presented during the test phase. All the paintings were
landscapes or skyscapes. We selected artists who would be rela-
tively unknown to the participants, although some of the paintings
by Braque and Seurat may have been familiar to some of the
participants (however, on the final test, average performance on
paintings by those two artists was not better than average perfor-
mance on paintings by all 12 artists). The paintings were cropped
to remove identifying characteristics such as names and signa-
tures, if necessary, and then resized to fit into a 15- 11-cm
rectangle on a computer screen.
Procedure and Design
Participants were instructed about the nature of the study and
test phases and were then shown 72 paintings, 6 paintings by
each of the 12 artists. Each painting was shown for 3 s on a
586 Volume 19—Number 6
Spacing and Induction
computer screen, with the last name of the artist displayed
below.
In Experiment 1a, the paintings by each of six of the artists
were presented consecutively (massed), whereas the paintings by
each of the other six artists were intermingled with paintings by
other artists (spaced). The artists assigned to the massed and
spaced conditions were determined randomly for each partici-
pant. Each successive block of six paintings consisted of six
paintings by a given artist (massed, or M) or one painting by each
of the six artists (spaced, or S). The order of the blocks was
MSSMMSSMMSSM (see Fig. 1). In Experiment 1b, depending on
the condition to which a participant was assigned, either all of the
paintings were presented in the massed condition or all of the
paintings were presented in the spaced condition.
At the end of the study phase, there was a 15-s distractor task,
during which participants counted backward by 3s from 547; the
test phase began when participants completed the distractor
task. On each test trial, an unfamiliar painting by one of the 12
artists was presented. Participants indicated who they thought
had created each painting by clicking their computer’s mouse on
1 of 13 buttons, 12 labeled with the names of the artists and 1
labeled ‘‘I don’t know.’’ After this response, feedback was pro-
vided : The word ‘‘correct’’ followed a correct selection, and the
correct artist’s name was presented following an error.
There were 48 test trials divided into four blocks of 12
paintings. Each block consisted of one new painting by each of
the 12 artists, presented in random order. After the test phase in
Experiment 1a, participants were told the meanings of the terms
massed and spaced and asked, ‘‘Which do you think helped you
learn more, massed or spaced?’’ They were given three response
options: ‘‘massed,’’ ‘‘about the same,’’ and ‘‘spaced.’’ The same
question could not be asked in Experiment 1b because partic-
ipants did not experience both conditions.
Results
In marked contrast to our expectations, spaced study resulted in
significantly better test performance than did massed study, as
measured by the proportion of artists identified correctly on the
test (Fig. 2). (Experiment 1b was conducted after we came up
with a convoluted conjecture that mixing massed and spaced
paintings in a single learning phase created a spacing effect in
Experiment 1a.) The advantage of spacing was significant in
both Experiment 1a, F(1, 119) 577.35, p<.0001, Zp25.39,
and Experiment 1b, F(1, 70) 515.63, p<.001, Zp25.18.
Not surprisingly, given that feedback was provided, test per-
formance increased across test blocks—Experiment 1a: F(3,
357) 526.99, p<.0001, Zp25.18; Experiment 1b: F(3, 210)
511.56, p<.0001, Zp25.14. The interaction of presentation
condition and test block was significant—Experiment 1a: F(3,
357) 513.25, p<.0001, Zp25.10; Experiment 1b: F(3, 210)
53.33, p<.05, Zp25.046. This interaction appears to reflect
the large increase from the first to the second test block in the
massed condition, which may have been a consequence of the
first test block acting as an additional, spaced study opportunity
that benefited previously massed items in particular. A planned
comparison of performance during the first test block, which
Fig. 1. The first 12 paintings presented to 1 of the participants in Ex-
periment 1a (the artists in each condition were determined randomly for
each participant). The first 6 paintings (left column) were all by the same
artist (massed, or M), and the next 6 paintings (right column) were all by
different artists (spaced, or S). In total, there were 12 blocks of 6
paintings in the order MSSMMSSMMSSM. Therefore, in the spaced
condition, a given artist was represented by 1 painting in each S block.
Volume 19—Number 6 587
Nate Kornell and Robert A. Bjork
was, presumably, largely unaffected by the presence of feed-
back, showed that participants performed significantly better in
the spaced condition than in the massed condition, in both
Experiment 1a (M5.61, SD 5.24 vs. M5.35, SD 5.24),
t(119) 510.82, p<.0001, p
rep
1.00, d50.99, and Ex-
periment 1b (M5.59, SD 5.22 vs. M5.36, SD 5.18), t(70) 5
4.94, p<.0001, p
rep
1.00, d51.28.
The advantage of spacing over massing is all the more sur-
prising given participants’ responses on the questionnaire ad-
ministered after the test. As Figure 3 shows, participants in
Experiment 1a judged massing to be more effective than spacing,
regardless of their performance in the two conditions. Overall,
78% of the participants did better with spaced presentations than
they did with massed presentations, but 78% of the participants
said that massing was as good as or better than spacing.
Discussion
The results of Experiments 1a and 1b pose two puzzles. First,
why did spacing, not massing, foster induction when there were
compelling reasons to expect otherwise? Second, why did par-
ticipants remain unaware that spacing was more effective than
massing, even after taking the test? With respect to the second
puzzle, we hypothesized that participants, while taking the test,
might not have remembered which artists had been presented in
which condition. To investigate this hypothesis, we presented 28
participants in Experiment 1a with a list of the artists’ names, and
asked them to indicate—after they had completed the test—how
each artist’s paintings had been presented (spaced or massed).
Accuracy on the identification task was significantly above chance
for artists whose paintings had been presented spaced (M5.74,
SD 5.17), t(27) 57.48, p<.0001, p
rep
1.00, d51.41, but
not for artists whose paintings had been presented massed (M5
.55, SD 5.21), t(27) 51.19, p5.25, p
rep
5.69,d50.23.
Participants’ inability to remember which artists’ paintings had
been presented massed suggests that participants often, if not
always, made their metacognitive judgments on the basis of their
subjective experience during the study phase.
With respect to the puzzle of why spacing enhanced induction,
one possible explanation is that the test required recalling a given
artist’s name, not just knowing his or her style, and spacing facil-
itates recall. It seems possible that participants did indeed induce
an artist’s style more effectively in the massed condition than in the
spaced condition, but recalled the name associated with that style
better in the spaced condition. Experiment 2 was designed to test
that possibility by assessing participants’ recognition, not recall, so
that they did not need to remember name-style associations.
EXPERIMENT 2
The learning phase in Experiment 2 was identical to the learning
phase in Experiment 1a. However, during the test phase, par-
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Spaced
Massed
0.2
0.3
0.4
Proportion Correct Proportion Correct
0.5
0.6
0.7
0.8
1234
Test Block
Spaced
Massed
Fig. 2. Proportion of artists selected correctly on the multiple-choice
tests in Experiments 1a (top panel) and 1b (bottom panel) as a function of
presentation condition (spaced or massed) and test block. Error bars
represent standard errors.
0
10
20
30
40
50
60
70
80
Massed >
Spaced
Massed =
Spaced
Spaced >
Massed
Judged Effectiveness
Number of Participants
Spaced > Massed
Massed = Spaced
Massed > Spaced
Actual Effectiveness
Fig. 3. Number of participants (out of 120) who judged massing as more
effective than, equally effective as, or less effective than spacing in Ex-
periment 1a. For each judgment, the number of participants is divided
according to their actual performance in the spaced condition relative to
the massed condition.
588 Volume 19—Number 6
Spacing and Induction
ticipants were given a style-recognition test. All of the tested
paintings were new paintings, as in Experiment 1, but partici-
pants were asked only to categorize a given test painting as by a
‘‘familiar artist’’ (i.e., by an artist whose paintings had been
presented during the study phase) or as by an ‘‘unfamiliar artist.’’
Thus, the test required remembering only studied artists’ styles,
not their names.
Method
Participants
The participants were 80 undergraduate students at the Uni-
versity of California, Los Angeles, who participated for course
credit.
Materials
The materials consisted of the same set of paintings used in
Experiment 1, plus, for each studied artist, an additional set of
four distractor paintings. Each distractor painting was chosen to
be stylistically similar to a studied artist’s paintings, and each
distractor was by a different artist (see Fig. 4).
Procedure
The study phase was exactly the same as in Experiment 1a, as
was the questionnaire at the end of the experiment. The only dif-
ference from Experiment 1a was in the test phase (and associ-
ated instructions).
During each trial of the test phase, a painting was presented
with two buttons on the computer screen; one button was labeled
‘‘familiar artist,’’ and one was labeled ‘‘unfamiliar artist.’’ Par-
ticipants were instructed to select the ‘‘familiar artist’’ button if
they thought the painting was by an artist whose paintings had
been presented during the study phase, and to select the ‘‘un-
familiar artist’’ button if they thought the painting was by an
artist whose paintings had not been presented during the study
phase. There were four test blocks, each of which included one
target painting and one distractor painting by a corresponding
nonstudied artist, making a total of 24 paintings per block. No
feedback was given during the test.
Results
Recognition test trials are, inevitably, also learning events. A
side effect of falsely endorsing a painting as by a familiar artist
was that a participant might alter his or her concept of the fa-
miliar artist’s style by incorporating aspects of a painting by an
unfamiliar artist into that concept. On each successive test
block, the potential for contamination created by false alarms
grew, resulting in a significant decrease in recognition accuracy
across test blocks, F(3, 237) 53.60, p<.05, Zp25.04.
Therefore, to gain maximum leverage on the question of interest,
we restricted our analyses to the first test block, which provided
the purest measure of the learning that occurred during the study
phase of the experiment.
Again, we were surprised to find that performance in the
spaced condition was superior to performance in the massed
condition. As Figure 5 shows, the spaced and massed conditions
produced similar rates of false alarms (i.e., saying that a painting
by a nonstudied artist was by a studied artist), but the hit rate
(i.e., correctly categorizing a new painting by a studied artist as
by a familiar artist) was higher in the spaced condition (M5.77,
SD 5.22) than in the massed condition (M5.67, SD 5.24),
t(79) 53.28, p<.01, p
rep
5.98, d50.41. Consequently, there
was a significant interaction between spacing condition and
response type, F(1, 79) 57.84, p<.01, Zp25.09. There was
also a main effect of response type, F(1, 79) 5177.82, p<
.0001, Zp25.69, with more hits than false alarms; this pattern
of results shows that participants could distinguish between the
target and distractor paintings. Thus, even in a situation that did
not require participants to recall name-style associations,
spacing led to more effective induction than did massing.
Fig. 4. Examples of four target (a) and four associated distractor (b) paintings from the test phase of Experiment 2. The target
paintings were the same as those used in Experiment 1. The distractors were all by different artists, and each was selected to be
similar to the paintings of a given studied artist.
Volume 19—Number 6 589
Nate Kornell and Robert A. Bjork
As in Experiment 1a, the participants’ metacognitive judg-
ments were strikingly at odds with their actual prior perfor-
mance. Of the 72 participants who did not say that learning in
the massed condition and learning in the spaced condition were
‘‘about the same,’’ 64 thought massing had been more effective
than spacing.
One possible explanation of the current findings is that
schema induction happened early in the study phase. For ex-
ample, the induction for each artist may have been ‘‘done’’ by the
third study trial, so that the next three study trials amounted to
either massed or spaced memory practice. To test this possi-
bility, we conducted an additional experiment. The test phase
was the same as in Experiment 2, but in the study phase, only two
paintings by each artist (instead of six) were presented. We
obtained the same pattern of results: There was a significant
benefit of spaced study, but participants thought massed study
had been more effective.
GENERAL DISCUSSION
A common way to teach students about an artist is to show, in
succession, a number of paintings by that artist. Counterintu-
itive as it may be to art-history teachers—and our participants
we found that interleaving paintings by different artists (spac-
ing) was more effective than massing all of an artist’s paintings
together. A possible key to understanding the present findings
involves the relationship between induction and discrimination.
Induction and Discrimination
Experiment 1 required that participants discriminate among
different artists’ styles; that is, on the test, they had to decide
which artist, among the 12 studied artists, had painted a given
new painting. The interleaving of artists that was intrinsic to the
spaced condition might have fostered such discrimination. For
example, the key to deciding whether a tree is a maple or an
oak (or some other tree) is learning to appreciate the differences
among trees, not learning about a given type of tree in isolation.
Interleaving had the effect of juxtaposing different paintings and
therefore might have enhanced discrimination learning. (In fact,
we have presented no evidence regarding the effects of temporal
spacing in the absence of interleaving, and it may be inter-
leaving, not spacing itself, that is the key to enhancing inductive
learning.)
With respect to this possibility, the following observation by
Kurtz and Hovland (1956) seems relevant: ‘‘When the degree of
discriminability is low, it might be expected that placing of in-
stances from different concepts in juxtaposition would facilitate
discrimination learning, whereas with greater discriminability,
like that obtaining in the present study, the reverse might ob-
tain’’ (p. 242). Thus, if discrimination is not difficult, as was the
case in Kurtz and Hovland’s experiment, massing may be ad-
vantageous, but if discrimination is difficult, as it was in our ex-
periments, spacing might be more effective.
This argument is appealing, but it is not entirely consistent
with the results from Experiment 2. The recognition test in Ex-
periment 2 required discriminating between paintings by pre-
viously studied artists and similar paintings by artists who had
not been studied; it did not require distinguishing among artists
whose work had been presented, and yet there was a benefit from
spacing.
It could be argued that a by-product of being better able to
distinguish among the presented artists is being able to distin-
guish those artists, as a group, from other artists. It may be rare,
in fact, that a concept or category (such as what psychology is or
how to fly a kite) is ever learned without the need to discriminate
it from other categories (such as sociology or ways to make a kite
fall).
Our results notwithstanding, there surely are situations in
which massing is more effective for induction than is spacing.
We attempted to create one such situation by asking participants
to figure out, and remember on a later test, the single word that
could be used to fill in the blanks in each of 12 sets of six words;
for example, in the case of _____ cracker, _____ wood, _____
side, _____ ant, _____ truck, _____ arm, the word to be gen-
erated and remembered was fire. The design was similar to the
design of Experiment 1a—that is, half of the sets that defined a
to-be-remembered word were presented massed, whereas the
other half were presented spaced—and 20 undergraduate par-
ticipants were tested. In this case, spacing made it nearly im-
possible to solve the problems, and, thus, later memory for the
target words was significantly better in the massed condition
than in the spaced condition (.34 vs. .22), t(19) 52.78, p<.05,
p
rep
5.94, d50.65.
Admittedly, this simple experiment was contrived to be a
situation in which massing, not spacing, would enhance the
0
0.2
0.4
0.6
0.8
1
Studied Artists
(Hits)
Nonstudied
Artists (FAs)
Proportion 'Familiar'
Massed
Spaced
Fig. 5. Results from the recognition test in Experiment 2: proportion of
paintings judged to be painted by a studied artist as a function of whether
the artist had been studied (hits) or had not been studied (false alarms, or
FAs), separately for the spaced and massed conditions. Only data from
the first test block were analyzed and plotted here. Error bars represent
standard errors.
590 Volume 19—Number 6
Spacing and Induction
generation and memory of the critical words. The experiment
demonstrates, however, that whether spacing is the friend or
enemy of induction is a matter for sophisticated theorizing, be-
cause induction is a product of conceptual and memory processes
that are open to multiple situational influences. The important
point, though, is that in less contrived and more complex real-
world learning situations, spacing appears to facilitate induction.
Practical Implications
Inductive learning—that is, learning from examples—is a key
element of formal education, and of how humans (and other
animals) informally learn about the world. There are many in-
ductive-learning situations that would seem, from an intuitive
standpoint, to lend themselves to massed study, but may not.
Examples include a baby learning what chair means by ob-
serving people talking about chairs; an older child learning the
rules of a language, such as that most plural English words end
in s, by listening to people speak the language; a student in
school learning how words are spelled by reading them (as well
as through more direct instruction); a quarterback learning to
recognize a complex pattern of motion that predicts an inter-
ception by gaining experience in practice and during games; a
monkey learning to recognize the warning signs that another
monkey is acting threateningly by observing other monkeys’
behavior; and a medical student learning to recognize warning
signs of lung cancer by reading x-rays under an expert’s su-
pervision. Our results cannot necessarily be generalized to all of
these situations, of course, but they do suggest that in inductive-
learning situations, spacing may often be more effective than
massing, even when intuition suggests the opposite.
Our results also suggest that individuals responsible for the
design and evaluation of instruction that involves induction are
susceptible to being very misled by their own intuitions and
subjective experiences. Although prior experiments (Baddeley &
Longman, 1978; Simon & Bjork, 2001; Zechmeister & Shaugh-
nessy, 1980) have shown that people can experience an illusion
that massing is effective, we know of no experiment that can
match the current findings in terms of sheer inaccuracy of
judgments. In Experiments 1a and 2 combined, 85% of the
participants did at least as well in the spaced condition as in the
massed condition, but 83% of the participants rated the massed
condition as equally effective as or more effective than the
spaced condition. The illusion of effective learning in the massed
condition, based, apparently, on a sense of fluency of induction,
was clearly powerful in the experiments presented here. In real-
world inductive-learning tasks, therefore, it seems likely that
people will be heavily influenced by the illusory benefits of
massing when making decisions about their own learning or the
learning of their students or children. That is, most people are
likely to prefer massing in inductive-learning situations, but our
results suggest that they may do so at their own (and their stu-
dents’ and children’s) peril from a learning standpoint.
CONCLUSION
Looking back at our own inability to foresee the benefits of
spacing, perhaps we fell victim to the same illusion that we have
railed against (e.g., Bjork, 1994, 1999; Kornell & Bjork, 2007),
namely, the illusion that a sense of ease or fluency accompanies
effective learning, whereas a sense of difficulty signifies in-
effective learning. In the case of induction, as in many other
types of learning, spacing appears to be sometimes, if not al-
ways, a desirable difficulty (Bjork, 1994).
Acknowledgments—We thank Makah Leal and Timothy Wong
for their invaluable contributions to all facets of the experiments,
Elizabeth Bjork for her insights, and Katherine Huang and Jeri
Little for their help carrying out the experiments. Grant 29192G
from the McDonnell Foundation supported this research.
REFERENCES
Appleton-Knapp, S., Bjork, R.A., & Wickens, T.D. (2005). Examining
the spacing effect in advertising: Encoding variability, retrieval
processes and their interaction. Journal of Consumer Research,
32, 266–276.
Baddeley, A.D., & Longman, D.J.A. (1978). The influence of length
and frequency of training session on the rate of learning to type.
Ergonomics,21, 627–635.
Bahrick, H.P., Bahrick, L.E., Bahrick, A.S., & Bahrick, P.E. (1993).
Maintenance of foreign language vocabulary and the spacing ef-
fect. Psychological Science,4, 316–321.
Benjamin, A.S., Bjork, R.A., & Schwartz, B.L. (1998). The mismeasure
of memory: When retrieval fluency is misleading as a metamne-
monic index. Journal of Experimental Psychology: General,127,
55–68.
Bjork, R.A. (1979). An information-processing analysis of college
teaching. Educational Psychologist,14, 15–23.
Bjork, R.A. (1994). Memory and metamemory considerations in the
training of human beings. In J. Metcalfe & A. Shimamura (Eds.),
Metacognition: Knowing about knowing (pp. 185–205). Cam-
bridge, MA: MIT Press.
Bjork, R.A. (1999). Assessing our own competence: Heuristics and
illusions. In D. Gopher & A. Koriat (Eds.), Attention and perfor-
mance XVII: Cognitive regulation of performance: Interaction
of theory and application (pp. 435–459). Cambridge, MA: MIT
Press.
Carew, T.J., Pinsker, H.M., & Kandel, E.R. (1972). Long-term habit-
uation of a defensive withdrawal reflex in Aplysia. Science,175,
451–454.
Cepeda, N.J., Pashler, H., Vul, E., Wixted, J.T., & Rohrer, D. (2006).
Distributed practice in verbal recall tasks: A review and quan-
titative synthesis. Psychological Bulletin,132, 354–380.
Dellarosa, D., & Bourne, L.E. (1985). Surface form and the spacing
effect. Memory & Cognition,13, 529–537.
Dempster, F.N. (1988). The spacing effect: A case study in the failure
to apply the results of psychological research. American Psy-
chologist,43, 627–634.
Dempster, F.N. (1996). Distributing and managing the conditions of
encoding and practice. In R. Bjork & E. Bjork (Eds.), Memory
(pp. 317–344). San Diego, CA: Academic Press.
Volume 19—Number 6 591
Nate Kornell and Robert A. Bjork
Ebbinghaus, H.E. (1964). Memory: A contribution to experimental
psychology (H.A. Ruger & C.E. Bussenius, Trans.). New York:
Dover. (Original work published 1885)
Gagne
´, R.M. (1950). The effect of sequence of presentation of similar
items on the learning of paired-associates. Journal of Experi-
mental Psychology,40, 61–73.
Glenberg, A.M. (1979). Component-levels theory of the effects of
spacing of repetitions on recall and recognition. Memory &
Cognition,7, 95–112.
Glover, J.A., & Corkill, A.J. (1987). Influence of paraphrased repeti-
tions on the spacing effect. Journal of Educational Psychology,
79, 198–199.
Hintzman, D.L. (1974). Theoretical implications of the spacing effect.
In R.L. Solso (Ed.), Theories in cognitive psychology: The Loyola
symposium (pp. 77–97). Potomac, MD: Erlbaum.
Kornell, N., & Bjork, R.A. (2007). The promise and perils of self-
regulated study. Psychonomic Bulletin & Review,14, 219–224.
Kurtz, K.H., & Hovland, C.I. (1956). Concept learning with differing
sequences of instances. Journal of Experimental Psychology,51,
239–243.
Melton, A.W. (1970). The situation with respect to the spacing of
repetitions and memory. Journal of Verbal Learning and Verbal
Behavior,9, 596–606.
Shea, J.B., & Morgan, R.L. (1979). Contextual interference effects on
the acquisition, retention, and transfer of a motor skill. Journal of
Experimental Psychology: Human Learning and Memory,5, 179–
187.
Simon, D.A., & Bjork, R.A. (2001). Metacognition in motor learning.
Journal of Experimental Psychology: Learning, Memory, and
Cognition,27, 907–912.
Wulf, G., & Shea, C.H. (2002). Principles derived from the study of
simple skills do not generalize to complex skill learning. Psy-
chonomic Bulletin & Review,9, 185–211.
Zechmeister, E.B., & Shaughnessy, J.J. (1980). When you know that
you know and when you think that you know but you don’t.
Bulletin of the Psychonomic Society,15, 41–44.
(RECEIVED 9/7/07; REVISION ACCEPTED 11/14/07)
592 Volume 19—Number 6
Spacing and Induction
... Typically, students learn categories by studying exemplars (Carvalho & Goldstone, 2017), e.g., a medical student can learn how to diagnose a type of lung infection by inspecting multiple X-rays that belong to a diagnostic category. In category learning, blocked and interleaved practice are two main strategies that students can use (Kornell & Bjork, 2008). Both strategies relate to the order of learning materials: Blocked practice entails a fixed study order, in which students inspect all the exemplars of the same category before inspecting exemplars from a different category. ...
... This helps students to detect differences between categories (Carvalho & Goldstone, 2017). In various domains, it has been shown that interleaved practice results in better category learning than blocked practice, such as science (Eglington & Kang, 2017), statistics (Sana et al., 2017), and arts (Kornell & Bjork, 2008). ...
... Several studies showed that over 60% of the students judged blocked practice to be a more effective strategy than interleaved practice, even though interleaved practice enhanced their category learning more than blocked practice (Birnbaum et al., 2013;Kornell et al., 2010;Zulkiply et al., 2012). This suggests that many students are unable to infer the efficacy of interleaved practice from their task experiences (Kornell & Bjork, 2008). Arguably, this metacognitive illusion results from the inaccurate monitoring of effort and learning during strategy execution . ...
Article
Full-text available
In higher education, many students make poor learning strategy decisions. This, in part, results from the counterintuitive nature of effective learning strategies: they enhance long-term learning but also cost high initial effort and appear to not improve learning (immediately). This mixed-method study investigated how students make learning strategy decisions in category learning, and whether students can be supported to make effective strategy decisions through a metacognitive prompt, designed to support accurate monitoring of effort and learning. Participants (N = 150) studied painting styles through blocked and interleaved practice, rated their perceived effort and perceived learning across time, and chose between either blocked or interleaved practice. Half of the participants (N = 74) were provided with a metacognitive prompt that showed them how their subjective experiences per strategy changed across time and required them to relate these experiences to the efficacy of learning strategies. Results indicated that subjective experiences with interleaved practice improved across time: students’ perceived learning increased as their perceived effort decreased. Mediation analysis revealed that the increased feeling of learning increased the likelihood to select interleaved practice. The percentage of students who chose interleaved practice increased from 13 to 40%. Students’ learning strategy decisions, however, did not benefit from the metacognitive prompt. Qualitative results revealed that students initially had inaccurate beliefs about the efficacy of learning strategies, but on-task experiences overrode the influence of prior beliefs in learning strategy decisions. This study suggests that repeated monitoring of effort and learning have the potential to improve the use of interleaved practice.
... For instance, the fluency of an instructor within a video (i.e., a well vs. a poorly prepared instructor) increased the subjective impression of learning which was reflected in higher predictions of test performance. Critically, however, this impression was not accompanied by an actual increase in learning (Carpenter et al., 2013;Toftness et al., 2018), showing that the participants had overrated their own learning success (see also Kornell & Bjork, 2008). With regard to actual learning, however, related research has argued that experimental conditions introducing disfluency could be beneficial for the learning outcome (e.g., Diemand-Yauman et al., 2011; but see Kühl et al., 2014). ...
... In other words, we were interested in whether the predicted test scores were sensitive to the same manipulations as the actual test scores. The main motivation for studying this was the remarkable mismatch between both scores in previous work, which seemed to arise from a tendency to misattribute the impression of fluency to successful learning (Carpenter et al., 2013;Kornell & Bjork, 2008;Ryffle & Wirth, 2020;Toftness et al., 2018). We investigated this question with our learning prediction hypothesis. ...
... In return, this suggests that our participants were less susceptible to the misattribution of experienced fluency to learning as the participants in previous samples (e.g., Carpenter et al, 2013;Kornell & Bjork, 2008). ...
Article
Full-text available
This study reports a field experiment investigating how instructional videos with and without background music contribute to the learning of examination techniques within a formal curriculum of medical teaching. Following a classroom teaching unit on the techniques for examining the knee and the shoulder joint, our participants (N = 175) rehearsed the studied techniques for either the knee or the shoulder joint with an instructional video with or without background music. As dependent measures, we collected a general questionnaire, a prediction of test performance, as well as performance on an exam-like knowledge test covering both joints. For both videos, the participants who had watched the particular video during rehearsal were more accurate in answering the corresponding questions than the participants who had seen the other video, signaling that instructional videos provide a useful tool for rehearsal (i.e., both groups reciprocally served as control groups). For the knee video (less difficult), we observed a detrimental effect of the background music, whereas we observed no such effect for the shoulder video (more difficult). Further explorations revealed that background music might be detrimental for learning, as it reduces the perceived demand characteristics. Because the impact of the demand characteristics might be more pronounced in less difficult instructional videos, we discuss video difficulty as a potential moderating factor. Overall, our study provides evidence that instructional videos could be usefully implemented in formal teaching curricula and that such instructional videos probably should be designed without background music.
... According to [2], an estimate of 25% to 40% of students experience test anxiety or high levels of stress, nervousness, and apprehension during testing and evaluative situations which significantly interfere with their performance, emotional and behavioral well-being, and attitude toward school. Recent studies, however, have focused on comparing shuffled practice and blocked practice to examine their effects in enhancing motor skills as demonstrated in [3][4][5][6][7] and in the few areas of mathematics education as demonstrated in [8,9,10,11]. In the blocked group, the same task was repeated over and over until all trials are repeated and then would start a different task while the shuffled group practiced all the tasks in an unpredictable order. ...
Article
The paper reports on a study about the blocked practice and shuffled practice formats in mathematics problem solving conducted at Xavier University High School during 2011. The independent variables are the two formats of practice: blocked practice, a method of practice involving problems from multiple lesson types that are arranged by lesson type, and shuffled practice, an unconventional method of practice, also involving problems from multiple lesson types but two problems of the same lesson types are not arranged consecutively. The dependent variables are mathematics achievement scores and mathematics test anxiety scores. The study was conducted at Xavier University High School, Cagayan de Oro City from June 28, 2011 to September 27, 2011 to eighty-eight second year students mixed boys and girls, each belonging to either of the two intact classes, with 44 students per class. The analyses yielded a significant difference on students' achievement scores and on mathematics test anxiety as influenced by the format of practice. Students who used shuffled practice outscored those who used blocked practice in mathematics achievement test (86.62% vs. 69.20%) and in mathematics test anxiety (8.99% vs.-0.90%). Therefore, shuffled practice should be used as reinforcer of mathematics learning, be adopted when conducting reviews for national assessments and be incorporated into the exercises portions of mathematics textbooks.
... Such beliefs are not easily and immediately changed through simple interventions such as a one-time demonstration of an effective learning strategy. 39,170 However, emerging research shows that learners can acquire more accurate beliefs about learning through comprehensive interventions that involve direct instruction on the research supporting effective learning strategies and how to use them, combined with continued use of those strategies over time and experience with the outcomes. Biology facts Short answer test with feedback 202 Biology processes Short answer test with feedback 53 Chemical engineering Scenario-based problem-solving practice test 203 Deductive inferences Fill-in-the-blank or free recall test, with feedback 66 Educational texts Short answer test with feedback 67 Face-name pairs Cued recall test 173 Foreign language translations Oral cued recall with feedback 204 History The Pythagorean theorem describes the relationship between the lengths of the three sides of a right triangle. ...
Article
Full-text available
[Full text link: https://rdcu.be/cSXY3] Research on the psychology of learning has highlighted some straightforward ways of enhancing learning. However, effective learning strategies are underutilized by learners. In this Review, we discuss key research findings on two specific learning strategies: spacing and retrieval practice. We focus on how these strategies enhance learning in a variety of domains across the lifespan, with an emphasis on research in applied educational settings. We also discuss key findings from research on metacognition—learners’ awareness and regulation of their own learning. Learners’ underutilization of effective learning strategies could stem from false beliefs about learning, lack of awareness of effective learning strategies, or the counter-intuitive nature of these strategies. Findings in learner metacognition highlight the need for improving learners’ subjective mental models of how to learn effectively. Overall, the research discussed in this Review has important implications for the increasingly common situations in which learners must effectively monitor and regulate their own learning. [Nature Reviews Psychology, August 2022]
Article
Increasing evidence has shown that summary visual statistics, such as the mean size or centroid of locations, can be perceived without focal attention. Here, we tested the role of attention in visual category learning – rapid learning of visual similarities among paintings of the same artist. Participants encoded paintings from two famous artists into memory while simultaneously monitoring a rapid serial visual presentation (RSVP) stream of colored squares, pressing the spacebar for target colors and making no response to distractor colors. Paintings encoded with the RSVP targets were better remembered than those encoded with the RSVP distractors, demonstrating an Attentional Boost Effect. Importantly, pairing one artist’s paintings with the RSVP targets led to better visual category learning – participants were more accurate at recognizing novel paintings from this artist, relative to another artist whose paintings were presented with the RSVP distractors. Thus, visual category learning is subjected to the same constraint of attention as exemplar memory, demonstrating common mechanisms for exemplar and category learning.
Article
Given the importance of collocational knowledge for second language learning, how collocation learning can be facilitated is an important question. The present study examined the effects of three different practice schedules on collocation learning: node massed, collocation massed, and collocation spaced. In the node-massed schedule, three collocations for the same node verb were studied on the same day. In the collocation-massed schedule, three collocations for the same node verb were studied in different weeks. In the collocation-spaced schedule, participants encountered multiple collocations for the same node verb within a single day; at the same time, multiple collocations for the same node verb were repeated each week. To examine whether the knowledge of studied collocations could be transferred to unstudied collocations containing the same node, posttests included novel collocations that were not encountered during the treatment. Results suggested that the collocation-spaced schedule led to the largest gains for both studied and unstudied collocations.
Article
Context: Health professions trainees must acquire a vast amount of clinical knowledge and skills, and a deliberate instructional design approach is needed to provide trainees with effective learning strategies. One powerful yet counterintuitive strategy that facilitates long-term learning is incorporating intentional difficulties during the learning process. Difficulties that require more effort from learners may impede short-term learning but are ultimately beneficial for long-term learning and are therefore termed, Desirable Difficulties. Objectives: In this cross-cutting edge paper, we describe the Desirable Difficulty effect from three theoretical perspectives originating in different fields, discuss common evidence-based Desirable Difficulty strategies used in Health Professions Education, and explore emerging research that could further optimize Desirable Difficulty-enhanced learning for health professions trainees. Methods: We synthesize theory and research from psychology, cognitive science, and health professions education literatures to further the understanding and application of Desirable Difficulties. We introduce three theoretical perspectives that provide a comprehensive overview of the theoretical underpinnings of the Desirable Difficulty effect: New Theory of Disuse, Challenge Point Framework, and Cognitive Load Theory. We then illustrate how three common Desirable Difficulty strategies in medical education research -- retrieval practice, spaced practice, and interleaved practice -- can be understood through these theoretical lenses. Finally, we provide relevant examples from the literature and explore emerging research in this area. Conclusions: This paper summarizes the theory and empirical research on Desirable Difficulties during the learning process, from explaining what they are and why they may be effective to how they have been applied in different contexts. We argue that providing educators and trainees with a comprehensive theoretical and applied understanding of Desirable Difficulty will promote deliberate instructional design decisions and lead to more effective learning.
Article
When metal nanoparticles are functionalized with charged ligands, the movement of counterions and conduction electrons is coupled, which enables us to develop electronic devices, including diodes, transistors, and logic gates, but dynamically modulating the conductivity of a synaptic device within these materials has proved challenging. Here we show that an artificial synapse can be created from thin films of functionalized metal nanoparticles using an active silver electrode. The electric-field-injected Ag+ coordinates with carboxyl ligands that sets up a conduction bridge to increase the nanoparticle conductivity by reducing the electron tunneling/hopping energy barriers. The dynamic modulation of conductivity allows us to implement several important synaptic functions such as potentiation/depression, paired-pulse facilitation, learning behaviors including short-term to long-term memory transition, self-learning, and massed leaning vs spaced learning. Finally, based on the nonvolatile characteristics, the metal nanoparticle synapse is used to build a single-layer hardware spiking neural network (SNN) for pattern recognition.
Article
Full-text available
We assessed the effects of removing some constraints that characterize traditional experiments on the effects of spaced, rather than massed, study opportunities. In five experiments—using lists of to-be-remembered words—we examined the effects of how total study time was distributed across multiple repetitions of a given to-be-remembered word. Overall, within a given list, recall profited from study time being distributed (e.g., four 1-second presentations or two 2-second presentations versus one 4-second presentation). Among the implications of these findings is that if students choose to engage in massed studying (by virtue of constraints on their study time or a failure to appreciate the benefits of spaced study sessions), then studying the information twice but for half the time may produce memory benefits in a single study session.
Chapter
Full-text available
examine 2 . . . contributors to nonoptimal training: (1) the learner's own misreading of his or her progress and current state of knowledge during training, and (2) nonoptimal relationships between the conditions of training and the conditions that can be expected to prevail in the posttraining real-world environment / [explore memory and metamemory considerations in training] (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
72 college students learned 3 motor tasks under a blocked (low interference) or random (high interference) sequence of presentation. Retention was measured after a 10-min or 10-day delay under blocked and random sequences of presentation. Subsequent transfer to a task of either the same complexity or greater complexity than the originally learned tasks was also investigated. Results showed that retention was greater following random acquisition than under changed contextual interference conditions. Likewise, transfer was greater for random acquisition groups than for blocked acquisition groups. This effect was most notable when transfer was measured for the transfer task of greatest complexity. Results are considered as support for W. F. Battig's (1978) conceptualization of contextual interference effects on retention and transfer. (13 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
This article constitutes an optimistic argument that basic research on human cognitive processes has yielded principles and phenomena that have considerable promise in guiding the design and execution of college instruction. To illustrate that point, four somewhat interrelated principles and phenomena arc outlined and some possible implications and applications of those principles and phenomena are put forward.
Article
Research on judgments of verbal learning has demonstrated that participants' judgments are unreliable and often overconfident The authors studied judgments of perceptual-motor learning. Participants learned 3 keystroke patterns on the number pad of a computer, each requiring that a different sequence of keys be struck in a different total movement time. Practice trials on each pattern were either blocked or randomly interleaved with trials on the other patterns, and each participant was asked, periodically, to predict his or her performance on a 24-hr test. Consistent with earlier findings, blocked practice enhanced acquisition but harmed retention. Participants, though, predicted better performance given blocked practice. These results augment research on judgments of verbal learning and suggest that humans, at their peril, interpret current ease of access to a perceptual-motor skill as a valid index of learning.
Article
The spacing effect would appear to have considerable potential for improving classroom learning, yet there is no evidence of its widespread application. I consider nine possible impediments to the implementation of research findings in the classroom in an effort to determine which, if any, apply to the spacing effect. I conclude that the apparent absence of systematic application may be due, in part, to the ahistorical character of research on the spacing effect and certain gaps in our understanding of both the spacing effect and classroom practice. However, because none of these concerns seems especially discouraging, and in view of what we do know about the spacing effect, classroom application is recommended.
Article
In a 9-year longitudinal investigation, 4 subjects learned and relearned 300 English-foreign language word pairs. Either 13 or 26 relearning sessions were administered at intervals of 14, 28, or 56 days. Retention was tested for 1.2.3. or 5 years after training terminated. The longer intersession intervals slowed down acquisition slightly, but this disadvantage during training was offset by substantially higher retention. Thirteen retraining sessions spaced at 56 days yielded retention comparable to 26 sessions spaced at 14 days. The retention benefit due to additional sessions was independent of the benefit due to spacing, and both variables facilitated retention of words regardless of difficulty level and of the consistency of retrieval during training. The benefits of spaced retrieval practice to long-term maintenance of access to academic knowledge areas are discussed.
Article
Four groups of postmen were trained to type alpha-numeric code material using a conventional typewriter keyboard. Training was based on sessions lasting for one or two hours occurring once or twice per day. Learning was most efficient in the group given one session of one hour per day, and least efficient in the group trained for two 2-hour sessions. Retention was tested after one, three or nine months, and indicated a loss in speed of about 30%. Again the group trained for two daily sessions of two hours performed most poorly. It is suggested that where operationally feasible, keyboard training should be distributed over time rather than massed.