ArticlePDF Available

Metacognitive strategies in student learning: Do students practise retrieval when they study on their own?


Abstract and Figures

Basic research on human learning and memory has shown that practising retrieval of information (by testing the information) has powerful effects on learning and long-term retention. Repeated testing enhances learning more than repeated reading, which often confers limited benefit beyond that gained from the initial reading of the material. Laboratory research also suggests that students lack metacognitive awareness of the mnemonic benefits of testing. The implication is that in real-world educational settings students may not engage in retrieval practise to enhance learning. To investigate students' real-world study behaviours, we surveyed 177 college students and asked them (1) to list strategies they used when studying (an open-ended free report question) and (2) to choose whether they would reread or practise recall after studying a textbook chapter (a forced report question). The results of both questions point to the same conclusion: A majority of students repeatedly read their notes or textbook (despite the limited benefits of this strategy), but relatively few engage in self-testing or retrieval practise while studying. We propose that many students experience illusions of competence while studying and that these illusions have significant consequences for the strategies students select when they monitor and regulate their own learning.
Content may be subject to copyright.
Metacognitive strategies in student learning:
Do students practise retrieval when they study
on their own?
Jeffrey D. Karpicke
Purdue University, West Lafayette, IN, USA
Andrew C. Butler and Henry L. Roediger III
Washington University in St. Louis, MO, USA
Basic research on human learning and memory has shown that practising retrieval of information (by
testing the information) has powerful effects on learning and long-term retention. Repeated testing
enhances learning more than repeated reading, which often confers limited benefit beyond that gained
from the initial reading of the material. Laboratory research also suggests that students lack
metacognitive awareness of the mnemonic benefits of testing. The implication is that in real-world
educational settings students may not engage in retrieval practise to enhance learning. To investigate
students’ real-world study behaviours, we surveyed 177 college students and asked them (1) to list
strategies they used when studying (an open-ended, free report question) and (2) to choose whether they
would reread or practise recall after studying a textbook chapter (a forced report question). The results
of both questions point to the same conclusion: A majority of students repeatedly read their notes or
textbook (despite the limited benefits of this strategy), but relatively few engage in self-testing or
retrieval practise while studying. We propose that many students experience illusions of competence
while studying and that these illusions have significant consequences for the strategies students select
when they monitor and regulate their own learning.
Keywords: Testing effect; Retrieval; Metacognition; Strategies.
A powerful way to enhance student learning is by
testing information. When students have been
tested on material they remember more in the
long term than if they had repeatedly studied it.
This phenomenon is known as the testing effect
and shows that the act of retrieving information
from memory has a potent effect on learning,
enhancing long-term retention of the tested
information (for review, see Roediger & Kar-
picke, 2006a). The testing effect is especially
striking in light of current findings showing
limited benefits of repeated reading for student
learning (see Callender & McDaniel, 2009;
McDaniel & Callender, 2008). Our recent re-
search has generalised the testing effect to
educational materials (Butler & Roediger, 2007;
Karpicke & Roediger, 2007, 2008; Roediger &
Karpicke, 2006b) and real-world classroom en-
vironments (see McDaniel, Roediger, & McDer-
mott, 2007). Testing enhances learning not only if
instructors give tests and quizzes in the classroom
but also if students practise recall while they study
#2009 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
Address correspondence to: Jeffrey D. Karpicke, Department of Psychological Sciences, Purdue University, 703 Third Street,
West Lafayette, IN 47907-2081, USA. E-mail:
We thank Stephanie Karpicke and Julie Evans for collecting and scoring the questionnaire data. This research was supported by a
Collaborative Activity Grant from the James S. McDonnell Foundation and a grant from the Institute of Education Science.
MEMORY, 2009, 17 (4), 471!479 DOI:10.1080/09658210802647009
Downloaded By: [Washington University School] At: 14:05 5 May 2009
on their own. If students were to practise retrieval
of information while studying this strategy would
have the potential to greatly improve academic
performance. However, we do not know the
extent to which students practise recall while
they study in real-world educational settings
(relative to other less-effective strategies like
repeated reading) or whether students who prac-
tise recall do so because they are aware of the
mnemonic benefits. These are important and
practically relevant research questions but few
studies have been aimed at answering them (see,
e.g., Kornell & Bjork, 2007).
The objective of this research was to determine
the extent to which students practise recall
relative to other study strategies in real-world
educational settings. In addition we wanted to
examine whether students who choose to engage
in retrieval practice do so because they know that
testing promotes long-term retention. Another
reason students may use testing during studying is
to determine what information is known and what
is not known so that future study time can be
allocated to the unknown material (see Dunlosky,
Hertzog, Kennedy, & Thiede, 2005; Dunlosky,
Rawson, & McDonald, 2002). This is a fine
justification for testing but it differs from using
testing as a learning device in its own right. To
accomplish these goals we created a new study
strategies questionnaire and surveyed a large
sample of undergraduate students. Although
there are a variety of study strategy inventories
in the education literature (see Entwistle &
McCune, 2004; Pintrich, Smith, Garcia, &
McKeachie, 1993; Weinstein, Schulte, & Palmer,
1987) these and other inventories do not specifi-
cally assess whether students practise retrieval
while studying. Our survey included a free report
question asking students to list the strategies they
use while studying and a forced report question
that asked them to choose between repeated
reading or repeated testing. The purpose of
including both forced and free report question
formats was to gain converging evidence aimed at
the target issue and to circumvent possible
response biases created by using either format
alone (see Schuman & Presser, 1996; Schwarz,
1999). We predicted that relatively few students
would report self-testing as a study strategy and
that the majority of students would report choos-
ing to reread or engage in some other non-testing
activity when forced to choose a study strategy.
We also predicted that most students who
selected self-testing would be unaware of the
mnemonic benefits of testing.
In the first section of this paper we provide a
brief overview of relevant research on repeated
reading, repeated testing, and students’ metacog-
nitive awareness of the testing effect. Next we
present the results of our survey of study strate-
gies. In the final section we interpret the survey
results in light of current theories of metacogni-
tion and self-regulated learning and then discuss
the practical and educational implications of our
The testing effect refers to the finding that taking
a test enhances long-term retention more than
spending an equivalent amount of time repeat-
edly studying. There are clear and direct implica-
tions of the testing effect for student learning.
One way for students to enhance their learning
would be to practise recalling information while
studying. However, research on the testing effect
has also shown that when students are asked to
assess their own learning they sometimes fail to
predict that testing enhances learning more than
repeated reading (e.g., Karpicke & Roediger,
2008). In short, there is a rapidly growing body
of research (briefly reviewed below) indicating
that testing has powerful effects on learning but
students lack metacognitive awareness of the
testing effect.
Students often report that they repeatedly read
their notes or textbook while studying (Carrier,
2003; Pressley, Van Etten, Yokoi, Freebern, &
Van Meter, 1998; Van Etten, Freebern, & Press-
ley, 1997). Yet there are several reasons to
question the effectiveness of repetitive reading
beyond reading a single time. Basic research on
memory has shown that spending extra time
maintaining or holding items in memory does
not by itself promote learning (Craik & Watkins,
1973) and students may spend large amounts of
additional time studying despite no gain in later
memory for the items, a phenomenon called
‘‘labour-in-vain’’ during learning (Nelson & Leo-
nesio, 1988). Recent research with educationally
relevant materials has shown that repeatedly
reading prose passages produces limited benefits
beyond a single reading (Amlund, Kardash, &
Kulhavy, 1986; Callender & McDaniel, 2009).
Downloaded By: [Washington University School] At: 14:05 5 May 2009
This is especially true when repeated readings are
massed together in a single learning session,
although spaced rereading tends to produce
positive effects (Rawson & Kintsch, 2005). In
short, memory research has shown many times
that repetitive reading by itself is not an effective
strategy for promoting learning and long-term
retention (for review, see McDaniel & Callender,
In contrast, several studies have shown that
repeated testing is a potent method for producing
robust learning. In one of our studies (Karpicke &
Roediger, 2008) we had students learn a set of
Swahili vocabulary words across alternating study
and test periods. In study periods students studied
a Swahili word and its English translation (ma-
shua !boat) and in test periods they saw the
Swahili words as cues to recall the English words
(mashua !?). The students learned the words in
one of four conditions and students in all condi-
tions took a final test 1 week after initial learning.
In two learning conditions, once a word was
correctly recalled it was dropped from further
test periods. The students who recalled each word
only once in these two conditions recalled just
35% of the items on the final test a week later. In
the other two conditions students continued to
repeatedly recall words even after they had
recalled them once. Students who repeatedly
recalled the words during learning recalled about
80% of the items on the final test. Repeated
retrieval practice*even after students were able
to successfully recall items in the learning phase*
produced large positive effects on long-term
Were students aware of the effect of repeated
testing on long-term retention? At the end of the
initial learning phase we asked students to predict
how many pairs they would recall on the final test
a week later. There was no difference in average
predictions across the four conditions: All groups
predicted they would recall about 50% of the
items. Despite the large effect of repeated retrie-
val on retention, students were not aware of the
mnemonic benefit of testing. Similar findings
have occurred in other experiments examining
the testing effect and students’ judgements of
learning (e.g., Agarwal, Karpicke, Kang, Roedi-
ger, & McDermott, 2008; Karpicke, McCabe, &
Roediger, 2006; Roediger & Karpicke, 2006b).
In sum, basic laboratory research on human
learning and memory has shown that (1) repeated
reading by itself is a questionable and often
ineffective study strategy, (2) repeated retrieval
practice produces robust learning and long-term
retention, but (3) students appear to lack meta-
cognitive awareness of the testing effect. The
implication of this basic research is that students
may not practise retrieval when they study in real-
world educational settings. Instead they may
spend their time repeatedly reading material
when they study. The objective of our survey
was to examine the prevalence of retrieval
practice, relative to other study strategies, in
students’ real-world study behaviours and stu-
dents’ metacognitive awareness of the benefits of
One reaction we have encountered when we
present our research on the testing effect goes
something like this: ‘‘This is completely obvious.
Of course testing enhances learning. We already
knew this. None of this is new or surprising.’’
Perhaps the testing effect is obvious to some
instructors*but is it obvious to students? If so
we would expect students to report that they
frequently practise recall while studying. But our
basic laboratory research has consistently shown
that students lack metacognitive awareness of the
testing effect. In fact students sometimes predict
that repeated reading will produce better long-
term retention than repeated testing (Roediger &
Karpicke, 2006b). The intent of our survey was to
determine whether students’ self-reported study
behaviours would converge with our laboratory
We surveyed 177 undergraduate students at
Washington University in St. Louis about the
strategies they use to study for exams. The
students were participants in various learning
and memory experiments in our laboratory and
they completed the survey at the very end of their
experimental session. Washington University stu-
dents are a select group with average SAT scores
greater than 1400 (Verbal"Quantitative). Our
survey included two questions aimed at identify-
ing how often the students practised recalling
information while studying. Question 1 was
an open-ended free report question in which
students listed the strategies they used when
Downloaded By: [Washington University School] At: 14:05 5 May 2009
studying and then rank ordered the strategies in
terms of how frequently they used them. All 177
students answered Question 1. Question 2 was a
forced report question that asked students to
imagine they were studying a textbook chapter
for an exam and to choose one of three alter-
natives: (1) repeated reading of the chapter, (2)
practising recall of material from the chapter
(with or without the opportunity to reread the
chapter, in different versions of this question), or
(3) engaging in some other study activity. A total
of 101 students answered Version 1 of Question 2
(testing without restudy) and the other 76 stu-
dents answered Version 2 (testing with restudy).
Students completed the entire questionnaire in
about 5 to 10 minutes. Our goals were to identify
students’ typical study strategies and to assess
how frequently they repeatedly read material or
engaged in retrieval practice, and our analysis
focused on the frequency with which students
reported these particular strategies.
Question 1: Students’ free report of study
strategies. The first question on the survey asked:
‘‘What kind of strategies do you use when you are
studying? List as many strategies as you use and
rank-order them from strategies you use most
often to strategies you use least often.’’ We
initially reviewed all responses from all students.
Based on our initial assessment we identified 11
strategies that occurred relatively frequently
(more than once across all student responses).
Two independent raters then categorised all
responses. There was close to 100% agreement
between the two raters and the first author
resolved any scoring discrepancies.
Figure 1 shows the frequency distribution of
the number of strategies listed by students in
response to Question 1. The figure shows that
most students listed and described three strate-
gies (M#2.9). Table 1 shows the 11 strategies and
the percent of students who listed each strategy.
The table also shows the percent of students who
ranked each strategy as their number one strategy
and the mean rank of each strategy. Repeated
reading was by far the most frequently listed
strategy with 84% of students reporting it. Not
only did students indicate that they repeatedly
read while studying but they also indicated that
rereading was a favoured strategy*55% of stu-
dents reported that rereading was the number one
strategy they used when studying. Table 1 also
shows another key finding: Only 11% of students
(19 of 177) reported that they practised retrieval
while studying. These students unambiguously
indicated in their list of strategies that they
practised testing themselves by recalling informa-
tion while they studied. Only 1% (2 of 177
students) identified practising recall as their
number one strategy. The results in Table 1
clearly show that a large majority of students
repeatedly read their textbook or notes but
relatively few students engage in self-testing by
practicing recall while studying.
Table 1 also shows that students reported other
strategies that could be interpreted as forms of
self-testing. For example, 43% of students indi-
cated that they answer practice problems while
studying and 40% reported using flashcards. Each
activity could be interpreted as a type of self-
testing, but of course there are ways students
might use these study methods without engaging
in retrieval practice. For example, students may
read practice questions and then look up and
copy answers from the text. This would qualify as
answering practice problems but students who do
this would not be practising or even attempting
recall of the answers. Likewise, students may
write facts on flashcards and repeatedly read
them rather than practising recall. A clear limita-
tion of the free response question is that our
procedure did not prompt each student to
Figure 1. Frequency distribution showing the number of
strategies listed by students.
Downloaded By: [Washington University School] At: 14:05 5 May 2009
elaborate on potentially ambiguous answers (cf.
the ethnographic interviewing technique of Press-
ley and colleagues; Pressley et al., 1998; Van
Etten et al., 1997). Nevertheless, even if we
considered the 40% of students who use flash-
cards or the 43% who answer practice problems
as students engaging in forms of self-testing, these
percentages are dwarfed by the 84% of students
who repeatedly read while studying.
The results of Question 1 indicate that re-
peated reading is the most popular study strategy
among college students (see too Carrier, 2003),
far more popular than practising retrieval, even
though retrieval practice is a more effective study
strategy. Students listed a variety of study strate-
gies but indicated that they use these alternative
study strategies far less frequently than repeated
reading. Question 2 asked students to choose
repeated reading or self-testing and prompted
them to explain the reasoning behind their choice.
By including a second question in forced report
format we hoped to find converging evidence and
to resolve ambiguities inherent in our first open-
ended free report question.
Question 2: Forced report questions about
repeated studying vs testing. Question 2 was a
forced report question about repeated studying
versus repeated testing. There were two versions
of the question. Version 1 asked students to
consider testing without going back and re-
studying, and Version 2 involved testing followed
by restudying (to get feedback after attempting
recall). The first version was given to 101 students
and the second version was given to 76 students.
Version 1 of Question 2 was as follows:
Imagine you are reading a textbook chapter for
an upcoming exam. After you have read the
chapter one time, would you rather:
A. Go back and restudy either the entire
chapter or certain parts of the chapter.
B. Try to recall material from the chapter
(without the possibility of restudying the
C. Use some other study technique.
The students were asked to select one alter-
native and write a brief explanation for their
choice. The scenario described in the question
was based directly on our research showing that
taking a recall test, even without feedback,
enhances long-term retention more than spending
the same amount of time restudying (Roediger &
Karpicke, 2006b).
Table 2 shows the percentage of students who
chose to restudy, self-test, or do something else
after reading a textbook chapter. Most students
unambiguously selected an alternative and ex-
plained their choice, but four students gave
ambiguous responses that could not be scored.
The table shows that 57% of students chose to
restudy (option A) and 21% indicated that they
would use some other study technique (option C).
Thus 78% of students indicated they would not
want to test themselves after reading a textbook
chapter. Only 18% of the students indicated that
they would self-test after studying (option B). To
examine students’ metacognitive awareness of the
Results of Question 1
Strategy Percent who list strategy Percent who rank as #1 strategy Mean rank
1. Rereading notes or textbook 83.6 (148) 54.8 (97) 1.5
2. Do practice problems 42.9 (76) 12.4 (22) 2.1
3. Flashcards 40.1 (71) 6.2 (11) 2.6
4. Rewrite notes 29.9 (53) 12.4 (22) 1.8
5. Study with a group of students 26.5 (47) 0.5 (1) 2.9
6. ‘‘Memorise’’ 18.6 (33) 5.6 (10) 2.0
7. Mnemonics (acronyms, rhymes, etc) 13.5 (24) 2.8 (5) 2.4
8. Make outlines or review sheets 12.9 (23) 3.9 (7) 2.1
9. Practise recall (self-testing) 10.7 (19) 1.1 (2) 2.5
10. Highlight (in notes or book) 6.2 (11) 1.6 (3) 2.3
11. Think of real life examples 4.5 (8) 0.5 (1) 2.8
Percent of students listing different learning strategies, percent who ranked strategies as their #1 strategy, and mean rankings of
strategies. Raw numbers of students are in parentheses.
Mean number of strategies listed was 2.9 (SD#0.96). Percentages of students indicating their #1 strategy do not add to 100%
because some students merged multiple strategies when reporting their #1 strategy (e.g., indicating that rereading and rewriting
notes were their #1 strategy).
Downloaded By: [Washington University School] At: 14:05 5 May 2009
mnemonic benefits of testing we separated stu-
dents’ responses based on their explanation for
why they chose self-testing. This analysis showed
that 10% of all students (or more than half of
those who chose self-testing) reported they would
self-test to generate feedback and guide their
future studying (even though Version 1 of this
question stated that students could not restudy
after testing). Only 8% of all students indicated
that they would test themselves because practis-
ing retrieval would help them do well on the
upcoming exam. This pattern of responding
suggests that most students were unaware of the
mnemonic benefits of self-testing. The results of
Version 1 of this forced report question provide
converging evidence with our first free report
question. Relatively few students reported that
they would test themselves after studying a text-
book chapter and even fewer indicated they
would test themselves because they knew the
act of practising recall was valuable for learning.
In Version 2 of Question 2 the scenario and
alternatives were identical to Version 1 except
that option B read ‘‘Try to recall material from
the chapter (with the possibility of restudying
afterward).’’ We imagined this would increase the
number of students choosing testing perhaps to
levels near ceiling if students recognised that
testing followed by rereading would produce far
superior learning to rereading without testing.
Table 3 shows the percent of students who chose
each option. The percentage of students choosing
self-testing increased when students could reread
after the test (42% in Question 2 vs 18% in
Question 1) and the percentage was about equal
to the percentage of students choosing repeated
reading (42% vs 41%). Students’ explanations of
their choices indicated that the increased like-
lihood of choosing testing was due to the possi-
bility of restudying after the test. Of the 32
students who chose self-testing, 25 provided
unambiguous explanations that we categorised
as testing for feedback or testing to practise
recall. A total of 23 students (30%) indicated
that they would test themselves to generate
feedback they could use when restudying
whereas only two students (3%) chose testing
because they believed the act of practising recall
would help them remember in the future. The
results of Version 2 of Question 2 expand on the
results of Version 1 by showing that students were
more likely to select self-testing when they could
restudy after testing but that very few students
are aware that the act of practising recall itself
enhances learning. What is perhaps most striking
about the data in Table 3 is that even when
students had the option of rereading after self-
testing, the majority of students (58%) continued
to indicate that they would not test themselves.
The objective of this research was to collect
benchmark data on college students’ real-world
study behaviours to assess how often students use
retrieval practice relative to other strategies and
whether they know about the mnemonic benefits
of self-testing. Our basic laboratory studies sug-
gested that students are not aware of the testing
effect, leading us to predict that they may not
practise retrieval while studying in real-world
settings. The results of our survey support this
prediction. The majority of students indicated
that they repeatedly read their notes or textbook
while studying. Relatively few reported that they
Version 1 of Question 2
Imagine you are reading a
textbook chapter for an upcoming
exam. After you have read the
chapter one time, would you rather: Overall
Test for
Test to
A. Go back and restudy either the entire
chapter or certain parts of the chapter
57.4 (58)
B. Try to recall material from the chapter
(without the possibility of restudying the material) 17.8 (18) 9.9 (10) 7.9 (8)
C. Use some other study technique 20.7 (21)
Percent of students who chose to restudy, self-test (without restudying), or do something else after reading a textbook chapter.
Raw numbers of students are in parentheses (N#101).
We were unable to score ambiguous responses given by four students.
Downloaded By: [Washington University School] At: 14:05 5 May 2009
tested themselves and of those who engaged in
self-testing only a handful reported doing so
because they believed the act of practising
retrieval would improve their learning. Our
survey results point to the conclusion that many
students do not view retrieval practice as a
strategy that promotes learning. If students do
practise recall or test themselves while studying
they do it to generate feedback or knowledge
about the status of their own learning, not
because they believe practising recall itself en-
hances learning.
Our results agree with laboratory experiments
showing that students lack metacognitive aware-
ness of the testing effect when they monitor their
own learning. A growing body of research has
shown that students sometimes predict that
practising retrieval will produce no effect on
retention (Karpicke & Roediger, 2008) or that
they will remember more in the long term if they
repeatedly study material rather than test it
(Agarwal et al., 2008; Karpicke et al., 2006;
Roediger & Karpicke, 2006b). If we assume that
metacognitive monitoring processes guide stu-
dents’ decisions to choose different learning
strategies*an assumption at the core of the
influential monitoring-and-control framework of
metacognition (Nelson & Narens, 1990)*then
the implication of these laboratory results is that
students may not choose to test themselves when
they regulate their own learning in real-world
educational settings. Our survey data confirm
that this lack of awareness of the testing effect
has consequences for students’ real-world study
In addition to agreeing with basic laboratory
findings our survey results also agree to some
extent with a recent survey by Kornell and Bjork
(2007). They surveyed college students about
their study behaviours and asked the students,
‘‘If you quiz yourself while you study ...why do
you do so?’’ The students selected one of four
alternatives: 18% selected ‘‘I learn more that way
than I would through rereading’’; 68% selected
‘‘To figure out how well I have learned the
information I’m studying’’; 4% indicated ‘‘I find
quizzing more enjoyable than rereading’’; and 9%
said ‘‘I usually do not quiz myself.’’ Kornell and
Bjork’s data indicate that the majority of students
(91%) do quiz themselves while studying but few
do so because they view the act of quizzing itself
as a method of enhancing learning (Kornell and
Bjork reasoned that the 18% of students who
selected ‘‘I learn more that way than I would
through rereading’’ believed that quizzing pro-
duced a direct mnemonic benefit; cf. Roediger &
Karpicke, 2006a). Likewise, our survey data
indicate that few students view practising recall
as an activity that enhances learning. However,
far more students indicated that they tested
themselves in the Kornell and Bjork survey than
in our study, and this may be due to a difference
in survey procedures. Whereas we used a combi-
nation of free and forced report questions to
gauge how often students practise retrieval,
Kornell and Bjork used one question focused on
why students might quiz themselves and the
framing of this question may have influenced
students’ responses (see Schuman & Presser,
1996; Schwarz, 1999). It is well known that a
single question can be framed in different ways
and alter the choices and decisions people make
(Tversky & Kahneman, 1981). Nevertheless our
results generally agree with those of Kornell and
Bjork in showing that few students view retrieval
practice as a method of enhancing learning.
Further, the differences between the two sets of
results highlight potentially important differences
between free and forced report methods of
Version 2 of Question 2
Imagine you are reading a textbook
chapter for an upcoming exam. After
you have read the chapter one time,
would you rather: Overall Test for feedback Test to practise recall
A. Go back and restudy either the entire
chapter or certain parts of the chapter
40.8 (31)
B. Try to recall material from the chapter
(with the possibility of restudying afterward) 42.1 (32) 30.3 (23) 2.6 (2)
C. Use some other study technique 17.1 (13)
Percent of students who chose to restudy, self-test followed by restudying, or do something else after reading a textbook chapter.
Raw numbers of students are in parentheses (N#76).
Downloaded By: [Washington University School] At: 14:05 5 May 2009
Our results fit with the broad theoretical
notion that students experience illusions of com-
petence when monitoring their own learning
(Bjork, 1999; Jacoby, Bjork, & Kelley, 1994;
Koriat & Bjork, 2005). Koriat and Bjork (2005)
argued that illusions of competence tend to occur
when students’ judgements of learning are biased
by information available during study but not
available during testing (see also Jacoby et al.,
1994). Several experimental findings are consis-
tent with this view. For example, students’ judge-
ments of learning are less accurate when made in
study trials than in test trials (Dunlosky &
Nelson, 1992). Students are less accurate at
judging the difficulty of anagrams when the
solution is present than when it is not (Kelley &
Jacoby, 1996). We believe repeated reading pro-
duces a similar illusion of competence. Specifi-
cally, repeatedly reading material like text
passages increases the fluency or ease with which
students process the text. Students may base their
assessments of their learning and comprehension
on fluency even though their current processing
fluency with the text right in front of them, is
not diagnostic of their future retention. Our
survey results show that the illusions students
experience during learning may have important
consequences and implications for the decisions
they make and the strategies they choose when
studying on their own.
Students generally exhibit little awareness of
the fact that practising retrieval enhances learn-
ing. A clear practical implication is that instruc-
tors should inform students about the benefits of
self-testing and explain why testing enhances
learning. When students rely purely on their
subjective experience while they study (e.g., their
fluency of processing during rereading) they may
fall prey to illusions of competence and believe
they know the material better than they actually
do. A challenge for instructional practice is to
encourage students to base their study strategies
on theories about why a particular strategy*like
practising repeated retrieval*promotes learning
and long-term retention.
Manuscript received 5 October 2008
Manuscript accepted 20 November 2008
Agarwal, P. K., Karpicke, J. D., Kang, S. H. K.,
Roediger, H. L., & McDermott, K. B. (2008).
Examining the testing effect with open- and
closed-book tests. Applied Cognitive Psychology,
22, 861!876.
Amlund, J. T., Kardash, C. A. M., & Kulhavy, R. W.
(1986). Repetitive reading and recall of expository
text. Reading Research Quarterly,21, 49!58.
Bjork, R. A. (1999). Assessing our own competence:
Heuristics and illusions. In D. Gopher & A. Koriat
(Eds.), Attention and peformance XVII. Cognitive
regulation of performance: Interaction of theory and
application (pp. 435!459). Cambridge, MA: MIT
Butler, A. C., & Roediger, H. L. (2007). Testing
improves long-term retention in a simulated class-
room setting. European Journal of Cognitive Psy-
chology,19, 514!527.
Callender, A. A., & McDaniel, M. A. (2009). The
limited benefits of rereading educational texts.
Contemporary Educational Psychology,34, 30!41.
Carrier, L. M. (2003). College students’ choices of study
strategies. Perceptual and Motor Skills,96, 54!56.
Craik, F. I. M., & Watkins, M. J. (1973). The role of
rehearsal in short-term memory. Journal of Verbal
Learning and Verbal Behavior,12, 599!607.
Dunlosky, J., Hertzog, C., Kennedy, M. R. F., & Thiede,
K. W. (2005). The self-monitoring approach for
effective learning. Cognitive Technology,10,4!11.
Dunlosky, J., & Nelson, T. O. (1992). Importance of the
kind of cue for judgments of learning (JOL) and the
delayed-JOL effect. Memory & Cognition,20, 374!
Dunlosky, J., Rawson, K. A., & McDonald, S. L. (2002).
Influence of practice test on the accuracy of
predicting memory performance for paired associ-
ates, sentences, and text material. In T. J. Perfect &
B. L. Schwartz (Eds.), Applied metacognition (pp.
68!92). Cambridge, UK: Cambridge University
Entwistle, N., & McCune, V. (2004). The conceptual
bases of study strategy inventories. Educational
Psychology Review,16, 325!345.
Jacoby, L. L., Bjork, R. A., & Kelley, C. M. (1994).
Illusions of comprehension, competence, and re-
membering. In D. Druckman & R. A. Bjork (Eds.),
Learning, remembering, believing: Enhancing hu-
man performance (pp. 57!80). Washington, DC:
National Academy Press.
Karpicke, J. D., McCabe, D. P., & Roediger, H. L.
(2006). Testing enhances recollection: Process dis-
sociation estimates and metamemory judgments.
Poster presented at the 47th Annual Meeting of
the Psychonomic Society, Houston, TX.
Karpicke, J. D., & Roediger, H. L. (2007). Repeated
retrieval during learning is the key to long-term
retention. Journal of Memory and Language,57,
Karpicke, J. D., & Roediger, H. L. (2008). The critical
importance of retrieval for learning. Science,319,
Kelley, C. M., & Jacoby, L. L. (1996). Adult egocentr-
ism: Subjective experience versus analytic bases for
judgment. Journal of Memory and Language,35,
Koriat, A., & Bjork, R. A. (2005). Illusions of compe-
tence in monitoring one’s knowledge during study.
Downloaded By: [Washington University School] At: 14:05 5 May 2009
Journal of Experimental Psychology: Learning,
Memory, and Cognition,31, 187!194.
Kornell, N., & Bjork, R. A. (2007). The promise and
perils of self-regulated study. Psychonomic Bulletin
& Review,14, 219!224.
McDaniel, M. A., & Callender, A. A. (2008). Cogni-
tion, memory, and education. In H. L. Roediger
(Ed.), Cognitive psychology of memory, Vol. 2 of
Learning and memory: A comprehensive reference
(pp. 819!843). Oxford, UK: Elsevier.
McDaniel, M. A., Roediger, H. L., & McDermott, K. B.
(2007). Generalizing test-enhanced learning from
the laboratory to the classroom. Psychonomic Bul-
letin & Review,14, 200!206.
Nelson, T. O., & Leonesio, R. J. (1988). Allocation of
self-paced study time and the ‘‘labor-in-vain effect’’.
Journal of Experimental Psychology: Learning,
Memory, and Cognition,14, 676!686.
Nelson, T. O., & Narens, L. (1990). Metamemory: A
theoretical framework and new findings. In G. H.
Bower (Ed.), The psychology of learning and
motivation (Vol. 26, (pp. 125!141)). New York:
Academic Press.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKea-
chie, W. J. (1993). Reliability and predictive validity
of the Motivated Strategies for Learning Question-
naire (MSLQ). Educational and Psychological Mea-
surement,53, 801!813.
Pressley, M., Van Etten, S., Yokoi, L., Freebern, G., &
Van Meter, P. (1998). The metacognition of college
studentship: A grounded theory approach. In D. J.
Hacker, J. Dunlosky, & A. C. Graesser (Eds.),
Metacognition in educational theory and practice
(pp. 347!366). Mahwah, NJ: Lawrence Erlbaum
Associates Inc.
Rawson, K. A., & Kintsch, W. (2005). Rereading effects
depend on time of test. Journal of Educational
Psychology,97, 70!80.
Roediger, H. L., & Karpicke, J. D. (2006a). The power
of testing memory: Basic research and implications
for educational practice. Perspectives on Psycholo-
gical Science,1, 181!210.
Roediger, H. L., & Karpicke, J. D. (2006b). Test
enhanced learning: Taking memory tests improves
long-term retention. Psychological Science,17, 249!
Schuman, H., & Presser, S. (1996). Questions and
answers in attitude surveys. Thousand Oaks, CA:
Schwarz, N. (1999). Self-reports: How the questions
shape the answers. American Psychologist,54, 93!
Tversky, A., & Kahneman, D. (1981). The framing of
decisions and the psychology of choice. Science,211,
Van Etten, S., Freebern, G., & Pressley, M. (1997).
College students’ beliefs about exam preparation.
Contemporary Educational Psychology,22, 192!212.
Weinstein, C. E., Schulte, A. C., & Palmer, D. R.
(1987). Learning and Study Strategies Inventory
(LASSI). Clearwater, FL: H&H Publishing.
Downloaded By: [Washington University School] At: 14:05 5 May 2009
This article was downloaded by:
[Washington University School]
5 May 2009
Access details:
Access Details: [subscription number 908307764]
Psychology Press
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK
Publication details, including instructions for authors and subscription information:
Metacognitive strategies in student learning: Do students practise retrieval
when they study on their own?
Jeffrey D. Karpicke a; Andrew C. Butler b; Henry L. Roediger III b
a Purdue University, West Lafayette, IN, USA b Washington University in St. Louis, MO, USA
Online Publication Date: 01 May 2009
To cite this Article Karpicke, Jeffrey D., Butler, Andrew C. and Roediger III, Henry L.(2009)'Metacognitive strategies in student learning:
Do students practise retrieval when they study on their own?',Memory,17:4,471 — 479
To link to this Article: DOI: 10.1080/09658210802647009
Full terms and conditions of use:
This article may be used for research, teaching and private study purposes. Any substantial or
systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or
distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses
should be independently verified with primary sources. The publisher shall not be liable for any loss,
actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly
or indirectly in connection with or arising out of the use of this material.
... While some students use better learning techniques, many students rely on relatively ineffective approaches. Rereading notes or textbooks is one of the most popular learning techniques (Karpicke et al., 2009). Brown et al. (2014) identified the following disadvantages students encounter when rereading: (1) it is time consuming, (2) the content learned does not 'stick', especially when rereading is massed (see also Callender & McDaniel, 2009), and (3) it induces self-deception because familiarity with the text implies mastery of the content. ...
Full-text available
This study explored long-term changes in learning behavior of students participating in an introduction to learning how to learn. The intervention that took place during the first semester included the Massive Open Online Course (MOOC) ‘Learning How to Learn’, discussions on the content of the MOOC, and hands-on practice using different learning platforms in the computer-assisted language learning (CALL) component of a Spanish 1 course. The CALL component focused on grammar, vocabulary and listening comprehension training throughout the four-semester observation period and was used in this study to verify changes in students’ learning behavior. The purpose of the study was to investigate if the changes that had already been observed during the first year, would still be noticeable in the second year and how these changes influence outcomes of students’ foreign language learning. Analyses of students’ questionnaires, observations of their study behavior when using the learning platforms, and their academic performance during the study showed that the positive changes students of the experimental group had exhibited during the first year were stable or even increased as more differences became significant compared to the first year and compared to the control group. Different from students in the control group who had not taken part in the intervention, students reported to use more learning techniques, to redo exercises with answers covered and to space repetitions instead of cramming before exams. The only exception was the decline in smartphone use while studying that had been reported in the first year. This was back to pre-intervention level in the second year. Observations of students’ use of the learning platforms confirmed answers to the questionnaires. This far more effective use of the CALL components by students of the experimental group led to better results in their academic performance. Thus, time spent on ‘Learning how to learn’ was time well spent.
... Psychologists have also developed numerous principles of good teaching that are supported by substantial bodies of evidence and can be implemented with relative ease in classrooms (Dunn, Saville, Baker, & Marek, 2013). These practices include (a) testing to enhance memory (e.g., Butler and Roediger, 2007;Karpicke, Butler, & Roediger, 2009) and transfer (e.g., Carpenter, 2012), (b) distributed practice or spaced learning (e.g., Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006;Wahlheim, Dunlosky, & Jacoby, 2011), (c) metacognitive skills such as reflective note-taking (e.g., Dunn, 2011), (d) writing to learn (e.g., Berninger, 2012;Dunn 1994Dunn , 2011, and (e) interteaching (e.g., Boyce & Hineline, 2002;Saville, Lambert, & Robertson, 2011;Saville & Zinn, 2009). Students profit from guidance and reminders from their instructors on the benefits of these strategies, as well as on the effects of other habits and behaviors known to impact academic success (Dunn et al., 2013). ...
Full-text available
Educational psychology is a field that straddles two large domains: education and psychology. Reaching far back into antiquity, the field was borne from philosophies and theories that weaved back and forth between each domain all with the intent of understanding the way learners learn, teachers teach, and educational settings should be effectively designed. This chapter tells the story of educational psychology – its evolution, its characteristics, and the insights it provides for understanding it as a field of study, teaching it at the tertiary level of education, and leveraging its findings in the classroom. The chapter begins with a rationale for a curriculum of educational psychology, tracing its core teaching and learning objectives. It describes the topics that are core to the field, as well as the theory-based and evidence-based strategies and approaches for teaching it effectively. It discusses the basic principles of effective teaching, including problem-based learning, inquiry-based learning, and small-group and service-based learning, among others. Finally, it addresses technology in learning, open-university teaching and learning, and closes with a discussion of the best approaches – both theory-based and evidence-based – for assessing the core competencies of the field.
Flashcards are a popular study tool, however learner decisions can lower their effectiveness. One such decision is whether or not to drop a concept from study. Using objective mastery criteria that adaptively determine when to add or drop an item from study based on performance may improve learning outcomes in flashcard-based tasks. The effectiveness of adaptive flashcard-based learning may also vary based on the cognitive ability of the learner. The current study examined the impact of adaptive mastery instructional strategies on learning butterfly species and whether or not the impact of adaptive mastery varies by cognitive ability. Three learning conditions were compared: a No Add/Drop group (all items remain in the deck throughout study), a Mastery Drop group (start with all items, then drop after an item is mastered), and a Mastery Add group (start with three items, add items once mastered). A pre-post-transfer test design was used both immediately after training and one week later. Participants also completed the symmetry span task and a change detection task to evaluate cognitive ability. Results show the worst overall immediate pre-post learning gains in the Mastery Drop condition compared to the Mastery Add and No Add/Drop conditions which showed similar learning gains. This pattern went away when looking at delayed pre-post learning gains. Cognitive ability did not have any impact on learning performance, suggesting that similar strategies work equally well across all levels of cognitive ability. These results suggest adaptively adding cards is better than dropping them, though if there are no time constraints, leaving all concepts in the deck leads to the best overall learning in the short term.
In an effort to modernize training, the United States Navy and Marine Corps have placed an emphasis on identifying effective, learner-centric instructional methods. One avenue is to apply individualized training techniques, such as adaptive sequencing to flashcard-based study, a popular tool used for independent study. Therefore, the goal of this research was to compare two adaptive sequencing methods to identify the most efficient and effective approach for long-term retention. Participants learned armored vehicle identification in one of three conditions, 1) Adaptive Response-Time Based-Sequencing (ARTS), which uses accuracy and reaction time to prioritize flashcards adaptively, 2) Leitner, which uses accuracy to create decks and prioritize flashcards adaptively, and 3) Random, which was the control condition that sequenced the flashcards randomly. We found no differences between the three conditions in terms of learning efficiency and delayed learning gains. However, we found that participants in the Leitner condition completed training significantly faster those in the other conditions. After controlling for in-training measures, we found participants in the Leitner condition had the lowest delayed learning gains and there were no significant differences between the ARTS and Random conditions. These data suggest that the “efficiency” associated with the Leitner condition translated to the worst long-term retention. Additionally, neither adaptive approach outperformed the random sequencing condition, suggesting that random sequencing may provide its own spacing due to the number of cards in the deck. More research is needed to determine whether adaptive sequencing provides added value as an adaptive approach for single-session flashcard study.
Many science, technology, engineering, and math (STEM) community college students do not complete their degree, and these students are more likely to be women or in historically excluded racial or ethnic groups. In introductory courses, low grades can trigger this exodus. Implementation of high-impact study strategies could lead to increased academic performance and retention. The examination of study strategies rarely occurs at the community college level, even though community colleges educate approximately half of all STEM students in the United States who earn a bachelor's degree. To fill this research gap, we studied students in two biology courses at a Hispanic-serving community college. Students were asked their most commonly used study strategies at the start and end of the semester. They were given a presentation on study skills toward the beginning of the semester and asked to self-assess their study strategies for each exam. We observed a significantly higher course grade for students who reported spacing their studying and creating drawings when controlling for demographic factors, and usage of these strategies increased by the end of the semester. We conclude that high-impact study strategies can be taught to students in community college biology courses and result in higher course performance.
Full-text available
Purpose: To understand how educational disparities unfolded after COVID-19 broke out, this study examined differences in effects of relevant factors on academic achievements before and after the COVID-19 outbreak. Design/methodology/data/approach: Data collected in two time points (2019 and 2020) from 2,699 middle school students as part of the Busan Education Longitudinal Study were used. Structural equation modeling (SEM) was applied to investigate the changes in relationship of the educational outcomes in Korean Language, English Language, and Math, with parental education, parental support, education expenditure, learning attitude, self-directed learning, and previous achievement. Findings/Results: The distributions of test scores demonstrated the possibilities of learning loss in subjects of Korean and English and educational disparities in the subjects of Korean and math. The SEM results revealed that student variables became more influential than parent variables after the pandemic. Especially, the effect of previous achievement on current achievement and the effect of self-directed learning on learning attitude became more prominent. Value: This study provides empirical evidence of the possible effects of COVID-19-related disruptions in education on educational achievements and disparities.
Full-text available
This paper reports on a new self-report, Likert-scaled instrument that was designed to assess motivation and use of learning strategies by college students. The motivation scales tap into three broad areas: (1) value (intrinsic and extrinsic goal orientation, task value), (2) expectancy (control beliefs about learning, self-efficacy); and (3) affect (test anxiety). The learning strategies section is comprised of nine scales which can be distinguished as cognitive, metacognitive, and resource management strategies. The cognitive strategies scales include (a) rehearsal, (b) elaboration, (c) organization, and (d) critical thinking. Metacognitive strategies are assessed by one large scale that includes planning, monitoring, and regulating strategies. Resource management strategies include (a) managing time and study environment; (b) effort management, (c) peer learning, and (d) help-seeking. Scale reliabilities are robust, and confirmatory factor analyses demonstrated good factor structure. In addition, the instrument shows reasonable predictive validity to the actual course performance of students.
Based on findings in basic memory and comprehension research, we note implications for enhancing acquisition of factual information. For each implication, we identify techniques that could be implemented to assist students’ acquisition of material in fact-laden courses. We evaluate the potential effectiveness of those techniques for education practice by reviewing translational research that uses educationally relevant materials and test tasks. Specifically, extending basic memory research, we discuss implications from findings on rote repetition (rereading text), elaboration (elaborative interrogation, advanced organizers, and imagery mnemonics), and desirable difficulty (creating interference, spacing, and generation). We suggest a contextualistic framework for understanding when difficulty is and is not desirable, including considerations of transfer-appropriate processing, material-appropriate processing, and individual learner differences. Building on basic research in comprehensive, we identify study methods that are sensitive to individual differences, techniques to improve successful inferencing, and principled text revisions to enhance comprehension. The final section touches on the mnemonic benefits of retrieval and discusses recent research demonstrating the value of using repeated testing to enhance learning (i.e., to improve performance on final criterial tests). The research discussed in this chapter supports the efforts to apply basic findings in memory and cognition to enhance educational practice.
This chapter focuses on research program, providing a description of a theoretical framework that has evolved out of metamemory research, followed by a few remarks about the methodology. Research in metamemory is initiated by the paradoxical findings that people can accurately predict their subsequent likelihood of recognizing nonrecallable items and that they can quickly and accurately decide-on the basis of no more than a cursory search through memory-that they will not retrieve particular sought after items. Those findings lead to develop a methodology based on psychophysical methods that are used to empirically investigate people's feeling of knowing. The results of the experiments convinced that for dealing with only a part of a complex metacognitive system and to account adequately for feeling-of-knowing phenomena, a larger perspective was needed. This eventuated in the present theoretical framework that emphasizes the role of control and monitoring processes. The embedding of the feeling of knowing in a richer framework helped to dissipate the paradoxical nature of the feeling of knowing. The chapter discusses that today there are many capable, active investigators and a wealth of solid empirical findings.
Sixty graduate students instructed to read a 669-word expository passage one, two, or three times completed both free and cued recall measures on three test occasions. Effects of repeated reading of to-be-learned material on recall and retention, as well as persistence of errors over time, were examined. Both quantitative and qualitative differences in recall were found as a function of the number of times text was read. Subjects who read the passage twice prior to first recall remembered significantly more text information than subjects reading the passage one or three times. Subjects who read the passage three times exhibited disproportionate recall of details versus main ideas. Persistence of initial encoding errors was remarkably stable for all groups despite the fact that all subjects reread the passage after the first recall. As expected, errors on the first cued recall were more likely to be repeated on a delayed recall, once they had been repeated on an immediate retest. /// [French] Soixante étudiants gradués à qui on avait demandé de lire un passage narratif de 669 mots une fois, deux fois ou trois fois ont complété des mesures de rappel libre et dirigé selon trois possibilités de test. On a examiné les effets de lecture répétée du matériel à apprendre sur le rappel et le maintien aussi bien que la persistence des erreurs au cours du temps. On a trouvé des différences quantitative et qualitative en rappel comme fonction du nombre de fois que le texte était lu. Les sujets qui avaient lu le passage deux fois avant le premier rappel se sont souvenus de beaucoup plus d'information de texte que les sujets qui avaient lu le passage une ou trois fois. Les sujets qui avaient lu le passage trois fois ont montré un rappel disproportionné d'idées principales par rapport aux détails. La persistance d'erreurs de codification initiales était remarquablement stable pour tous les groupes en dépit du fait que tous les sujets ont relu le passage aprés le premier rappel. Comme prévu, les erreurs du premier rappel dirigé étaient plus susceptibles d'être répétées en rappel différé, une fois qu'elles avaient été répétées sur un test réadministré immédiatement. /// [Spanish] Se aplicaron tres tests de recuerdo libre y asociado a 60 estudiantes de post-grado despues de que estos leyeron un pasaje expositorio de 669 palabras una, dos, o tres veces. Los resultados fueron examinados para observar los efectos que la tarea de lectura repetida de un material por aprenderse tuvo sobre el recuerdo y la retención, así como sobre la persistencia de errores en el tiempo. Se encontraron diferencias cuantitativas y cualitativas en la capacidad de recuerdo de los sujetos como una función del número de veces que el texto fue leído. Los sujetos que leyeron el pasaje dos veces previo a la primera prueba recordaron significativamente más información sobre el texto que aquellos sujetos que leyeron el pasaje una o tres veces. Los sujetos que leyeron el pasaje tres veces mostraron un recuerdo desproporcionado de detalles comparado con su recuerdo de ideas principales. La persistencia de errores de codificación incurridos al principio fue marcadamente estable para todos los grupos sin importar el hecho de que todos los sujetos releyeron el pasaje después de la primera prueba. Como se esperaba, los errores en la primera prueba de recuerdo asociado aparecieron con mayor probabilidad en la tarea de recuerdo retardado, una vez que se habían repetido en un re-test inmediato. /// [German] Sechzig hochschulstudenten, die gebeten wurden, ein-, zwei-, oder dreimal einen 669 Wörter langen Sachtext zu lesen, erfüllten sowohl die freie wie auch direkte Abrufmessungen in drei durchgeführten Tests. Untersucht wurde die Wirkung wiederholten Lesens von zu lernendem Material auf sowohl Abrufungs- und Erinnerungskapazität als auch Fehlerkonsistenz über einen Zeitraum. Sowohl quantitative wie qualitative Unterschiede in der Abrufung erwiesen sich als Funktionen der Anzahl der Lesungen. Testpersonen, die den Text zweimal von der ersten Gedächtnisprüfung lasen, erinnerten sich an bedeutend mehr Informationen aus dem Text als Testpersonen, die den Text einmal oder dreimal lasen. Testpersonen, die den Text dreimal gelesen hatten, wiesen unverhältnismäßig gute Erinnerung von Einzelheiten gegenüber der Hauptgedanken auf. Die Hartnäckigkeit der beim ersten Lesen gemachten Fehler war erstaunlich gleichbleibend für alle Gruppen, trotz der Tatsache, daß alle Testpersonen den Text nach der ersten Gedächtnisprobe nochmal lasen. Wie zu erwarten, wurden Fehler, die bei der ersten direktiven Gedächtnisprüfung gemacht wurden, mit großer Wahrscheinlichkeit bei späterer Erinnerung wiederholt, nachdem sie sich einmal bei sofortiger nochmaliger Testdurchführung wiederholt hatten.