ArticlePDF Available

Abstract and Figures

Previous survey research has documented students' use of self-regulated study strategies, with a particular interest in self-testing. These surveys indicate that students frequently use flashcards to self-test and that self-testing is primarily used as a way to monitor learning. Whereas previous surveys provide information about whether and why students self-test, they provide minimal information about how and when students choose to self-test. Accordingly, the primary purpose of the current survey was to explore how and when students engage in self-testing. We surveyed 374 undergraduates about the amount of practice and the timing of practice, two factors that strongly affect the efficacy of self-testing. Results indicate that students understand the benefits of practising to higher criterion levels (amount of practice) but do not typically implement or understand the benefits of practising with longer lags (timing of practice). We discuss practical implications for supporting more successful student learning.
Content may be subject to copyright.
How and when do students use flashcards?
Kathryn T. Wissman
1
, Katherine A. Rawson
1
, and Mary A. Pyc
2
1
Department of Psychology, Kent State University, Kent, OH, USA
2
Department of Psychology, Washington University in St. Louis, St. Louis, MO, USA
Previous survey research has documented students’ use of self-regulated study strategies, with a
particular interest in self-testing. These surveys indicate that students frequently use flashcards to self-
test and that self-testing is primarily used as a way to monitor learning. Whereas previous surveys
provide information about whether and why students self-test, they provide minimal information about
how and when students choose to self-test. Accordingly, the primary purpose of the current survey was
to explore how and when students engage in self-testing. We surveyed 374 undergraduates about the
amount of practice and the timing of practice, two factors that strongly affect the efficacy of self-
testing. Results indicate that students understand the benefits of practising to higher criterion levels
(amount of practice) but do not typically implement or understand the benefits of practising with
longer lags (timing of practice). We discuss practical implications for supporting more successful
student learning.
Keywords: Self-testing; Criterion learning; Lag effects; Survey.
A wealth of previous research has shown that
testing is beneficial for learning and memory (for
recent reviews, see Rawson & Dunlosky, 2011;
Roediger & Butler, 2011). However, the effec-
tiveness of self-testing depends on several factors,
including the amount and timing of practice.
Although the majority of research on testing
effects has been conducted in laboratory settings
in which practice tests are under experimental
control, in real-world contexts decisions of how
and when to self-test are left up to students. Thus,
the actual benefits of self-testing in most learning
environments will depend on the extent to which
students’ choices reflect those conditions that
facilitate learning. Given the sizeable amount of
laboratory research exploring conditions under
which self-testing is most effective, it is perhaps
surprising that little is known about how students
implement self-testing on their own. Accordingly
the main focus of the current survey was to
explore how and when students self-test, with a
particular interest in students’ self-reported use of
flashcards (given that students often report using
flashcards during self-regulated learning).
HOW SHOULD STUDENTS IMPLEMENT
SELF-TESTING?
Two critical factors that affect the efficacy of self-
testing are the amount of practice and the timing
of practice. Concerning the amount of practice,
increasing the number of times an item is
correctly recalled during practice benefits later
retention (e.g., Karpicke, 2009; Karpicke &
Roediger, 2008; Pyc & Rawson 2009; Vaughn &
Address correspondence to: Kathryn Wissman, Kent State University, Department of Psychology, P.O. Box 5190, Kent, OH
44242-0001, USA. E-mail: kwissman@kent.edu
The research reported here was supported by supported by a Collaborative Award from the James S. McDonnell Foundation
21st Century Science Initiative in Bridging Brain, Mind and Behavior. Thanks to Mike Appleman, Nicole Gonzalez, Jeremy Meduri,
Caitlin Metelko, Dan Molnar, Rochelle O’Neil, Katie Senko, and Sara Smith for assistance with data collection and data scoring.
MEMORY, 2012, 20 (6), 568579
#
2012 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
http://www.psypress.com/memory http://dx.doi.org/10.1080/09658211.2012.687052
Rawson, 2011). For example, Vaughn and Rawson
(2011) presented learners with Lithuanian
English word pairs for testrestudy practice until
items were correctly recalled one to five times.
Correctly recalling an item more than once during
encoding significantly improved final cued recall
two days later (e.g., 31% versus 71% for items
correctly recalled once versus four to five times).
In addition to effects of the amount of practice
within a given learning session, increasing the
number of learning sessions benefits long-term
retention. For example, Rawson and Dunlosky
(2011) presented students with key concept defi-
nitions for testrestudy practice until items were
correctly recalled during initial learning, followed
by anywhere from one to five relearning sessions
on subsequent days. Final cued recall 1 month
later improved substantially as the number of
relearning sessions increased (e.g., 35% versus
62% after one versus four to five relearning
sessions).
Concerning the timing of practice, self-testing
is most effective when using longer versus shorter
lags between practice trials for a given item, both
within and between sessions (e.g., Bahrick & Hall,
2005; Cepeda, Vul, Rohrer, Wixted, & Pashler,
2008; Karpicke & Bauernschmidt, 2011; Kornell,
2009; Pashler, Zarow, & Triplett, 2003; Pyc &
Dunlosky, 2010; Pyc & Rawson, 2007, 2009, 2012).
Pyc and Rawson (2009) presented learners with
SwahiliEnglish word pairs for testrestudy prac-
tice trials that were separated by either 6 or 34
other items. Items practised with a longer versus
shorter lag were more likely to be recalled on a
final test 25 minutes later (76% versus 55%) or
one week later (30% versus 5%). In sum, self-
testing is most effective when implemented with
higher criterion levels within and across sessions
and longer lags between practice trials.
HOW DO STUDENTS IMPLEMENT
SELF-TESTING?
In contrast to the sizeable amount of laboratory
research in which practice tests are under experi-
menter control, minimal research has explored
how students implement self-testing on their own.
A few studies have used experimenter-controlled
environments to examine students decisions
about the amount or timing of self-testing
(Karpicke, 2009; Kornell & Bjork, 2008a; Kornell
& Metcalfe, 2006). Kornell and Bjork (2008a,
Experiment 3) gave learners 10 minutes to learn
20 SwahiliEnglish word pairs (in anticipation of
a final cued recall test after a 5-minute delay).
After initial study, items were presented for test
restudy trials. After each trial learners were given
the option to remove a word pair or keep it in the
list for further practice. Learners dropped 63% of
word pairs after one correct recall rather than
practising to a higher criterion. These results
appear to suggest that students do not understand
the benefits of higher criterion levels. However, a
time limit of 10 minutes might have compelled
students to drop items after only one correct
recall to ensure enough time was left to practise
unlearned items. More generally, all previous
experimental studies have only given students
control over limited aspects of self-testing in
which much of the implementation of testing is
still under experimenter control.
Survey methods provide a complementary
approach to investigating students use of self-
testing (Carrier, 2003; Hartwig & Dunlosky, 2012;
Karpicke, Butler, & Roediger, 2009; Kornell &
Bjork, 2007; Kornell & Son, 2009; McCabe, 2010).
Surveys avoid the disadvantage of limited control
in experimental studies by providing students
with opportunities to report decisions used in
real-world contexts. Several recent survey studies
have established that students use self-testing and
frequently report using flashcards in particular.
Hartwig and Dunlosky (2012) had students report
which study strategies they use on a regular basis,
and 62% of learners reported using flashcards.
Likewise Karpicke et al. (2009) had students
freely list strategies used during study; 40% of
learners reported using flashcards. Prior surveys
also provide information about why students self-
test. Kornell and Son (2009) asked learners why
they choose to self-test during study; 66% of
learners reported self-testing to determine how
well they knew the information, whereas only
20% said because they learn more than with
restudy. Similarly, in surveys by Hartwig and
Dunlosky (2012) and Kornell and Bjork (2007)
54% and 68% of learners reported self-testing to
figure out how well they had learned information,
and only 27% and 18% said because they learned
more that way than rereading. These findings
suggest that students primarily self-test to moni-
tor how well target information has been learned
rather than as a learning strategy.
Although prior survey research is informative
about whether and why students choose to
self-test, prior survey studies provide minimal
HOW AND WHEN STUDENTS USE FLASHCARDS 569
information about how and when students engage
in self-testing. Given that the efficacy of self-
testing is highly dependent on how and when it is
used, the present survey focuses on the details of
these factors in students flashcard use.
METHOD
Surveys were administered via computer as part
of six larger studies. Across all six studies, 374
undergraduates from Kent State University com-
pleted the survey in return for course credit. Each
participant was given 20 minutes to complete the
survey. Due to the time limit some participants
did not respond to every question. Furthermore
the particular set and sequence of questions
varied somewhat across studies. Accordingly the
number of respondents varies across questions;
sample size for each question is indicated in the
tables below.
For the majority of the survey questions,
response format was open-ended. The few ques-
tions involving an alternate response format are
denoted in the tables below. For scoring purposes,
all open-ended responses were coded by one of
two raters. Data were initially coded by frequency
of response exemplars. The tally for each re-
sponse exemplar included verbatim or close
paraphrases of a particular idea. For example,
Question 1A asked learners why they use flash-
cards to study for classes; responses of ‘‘easy for
me to learn’’, ‘‘learning is easier this way’’, and
‘‘easiest way to learn’’ were all coded under ‘‘easy
to learn’’. Two authors then reviewed the list of
response exemplars and decided which exemplars
could be collapsed into response categories.
Response categories were developed for exem-
plars that indicated a similar kind of information
(e.g., for Question 2C, we collapsed responses of
‘‘quickly recalled’’ and ‘‘easily recalled’’ into one
response category) or to combine infrequent
responses (e.g., for Question 2G, we created the
category ‘‘6 times’’). Overall, response cate-
gories were used infrequently; all responses pre-
sented in the tables are exemplars unless
otherwise denoted as a response category.
RESULTS AND DISCUSSION
Before reporting the primary outcomes of interest
concerning how and when students self-test, we
briefly discuss relevant background information
about respondents and replications of findings
from prior survey studies.
Background information
Appendix A of the online supplement reports
background information about the survey respon-
dents. Most students were freshmen, and a
majority of the sample maintained above a 3.00
grade point average. A significant number of
students reported that teachers and/or professors
taught them study strategies (A13). Interest-
ingly, although a large majority of students
reported having received instruction from tea-
chers about how to study, students reported that
only 23% of instructors recommended flashcards,
and only 25% recommended completing practice
problems or quizzes.
Whether and why students use
flashcards
Replicating prior survey research, Table 1 shows
that a large number of students report using
flashcards to study (Question 1A). In contrast to
previous findings, our results differ somewhat in
regard to reasons students gave for why they use
flashcards. Whereas the most common reason
students reported self-testing in prior studies
was to evaluate how well target information has
been learned, the most common reason reported
in the current study was to help memorise
information (although see Question 2J for some
evidence that students revisit dropped flashcards
for purposes of checking memory). These diver-
gent outcomes may be due to methodological
differences between studies. For example,
whereas we asked students why they use flash-
cards in particular, prior surveys asked students
why they choose to quiz themselves more gen-
erally (which might or might not involve the use
of flashcards). Additionally, the three prior survey
studies all used a multiple choice format in which
possible reasons were provided but that only
permitted selection of one response option. In
contrast we used an open-ended question. Given
the open-ended nature of the responses it is
unclear whether students meant the act of testing
in and of itself helps memorisation or if testing
helps determine what information still needs to be
memorised. Further clarification on this distinc-
tion awaits future research.
570 WISSMAN, RAWSON, PYC
Extending beyond prior research, Table 1
provides information about what kinds of materi-
als students use with flashcards*an overwhelm-
ing 80% of students report using flashcards to
learn vocabulary (Question 1B)*and also shows
that a majority of students (53%) use flashcards in
more than one class (Question 1C).
How students use flashcards: Amount
of practice
Learning to criterion. Table 2 includes questions
about how students use flashcards, focusing
TABLE 1
Whether and why students use flashcards
1A. Do you use flashcards to study for your classes? (n247)
Yes 67.6%
No 32.4
Why or why not? (n 154)*
Why (n 114) Why Not
(n 40)
Helps to memorise 60.5%
Easy way to learn 22.8
Lots of vocabulary 14.9
Convenient to carry 8.8
Isolates information 6.1
Helps organisation 2.6
Breaks up information 1.7
Forces rereading 1.7
Helps/forces rewriting 0.9
Not helpful 60.0%
Not useful 35.0
Didnt need to study 5.0
1B. Explain what type of information you put on them
(n216)*
Vocabulary 82.9%
Factual information from notes (people/places/
dates)
29.2
Key concepts 28.2
Formulas 7.4
Difficult information 3.7
Information from study guide 3.7
1C. In reference to the classes you took last semester, how
often did you use flashcards to study? (n 264)
0 classes 35.6%
1 class 11.4
2 classes 12.1
3 classes 22.3
4 classes 18.6
*For these questions the sum of response percentages
exceeds 100% because participants could report more than
one response.
For this question participants were given five corresponding
response buttons.
TABLE 2
How student use flashcards: Amount of practice
2A. Imagine that you are studying with a stack of flashcards.
How do you decide when you have studied a given flashcard
enough? (n327)*
I recall it correctly 39.4%
I recall it correctly more than once 25.7
I dont have to look at the other side 17.4
I dont have to think about it 15.6
I recall it quickly 12.5
I understand the information 8.6
Mix-up and still recall 5.5
Other 3.1
2B. Imagine you are studying a stack of flashcards. Which of
the following would influence how many times you studied an
item? (n272)*
How easily you can remember the item 80.0%
How many times you can correctly recall
item
68.7
Importance of the particular item 58.9
How long you planned to study that day 40.1
Other 4.0
2C. Do you ever remove cards from the stack of flashcards
when you are studying? (n 327)
Yes 73.7%
No 26.3
If yes, please explain when and why you
would drop out cards (n296)*
Need to focus on unknown 40.0%
Remembered multiple times 33.8
Dont need to study what is known 29.8
Quickly/easily recalled $ 12.0
Ones I need to study more 8.4
Helps divide concepts 3.1
Helps organise 2.7
Other 1.8
If no, please explain why you would not
drop out cards (n72)*
Need to know all the material 55.6%
Will remember better more times studied 36.1
Messes up the order 8.3
Dont see the point 8.3
2D. When you take a card out, how often is it because you
tested yourself and were able to remember it correctly?
(n133)
0.019.9% 8.3%
20.039.9% 3.8
40.059.9% 13.5
60.079.9% 21.8
80.0100.0% 52.6
2E. When you take a card out, how often is it because its too
hard and you dont feel that you can learn it? (n135)
0.019.9% 65.0%
20.039.9% 16.3
40.059.9% 5.9
60.079.9% 8.9
80.0100.0% 3.0
2F. When you take a card out, how often is it because its too
easy and you dont feel you need to practice it? (n135)
0.019.9% 20.7%
HOW AND WHEN STUDENTS USE FLASHCARDS 571
specifically on students self-reported use and
beliefs about amount of practice. Question 2A
demonstrates that a large majority of students
(65%) report practising until items are correctly
recalled (collapsing across ‘‘I recall it correctly’’
and ‘‘I recall it correctly more than once’’; these
two response exemplars were treated as mutually
exclusive for coding purposes). Although the
number of correct recalls was not always speci-
fied, this finding suggests that many students
learn to a criterion of at least one correct recall
when using flashcards. With that said, a discon-
certing number of students report making deci-
sions without checking to make sure they are
correct (‘‘I dont have to look at the other side’’,
17%) or based on cue familiarity (‘‘I dont have to
think about it’’, 16%) and/or fluency of retrieval
(‘‘I recall it quickly’’, 13%). Although fluency
of retrieval can accurately predict subsequent
TABLE 2 (Continued )
20.039.9% 11.1
40.059.9% 8.1
60.079.9% 25.2
80.0100.0% 34.8
2G. When using flashcards to self-test your memory, how
many times do you think you should be able to correctly recall
an item during flashcard practice so that you will be able to
remember it later for an exam? (n243)
1 time 0.8%
2 times 4.5
3 times 22.2
4 times 8.2
5 times 22.0
6 times $ 29.6
Non-numeric response $ 12.7
2H. Imagine that you are self-testing using flashcards and
you correctly recall an item once. How much more likely
would you be to remember that item later if you continued
practising until you correctly recall that item again a second
time? (n143)
0.019.9% 2.1%
20.039.9% 7.7
40.059.9% 9.1
60.079.9% 32.3
80.0100.0% 49.0
2I. Do you think there is a point at which correctly recalling an
item more times during practice would no longer benefit your
ability to remember that item later? If so, what do you think
that point is? (n 143)
No, there is not a point 44.8%
Yes, there is a point 55.2
If so, what is the point? (n 78)*
I recall it X number of times 52.6%
I know answer without thinking about it 30.8
I remember the item the next day 12.8
I start to become unfocused 11.5
Other 2.6
2J. If you were to drop out cards from your stack of flashcards
when you are studying, once an item is dropped from practice,
does it ever get put back in the stack of flashcards within that
same session? (n 315)
Yes 84.1%
No 15.9
Why or why not?*
Why (n257) Why Not
(n37)
To check memory 41.0%
Always useful to restudy 37.7
Need to keep all to learn
all
14.4
Helps to understand
completely
7.0
Forget otherwise 7.0
I know it 64.9%
Use it in different session 37.9
Need to focus on unknown 13.5
Confuses if re-added 8.1
TABLE 2 (Continued )
2K. Suppose you were using flashcards to prepare for an
upcoming exam,
and you wanted to get an A
¯
on the exam. On
how many different days would you study with your stack of
flashcards? (n126)
1 day 4.0%
2 days 13.5
3 days 15.1
4 days 13.5
5 days 15.1
6 days 4.8
7 days 14.3
810 days $ 12.7
Other/non-numeric $ 7.2
2L. When using flashcards to self-test, how do you decide if
you have correctly recalled an item? (n274)*
Self check by looking at external
information
48.9%
Self check in mind 27.4
Continuously recall correctly 16.4
Answer correctly without hesitation 12.0
Partner check 3.6
Write down correctly 2.5
*For these questions the sum of response percentages
exceeds 100% because participants could report more than
one response.
For this question participants were provided with the list of
options.
For Questions 3D3F participants were given were given a
sliding scale with endpoints marked as ‘‘0% of the time’’ and
‘‘100% of the time’’. For Question 3H participants were given
a sliding scale with endpoints marked as ‘‘100% more likely’’
to ‘‘0% more likely’’.
$Response category: see text for details.
572 WISSMAN, RAWSON, PYC
performance in some situations, fluency can be
misleading in other cases (e.g., if the fluently
retrieved information is incorrect, or if fluency is
due to retrieval from working memory rather
than long-term memory). Relying only on the
fluency with which information comes to mind
without evaluating the correctness of the fluently
retrieved information may be particularly proble-
matic if it leads students to incorrectly believe
they know more than they do, which is particu-
larly likely for more complex material (e.g.,
Dunlosky, Hartwig, Rawson, & Lipko, 2011;
Rawson, ONeil, & Dunlosky, 2011).
Nonetheless most students seem to understand
theimportanceofcorrectlyrecallingtargetinfor-
mation during study, which is one key component
of maximising the benefits of self-testing. Conver-
ging evidence for this conclusion comes from
Question 2B, with a majority of students (69%)
reporting that they practice to criterion when
studying with flashcards. Additionally, Question
2D shows that students commonly report dropping
flashcards in their stack because they reached
criterion. Along the same lines Question 2E
indicates that students are not likely to drop cards
prior to reaching criterion. Students also reported
removing items from practice when the item was
easy (Question 2F), but these results are somewhat
harder to interpret given that students might have
interpreted ‘‘too easy’’ to mean they were able to
recall target material correctly, quickly, or based on
some other source (e.g., cue familiarity).
Higher criterion is better. Results thus far
indicate that students typically choose to reach
criterion before dropping flashcards from study.
The next question of interest concerns the extent to
which students practise beyond a single correct
recall. In Question 2A, 26% of students sponta-
neously reported that they practise until informa-
tion is correctly recalled more than once. Question
2C indicates that of the 74% of students who report
dropping flashcards from their stack during study,
34% of these students explicitly stated that they
practised recalling information multiple times be-
fore dropping it. However, these outcomes may
underestimate the prevalence of practising to high-
er criterion levels, in that some students might
simply not have thought to specify a particular
criterion level. Indeed, when asked directly, almost
every single student indicated belief that practising
beyond one correct recall will help later memory of
target material for an exam (Question 2G). With
that said, by asking ‘‘how many times’’, the phras-
ing of Question 2G might have biased some
students to report more than one correct recall
versus only one (although students preferences for
more versus fewer correct recalls are harder to
explain as due to biasing from the prompt).
Question 2H shows that a majority of students
expect to benefit substantially from correctly
recalling information for a second time. Question
2I suggests that some students do not anticipate
diminishing returns from continuing to correctly
recall target material (although a majority of
students do correctly believe that once information
has been recalled a certain number of times, there is
no need to continue to practise it). Finally, Ques-
tion 2J indicates almost all students (84%) report
putting dropped flashcards back into their stack
during a practice session. Moreover, when students
report not putting a flashcard back into their stack,
it is interesting to note that a common reason is
because they intend to practise it at a later time
(38%). Lastly, Question 2K indicates that students
believe it is important to restudy flashcards on
more than one day in order to obtain an A on an
exam, suggesting that students understand the
benefit of engaging in multiple learning sessions.
Concerning criterion learning and amount of
practice, one related aspect is important to
consider. Students report learning to criterion
and that a higher criterion is beneficial, but how
do students know when they have correctly
recalled information? Question 2L shows that
fewer than half of students report checking
answers against external information (e.g., turn-
ing the flashcard over or checking against notes).
Importantly, even when students check their
answers against external information, they often
still overestimate the correctness of their own
answer (e.g., Dunlosky et al., 2011; Lipko
et al., 2009; Rawson et al., 2011). Even more
disconcerting is the fact that 27% of students
reported checking answers ‘‘in their mind’’, given
that research has shown that evaluating the
correctness of a response without comparison to
external information produces even higher levels
of overconfidence (Dunlosky & Rawson, in press;
Lipko et al., 2009). In either case, overconfidence
can produce detrimental effects on learning
because students do not effectively regulate
practice (Dunlosky & Rawson, in press; Rawson
et al., 2011). For present purposes this outcome
suggests that students may not actually achieve
their intended criterion level if they prematurely
terminate practice due to overconfidence in the
accuracy of their responses.
HOW AND WHEN STUDENTS USE FLASHCARDS 573
When students use flashcards: Timing
of practice
Table 3 includes questions about when students
use flashcards with specific respect to the timing
of practice. Most strikingly, several outcomes
indicate that students do not understand the
benefits of longer lags during study. For example,
Question 3A indicates that a majority of students
prefer to use a smaller versus larger stack of
flashcards when studying. Of those students who
reported a preference for using a smaller stack, a
TABLE 3
How students use flashcards: Timing of practice
3A. Imagine that you have a total of 60 flashcards for a class. You need to know all 60 items for the next exam. When studying the
flashcards, how many cards would you practise at a time? In other words, would you practise all 60 in one big stack, or would you
divide them up into smaller stacks and then study one stack at a time? Below indicate how many flashcards you would practise at a
time (n279)
60 28.7%
30 9.0
20 21.0
15 14.0
10 18.3
Other 9.0
Why? (n 253)*
1015 (n87) 2030 (n72) 60 (n 74)
Need to know them all 4.6% 11.1% 63.5%
Easier to remember a smaller stack 67.8 44.4 2.7
Less overwhelming 34.5 40.3
Divides into ones I know/dont know 15.3 20.3
Influences how information is encoded/learned 4.6 4.2 8.1
Needs to be somewhat difficult 1.1 6.9 6.8
Confusing to divide 1.1 1.4 2.7
Split into categories/chapters $ 5.7 2.8 1.3
See card more times 4.6 4.2
3B. Do you think having a larger or small stack of flashcards is better for studying? (n 263)*
Small 72.2%
Medium 1.9
Large 16.3
Doesnt matter 6.8
Depends 4.2
Why? (n 229)* Small (n176) Medium/Large (n42)
Less to remember 53.7% 9.5%
Learn quicker 23.9 4.8
Narrows focus 19.1 11.9
See card more times 8.5 2.4
Gets confusing 8.0
Helps to feel accomplished 6.4 4.8
Brain only holds so much 6.4
Need to know all the information 3.2 64.3
Time between cards better 1.1 4.8
3C. If you were going to study a stack of flashcards, what do you think would work best and why? Study them one at a time
repeatedly (for example, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4). Study them mixed together (for example, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4).
(n140)*
Mixed 80.9
Repeated 17.1
Other 3.9
*For these questions the sum of response percentages exceeds 100% because participants could report more than one response.
For this question participants were given six corresponding response buttons.
$Response category: see text for details.
574 WISSMAN, RAWSON, PYC
substantial percentage (68%) said the reason was
because it is easier. In addition, even when
students said they would use a larger stack,
none of them reported doing so because it would
improve learning. Question 3B provides conver-
ging evidence showing that when students are
specifically asked whether using a smaller versus
larger stack of flashcards is better for studying, an
overwhelming 72% believe smaller is superior,
whereas research has shown that retention is
better after practising with a larger versus smaller
stack (e.g., Kornell, 2009). Furthermore, not a
single student indicated that the reason for their
preference on stack size was because it is better
for retention. However, although students do not
appear to understand the advantage of long
versus short spacing, Question 3C shows that at
least most students (81%) believe that spaced
study is better than massed study.
Although students understanding of lag effects
was of primary interest, Appendix A of the online
supplement provides results for questions that are
relevant to another aspect of timing. Of interest, a
recently emerging issue in the testing effect
literature focuses on interleaving versus blocking
of practice for items from different categories
(e.g., Kornell & Bjork, 2008b; Taylor & Rohrer,
2010). Despite frequent reports of using flash-
cards for more than one class (Question 1C and
A14), Question A15 indicates that almost no
one said they would mix flashcards of different
topics when studying.
Putting the pieces together: Calendar
question
Tables 2 and 3 report outcomes from fine-grain
questions that prompted students to report either
on amount of practice or on timing of practice.
Table 4 reports outcomes from a more global
survey question that was designed to explore how
students would make decisions about both the
amount and timing of practice and how flashcard
use might be combined with the use of other
strategies. For this question students were shown
a visual calendar display for the month of
February along with the informational prompt
and questions shown in Table 4.
Not surprisingly almost all students said that
they would study the night before an exam (93%)
and a fair number reported studying the morning of
an exam (38%). Although prior research has shown
that massed study can enhance performance on
immediate tests, cramming before an exam is not
sufficient for longer-term retention (e.g., Balota,
Duchek, & Paullin, 1989; Karpicke & Roediger,
2007; Rawson & Kintsch, 2005). Responses to
Question 4A are consistent with outcomes de-
scribed above, in that almost all students reported
that they would study on more than one day.
However, results also extend the finding that
students do not understand lag effects. For students
who reported they would study in two or more
sessions, the majority reported that they would only
separate their study sessions by about one day.
Responses to Question 4B indicate that a
majority of students (65%) said they would use
two or three different strategies when studying for
the exam. However, students also reported that
they would rely more heavily on restudying than
on self-testing. Restudying notes had the highest
reported percentage for strategies used the morn-
ing of an exam, the night before an exam, or any
other day, and 52% of students said they would
restudy notes during every study session. In
contrast only 34% said they would use flashcards
the morning of an exam and 38% said they would
use flashcards the night before the exam. Like-
wise only 34% said they would use flashcards on
other days, and only a minority (24%) said they
would use flashcards on every other day they
studied. Although the percentage of students who
said they would use flashcards is lower than in
other survey questions described above (e.g., 68%
in Question 1A), note that Question 4B asked
students specifically about preparing for an exam
in General Psychology. To revisit Question 1B, a
majority of students reported using flashcards for
vocabulary (83%) versus information from notes
or key concepts (29% and 28%, respectively).
Given that General Psychology typically involves
learning key concepts and related material rather
than vocabulary (as in foreign language classes),
the lower percentage of students who reported
they would use flashcards in Question 4B is
consistent with the percentage who reported
using flashcards for key concepts in Question 1B.
DISCUSSION
The current survey extends beyond previous survey
research by exploring how and when students self-
test, with an emphasis on flashcard use. Of primary
interest, two key patterns emerged: First, most
students report engaging in criterion learning and
HOW AND WHEN STUDENTS USE FLASHCARDS 575
understand the benefits of practising to higher
criterion levels. Second, students do not appear to
implement or understand the benefits of longer lags
when self-testing with flashcards. Concerning learn-
ing to criterion, a majority of students choose to
reach at least one correct recall during practice
before dropping a given flashcard (Questions 2A,
2B, and 2D), although it is interesting to note that
students do not always check their answer against
external information (Question 2L). Concerning
criterion level, some students spontaneously re-
ported going beyond one correct recall during
practice (Questions 2A and 2C) and almost all
students believe that going beyond one correct
recall is important (Questions 2G and 2H). In
addition a majority of students report using more
than one practice session (Question 2K). Concern-
ing lag, responses indicate a majority of students opt
tousesmallerversuslargerstacksizeswhenself-
testing (Questions 3A and 3B). Of critical impor-
tance is that not one student (regardless of preferred
stack size) indicated awareness of the relationship
between lag and learning (Questions 3A and 3B).
Overall the results suggest that when students
self-test with flashcards they make advantageous
decisions regarding the amount of practice but detri-
mental decisions regarding the timing of practice.
Note that some questions in the current survey
tapped students beliefs whereas others tapped
students behaviours. We used this multi-faceted
approach because what students believe about
effective study strategies and what they actually
do when studying likely do not completely align
(e.g., due to time pressure). For example, nearly all
students believe that correctly recalling an item
more than once has a substantial benefit on later
retrieval (Question 2H), whereas only 34% of
students spontaneously reported practising to a
TABLE 4
Responses to calendar question
Here is a calendar for the month of February. Imagine that
today is Feb 1st and you have an exam in General Psychology
on Feb 27th. Suppose you wanted to get an A on this exam.
In the field to the right, answer these questions in as much
detail as possible:
4A. Which day or days would you study for the exam (write
exact dates)? (n286)
Participants reporting they would study the
morning of the exam
38.5%
Participants reporting they would study the
night before the exam
93.0
Not counting the morning of the exam or the night before the
exam, number of other days participants reported they would
study:
0 days 6.3%
1 day 13.0
2 days 9.1
3 days 13.3
4 days 8.7
5 days 7.3
6 days 9.8
78 days 10.8
9 days 21.7
For participants who reported studying on two or more
sessions (n273), the mean lag between sessions:
1 day (1.01.5) 66.3%
2 days (1.62.5) 21.2
3 days (2.63.5) 7.0
4 days (3.64.7) 3.3
7 days (6.57.5) 2.2
4B. What exactly would you be doing in each of these study
sessions to prepare for the exam? Next to each date you wrote
down, write what you would be doing during that session to
prepare for the exam (n 269)
Number of strategies participants reported using to study
across sessions:
1 strategy 20.0%
2 strategies 32.6
3 strategies 31.9
4 strategies 11.8
5 strategies 3.3
6 strategies 0.4
For participants who indicated they would study the morning
of the exam (n108), how did they say they would study? For
participants who indicated they would study the night before
the exam (n 247), how did say they would study? For
participants who indicated they would study on at least one
other day (n 252), how did they say they would study?*
Morning Night
before
Other
day(s)
Restudy notes 70.4% 68.8% 65.6%
Restudy textbook 30.6 36.4 40.0
Flashcards 34.3 38.4 34.0
Practice test/
problems
15.7 14.6 10.2
Other 21.3 25.5 21.2
TABLE 4 (Continued )
How many sessions on other day(s) participants reported they
would use flashcards, restudy the textbook, and restudy notes
to prepare for exam (n 252)
Flashcards Restudy
textbook
Restudy
notes
Every session 24.2% 30.6% 51.6%
No session 55.6 48.8 23.0
Some but not all
sessions
20.2 20.6 25.4
*This question only includes participants who provided
responses for both how and when they would study. The sum
of response percentages exceeds 100% because participants
could report more than one response
576 WISSMAN, RAWSON, PYC
higher criterion when studying (Question 2C).
Additionally, if students do not report engaging in
a particular behaviour, questions about beliefs help
to diagnose whether suboptimal behaviour is due
to a metacognitive knowledge deficit or an im-
plementation deficit. For example, 71% of students
report dividing up information into smaller stacks
(Question 3A); the finding that 72% of students
believe using a smaller versus larger stack of
flashcards is better (Question 3B) suggests that
the suboptimal behaviour is primarily due to a
metacognitive knowledge deficit. More generally,
we would advocate for the inclusion of both belief
and behaviour questions in future survey research.
Our findings concerning how and when stu-
dents use flashcards have several important im-
plications for educational practice. Perhaps most
importantly the results showing a general lack of
understanding about how lag affects learning, and
that relatively few students effectively implement
lag either within or between practice sessions,
indicate that students should be educated about
the benefits of lag and how to effectively imple-
ment lag when self-testing. Another unanticipated
but important outcome concerns the extent to
which students reported using flashcards for a
relatively restricted range of materials (in Ques-
tion 1B, 83% of students report using flashcards
for vocabulary whereas only 29% report using
flashcards for key concepts). Thus students would
also likely benefit from education about the
utility of flashcards for learning a wider range of
material. In addition, responses concerning how
students evaluate the correctness of their answers
(Question 2A) suggest that students should also
be educated about the importance of explicitly
checking their answers, particularly when using
flashcards to learn complex material.
Beyond implications for more effective flash-
card use in particular, the current survey results
have practical implications for supporting success-
ful student learning with the use of self-testing
more generally. Although a majority of students
choose to self-test during study, the current results
suggest that students would benefit from more
information or training about how to effectively
implement self-testing. One straightforward ap-
proach would be for instructors to teach students
about the importance of the amount and timing of
self-testing. Unfortunately the current results
suggest that relatively few instructors advise
students to use self-testing as a learning strategy
(Question A13), much less how to self-test
effectively. Given that few instructors recommend
self-testing to students, educators themselves may
be unaware of the robust effects of self-testing
and the factors that influence the efficacy of self-
testing. Ideally teacher training programmes
could educate instructors on the benefits of self-
testing, factors that influence the effectiveness of
self-testing, and ways to incorporate self-testing
methods into curricula. Another approach to
enhance the effective use of self-testing is through
study skills classes or workshops increasingly
offered by universities to support student achieve-
ment. Ideally these programmes would target
first-semester freshmen, who may have never
been taught about the use of self-testing. The
goal would be to educate students on how and
when to self-test in ways that facilitate learning
(e.g., higher criterion levels and longer lags).
More generally, promoting students awareness
of the benefits of self-testing and providing
knowledge of how to effectively self-test in real-
world contexts will support more successful
student learning.
Manuscript received 30 November 2011
Manuscript accepted 12 April 2012
First published online 8 June 2012
REFERENCES
Bahrick, H. P., & Hall, L. K. (2005). The importance of
retrieval failures to long-term retention: A meta-
cognitive explanation of the spacing effect. Journal
of Memory and Language, 52, 566577.
Balota, D. A., Duchek, J. M., & Paullin, R. (1989). Age-
related differences in the impact of spacing, lag, and
retention interval. Psychology and Aging, 4,39.
Carrier, M. L. (2003). College students choices of study
strategies. Perceptual and Motor Skills, 96,5456.
Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., &
Pashler, H. (2008). Spacing effects in learning: A
temporal ridgeline of optimal retention. Psycholo-
gical Science, 19, 10951102.
Dunlosky, J., Hartwig, M. K., Rawson, K. A., &
Lipko, A. R. (2011). Improving college students
evaluation of text learning using idea-unit standards.
Quarterly Journal of Experimental Psychology, 64,
467484.
Dunlosky, J., & Rawson, K. A. (in press). Overconfi-
dence produces underachievement: Inaccurate self
evaluations undermine students’ learning and reten-
tion. Learning and Instruction.
Hartwig, M. K., & Dunlosky, J. D. (2012). Study
strategies of college students: Are self-testing and
scheduling related to achievement? Psychonomic
Bulletin & Review, 19, 126134.
Karpicke, J. D. (2009). Metacognitive control and
strategy selection: Deciding to practice retrieval
HOW AND WHEN STUDENTS USE FLASHCARDS 577
during learning. Journal of Experimental Psychol-
ogy: General, 138, 469486.
Karpicke, J. D., & Bauernschmidt, A. (2011). Spaced
retrieval: Absolute spacing enhances learning re-
gardless of relative spacing. Journal of Experimental
Psychology: Learning, Memory,and Cognition, 37,
12501257.
Karpicke, J. D., Butler, A. C., & Roediger, H. L. III.
(2009). Metacognitive strategies in student learning:
Do students practice retrieval when they study on
their own? Memory, 17, 471479.
Karpicke, J. D., & Roediger, H. L. III. (2007). Expand-
ing retrieval practice promotes short-term retention,
but equally spaced retrieval enhances long-term
retention. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 33, 704719.
Karpicke, J. D., & Roediger, H. L. III. (2008). The
critical importance of retrieval for learning. Science,
319, 966968.
Kornell, N. (2009). Optimising learning using ash-
cards: Spacing is more effective than cramming.
Applied Cognitive Psychology, 23, 12971317.
Kornell, N., & Bjork, R. A. (2007). The promise and
perils of self-regulated study. Psychonomic Bulletin
& Review, 14, 219224.
Kornell, N., & Bjork, R. A. (2008a). Optimising self-
regulated study: The benets and costs of
dropping ashcards. Memory, 16, 125136.
Kornell, N., & Bjork, R. A. (2008b). Learning concepts
and categories: Is spacing the ‘‘enemy of induc-
tion’’? Psychological Science, 19, 585592.
Kornell, N., & Metcalfe, J. (2006). Study efcacy and
the region of proximal learning framework. Journal
of Experimental Psychology: Learning,Memory and
Cognition, 32, 609622.
Kornell, N., & Son, L. K. (2009). Learners choices and
beliefs about self-testing. Memory, 17, 493501.
Lipko, A. R., Dunlosky, J., Hartwig, M. K.,
Rawson, K. A., Swan, K., & Cook, D. (2009). Using
standards to improve middle-school students accu-
racy at evaluating the quality of their recall. Journal
of Experimental Psychology: Applied, 15, 307318.
McCabe, J. (2010). Metacognitive awareness of learn-
ing strategies in undergraduates. Memory & Cogni-
tion, 39, 462476.
Pashler, H., Zarow, G., & Triplett, B. (2003). Is
temporal spacing of tests helpful when it inates
error rates? Journal of Experimental Psychology:
Learning, Memory and Cognition, 29, 10511057.
Pyc, M. A., & Dunlosky, D. (2010). Toward an under-
standing of students allocation of study time: Why
do they decide to mass or space their practice?
Memory & Cognition, 38, 431440.
Pyc, M. A., & Rawson, K. A. (2007). Examining the
efciency of schedules of distributed retrieval prac-
tice. Memory and Cognition, 35, 19171927.
Pyc, M. A., & Rawson, K. A. (2009). Testing the
retrieval effort hypothesis: Does greater difculty
correctly recalling information lead to higher levels
of memory? Journal of Memory and Language, 60,
437444.
Pyc, M. A., & Rawson, K. A. (2012). Why is test
restudy practice benecial for memory? An evalua-
tion of the mediator shift hypothesis. Journal of
Experimental Psychology: Learning, Memory, and
Cognition, 38, 737746.
Rawson, K. A., & Dunlosky, J. (2011). Optimising
schedules of retrieval practice for durable and
efcient learning: How much is enough? Journal of
Experimental Psychology: General, 140, 283302.
Rawson, K. A., & Kintsch, W. (2005). Rereading effects
depend upon time of test. Journal of Educational
Psychology, 97,7080.
Rawson, K. A., ONeil, R. L., & Dunlosky, J. (2011).
Accurate monitoring leads to effective control and
greater learning of patient education materials.
Journal of Experimental Psychology: Applied, 17,
288302.
Roediger, H. L. III., & Butler, A. C. (2011). The critical
role of retrieval practice in long-term retention.
Trends in Cognitive Science, 15,2027.
Taylor, K., & Rohrer, D. (2010). The effects of
interleaved practice. Applied Cognitive Psychology,
24, 837848.
Vaughn, K. E., & Rawson, K. A. (2011). Diagnosing
criterion-level effects on memory: What aspects of
memory are enhanced by repeated retrieval? Psy-
chological Science, 22, 11271131.
APPENDIX A
Background information and outcomes of secondary interest.
A11. What is your college standing, based on number of
credit hours completed? (n372)
Freshman 66.1%
Sophomore 15.9
Junior 8.9
Senior 9.1
A12. What was your approximate GPA in high school?
(n370) What is your approximate GPA in college? (n 308)
High
school
College
Below 2.00 0.8% 2.9%
2.00 2.49 4.1 10.7
2.50 2.99 18.4 25.3
3.00 3.49 35.1 32.5
3.50 4.00 38.4 28.6
4.00 3.2 n/a
A13. Has anyone ever given you advice or taught you how
you should study? (n135)
Yes 66.7%
No 33.3
When was it (what grade were you in)? (n 67)*
Grade school 13.4%
Middle school 11.0
High school 47.8
College 40.3
All grades 6.0
Who was it (for example, a teacher or a friend)? (n 86)*
Teacher/professor 75.6%
Friend 23.3
Parent 19.8
Other 7.0
578 WISSMAN, RAWSON, PYC
TABLE 5 (Continued )
What did they tell you to do when studying? (n 73)*
Overall Teacher/professor
(n40)
Flashcards 37.0% 22.5%
Make outline 27.4 27.5
Read/reread material 27.4 30.0
Practice problems/quizzes 24.7 25.0
Find out best method for self 17.8 17.5
Make up system (rhymes,
colour code)
12.3 12.5
Work in groups 6.8 5.0
Make charts/graphs 4.1 5.0
Other 4.1 5.0
A14. Have you ever been in a situation in which you were
using flashcards to study two topics at the same time (for
example, might have had two exams, in Spanish and
Chemistry)? (n149)
Yes 59.1%
No 40.9
A15. Suppose you had sets of flashcards for two different
classes. Would you mix the cards from the two topics and study
them together or would you keep the cards from the two
topics separate? (n150)
Dont mix 97.8%
Mix 2.2
If you said dont mix, explain why you would study them
separately (n141)*
Confusing 63.1%
Different subjects 53.9
Mixed hurts memorisation 17.0
Separate helps memorisation 6.4
Makes more challenging 3.5
Classes coincide 2.1
*For these questions the sum of response percentages
exceeds 100% because participants could report more than
one response.
APPENDIX A (Continued)
HOW AND WHEN STUDENTS USE FLASHCARDS 579
Copyright of Memory is the property of Psychology Press (UK) and its content may not be copied or emailed to
multiple sites or posted to a listserv without the copyright holder's express written permission. However, users
may print, download, or email articles for individual use.
... students do not always check the accuracy of their responses when using flashcards (Wissman et al., 2012). This pattern is especially problematic when considering that students sometimes drop flashcards even before a single successful retrieval (possibly due to inadequately assessing the correctness of their responses; e.g., Kornell & Bjork, 2008, Experiment 3). ...
Preprint
Full-text available
We investigated the benefits of two ways to use flashcards to perform retrieval practice: aloneversus with a partner. In three experiments, undergraduate students learned word-definition pairsusing flashcards alone (Individual condition) or with another student (Paired condition).Participants then made global judgments of learning (gJOLs; Experiments 1-3), and item-leveljudgments of learning (iJOLs; Experiment 3). Finally, participants took a cued-recall test after a5-min delay (Experiments 1-3) and a 24-hour delay (Experiments 2-3). In Experiment 1, studentsin the Paired condition dropped flashcards less often than in the Individual condition (droppingwas prohibited in Experiments 2-3). In addition, although final test performance tended to besimilar across conditions, inaccurate gJOLs for the immediate test—inflated by ~20% relative toactual immediate test performance—were common in the Individual condition but not in thePaired condition in Experiments 1-2. In Experiment 3, we tested whether this difference inmetacognitive calibration was due to the Paired condition requiring overt retrieval by instructingparticipants in the Individual condition to retrieve out loud. With this change, participants in theIndividual and Paired conditions reported similarly accurate gJOLs and iJOLs. Taken together,these findings suggest that although performing retrieval practice with flashcards alone versuswith a partner yields comparable amounts of learning, doing so with a partner can increasemetacognitive accuracy, a benefit possibly driven by the facilitation of overt retrieval. Overall,these findings have implications for self-regulated learning and effective exam preparation.
... However, it is a strategy that students may ignore, choosing instead to use strategies like rereading that give them an illusion of competence [30]. Rereading is a technique students often report using [31]. Although rereading has some effect on learning, it is not the most efficient or effective strategy [30,32,33] and therefore was used as a contrasting strategy to self-testing in the present study. ...
Article
Full-text available
Study skills are important for success in college. However, students may not be aware of or willing to use effective strategies such as spaced practice and self-testing. This replication-extension study of Susser and McCabe (Susser and McCabe in Instr Sci 41:345–363, 2013) supports their original findings regarding spaced practice and extends the investigation to self-testing. Using the same survey method as the previous researchers, we found that college students showed awareness of the efficacy of spaced practice over massed practice and reported using it in ideal situations. Students reported greater use of spaced studying when a test was difficult, more heavily weighted, or over material of future use. Greater metacognitive self-regulation and elaboration was related to greater reported use of spaced practice. Extending beyond the original study to look self-testing, students were aware of the usefulness of self-testing. Similar to spaced practice, students reported being more likely to use self-testing for more difficult tests and heavily weighted tests. Those with greater metacognitive self-regulation were also more likely to report greater use of self-testing, although greater elaboration was not correlated with reported use of self-testing. Students with a higher GPA were also more likely to report spaced studying and self-testing. Findings suggest that students do know about some effective study strategies. Use of effective strategies may be helpful but students do not report using them in all settings. Educators may be interested in creating circumstances that encourage students to use effective strategies.
... Flashcards, renowned for their dynamic and self-paced nature, emerge as effective tools for memorization, employing active recall and retrieval practice to render learning both efficient and enjoyable [20,53]. In subjects such as gross anatomy, where fast information retention is crucial, flashcards stand out as invaluable study aids, providing effective learning strategies [15]. ...
Conference Paper
Full-text available
Traditional anatomy flashcards, with their recognizable static illustrations on the front side and comprehensive lists of concepts on the back, are a long-standing tool for memorizing and refreshing anatomical concepts. This study repurposes such established tool by introducing two key elements: (i) Augmented Reality (AR) lenses acting as magic mirrors enabling users to view anatomical illustrations mapped onto their own faces, and (ii) a knowledge map layer acting as the card's backside to visually and explicitly illustrate conceptual connections between anatomical reference points. Using Snapchat's Lens Studio, we crafted a deck of interactive facial anatomy flashcards to assess the potential of AR and knowledge maps for retaining and refreshing anatomical concepts. We conducted a user study involving 44 university-level students. Divided into two groups, participants utilized either flashcard lenses with knowledge maps or traditional flashcards to quickly grasp and refresh anatomical concepts. By employing an approach that integrates anatomical quizzes for objective assessment with surveys and interviews for subjective feedback, our results indicate that anatomy flashcard lenses with knowledge maps offer a more engaging educational experience, yielding higher user preferences and satisfaction levels compared to traditional flashcards. While both approaches showed similar effectiveness in quiz scores, anatomy flashcard lenses with knowledge maps were favored for their usability, significantly reducing temporal demand. These findings underscore the engaging and effective nature of anatomy flashcard lenses with knowledge maps, highlighting them as an alternative tool for the quick retention and review of anatomical concepts.
... The finding that participants' confidence in performance and actual performance were more closely aligned when referencing similar tasks given in the future (i.e., relative monitoring accuracy) is noteworthy. Wissman, Rawson, and Pyc (2012) argued that questions about beliefs help to establish whether performing beyond what is expected is due to a deficit in metacognitive knowledge or a deficit in its implementation. Similarly, some studies support the need to establish mechanisms that encourage students to question themselves in relation to their conditional knowledge, which allows them to improve their metacognitive monitoring to determine more effectively what they know and do not know about their learning (e.g., Gutiérrez & Schraw, 2015;Schraw, Kuch, & Gutiérrez, 2013;Gutiérrez et al., 2016). ...
Article
Full-text available
While research on metacognition and cognitive styles is robust for either field alone, few studies have broached the two together. In addition, no studies to date have examined finer-grained objectives related to specific aspects of metacognition such as monitoring skill and its relation to cognitive style. Thus, the present study investigated confidence, performance, and accuracy measures for three types of metacognitive judgments (prediction, concurrent and postdiction) and three different types of metacognitive questions-questions about the task, questions about the self, and questions at different moments (before, during, and after)-and how these are related to cognitive style (field dependent, intermediate, field independent) in a sample of 57 Colombian university students. Results revealed that there were differences in metacognitive monitoring accuracy and bias as a function of cognitive style, and that these findings were similar both between different moments and across metacognitive judgments. Regarding cognitive style, those with an intermediate or field independent cognitive style reported greater monitoring accuracy and less bias than individuals with a field dependent style. Implications for research, theory, and practice are discussed.
... Perhaps most strikingly, only one faculty recommended more than one EBLS. Given that faculty awareness and adoption of EBLSs is limited, it is perhaps not surprising that students report using suboptimal strategies (Blasiman et al., 2017Hartwig & Dunlosky, 2012Karpicke et al., 2009;McCabe, 2011;Wissman et al., 2012). Taken together, survey research indicates that students do Asking learners to add new information to their existing memory, often by having them explain how and why things work Students ask and explain why Botox prevents wrinkles (i.e., The nervous system cannot send messages to move certain muscles). ...
... Unfortunately, however, learners' misinterpretation of difficulty as detrimental leads them astray. For example, given the choice between more versus less spacing, students choose less (Wissman et al. 2012). Given the option to mass together or space out learning events, students choose massing (Tauber et al. 2012). ...
Article
Full-text available
Performance during training is a poor predictor of long-term retention. Worse yet, conditions of training that produce rapidly improving performance typically do not produce long-lasting, generalizable learning. As a result, learners and instructors alike can be misled into adopting training or educational experiences that are suboptimal for producing actual learning. Computer-based educational training platforms can counter this unfortunate tendency by providing only productive conditions of instruction—even if they are unintuitive (e.g., spacing instead of massing). The use of such platforms, however, introduces a different liability: being easy to interrupt. An assessment of this possible liability is needed given the enormous disruption to modern education brought about by COVID-19 and the subsequent widespread emergency adoption of computer-based remote instruction. The present study was therefore designed to (a) explore approaches for detecting interruptions that can be reasonably implemented by an instructor, (b) determine the frequency at which students are interrupted during a cognitive-science-based digital learning experience, and (c) establish the extent to which the pandemic and ensuing lockdowns affected students’ metacognitive ability to maintain engagement with their digital learning experiences. Outliers in time data were analyzed with increasing complexity and decreasing subjectivity to identify when learners were interrupted. Results indicated that only between 1.565% and 3.206% of online interactions show evidence of learner interruption. And although classroom learning was inarguably disrupted by the pandemic, learning in the present, evidence-based platform appeared to be immune.
Conference Paper
Full-text available
We present ALeA, an adaptive learning assistant for university courses. The intelligent tutoring system (ITS) provides services for both learners and instructors, based on semantically annotated course material. The system can generate learning tools like flashcards, guided tours, and quizzes, which can be tailored to individual learning progress. Currently, the system is only available for courses from the field of computer science, like artificial intelligence. We plan to expand it for further courses from different fields.
Article
Full-text available
Previous research has shown better text learning after rereading versus 1 reading of a text. However, rereading effects have only been explored using immediate tests, whereas most students face delays between study and test. In 2 experiments, 423 college students read a text once, twice in massed fashion, or twice with 1 week between trials. Students were tested either immediately or 2 days after study. On an immediate test, performance was greater after massed versus single reading, whereas performance for distributed rereading was not significantly greater than after single reading. On a delayed test, performance was greater after distributed versus single reading, whereas performance for massed rereading and single reading no longer differed significantly. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The spacing effect—that is, the benefit of spacing learning events apart rather than massing them together—has been demonstrated in hundreds of experiments, but is not well known to educators or learners. I investigated the spacing effect in the realistic context of flashcard use. Learners often divide flashcards into relatively small stacks, but compared to a large stack, small stacks decrease the spacing between study trials. In three experiments, participants used a web-based study programme to learn GRE-type word pairs. Studying one large stack of flashcards (i.e. spacing) was more effective than studying four smaller stacks of flashcards separately (i.e. massing). Spacing was also more effective than cramming—that is, massing study on the last day before the test. Across experiments, spacing was more effective than massing for 90% of the participants, yet after the first study session, 72% of the participants believed that massing had been more effective than spacing. Copyright © 2009 John Wiley & Sons, Ltd.
Article
Full-text available
Although the memorial benefits of testing are well established empirically, the mechanisms underlying this benefit are not well understood. The authors evaluated the mediator shift hypothesis, which states that test-restudy practice is beneficial for memory because retrieval failures during practice allow individuals to evaluate the effectiveness of mediators and to shift from less effective to more effective mediators. Across a series of experiments, participants used a keyword encoding strategy to learn word pairs with test-restudy practice or restudy only. Robust testing effects were obtained in all experiments, and results supported predictions of the mediator shift hypothesis. First, a greater proportion of keyword shifts occurred during test-restudy practice versus restudy practice. Second, a greater proportion of keyword shifts occurred after retrieval failure trials versus retrieval success trials during test-restudy practice. Third, a greater proportion of keywords were recalled on a final keyword recall test after test-restudy versus restudy practice.
Article
Full-text available
Effective management of chronic diseases (e.g., diabetes) can depend on the extent to which patients can learn and remember disease-relevant information. In two experiments, we explored a technique motivated by theories of self-regulated learning for improving people's learning of information relevant to managing a chronic disease. Materials were passages from patient education booklets on diabetes from NIDDK. Session 1 included an initial study trial, Session 2 included self-regulated restudy, and Session 3 included a final memory test. The key manipulation concerned the kind of support provided for self-regulated learning during Session 2. In Experiment 1, participants either were prompted to self-test and then evaluate their learning before selecting passages to restudy, were shown the prompt questions but did not overtly self-test or evaluate learning prior to selecting passages, or were not shown any prompts and were simply given the menu for selecting passages to restudy. Participants who self-tested and evaluated learning during Session 2 had a small but significant advantage over the other groups on the final test. Secondary analyses provided evidence that the performance advantage may have been modest because of inaccurate monitoring. Experiment 2 included a group who also self-tested but who evaluated their learning using idea-unit judgments (i.e., by checking their responses against a list of key ideas from the correct response). Participants who self-tested and made idea-unit judgments exhibited a sizable advantage on final test performance. Secondary analyses indicated that the performance advantage was attributable in part to more accurate monitoring and more effective self-regulated learning. An important practical implication is that learning of patient education materials can be enhanced by including appropriate support for learners' self-regulatory processes.
Article
Although substantial research has demonstrated the benefits of retrieval practice for promoting memory, very few studies have tested theoretical accounts of this effect. Across two experiments, we tested a hypothesis that follows from the desirable difficulty framework [Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe, A. Shimamura, (Eds.), Metacognition: Knowing about knowing (pp. 185–205). Cambridge, MA: MIT Press], the retrieval effort hypothesis, which states that difficult but successful retrievals are better for memory than easier successful retrievals. To test the hypothesis, we set up conditions under which retrieval during practice was successful but differentially difficult. Interstimulus interval (ISI) and criterion level (number of times items were required to be correctly retrieved) were manipulated to vary the difficulty of retrieval. In support of the retrieval effort hypothesis, results indicated that as the difficulty of retrieval during practice increased, final test performance increased. Longer versus shorter ISIs led to more difficulty retrieving items, but higher levels of final test performance. Additionally, as criterion level increased, retrieval was less difficult, and diminishing returns for final test performance were observed.
Article
Applied a self-report approach to 2 examinations of different types of study strategies in an upper-division college course. The 1st examination was a closed-book multiple-choice tests of 60 items. 46 students returned questionnaires. The 2nd examination was an open-book, open-note multiple-choice test of 30 items. For a closed-book examination, study strategies that could promote deep processing correlated positively with scores but were not likely to be used by the students. For an open-book, open-note examination, strategies that might have led to confusion regarding the locations of material in the textbook and lecture notes correlated negatively with scores, although they were not likely to be used by the 58 students. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Previous research shows that interleaving rather than blocking practice of different skills (e.g. abcbcacab instead of aaabbbccc) usually improves subsequent test performance. Yet interleaving, but not blocking, ensures that practice of any particular skill is distributed, or spaced, because any two opportunities to practice the same task are not consecutive. Hence, because spaced practice typically improves test performance, the previously observed test benefits of interleaving may be due to spacing rather than interleaving per se. In the experiment reported herein, children practiced four kinds of mathematics problems in an order that was interleaved or blocked, and the degree of spacing was fixed. The interleaving of practice impaired practice session performance yet doubled scores on a test given one day later. An analysis of the errors suggested that interleaving boosted test scores by improving participants' ability to pair each problem with the appropriate procedure. Copyright © 2009 John Wiley & Sons, Ltd.
Article
Encoding strategies vary in their duration of effectiveness, and individuals can best identify and modify strategies that yield effects of short duration on the basis of retrieval failures. Multiple study sessions with long inter-session intervals are better than massed training at providing discriminative feedback that identifies encoding strategies of short duration. We report two investigations in which long intervals between study sessions yield substantial benefits to long-term retention, at a cost of only moderately longer individual study sessions. When individuals monitor and control encoding over an extended period, targets yielding the largest number of retrieval failures contribute substantially to the spacing advantage. These findings are relevant to theory and to educators whose primary interest in memory pertains to long-term maintenance of knowledge.
Article
Previous studies, such as those by Kornell and Bjork (Psychonomic Bulletin & Review, 14:219-224, 2007) and Karpicke, Butler, and Roediger (Memory, 17:471-479, 2009), have surveyed college students' use of various study strategies, including self-testing and rereading. These studies have documented that some students do use self-testing (but largely for monitoring memory) and rereading, but the researchers did not assess whether individual differences in strategy use were related to student achievement. Thus, we surveyed 324 undergraduates about their study habits as well as their college grade point average (GPA). Importantly, the survey included questions about self-testing, scheduling one's study, and a checklist of strategies commonly used by students or recommended by cognitive research. Use of self-testing and rereading were both positively associated with GPA. Scheduling of study time was also an important factor: Low performers were more likely to engage in late-night studying than were high performers; massing (vs. spacing) of study was associated with the use of fewer study strategies overall; and all students-but especially low performers-were driven by impending deadlines. Thus, self-testing, rereading, and scheduling of study play important roles in real-world student achievement.