Pretesting with Multiple-choice Questions Facilitates Learning

Abstract

Taking a test before study can improve subsequent learning of that pretested information. How the pretest affects subsequent learning of other information in the passage is less clear, however. In three experiments, we examined the consequences of taking a multiple-choice (MC) pretest on the later recall of both pretested and non-pretested related information, finding that pretesting improved recall of pretested information without impairing recall of non-pretested information. In addition, we compared a pretest condition to conditions in which subjects were told to memorize the questions and in which subjects studied facts prior to reading. Although taking a pretest was not significantly more effective than memorizing questions or studying facts for the pre-exposed information, it did not impair the learning of related information, whereas studying facts did. Thus, even when an MC pretest takes time away from study, that pretest appears to make subsequent study more effective than other types of activities that pre-expose students to to-be-tested information.

Full text

Pretesting with Multiple-choice Questions Facilitates Learning
Jeri L. Little (jerilittle@ucla.edu)
Department of Psychology, 1285 Franz Hall, Box 951563
Los Angeles, CA 90024 USA
Elizabeth Ligon Bjork (elbjork@psych.ucla.edu)
Department of Psychology, 1285 Franz Hall, Box 951563
Los Angeles, CA 90024 USA
Abstract
Taking a test before study can improve subsequent learning of
that pretested information. How the pretest affects
subsequent learning of other information in the passage is less
clear, however. In three experiments, we examined the
consequences of taking a multiple-choice (MC) pretest on the
later recall of both pretested and non-pretested related
information, finding that pretesting improved recall of
pretested information without impairing recall of non-
pretested information. In addition, we compared a pretest
condition to conditions in which subjects were told to
memorize the questions and in which subjects studied facts
prior to reading. Although taking a pretest was not
significantly more effective than memorizing questions or
studying facts for the pre-exposed information, it did not
impair the learning of related information, whereas studying
facts did. Thus, even when an MC pretest takes time away
from study, that pretest appears to make subsequent study
more effective than other types of activities that pre-expose
students to to-be-tested information.
Keywords: pretesting, testing effects, multiple-choice
Introduction
In addition to assessing learning, tests can enhance learning.
Testing information after study improves later recall more
than additional study (see, e.g., Roediger & Karpicke,
2006). Likewise, testing information before study (i.e.,
pretesting) has been shown to improve subsequent learning
of the pretested information (e.g., Kornell, Hays, & Bjork,
2009; Richland, Kornell, & Kao; 2009; Rothkopf, 1966),
although some evidence suggests that pretesting can have
negative consequences (e.g., persistence of errors, Fritz,
Morris, Bjork, Gelman, & Wickens, 2000). How pretest-
taking affects subsequent learning of information not
tested—in particular, related information—is less clear, but
of concern for both practical and theoretical reasons. In the
present work, we examine the effects of pretesting on the
later recall of both tested and nontested related information.
The Effect of Pretesting on Pretested Information
Testing information after study may improve its later recall
because the act of retrieving information from memory
modifies its representation in such a way as to make it more
recallable in the future than it would have been otherwise
(e.g., Bjork, 1975; Bjork & Bjork, 1992). When given a
pretest, however, correct retrieval of answers is unlikely
given that students have not yet been exposed to the to-be-
learned material. Thus, the observed improved recall of
pretested information should reflect the consequence of
processes other than successful retrieval.
Pretesting may be beneficial because it encourages more
active involvement in learning, perhaps by increasing
general interest in the topic. Additionally, the pretest may
help students to discern what information is most important
or what type of information the teacher is likely to test later.
Thus, a pretest may lead to better recall for the previously
tested information because it directs attention to the need to
encode that information when encountered again during
subsequent study (Hamaker, 1986).
Possible Negative Consequences of Pretesting with
Multiple-choice (MC) Tests Answering a question
incorrectly may strengthen the erroneous response and
decrease one’s ability to learn the correct information later
(e.g., Fritz et al., 2000). A negative characteristic of MC
tests, in particular, is that they expose students to incorrect,
but often attractive, alternatives, which can lead students to
intrude those incorrect alternatives on later cued-recall tests
(e.g., Roediger & Marsh, 2005). Of particular concern
when using MC pretests is the finding by Butler and
Roediger (2008) that intrusions increase when participants
take a test without having studied the relevant information
beforehand. On the other hand, Butler and Roediger also
found that students were likely to change erroneous
responses made on the pretest—to correct answers on a later
test—when feedback was given. Therefore, the intrusion of
errors on a later test might be reduced or eliminated as a
consequence of being able to read the passage after taking
the MC pretest.
The Effect of Pretesting on Related Information
The effect of pretesting on the subsequent learning of
related information not tested on the pretest has been
investigated in a variety of studies (e.g., see Anderson &
Biddle, 1975; and Hamaker, 1986 for meta-analyses).
Evidence suggests that pretesting directs attention towards
the processing of pretested information. Pretesting appears
to improve subsequent learning of information that is related
to pretested information, but perhaps only when the tested
and nontested information are related in certain ways (e.g.,
the related information aids in searching for the pretested
information during subsequent study). To the extent that
non-pretested information does not aid in the search task,
294

however, such non-pretested information may not be better
learned during subsequent study, even if the pretested and
non-pretested information have a systematic relationship.
For example, non-pretested information that has a
competitive relationship with the pretested information
(e.g., the answer to one question in a pair of related
questions would be a plausible, although incorrect answer,
to the other question in the pair) might not be useful in the
search task. A pretest might thereby impair the learning of
competitive related information. To our knowledge, past
research has not examined the effect of pretesting on the
later learning of competitive non-pretested information.
Benefits for Related Information with Multiple-Choice
Tests Little and Bjork (2010) demonstrated that taking an
MC test following the study of a text passage improves the
later recall of both tested and competitive nontested
information—if the answer to a competitive question
appears as a incorrect alternative in the initial MC test.
They argued that MC alternatives with competitive incorrect
alternatives encourage students to recall information that not
only confirms why the right answer is correct, but also why
the other alternatives are incorrect. In a pretesting situation,
students would not have access to information that would
allow them to reject incorrect alternatives. The MC pretest,
however, would still encourage participants to examine the
alternatives thoroughly (perhaps in hope that they have
some background information that would enable them to
reject one or more alternatives to make a better guess). In
this process, students may encode not only the question, but
also the alternatives. Consequently, when reading the
passage after the pretest, students may direct their attention
not only to the processing of information that would answer
the pretested questions, but also to the processing of
information having to do with the alternatives—information
that might otherwise interfere with the search for pretested
information. From this perspective, it seems likely that
pretest MC questions could be effective in improving
learning of information that has a competitive relationship
with the pretested information.
The Present Work
In the present research, we aimed to assess whether MC
pretests provide benefits for tested information that
outweigh the potential costs of errors made on the initial test
and whether such tests can improve learning of nontested
competitive information. In addition, we explored whether
benefits or costs occur simply as a consequence of being
exposed to to-be-tested information before study or whether
the act of trying to answer questions engages processing that
leads to these effects. Specifically, in Experiments 2 and 3,
we investigated whether a benefit of test-taking (although
mostly unsuccessful) exceeds that of exposure to material,
by comparing pretesting to a condition in which questions
are memorized, but not answered, before reading the
passage (Experiment 2) and a condition in which
participants study facts that could have been tested prior to
reading (Experiment 3). Our expectation was that testing
would provide benefits (for both tested and related
information) not afforded by spending the full time
studying, and that these benefits would be larger than those
obtained with other pre-exposure activities.
Experiment 1
Method
Participants and design Twenty-five students at the
University of California, Los Angeles, participated for
partial course credit. Condition of study (pretested vs.
extended-study) was manipulated within-subjects. On the
final cued-recall test, all participants answered pretested and
non-pretested related questions from the passage that was
preceded by a pretest and questions from the passage that
was not preceded by a pretest (baseline control).
Materials Two passages were constructed, one about
Saturn and one about Yellowstone National Park (~800
words), and ten pairs of MC questions were created for each
passage. The two questions in each pair tested the same
topic (e.g., geysers) and had the same four alternatives (e.g.,
Old Faithful, Steamboat Geyser, Castle Geyser, and Daisy
Geyser), but different correct answers (e.g., What is the
tallest geyser in Yellowstone National Park? Answer:
Steamboat Geyser; and, What is the oldest geyser in
Yellowstone National Park? Answer: Castle Geyser). The
questions from each passage were randomly divided into
two 10-item sets (for each pair, one question was in Set A
and one question was in Set B), such that one set would be
tested on the pretest and the other set would serve as related
questions on the final test (counterbalanced). Passage order
and condition order were also counterbalanced across
participants.
Procedure Each participant read two passages—one that
would be preceded by an initial MC test (4 min test; 10 min
study) and one that would be studied for the full time (14
min study). For the pretested condition, the ten MC
questions (i.e., all the items in one of the question sets for
that passage) were presented one at a time on the computer
screen for 22 s. Participants were told that although they
had not yet read the passage, the experimenters wanted to
assess how much they already knew about the topic. In
addition, participants were told that they would take a later
test on the topic and that the questions on the pretest would
provide them with an idea of the type of information that
would be tested later. No corrective feedback was given
during the test. For the passage that was not tested
(control), participants were given the full 14 min to read the
passage, and they were told that if they finished reading
early, they should spend the remainder of the time studying.
Finally, after a 5-min retention interval during which they
played Tetris (a spatial-reasoning puzzle game), participants
received a 40-item final cued-recall test, with the questions
presented one at a time on the computer screen. For the
pretested condition, except for the absence of alternatives,
295

half of the questions were identical to the MC questions
(i.e., pretested) and half were the non-pretested related items
(i.e., the 10 questions from the set that had not been tested).
Related questions were always tested in the first half of the
test, along with half of the control questions, to which their
performance would be compared. Similarly, previously
tested questions were tested in the second half of the test,
along with half of the control questions, to which their
performance would be compared.
Results and Discussion
Pretest Performance Performance on the MC pretest (M =
27%, SD = 12%) was not significantly different from chance
performance (25%), t(24) = 0.89, p > .05.
Final-test Performance We found that taking an MC
pretest in lieu of additional time spent studying improved
recall of that pretested information, but not recall of related
information.
Performance in the pretested condition was compared to
that in the corresponding extended-study control condition
via planned paired-samples t tests. Specifically, these
comparisons revealed that pretested questions (M = 61%, SE
= 4%) were answered correctly more often than were
control questions from the topic that did not receive a
pretest (M = 43%, SE = 4%), t(24) = 5.26, p < .05. No such
benefit occurred for questions of non-tested related
information (M = 48%, SE = 4%) as compared to control
questions (M = 46%, SE = 5%), t(24) = 0.35, p > .05.
Our results suggest that taking a pretest is beneficial for
learning. Previous work, however, has suggested that
spending part of one’s study time taking a test, for which
one is unlikely to know the correct answers might not lead
to the pattern of results that we obtained: Rather, that errors
would persist (e.g., Fritz et al., 2000).
Indeed, participants did sometimes recall previously
incorrect alternatives on the final test. Because all of the
alternatives were contained in the text, however, participants
intruded these responses in the extended study condition as
well. Of interest regarding the costs of MC testing is
whether intrusions are greater after taking a pretest than
after extended study. Table 1 shows recall (intrusion) rates
for incorrect alternatives (or items than could have been
incorrect alternatives) on the final test for the experimental
and control conditions in Experiments 1, 2, and 3. As
shown in Table 1, taking a MC test did not increase
intrusion of incorrect information.
Table 1: Intrusion Percentages for Experiments 1, 2, & 3
Exp.
Activity
Previously Tested
Extended-
Type
Memorized, or Studied
study Control
1
Pretest
10%
13%
2
Pretest
11%
13%
Memorize
14%
13%
3
Pretest
8%
14%
Fact-Study
7%
14%
Additionally, participants were not—as a consequence of
spending some study time taking a test—impaired in their
ability to answer correctly related, but initially non-pretested
questions. Students were told that the pretest would “give
them an idea of the questions to expect,” in order to steer
them away from focusing on the pretested information
during subsequent study. It was clear from the high
performance on pretested items, however, that—
purposefully or not—they paid significant attention to
pretested information. In fact, it is more likely that
participants directed attention towards the processing of
pretested information than towards the processing of non-
pretested information. For this reason, we believe that a
numerical benefit for related information given significantly
less study time for that tested passage (10 min compared to
14 min, respectively) is worth consideration.
Experiment 2
In Experiment 1, we found that taking a pretest improved
the effectiveness of a subsequent reading session, even
though the pretest took time away from reading and
participants answered a majority of the questions incorrectly
on the pretest. Specifically, recall of pretested information
was improved. Moreover, recall of related information was
not hurt, suggesting that the 10 min of studying after a test
was as effective as studying for the full 14 min without a
test.
It is uncertain, however, whether trying to answer
questions would increase the effectiveness of study beyond
what would occur from simply being exposed to those
questions beforehand. Previous work has demonstrated that
testing (with cued-recall questions) improves retention more
so than does reading those questions before study (Richland,
Kornell, & Kao, 2009). In Experiment 2, we explored
whether trying to answer an MC question leads to improved
performance as compared to memorizing that question and
what impact this difference in processing would have on the
learning of competitive related information. On the one
hand, if trying to answer a question engages a deeper level
of processing than memorizing a question, then answering
questions should provide a benefit that outweighs that of
memorizing questions. On the other hand, perhaps
answering questions would increase the encoding of
misinformation as compared to memorizing questions.
Method
Participants and design Sixty-four students at the
University of California, Los Angeles, participated for
partial course credit. Condition of study (question-study vs.
extended-study) was manipulated within-subjects. Type of
instructions for the question-study condition (pretest vs.
memorize) was manipulated between subjects. On the final
cued-recall test, all participants answered pretested or
memorized questions and non-pretested or non-memorized
related questions from the passage that was preceded by
questions, in addition to questions from the passage that was
not preceded by questions (baseline control).
296

Materials and procedure The materials and procedure
were the same as those used in Experiment 1, with one
exception: the addition of a between-subject variable (type
of instructions). For half of the participants, instead of
answering the questions, they were told to memorize the
questions. Specifically, they were told that they would see
questions that could be asked about the to-be-read passage,
and that they should memorize the questions and the answer
choices. They were given this instruction to insure that they
processed the questions. The pretesting group was not told
anything about a later test, just that the experimenters
wanted to assess their knowledge of the to-be-learned topic.
Results and Discussion
Pretest Performance Performance on the MC test (M =
27%, SD = 18%) was not significantly different from chance
performance (25%), t(31) = 0.68, p > .05.
Final-test Performance Correct recall performance on the
final test is presented in Figure 1, and, as indicated there,
performance on previously exposed questions (in both the
pretest and memorize conditions) was improved, as
compared to spending the full time studying (i.e., control
condition). The recall of related information, however,
appears only to have been improved when participants
answered questions on an initial MC pretest, not when they
simply memorized those questions.
0
10
20
30
40
50
60
70
Prete ste d Re late d Exten ded -
Study Control
Mem orize d Re lated Exten ded -
Study Control
Answere d Que stions (Pretes t) Mem orized Ques tions
Correct Recall Performance
Percentages
Figure 1: Correct recall performance percentages as a
function of instruction type and item type in Experiment 2.
The white bars show the average performance for extended
study control questions tested in the first half and second
half of the test for each condition. Error bars represent +/-1
SE.
Correct performance for the pretested and memorized
items was compared to that for the corresponding items in
the extended study control condition via planned paired-
samples t tests. Benefits were found for both pretested
items (M = 54%, SE = 5%) as compared to control (M =
37%, SE = 4%), as well as for memorized items (M = 46%,
SE = 4%) as compared to control (M = 36%, SE = 5%),
t(31) = 3.64, p < .01 and t(31) = 2.54, p < .05, respectively.
A 2 × 2 repeated-measures ANOVA did not reveal a
significant interaction between type of instructions
(pretested vs. memorized) and condition of study (question-
study vs. extended study control), F(1,62) = 1.55, p > .05.
Additionally, a 2 × 2 repeated-measures ANOVA also did
not reveal a significant interaction between type of
instructions (pretested vs. memorized) and condition of
study (question-study vs. extended study control) for related
information, F(1,62) = 2.32, p > .05. In both cases,
however, the interaction trended towards testing being better
than memorizing.
In Experiment 2, we found that although taking a pretest
and memorizing information both led to improved recall of
correct answers to those pre-exposed questions, the benefit
afforded by actually trying to answer questions was
numerically greater than that afforded by a comparable pre-
exposure activity (i.e., memorizing questions). We found
that this trend occurred for related information as well. In
neither case, however, was the benefit of taking a test
significantly better than the benefit afforded by memorizing
the questions, a result that we believe may have occurred as
a consequence of participants in the memorize condition
spontaneously trying to answer the questions. In
Experiment 3, we attempted to control for this possible
problem with a modified method that would remove the
propensity to endorse an answer, with the expectation that
testing would then reveal a greater overall benefit.
Experiment 3
In Experiment 3, our aim was similar to that in Experiment
2: to compare the effects of pretesting with another task that
also exposes participants to to-be-tested information before
reading. In this case, however, we wanted to ensure that the
non-test activity did not encourage the type of processing
occurring when answering questions, as might have been
the case in Experiment 2. Thus, in Experiment 3, we
compared taking a pretest to being exposed to comparable
facts that contained the correct answer as well as
competitors (e.g., The oldest geyser in Yellowstone National
Park is Castle Geyser, not Old Faithful, Steamboat Geyser,
or Daisy Geyser). Because the facts contained the correct
answer to the matched question, we also gave feedback in
the pretest condition. In addition, we manipulated all
variables within subjects. To do so, we developed a third
passage so that all participants would take a pretest before
one passage, study facts before another passage, and receive
extra time to study a third passage
Method
Participants and design Seventy-two students at the
University of California, Los Angeles, participated for
partial course credit. Activity type (pretest vs. facts vs.
extended-study) was manipulated within subjects. On the
final cued-recall test, all participants answered questions
pertaining to the pretest or studied facts and questions
related to the pretest or studied facts, in addition to
questions from the passage that received extended study
time (baseline control).
Materials The materials were variations of those used in
Experiments 1 and 2. We shortened the passages about
297

Saturn and Yellowstone (~600 words), without removing
any information contained in the questions. In addition, we
added a passage about stimulant drugs. For each passage,
we used ten pairs of MC questions. Questions for Saturn
and Yellowstone were the same as those used in Experiment
1 and 2. Questions for the passage about stimulants were
constructed in the same manner.
Finally, Experiment 3 differed from Experiment 2 in that
the question-memorization condition from Experiment 2
was replaced with a fact-study condition. For example, in
the fact-study condition, participants would see: The oldest
geyser in Yellowstone National Park is Castle Geyser, not
Steamboat Geyser, Daisy Geyser, or Old Faithful. The facts
contained all of the information contained in the matched
question, including the competitors.
Procedure The procedure for Experiment 3 was the same
as that of Experiment 1, with the following exceptions. All
participants learned about three topics. For one topic, the
reading of the passage (6 min) was preceded by a 10-item
MC test (4 min); for a second topic, the reading of the
passage (6 min) was preceded by a 10-fact study session (4
min); for a third topic, participants were given the full 10
min to read the passage.
On the pretest, participants were given 20 s to answer
each question. After typing in their response, they
continued to view the question until the 20 s elapsed. After
each question, participants received feedback (i.e., the
correct answer presented below the question) for 4 s. For
the fact-study condition, participants were presented with 10
facts for 24 s each and were told to think about the fact for
the full time that it was presented.
After a 5-min non-verbal distractor task (i.e., playing
Tetris), participants received a final cued-recall test with
sixty questions, ten questions for each of the following
(previously tested, related to tested, previously studied as
facts, related to studied facts) and twenty questions from the
extended study control condition.
Results and Discussion
Pretest Performance Performance on the MC test (M =
30%, SD = 15%) was higher than chance performance, t(71)
= 2.69, p < .01, and resulted as a consequence of high
performance on the pretest for the stimulants topic (M =
38%). Performance for Saturn (M = 25%) and Yellowstone
(M = 26%) were comparable to that found in Experiments 1
and 2.
Final-test Performance Correct recall performance on the
final test is presented in Figure 2, and, as indicated there,
participants’ ability to answer questions for information that
was exposed during the pretest and fact-study conditions
was improved, as compared to spending the full time
studying (i.e., control condition). The recall of related
information appeared not to be hurt as a consequence of
taking an initial pretest, but appeared to be hurt as a
consequence of studying facts before reading.
Correct performance for the information pretested or
studied as facts was compared to that for the corresponding
items in the extended study control condition via planned
paired-samples t tests. Benefits were found for both
pretested items (M = 69%, SE = 3%) and items studied as
facts (M = 66%, SE = 2%) as compared to control (M =
39%, SE = 3%), t(71) = 10.67, p < .01 and t(71) = 9.21, p <
.01, respectively. Pretested information was not recalled
correctly more often than was information studied as facts.
0
10
20
30
40
50
60
70
80
Pretested Related Studied as
Facts
Related
Pretest Fact Study Extended-
Study Control
Correct Recall
Performance Pe rcentages
Figure 2: Correct recall performance percentages as a
function of activity type and item type in Experiment 3.
The white bar shows the average performance for extended
study control questions tested in the first half and second
half of the test. Error bars represent +/-1 SE.
Correct performance on questions pertaining to
information that was related to the pre-exposed information
was not better than that for the corresponding information in
the extended-study control condition. Interestingly,
however, although correct performance on items related to
pretested information (M = 40%, SE = 2%) was not
significantly different than that for control items (M = 41%,
SE = 3%), correct performance on items related to studied
facts (M = 34%, SE = 2%) was significantly worse than that
for control items, t(71) = 2.57, p < .05, and, in fact, was
significantly worse than that for questions related to
pretested information, t(71) = 2.81, p < .01. Thus, although
answering questions before study did not impair one’s
ability to learn related information, studying comparable
facts before study did—even when the facts also provided
participants with the answers to the related questions.
When assessing the total effect of taking a pretest (as
compared to studying facts beforehand or spending the full
time studying) across both tested and related information (M
= 56%, SE = 2%), there was a clear benefit for pretesting
over studying facts (M = 50%, SE = 2%), t(71) = 2.67, p <
.01 and for pretesting compared to extended study (M =
40%, SE = 2%), t(71) = 6.91, p < .05.
In Experiment 3, we found that taking a test with
feedback prior to the reading of a passage provided a
learning benefit that outweighed that of studying facts
beforehand or of spending additional time studying.
General Discussion
Although incorrect answers were endorsed for a majority of
the questions on the MC pretests, we found that taking a
pretest made subsequent study more effective, as
demonstrated by improved recall of that pretested
298

information as compared to spending the full time studying;
additionally, the pretest appeared not to lead to
misinformation.
This result cannot be the sole consequence of pre-
exposure to to-be-tested information. The trend towards a
benefit for pretesting over studying facts in Experiment 3 is
intriguing because the fact-study condition provided
participants with access to the correct answer for each
question, in the form of a fact, for 24 s (as opposed to
receiving the correct answer for only 4 s after spending 20 s
choosing an incorrect answer for the majority of questions).
This pattern suggests that testing can serve as an effective
learning event before study, even when retrieval fails.
Additionally, although we did not find that related
information was learned better in the pretest condition than
in the extended-study condition, we never found it to be
impaired, even though the extended-study condition actually
provides a very conservative control for total time on task.
In the pretesting condition, participants received
substantially less time to read the passage than they did for
the extended-study condition. We would thus contend that
comparable recall across the two conditions is likely to be
under-representative of relative learning (i.e., correct recall
per minute of reading time). Evidence from Experiment 3
would further support this contention because reduced time
to read the passage in the fact-study condition led to
impaired recall of related information. Finally, compared to
this fact-study condition (which also offers a valid control
for time-on-task), testing did improve recall of related
information.
The present pattern of results speaks to the benefit of
pretests as learning events. The present experiments,
however, do not reveal what specific mechanisms might
lead to testing being more beneficial than studying facts
(Experiment 3). The results are consistent with the idea that
an MC pretest directs attention broadly, such that students
search not only for the correct answer to the pretested
question, but also for information pertaining to the other
choices. Additionally, it is the act of trying to answer
question that leads to enhanced recall of related
information—not simply pre-exposure—as facts containing
the competitors do not seem to direct attention so broadly.
That is, testing might lead to a deeper level of processing
than trying to memorize questions (or studying facts), thus
making it more likely that students will be reminded of
those questions and alternatives when reading the passage
during subsequent study. Although the present research
provides evidence that MC pretests can serve as effective
study events, future research should further investigate the
specific underlying processes that lead to such benefits of
pretesting.
Acknowledgements
We thank Robert Bjork, Nate Kornell, and Matt Hays for
inspiring this work. Grant 29192G from the McDonnell
Foundation supported this research.
References
Anderson, R. C., & Biddle, M C. (1975). On asking people
questions about about what they are reading. In G. H.
Bower (Ed.), The Psychology of Learning and Motivation,
Vol. 9. (pp. 89-132). New York, NY, US: Academic Press
Bjork, R. A. (1975). Retrieval as a memory modifier. In R.
Solso (Ed.), Information processing and cognition: The
Loyola Symposium (pp. 123-144). Hillsdale, NJ:
Lawrence Erlbaum Associates.
Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse
and an old theory of stimulus fluctuation. In A. F. Healy,
S. M. Kosslyn, & R. M. Shiffrin (Eds.), From Learning
Processes to Cognitive Process. (pp. 35-67) Hillsdale, NJ:
Erlbaum.
Butler, A. C., & Roediger, H. L (2008). Feedback enhances
the positive effects and reduces the negative effects of
multiple-choice testing. Memory & Cognition, 36, 604-
616.
Fritz, C. O., Morris, P. E., Bjork, R. A., Gelman, R., &
Wickens, T. D. (2000). When further learning fails:
Stability and change following repeated presentation of
text. British Journal of Psychology, 91, 493-511.
Kornell, N., Hays, M. J., & Bjork, R. A. (2009).
Unsuccessful retrieval attempts enhance subsequent
learning. Journal of Experimental Psychology: Learning,
Memory, and Cognition, 35, 989-998.
Hamaker, C. (1986). The effects of adjunct questions on
prose learning. Review of Educational Research, 56, 212-
242.
Little, J. L., & Bjork, E. L. (2010). Multiple-choice testing
can improve the retention of non-tested related
information. In S. Ohisson & R. Catrabone (Eds.)
Proceedings of the 32nd Annual Conference of the
Cognitive Science Society (pp. 1535-1540). Austin, TX:
Cognitive Science Society.
Richland, L. E., Kornell, N., & Kao, L. S. (2009). The
pretesting effect: Do unsuccessful retrieval attempts
enhance learning? Journal of Experimental Psychology:
Applied, 15, 243-257.
Roediger , H. L., & Karpicke, J. D. (2006). The power of
testing memory: Basic research and implications for
educational practice. Perspectives on Psychological
Science, 1, 181-210.
Roediger, H. L., & Marsh, E. J. (2005). The positive and
negative consequences of multiple-choice testing. Journal
of Experimental Psychology: Learning, Memory, and
Cognition, 31, 1155-1159.
Rothkopf, E. Z. (1966). Learning from written instructive
materials: An exploration of the control of inspection
behavior by test-like events. American Educational
Research Journal, 3, 241-249.
299