No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
When People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The "Delayed-JOL Effect"
Abstract
Judgments of learning (JOLs), which pertain to knowing what one knows and which help to guide self-paced study during acquisition, have almost never been very accurate at predicting subsequent recall. We recently discovered a situation in which the JOLs can be made to be extremely accurate. Here we report the conditions under which such high accuracy occurs, namely, when the JOL made on the stimulus cue is delayed until shortly after study rather than being made immediately after study. Discussion is focused both on theoretical explanations (to be explored in future research) and on potential applications of the delayed-JOL effect.
... The dual-memory monitoring hypothesis posits that making judgments of learning requires information from both working and episodic memories 8 . Alternatively, the memory strength hypothesis suggests that judgments of learning are based on the strength of working memory 9,10 . The monitoring dual-memories hypothesis and memory strength hypothesis www.nature.com/scientificreports/ ...
... Since immediate judgments of learning occur immediately after encoding, essentially involving immediate retrieval attempts, the relevant information remains in working memory. Previous theories highlight that immediate judgments of learning may incorporate working memory information 9,21 . Therefore, it can be inferred that individuals utilize working memory information to formulate their JOLs. ...
... As has been mentioned previously, making metacognitive monitoring predictions requires retrieval of the target word in working memory 9,10,14 . This is because at that time, the slow memory traces are weak, and participants will overestimate or underestimate their memory performance. ...
Metacognitive systematic bias impairs human learning efficiency, which is characterized by the inconsistency between predicted and actual memory performance. However, the underlying mechanism of metacognitive systematic bias remains unclear in existing studies. In this study, we utilized judgments of learning task in human participants to compare the neural mechanism difference in metacognitive systematic bias. Participants encoded words in fMRI sessions that would be tested later. Immediately after encoding each item, participants predicted how likely they would remember it. Multivariate analyses on fMRI data demonstrated that working memory and uncertainty decisions are represented in patterns of neural activity in metacognitive systematic bias. The available information participants used led to overestimated bias and underestimated bias. Effective connectivity analyses further indicate that information about the metacognitive systematic bias is represented in the dorsolateral prefrontal cortex and inferior parietal cortex. Different neural patterns were found underlying overestimated bias and underestimated bias. Specifically, connectivity regions with the dorsolateral prefrontal cortex, anterior cingulate cortex, and supramarginal gyrus form overestimated bias, while less regional connectivity forms underestimated bias. These findings provide a mechanistic account for the construction of metacognitive systematic bias.
... Although working memory has not been measured in prior studies, the relevance of differentiating between working memory and long-term memory contributions to JOLs has been highlighted previously. Nelson and Dunlosky (1991), for instance, used it to explain the delayed-JOL effect: the wellreplicated observation that JOLs are more accurate when solicitated with a delay (e.g., after the presentation of 10 intervening items) instead of immediately after item presentation. For example, in Nelson and Dunlosky's study, the correlation of the delayed JOLs with recall was 0.9, whereas this correlation was only 0.38 for immediate JOLs. ...
... A meta-analysis of the delayed JOL effect has shown that JOL accuracy substantially increases irrespective of the size of the delay, which usually ranges from 1 min to 10 min. Nelson and Dunlosky (1991) hypothesized that delayed JOLs are more accurate (than immediate JOLs) because both delayed JOLs and delayed performance are related to the strength of long-term memory representations. Conversely, immediate JOLs are less accurate because they are contaminated by working memory traces. ...
... This delay was caused by the sequential presentation of the memoranda, followed by the sequential cueing of the objects for the JOL rating. This delay is important to consider as the literature shows that delaying JOLs can lead to more accurate predictions of long-term memory, known as the delayed JOL effect (Nelson & Dunlosky, 1991). The second concern that can be raised is that the delay systematically differed between the sequence length conditions. ...
Judgments of learning (JOLs) are assumed to be made inferentially, based on cues. This cue-utilization approach substituted the theory that memory strength guides JOLs. The rejection of this theory ignores the existence of two memory systems: working memory (WM), which holds representations immediately accessible, and long-term memory (LTM), which is a permanent store. By manipulating and measuring WM strength, we tested a revised version of the memory-strength theory in which JOLs are guided by WM representations. In Experiment 1, participants memorized sequences of two or four colored objects, then they provided JOLs for an LTM test of these objects, and performed a WM test on the objects’ colors. After learning 200 objects, the LTM test followed. Sequence-length affected WM, but not LTM performance. JOLs, however, were higher for sequences of two than for four objects and correlated higher with WM than LTM performance. We replicated these results with a simultaneous presentation of the objects (Experiment 2), in the absence of a WM test (Experiment 3), and in a word-pair task (Experiment 4). Overall, our findings are consistent with the revised memory-strength theory. WM strength should therefore be considered when examining the factors guiding JOLs.
... remembered on the test). Nelson and Dunlosky (1991), however, found that increasing the amount of time allowed between study and JOL dramatically improved monitoring accuracy. Participants studied paired associates and then made JOLs either immediately after study (hereafter identified as "immediate JOLs") or several minutes later (hereafter identified as "delayed JOLs"). ...
... Theoretical explanations for such high delayed JOL accuracy have focused on the retrieval of target information at time of judgment. One specific account, known as the monitoring-dualmemories hypothesis, was proposed by Nelson and Dunlosky (1991). They suggested that participants monitor information in both short-term and long-term memory at time of JOL, although the recall test taps only long-term memory. ...
... Although theoretically distinct, Nelson and Dunlosky's (1991) hypothesis and Spellman and Bjork's (1992) explanation are not necessarily mutually exclusive. In fact, both views emphasize the importance of making a target retrieval attempt at time of JOL. ...
Three experiments examined metamemory for categorized lists of items. Judgments of learning (JOLs) were obtained from college students either immediately after study or following a brief (at least 30-s) delay. In contrast to past findings (e.g., T.O. Nelson & J. Dunlosky, 1991), no advantage was found for delayed JOLs in Experiment 1, using a standard, prediction-based metamemory cue. In Experiment 2, knowledge-based judgments were elicited, and delayed JOL accuracy improved significantly. The relative efficacy of 4 different metamemory cues was examined in Experiment 3. An interaction between the timing and phrasing of JOL cues was detected: Delayed JOLs were more accurate than immediate JOLs only when knowledge-based cues were used. These results are interpreted in A. Koriat’s (1997) cue-utilization framework for JOL accuracy, and they show that the phrasing of metamemory cues can have a substantial impact on monitoring accuracy.
... One kind of subjective report commonly used to measure monitoring during acquisition is judgments of learning (JOLs), a person's predictions of the likelihood of his or her own subsequent memory performance for recently studied items. JOLs have been documented as a standard measure of metamemory in many sources, such as in recent edited books by Metcalfe and Shimamura (1994), Nelson (1992a), and Reder (1996), tutorials on metamemory by Nelson (1992bNelson ( , 1993, and numerous journal articles (e.g., Begg, Martin, & Needham, 1992;Dunlosky & Connor, 1997;Dunlosky & Nelson, 1994Keleman & Weaver, 1997;Lovelace, 1984;Mazzoni & Nelson, 1995;Nelson & Dunlosky, 1991;Schwartz, 1994;Shaughnessy, 1981;Shaughnessy & Zechmeister, 1992). ...
... Both kinds of JOL yield above-chance accuracy in predicting subsequent recall performance. However, accuracy is greater for delayed JOLs than for immediate JOLs, which is called the delayed-JOL effect (Nelson & Dunlosky, 1991). ...
... The double-blind administration of an active or a placebo gas was made possible by use of an N 2 O delivery unit with a computer-controlled valve that could substitute the delivery of nitrogen for N 2 O while holding the oxygen concentration constant. As described by Ramsay et al. (1992), a slight almond odor was added to the placebo and N 2 O gas mixtures lo mask olfactory cues that might distinguish N 2 O from the placebo gas and to supply 1 Nelson and Dunlosky (1991) offered two other explanations for the delayed-JOL effect. A core assumption of the monitoring-dualmemories hypothesis is that when people make a JOL, they monitor information about the item both from short-term memory and from long-term memory. ...
Many hypotheses have been proposed to account for the effects of nitrous oxide on memory, with one emerging possibility being that it has a global effect on memory-related functioning. This possibility was explored by examining the effects of nitrous oxide on memory performance and on the accuracy of people's judgments about their memory performance. Participants inhaled 30% nitrous oxide or a placebo gas while items were studied and while judgments were made about the likelihood of recall for each item. Next, all participants inhaled the placebo during paired-associate recall. Although administration of nitrous oxide during study impaired recall, it did not affect the predictive accuracy of the metacognitive judgments. These results provide pharmacological evidence for a distinction between memory and metamemory.
... However, the accuracy of immediate JOLs is generally closer to 0 (which would indicate no association of predictions and subsequent item recall) than to 1 (which would indicate perfect ordinal association of the two variables). For example, a series of studies on JOLs for paired-associate learning by undergraduate students indicated a mean (across individuals) gamma correlation between JOLs and subsequent recall performance of approximately .40 (Dunlosky & Nelson, 1992Nelson & Dunlosky, 1991). In these studies, immediate JOLs were compared with JOLs delayed about 30 s after an item had been studied (the participant studied other pairs during the delay). ...
... Instead, delayed JOLs for stimulus-response pairs were equivalent in accuracy to immediate JOLs. A leading explanation for the close-to-perfect accuracy of stimulus-alone delayed JOLs is that they are based on monitoring the outcome of a covert retrieval attempt cued by the stimulus word (Nelson & Dunlosky, 1991;see Schwartz, 1994, for a review). That is, when presented with the cue after some delay (i.e., DOG-?), the individual first monitors retrieval of information about the response word of the pair from memory and then maps that information onto the scale imposed by the JOL . ...
... In summary, the present three experiments were designed to accomplish a variety of interrelated goals: (a) to demonstrate that the accuracy of global judgments is at least partially dependent on die level of recall performance, (b) to replicate the age-differential upgrade of the correlational accuracy of global predictions, (c) to evaluate the possibility of age-related differences in the accuracy of immediate and delayed JOLs, and (d) to determine whether the delayed-JOL effect occurs for older adults. To accomplish these goals, we used both kinds of judgments (global and item by item) in a single experimental task, integrating methods developed in earlier investigations of global predictions (e.g., and in investigations of immediate and delayed JOLs (Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991). ...
In 3 experiments, the effects of age on different kinds of metacognitive prediction accuracy were assessed. Participants made global memory predictions and item-by-item memory predictions in a single experimental task. Metacognitive accuracy was evaluated with correlational and more traditional difference-score measures. Difference-score measures were found, in some cases, to be sensitive to level of recall performance. Correlational techniques revealed that older adults monitored learning effectively. Relative to younger adults, they showed equally accurate immediate judgments of learning (JOLs), produced an equivalent delayed-JOL effect, and showed equivalent upgrading in the accuracy of their global prediction from before to after study of test materials.
... Target-absent JOLs, in contrast, involve presenting only the cue or question. Unlike target-present JOLs, target-absent JOLs are thought to elicit covert retrieval practice, encouraging participants to search their memory for the answer as part of the process of judging what they know and will remember later Kimball and Metcalfe 2003;Nelson and Dunlosky 1991). Target-absent and target-present JOLs are therefore analogous to covert retrieval practice and restudy opportunities, respectively (e.g., Jönsson et al. 2014). ...
... Our prediction that delayed JOLs would enhance text comprehension measured on a test after a 1-week retention interval was based on the hypothesis that target-absent JOLs elicit covert retrieval Nelson and Dunlosky 1991;Spellman and Bjork 1992). However, research suggests that JOLs do not spontaneously elicit covert retrieval practice, or at least not an exhaustive memory search. ...
... This outcome was surprising given that the present study used learning and test conditions that should have maximized the benefits of JOLs for learning, according to prior research and theory. JOLs were made with the target information absent Kimball and Metcalfe 2003;Nelson and Dunlosky 1991), and following a brief delay, after the participants had read all the texts, rather than after each text (for a meta-analysis, see (Rhodes and Tauber 2011)). The final test also occurred after a 1-week retention interval given that learners reap the benefits of covert retrieval after long delays (for a meta-analysis, see (Rowland 2014)). ...
Making judgments of learning (JOLs) after studying can directly improve learning. This JOL reactivity has been shown for simple materials but has scarcely been investigated with educationally relevant materials such as expository texts. The few existing studies have not yet reported any consistent gains in text comprehension due to providing JOLs. In the present study, we hypothesized that increasing the chances of covert retrieval attempts when making JOLs after each of five to-be-studied text passages would produce comprehension benefits at 1 week compared to restudy. In a between-subjects design, we manipulated both whether participants (N = 210) were instructed to covertly retrieve the texts, and whether they made delayed target-absent JOLs. The results indicated that delayed, target-absent JOLs did not improve text comprehension after 1 week, regardless of whether prior instructions to engage in covert retrieval were provided. Based on the two-stage model of JOLs, we reasoned that participants’ retrieval attempts during metacomprehension judgments were either insufficient (i.e., due to a quick familiarity assessment) or were ineffective (e.g., due to low retrieval success).
... More favorable encoding conditions lead to more optimistic forecasts and to higher suspect-identification accuracy than do less favorable encoding conditions (e.g., Molinaro et al., 2021;Nguyen et al., 2018). This finding is consistent with a broader literature on judgments of learning (e.g., Nelson & Dunlosky, 1991;Rhodes & Tauber, 2011). For example, judgments of learning are associated with accuracy when there is a slight delay between learning and forecasting (Nelson & Dunlosky, 1991) and when task fluency is a valid cue for later accuracy (Arbuckle & Cuddy, 1969) (but not when task fluency is negatively related to accuracy; Benjamin et al., 1998). ...
... This finding is consistent with a broader literature on judgments of learning (e.g., Nelson & Dunlosky, 1991;Rhodes & Tauber, 2011). For example, judgments of learning are associated with accuracy when there is a slight delay between learning and forecasting (Nelson & Dunlosky, 1991) and when task fluency is a valid cue for later accuracy (Arbuckle & Cuddy, 1969) (but not when task fluency is negatively related to accuracy; Benjamin et al., 1998). ...
General Audience Summary
In the process of solving crimes, police sometimes present witnesses with objects, such as vehicles, that they suspect were used in the commission of a crime. In the only previously published study of vehicle-identification procedures, witnesses demonstrated a poor ability to discriminate between correctly suspected and incorrectly suspected vehicles (Smith, Mackovichova, et al., 2020). We reasoned that this relatively low discriminability might have been attributable to witnesses attending to other aspects of the crime and away from the getaway vehicle. Accordingly, we evaluated the role of selective attention to the getaway vehicle on the ability of witnesses to discriminate correctly suspected vehicles from incorrectly suspected vehicles. As predicted, witnesses who allocated their attention toward getaway vehicles had relatively high levels of discriminability. In Experiment 1, witnesses who were instructed to attend to the getaway vehicle were better able to discriminate correctly suspected vehicles from incorrectly suspected vehicles than were witnesses who were instructed to attend to the culprit or video (control). In Experiment 2, both self-appraised focus on the getaway vehicle and preidentification confidence distinguished witnesses with relatively high discriminability from witnesses with relatively low discriminability. Finally, suspect-identification accuracy was strongly related to postidentification confidence. These results suggest that when law enforcement personnel follow science-based best-practice recommendations for administering lineups, witnesses might be able to forecast their discriminability on a later vehicle lineup and that police can rely on postidentification confidence for assessing likely identification accuracy.
... The use of a 100-point scale is beneficial as it allows for a straightforward comparison between predicted memory (measured via JOLs) and actual memory (measured via test performance). Thus, JOL accuracy can be easily assessed in terms of calibration by directly comparing JOL ratings with the percentage of targets that participants correctly recall at test. 1 Several factors have been shown to affect JOL accuracy, including perceptions of fluency by studying identical cue-target word pairs (Castel et al., 2007;Mueller et al., 2016), changes in font size (Rhodes & Castel, 2008), increasing the time participants spend studying word pairs (Koriat & Ma'ayan, 2005), and changing JOL timings (i.e., delayed vs. immediate JOLs; Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991). A highly impactful factor on JOL calibration is the direction of the association between the cue-target word pair (i.e., probability of a cue word eliciting a specific target as response; see Nelson et al., 2000). ...
... Finally, because of our interest in the effects of item-specific/relational encoding on JOL calibration, each experiment additionally included a set of calibration plots modeled after Maxwell and Huff (2021) which assessed changes in calibration across each item type as a function of encoding strategy by visualizing changes in mean recall at various JOL increments (see also Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991). Commonly, these plots use JOL increments of 10, allowing for a comparison across 11 total levels (i.e., 0-100 in multiples of 10). ...
Metamemory, or the ability to understand the capacities of one’s own memory, is important for learning. To investigate questions surrounding metamemory, researchers commonly have participants make judgments of learning (JOLs) at encoding, in which participants rate their likelihood of recalling the target in a cue–target word pair when shown only the cue at test. However, the associative direction of cue–target pairs can affect the calibration of JOLs. Unlike forward associates (e.g., credit–card), in which JOLs often accurately predict recall, an illusion of competence has been reported for backward associates (e.g., card–credit), symmetrical associates (e.g., salt–pepper), and unrelated cue–target pairs (e.g., artery–bronze) such that JOLs overestimate later recall. The present study evaluates whether the illusion of competence can be reduced when participants apply deep item-specific or relational encoding tasks relative to silent reading. Across two experiments, we show that both item-specific and relational encoding strategies reduce the illusion of competence for backward associates and unrelated pairs while improving the calibration between JOLs and recall. Our findings suggest that these encoding strategies are effective at reducing the illusion of competence, with increased calibration primarily reflecting improved recall. Thus, item-specific and relational encoding strategies primarily affect retrieval processes rather than metacognitive processes that participants engage in at encoding.
... for the short and long intervals, respectively. The somewhat higher correlation for the longer interval is consistent with Nelson and Dunlosky's (1991) finding that judgments of learning are more accurate when made shortly after study than when made immediately after study. ...
... One implication of this idea is that FOK accuracy may be improved by delaying FOK judgments. Indeed, with regard to judgments of learning, Nelson and Dunlosky (1991) observed that when these judgments are delayed until shortly after study, they are more accurate than when they are made immediately after study. ...
Even when Ss fail to recall a solicited target, they can provide feeling-of-knowing (FOK) judgments about its availability in memory. Most previous studies addressed the question of FOK accuracy, only a few examined how FOK itself is determined, and none asked how the processes assumed to underlie FOK also account for its accuracy. The present work examined all 3 questions within a unified model, with the aim of demystifying the FOK phenomenon. The model postulates that the computation of FOK is parasitic on the processes involved in attempting to retrieve the target, relying on the accessibility of pertinent information. It specifies the links between memory strength, accessibility of correct and incorrect information about the target, FOK judgments, and recognition memory. Evidence from 3 experiments is presented. The results challenge the view that FOK is based on a direct, privileged access to an internal monitor.
... These are metacognitive judgments in which learners see a cue (e.g., a foreign word) and predict their posttest performance (e.g., their recall of the translation). When cue-only judgments are made some time after initial studying, they are thought to induce covert retrieval (e.g., Dunlosky & Nelson, 1992;Kimball & Metcalfe, 2003;Nelson & Dunlosky, 1991) and thereby benefit learning and metacognitive accuracy compared to judgments that are made without retrieval opportunity (e.g., when the judgment is made while seeing the complete word pair; for a meta-analysis, see Rhodes & Tauber, 2011). Parallel to this phenomenon, we expect that an opportunity for cued recall will allow learners to make more accurate judgments when working with MCQs. ...
... Thus, we did not find support for our prediction that stepwise MCQs would provide learners with more diagnostic cues regarding their level of knowledge, thereby improving their monitoring accuracy. On the one hand, this outcome is surprising because retrieval opportunities have been shown to markedly enhance the accuracy of metacognitive judgments (e.g., Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991; for a meta-analysis, see Rhodes & Tauber, 2011). On the other hand, much previous research has focused on by-item judgements of learning, where learners can use their performance during retrieval of individual items to predict performance for that same item on the posttest (Rhodes & Tauber, 2011). ...
Multiple-choice questions (MCQs) are popular in vocabulary software because they can be scored automatically and are compatible with many input devices (e.g., touchscreens). Answering MCQs is beneficial for learning, especially when learners retrieve knowledge from memory to evaluate plausible answer alternatives. However, such retrieval may not always occur (e.g., with easy-to-guess answers). Therefore, we tested whether we could optimize MCQs for retrieval practice with a stepwise display, which presents the question before the answer options. This creates an opportunity for cued recall. In an experimental classroom study (N = 75) and three online experiments with adult participants (N = 45, N = 77, and N = 79), participants practiced vocabulary with either standard MCQs or stepwise MCQs: In standard MCQs, a word was presented with three possible translations; in stepwise MCQs, it was shown for 4 s before the translations appeared. In three of the four experiments (Experiments 1, 3a, and 3b), stepwise MCQs enhanced retention significantly compared to standard MCQs, measured by a posttest several days after practice. Benefits of stepwise MCQs were found for different translation directions (Experiments 1 and 3) and both with easy-to-guess answers (noncompetitive answer options, Experiment 1) and hard-to-guess answers (competitive answer options, Experiment 3). However, the effect was most robust for questions with noncompetitive answer options (Experiments 1, 3a, and 3b) and for learners who reported trying to retrieve the answer from memory during the stepwise display (Experiments 1 and 3). Moreover, retention was generally enhanced by competitive, hard-to-guess answer options (Experiments 3a and 3b). Overall, a stepwise display is a promising and easy-to-implement manipulation to optimize MCQs for retrieval practice.
... The SRL model, however, has generated several explanations of how much time a learner should allocate in a related task such as studying. The discrepancy reduction model specified that humans make judgments to allocate study time toward information that is most discrepant from some internal criterion (Nelson and Dunlosky, 1991;Dunlosky and Hertzog, 1998;Dunlosky et al., 2013). A limitation of this explanation though is that it cannot explain why individuals who allocate a great deal of time to very difficult items have such little gain, called the labor-in-vain effect (Nelson and Leonesio, 1988). ...
... The findings that many college students continued attempts at learning instead of ceasing the overlearning and unsolvable problems do not fit within existing explanations of self-regulated learning, such as the discrepancy reduction model (e.g., Nelson and Dunlosky, 1991), the diminishing criterion model (e.g., Ackerman, 2014) or the region of proximal learning model (e.g., Metcalfe and Kornell, 2005). One obvious reason for the lack of fit is that these models have been developed to explain study time allocation rather that feedback-based learning tasks. ...
Introduction
Effective learning involves the acquisition of information toward a goal and cessation upon reaching that goal. Whereas the process of learning acquisition is well understood, comparatively little is known about how or when learning ceases under naturalistic, open-ended learning conditions in which the criterion for performance is not specified. Ideally, learning should cease once there is no progress toward the goal, although this has never been directly tested in human learners. The present set of experiments explored the conditions under which college students stopped attempting to learn a series of inductive perceptual discrimination problems.
Methods
Each problem varied by whether it was solvable and had a criterion for success. The first problem was solvable and involved a pre-determined criterion. The second problem was solvable, but with no criterion for ending the problem so that learners eventually achieved a highly accurate level of performance (overlearning). The third problem was unsolvable as the correct answer varied randomly across features. Measures included the number of trials attempted and the outcome of each problem.
Results and Discussion
Results revealed that college students rarely ceased learning in the overlearning or unsolvable problems even though there was no possibility for further progress. Learning cessation increased only by manipulating time demands for completion or reducing the opportunity for reinforcement. These results suggest that human learners show laudable, but inefficient and unproductive, attempts to master problems they should cease.
... There are several accounts for the JOL reactivity (for reviews, see Double et al., 2018;Double & Birney, 2019). One of the main accounts is the covert-retrieval hypothesis (Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991;Spellman & Bjork, 1992). Making metacognitive judgments can encourage learners to covertly retrieve the information (Son & Metcalfe, 2005), and such retrieval attempts can alter the memory. ...
... Making metacognitive judgments can encourage learners to covertly retrieve the information (Son & Metcalfe, 2005), and such retrieval attempts can alter the memory. Indeed, Nelson and Dunlosky (1991) reported many people attempted retrieval before making metacognitive judgments. Prior studies demonstrating the memory benefits of JOLs often used a list of paired associates and asked participants to predict their future performance when providing incomplete information. ...
For successful learning, students need to evaluate their learning status relative to their learning goals and regulate their study in response to such monitoring. The present study investigated whether making metacognitive judgments on previously studied text would enhance the learning of that studied (backward effect) and newly studied text material (forward effect). We also examined how different learning goals would orient learners to adopt different study strategies and, in turn, affect learning performance by asking learners to make different types of metacognitive judgments. In two experiments, participants studied two different passages across two sections (Sections A and B). They were asked to make either inference-based or memory-based metacognitive judgments on the studied passage of Section A before studying a new passage in Section B. The study-only control group did not make any metacognitive judgments between sections. On completion of Section B, all participants were given final retention and transfer tests on both sections. The meta-analytic results from the two experiments revealed that making inference-based metacognitive judgments was more beneficial than simply studying the material for both retention and transfer of the previously studied and newly studied text material. However, the benefit of memory-based metacognitive judgments was limited, in that it did not enhance retention performance of the previously studied material compared to the control condition. The current findings suggest that the effectiveness of metacognitive judgments varies depending on the learning goal. Highlighting a high-level learning goal seems to influence learners’ all knowledge levels, showing a cascading effect.
... JoLs can be made immediately after learning or made following a delay. Delayed JoLs are shown to have greater accuracy than immediate JoLs (Rhodes & Tauber, 2011), an effect dubbed the delayed JoL effect (Nelson & Dunlosky, 1991). ...
... The following section discusses each of the three non-relational theoretical frameworks to further understand Spellman and Bjork (1992) hypothesised that when participants make a JoL, they engage in covert retrieval of the item. This theory was based on early studies demonstrating that participants attempt to retrieve targets before making a JoL about the target (Nelson & Dunlosky, 1991). built upon Spellman and Bjork's hypothesis and suggested that the retrieval opportunity could boost memory by strengthening the memory trace and improving subsequent retrieval. ...
Judgments of learning (JoLs) are predictions about the likelihood of recalling learnt material. JoLs have been a standard self-report tool in memory research for over 50 years, but recent research has observed that JoLs can affect memory in and of themselves: an effect termed JoL reactivity. JoL reactivity is typically observed in word pair experiments, in which participants who give a JoL for related word pairs (e.g., dog-cat) recall more targets than participants without a JoL. Thus, JOLs appear to improve recall for related word pairs. However, despite this finding, little is understood about when or why JoL reactivity occurs. Subsequently, this thesis provides an investigation into JoL reactivity across two papers.
The first paper provides a systematic review of the JoL reactivity literature. JoL reactivity research has grown rapidly since the last systematic review, but with contradictions in the literature: some papers report positive reactivity (improved performance), others negative reactivity (impaired performance) and some no reactivity. In addition, contrasting theoretical frameworks have been put forward to explain the mechanisms that result in JoL reactivity. The systematic literature review assesses the evidence and theoretical accounts of JoL reactivity. We observed that word pair relatedness appears to moderate the reactive effect and that there is a growing consensus that JoLs produce positive reactivity with semantically related word pairs. We also observed that relational accounts of reactivity are most common in the literature but have inconsistent evidence. There are emerging non-relational accounts, but these are tentative frameworks. Future areas for research are suggested.
The second paper investigates JoL reactivity in a transfer appropriate processing (TAP) paradigm. In an initial encoding phase, we presented participants with related, rhyming, or unrelated word pairs to induce different levels of processing. Half of the participants made a JoL after studying each word pair, while the remaining participants simply studied each word pair for an equivalent duration. Afterwards, all participants completed either standard or rhyme recognition tests. We successfully replicated the TAP effect. In the rhyme recognition test, the participants successfully recognised more rhymes of targets from the rhyming pairs than the related and unrelated pairs. However, no significant evidence of JoL reactivity was seen, regardless of encoding or test condition. The study is the first to investigate JoL reactivity using a TAP paradigm with word pairs and provides a foundation for future work to examine the role of the test on JoL reactivity and JoL reactivity in alternative paradigms.
... Albeit this revisited version is on the teachers' judgment accuracies just like Thiede and his colleagues described, it is also possible to see the learners' particular metacognitive judgments in the model. For instance, students' immediate or delayed score estimations of their test performance may provide evidence of how accurately they monitor their knowledge (Basokcu & Guzel, 2022;Prohaska, 1994;Svanum & Bigatti, 2006) just like those measured in the studies of judgments of learning ([JOL]; e.g., Nelson & Dunlosky 1991;Dunlosky & Metcalfe, 2009a;Rhodes, 2016). ...
... As we are aware, none of the previous works on teachers' JAs has tested these three factors together. Additionally, we herein defined this accuracy similar to the absolute accuracies and so measured them as the divergence between teachers' score estimations and their classes' actual scores that are obtained from a given test (see, e.g., Urhahne & Wijnia 2021 for further details on how absolute and relative JAs are measured), though JAs have been operationalised and measured somewhat variably in different kinds of literature, such as in cognitive psychology and education (Dunlosky & Metcalfe, 2009b;Nelson & Dunlosky, 1991;Thiede et al., 2015). Unlike the previous studies that measured teachers' absolute accuracies, the current study did not ask teachers to estimate the scores of a particular group of students in their classes or each student individually, but it asked teachers to estimate their whole class(es)' performance (i.e., expected 'average exam score of the class'). ...
Besides learners? awareness of their knowledge, a growing number of studies also emphasise the importance of teachers? awareness of how well their students perform to adjust their teaching strategies accordingly. Therefore, proposing a multi-layered metacognitive regulatory model in teaching first, we investigated whether estimation type, item difficulty, and class performance affect teachers? judgment accuracies ([JAs], i.e., score estimations). Teachers (N=38) of 86 classes made item-by-item and overall estimations of their classes? test scores (N=2608 sixth-graders native in Turkish) at a PISA-equivalent mathematics test that was developed in the earliest phase of the current long-term research project. The results showed that teachers? item-by-item estimations were below their classes? actual performance, unlike their overall estimations. Teachers of low-performance classes were less accurate than those of high-performance classes. These teachers also showed the clearest underestimation for the easy questions, whereas teachers of high-performance classes overestimated their classes? scores for the difficult questions. This dissociation implied that the teachers ?must have? primarily used their perceptions about their classes (e.g., classes? existing performance) as a mnemonic judgment cue rather than item difficulty as an external cue when making their score estimations. The implications of the results were discussed in the light of existing literature and suggestions for prospective research were given.
... Judgments of learning (JOLs) refers to people's predictions about the likelihood of remembering a studied item in a future memory test (Nelson & Dunlosky, 1991). Over the past few decades, numerous studies have employed JOLs to measure people's metacognitive monitoring (e.g., Besken & Mulligan, 2013;Hu et al., 2015;Rhodes & Castel, 2008;Yang et al., 2018), which is essential for learners to effectively regulate their learning activities (Bjork et al., 2013;Thiede et al., 2003). ...
Successful recognition is generally thought to be based on both recollection and familiarity of studied information. Recent studies found that making judgments of learning (JOLs) can reactively facilitate recognition performance, a form of reactivity effect on memory. The current study aimed to explore the roles of recollection and familiarity in the reactivity effect on recognition performance. Experiment 1 replicated the positive reactivity effect on recognition performance. Experiment 2 used the sequential remember/know (R/K) procedure, Experiment 3 utilized the simultaneous R/K procedure, and Experiment 4 inserted a long study-test interval (i.e., 24-h) to determine the roles of recollection and familiarity in the reactivity effect. These three experiments converged in demonstrating that making JOLs reactively facilitated recognition performance through enhancing both recollection and familiarity. Furthermore, there was minimal difference between the reactive influences on recollection and familiarity. The documented findings imply that the JOL reactivity effect on recognition is supported by two underlying mechanisms: greater recollection induced by enhanced distinctiveness, and superior familiarity induced by enhanced learning engagement.
... show lower relative accuracy than immediate JOLs (Kelemen et al., 2000;Leonesio & Nelson, 1990;Pieger et al., 2016). Furthermore, JOLs that are elicited immediately after studying each item are less accurate than JOLs elicited with a delay (Dunlosky & Nelson, 1992, 1994Nelson & Dunlosky, 1991). This is because the cues available after learning might be more diagnostic than those during learning where item information is still present in working memory. ...
There are conflicting findings regarding the accuracy of metamemory for scene pictures. Judgements of stimulus memorability in general (memorability judgements [MJs]) have been reported to be unpredictive of actual image memorability. However, other studies have found that judgements of learning (JOLs)—predictions of one’s own later memory performance for recently studied items—are moderately predictive of people’s own actual recognition memory for pictures. The current study directly compared the relative accuracy and cue basis of JOLs and MJs for scene pictures. In Experiments 1 and 2, participants completed an MJ task and a JOL task in counterbalanced order. In the MJ task, they judged the general memorability of each picture. In the JOL task, they studied pictures and made JOLs during a learning phase, followed by a recognition memory test. Results showed that MJs were predictive of general scene memorability and relied on the same cues as JOLs, but MJ accuracy considerably improved after the JOL task. Experiment 3 demonstrated that prior learning experiences drove this increase in MJ accuracy. This work demonstrates that people can predict not only their own future memory performance for scene pictures with moderate accuracy but also the general memorability of scene pictures. In addition, experiences with one’s own learning and memory support the ability to assess scene memorability in general. This research contributes to our understanding of the basis and accuracy of different metamemory judgements.
... Judgments of learning (JOLs) are metacognitive assessments where people indicate the likelihood of later remembering studied items (Dunlosky & Nelson, 1992). Although people tend to be overconfident in their future memory abilities (e.g., Koriat et al., 2004), JOLs are often well-calibrated with performance (e.g., Arbuckle & Cuddy, 1969;Dunlosky & Nelson, 1992;Nelson & Dunlosky, 1991;Rhodes & Tauber, 2011). This confidence-accuracy relationship exists for more than verbal materials: Researchers have also found that JOLs are predictive of future performance using faces as stimuli (Nomi et al., 2013;Sommer et al., 1995). ...
Although the confidence-accuracy relationship is now well established, confidence assessments are usually taken after the lineup identification procedure. Witnesses, however, often express confidence in their potential identification accuracy at other times, such as prior to seeing a lineup. Recent research has shown that these post-identification confidence statements are not consistently interpreted in the manner witnesses intend them. The present studies compare interpretations of pre-and post-identification confidence statements, and examine whether these interpretations are similarly affected by numerical statements and featural justifications. Across four studies, participants read eyewitness confidence statements and judged how confident and accurate they perceived witnesses to be. We manipulated expression type (verbal, numerical), statement type (confidence only, confidence paired with justification), and statement time (pre-and post-identification). Pre-identification confidence statements were perceived as less confident and less likely to be accurate. Unlike post-identification statements, pre-identification statements were not discounted when accompanied by featural justifications. K E Y W O R D S accuracy, confidence, confidence statements, eyewitness, perceptions
... Despite widespread use by metamemory researchers, early studies often regarded JOLs as having no direct effect on memory performance and instead focused on factors influencing their accuracy (e.g., associative direction, Koriat & Bjork, 2005;Maxwell & Huff, 2021; font size, Rhodes & Castel, 2008;JOL timing, Dunlosky & Nelson, 1994;Nelson & Dunlosky, 1991). A growing body of evidence, however, indicates that JOLs are reactive on learning, particularly when participants provide them concurrently with or immediately following study of cue-target word pairs (see Double et al., 2018, for review). ...
Judgments of learning (JOLs) are often reactive on memory for cue-target pairs. This pattern, however, is moderated by relatedness, as related but not unrelated pairs often show a memorial benefit compared to a no-JOL control group. Based on Soderstrom et al.’s, Journal of Experimental Psychology: Learning, Memory, and Cognition 41, 553-558, (2015) cue-strengthening account, JOLs direct attention towards intrinsic cues which aid retrieval. However, reactivity may also reflect specific processing of cue-target associations, which is applied whenever semantic associations are available, even when these associations are indirect. The present study tested this possibility using mediated associates (e.g., lion – stripes) which are directly unrelated to each other and indirectly related through a non-presented mediator (e.g., tiger). Based on a cue-strengthening account, no reactivity would be expected for mediated associates. Alternatively, if cue strengthening primarily reflects enhanced processing of cue-target relations, memory benefits would be expected whenever pairs are semantically related, even if pairs are indirectly related through mediators. Overall, reactivity extended to mediated associates in cued-recall (Experiment 1) and recognition tests (Experiments 2 and 3). Interestingly, JOL reactivity was consistently found on recognition of non-mediated unrelated pairs (Experiments 2–4). Thus, positive reactivity on related pairs for cued-recall testing likely reflects increased activation of cue-target associations. However, because recognition is based on familiarity cues, reactivity occurs globally for all pair types, regardless of cue-target relations.
... Retrieval practice is a crucial aspect in learning and integrating retrieval practice during study sessions reduces underconfidence in JOLs thereby helping to choose or move to other to-be-learned items accurately (Kubik et al., 2022). Additionally, having people assess their subsequent memory after a retrieval attempt often improves one's insight into subsequent memory accuracy (Dunlosky & Nelson, 1994;Metcalfe & Finn, 2008;Nelson & Dunlosky, 1991, Van Overschelde & Nelson, 2006 for a review, see . One type of metamemory judgment that accomplishes this goal includes FOK judgments. ...
Metamemory judgments, defined as predictions of memory performance, are often influenced by misleading cues, such as fluency. However, how fluency cues compete to influence retrospective metamemory judgments is still unclear. The present study investigated how multiple fluency cues concurrently influence immediate feeling of knowing (FOK) judgments with two fluency manipulations—font size (large vs. small font size) as a perceptual cue and level of processing (deep vs. shallow processing) as a conceptual cue. In Experiment 1, participants studied large or small unrelated word pairs and were either directed to process the conceptual aspects of each word pair (deep) or to focus on the perceptual aspects of the word pairs (shallow). Then participants were presented with a cued recall test and asked to make an FOK judgment. Lastly, participants received a five alternative- forced-choice recognition test. Experiment 2 was similar except the deep condition was replaced with a no-processing (no instruction) condition. Results revealed that perceptual fluency (large font size) influenced FOK judgments only when word pairs were processed in the shallow condition in both experiments compared to no-processing condition. This interaction of multiple cues suggests that, participants rely on information which is easily accessible to them (perceptual fluency) for FOK judgements in presence of certain secondary cues despite those cues being less diagnostic of future memory performance. These new insights inform how people integrate different sources of information in metamemory decisions and have broad implications for settings including academic learning and everyday decision making.
... For instance, delayed predictions in LTM tasks are more accurate than immediate predictions [16][17][18] and vividness ratings of WM content are more affected by dual tasks when stimuli are nonsensical (i.e., when the information is not consolidated in LTM) 19 . Moreover, active manipulation of WM contents can impair metacognition for perceptual tasks 20 , and clarity of content in visual WM can impact metacognitive parameters 21 . ...
Although the study of metamemory monitoring originated in predictions for simple span tasks, the study of metacognition for working memory (WM) has been somewhat neglected in comparison with long-term memory. We aimed to fill this gap by exploring the ability to self-assess WM operations. Thirty-four participants performed 16 series of complex span tasks and rated their confidence in a verbal recall paradigm. We manipulated the cognitive load based on the TBRS model in order to analyze the role of attentional resources on both WM and metacognitive evaluations. As expected, we found that recall is affected by cognitive load and we found standard serial position effects. Interestingly, metacognitive evaluations followed the same pattern, and measures of metacognitive sensitivity suggest that participants are able to make item-by-item retrospective judgments reflective of their performance. We discuss how these results contribute to our understanding of metacognitive access to newly-formed WM contents.
... Specifically, participants' predictions about their fact memory were more closely differentiating actually remembered and actually forgotten facts when they had an opportunity to self-test rather than just re-read the facts. This may allow them to efficiently allocate more study effort to notyet-mastered items (Nelson and Dunlosky, 1991;Metcalfe and Kornell, 2005;Metcalfe and Finn, 2008a;Little and McDaniel, . /fcogn. . ...
Introduction
Is retrieval practice always superior to restudy? In a classic study by Roediger and Karpicke, long-term retention of information contained in prose passages was found to be best when opportunities to restudy were replaced with opportunities to self-test. We were interested whether this striking benefit for repeated testing at the expense of any restudy replicates when study opportunities are brief, akin to a single mention of a fact in an academic lecture. We were also interested in whether restudying after a test would provide any additional benefits compared to restudying before test.
Method
In the current study, participants encountered academically relevant facts a total of three times; each time either studied (S) or self-tested (T). During study, participants predicted how likely they were to remember each fact in the future. During self-test, participants performed covert cued recall and self-reported their recall success. Final test followed immediately or after a delay (Experiment 1: 2 days, Experiment 2: 7 days).
Results
Contrary to prior work, long-term memory was superior for facts the were restudied in addition to self-tested (SST > STT = SSS). We further investigated whether restudy after a test (STS) provides additional benefits compared to restudy before test (SST). Restudying after a retrieval attempt provided an additional benefit compared to restudying before a retrieval attempt on an immediate test, but this benefit did not carry over a delay. Finally, exploratory analyses indicated that restudy after test improved the accuracy of participants' subjective predictions of encoding success.
Discussion
Together, our results qualify prior work on the benefits of repeated testing, indicating that balancing testing with repetition may allow for more information to be learned and retained. These findings offer new insights into the conditions that promote encoding and long-term retention, provide new constraints for existing cognitive theories of testing effects, and have practical implications for education.
... At the same time, metacognitive evaluations are prone to errors (Dunlosky & Lipko, 2007;Dunning et al., 2004), and metacognitive accuracy might be comparably low without specific instructional practices aiming to improve accuracy. For example, summarising, re-reading, retrieval practice or delaying time between study phase and metacognitive evaluation (delayed judgment of learning effect) lead to an increase in metacognitive accuracy (Miller & Geraci, 2014;Nelson & Dunlosky, 1991;Rawson et al., 2000;Thiede et al., 2003;Thiede et al., 2005). In detail, delaying judgment intervals and retention time, matching judgment of learning items with test questions, and applying cued recall tasks in learning tests are associated with higher metacognitive accuracy (Rhodes & Tauber, 2011). ...
Metacognitive accuracy is understood as the congruency of subjective evaluation and objectively measured learning performance. With reference to the cue utilisation framework and the embedded-processes model of working memory, we proposed that prompts impact attentional processes during learning. Through guided prompting, learners place their attention on specific information during the learning process. We assumed that the information will be taken into account when comprehension judgments are formed. Subsequently, metacognitive accuracy will be altered. Based on the results of this online study with pre-service biology teachers, we can neither confirm nor reject our main hypothesis and assume small effects of prompting on metacognitive accuracy if there are any. Learning performance and judgment of comprehension were not found to be impacted by the use of resource- and deficit-oriented prompting. Other measurements of self-evaluation (i.e. satisfaction with learning outcome and prediction about prolonged comprehension) were not influenced through prompting. The study provides merely tentative evidence for altered metacognitive accuracy and effects on information processing through prompting. Results are discussed in light of online learning settings in which the effectiveness of prompt implementation might have been restricted compared to a classroom environment. We provide recommendations for the use of prompts in learning settings with the aim to facilitate their effectiveness, so that both resource-oriented and deficit-oriented prompts can contribute to metacognitive skill development if they are applied appropriately.
... This is typically expressed by indicating the degree of confidence in the first-order decision. Metacognition is fundamental for humans to adapt to changes in the environment and to update their internal representations of the world (Nelson & Dunlosky, 1991). While individuals' metacognitive ability to make perceptual decisions has been studied extensively (Ais, Zylberberg, Barttfeld, & Sigman, 2016;De Gardelle, Le Corre, & Mamassian, 2016;Deroy et al., 2016;Fleming, Huijgen, & Dolan, 2012), little is known about the metacognition of sense of agency. ...
Intelligent agents need to understand how they can change the world, and how they cannot change it, in order to make rational decisions for their forthcoming actions, and to adapt to their current environment. Previous research on the sense of agency, based largely on subjective ratings, failed to dissociate the sensitivity of sense of agency (i.e., the extent to which individual sense of agency tracks actual instrumental control over external events) from judgment criteria (i.e., the extent to which individuals self-attribute agency independent of their actual influence over external events). Furthermore, few studies have examined whether individuals have metacognitive access to the internal processes underlying the sense of agency. We developed a novel two-alternative-forced choice (2FAC) control detection task, in which participants identified which of two visual objects was more strongly controlled by their voluntary movement. The actual level of control over the target object was manipulated by adjusting the proportion of its motion that was driven by the participant's movement, compared to the proportion driven by a pre-recorded movement by another agent, using a staircase to hold 2AFC control detection accuracy at 70%. Participants identified which of the two visual objects they controlled, and also made a binary confidence judgment regarding their control detection judgment. We calculated a bias-free measure of first-order sensitivity (d') for detection control at any given level of participant's own movement. The proportion of pre-recorded movements determined by the stairecase could then be used as an index of control detection ability. We identified two distinct processes underlying first-order detection of control: one based on instantaneous sensory predictions for the current movement, and one based on detection of a regular motor-visual relation across a series of movements. Further, we found large individual differences across 40 particpants in metacognitive sensitivity (meta-d') even though first-order sensitivity of control detection was well controlled. Using structural equation modelling (SEM), we showed that metacognition was negatively correlated with the predictive process component of detection of control. This result is inconsistent with previous hypotheses that detection of control relies on metacognitive monitoring of a predictive circuit. Instead, it suggests that predictive mechanisms that compute sense of agency may operate unconsciously.
... Methodologists (e.g., Juslin et al., 1996;Lichtenstein & Fischhoff, 1977;Murphy, 1973;Nelson, 1996;Nelson & Dunlosky, 1991;Schraw, 2009;Yates, 1982) have delineated how monitoring accuracy can be assessed in terms 16 Fraundorf et al. Cognitive Research: Principles and Implications (2023) 8:58 of both calibration and resolution. ...
Is self-assessment enough to keep physicians' cognitive skills-such as diagnosis, treatment, basic biological knowledge, and communicative skills-current? We review the cognitive strengths and weaknesses of self-assessment in the context of maintaining medical expertise. Cognitive science supports the importance of accurately self-assessing one's own skills and abilities, and we review several ways such accuracy can be quantified. However, our review also indicates a broad challenge in self-assessment is that individuals do not have direct access to the strength or quality of their knowledge and instead must infer this from heuristic strategies. These heuristics are reasonably accurate in many circumstances, but they also suffer from systematic biases. For example, information that feels easy to process in the moment can lead individuals to overconfidence in their ability to remember it in the future. Another notable phenomenon is the Dunning-Kruger effect: the poorest performers in a domain are also the least accurate in self-assessment. Further, explicit instruction is not always sufficient to remove these biases. We discuss what these findings imply about when physicians' self-assessment can be useful and when it may be valuable to supplement with outside sources.
... Goodman-Kruskal gamma correlations were computed to assess the rank-order relationship between performance and confidence (relative metacognitive accuracy; see Nelson 1984). It is common in metamemory research to calculate gamma coefficients for each participant and then to calculate central tendency (means or medians) of those coefficients across participants to assess relative metacognitive accuracy (e.g., Dunlosky and Nelson 1992;Koriat 2008bKoriat , 2019Nelson and Dunlosky 1991). Coefficients were calculated for each participant for the basic condition, NOTA condition, and for the combination of the two conditions (i.e., the critical items). ...
Students claim that multiple-choice questions can be tricky, particularly those with competitive incorrect choices or choices like none-of-the-above (NOTA). Additionally, assessment researchers suggest that using NOTA is problematic for assessment. In experiments conducted online (with trivia questions) and in the classroom (with course-related questions), I investigated the effects of including NOTA as a multiple-choice choice alternative on students’ confidence and performance. In four experiments, participants answered two types of questions: basic multiple-choice questions (basic condition) and equivalent questions in which one incorrect choice was replaced with NOTA (NOTA condition). Immediately after answering each question, participants rated their confidence in their answer to that question (item-by-item confidence). At the end of the experiments, participants made aggregate confidence judgments for the two types of questions and provided additional comments about the use of NOTA as an alternative. Surprisingly, I found no significant differences in item-by-item confidence or performance between the two conditions in any of the experiments. However, across all four experiments, when making aggregate judgments, participants provided lower confidence estimates in the NOTA condition than in the basic condition. Although people often report that NOTA questions hurt their confidence, the present results suggest that they might not—at least not on a question-by-question basis.
... This assumption might be supported by research addressing the delayed JOL effect. Immediate JOLs are less accurate than delayed JOLs by showing less agreement with subsequent performance (Nelson & Dunlosky, 1991). That is, delayed judgments of learning are more likely to match subsequent recognition than immediate judgments of learning. ...
When young children evaluate their confidence, their monitoring is often overoptimistic, that is, inaccurate. The present study investigated a potential underlying mechanism for kindergarteners’ and second graders’ overconfidence within a paired associates learning paradigm. We implemented a pre-monitoring phase motivating children to differentially evaluate their confidence for each alternative before children could choose an answer in the subsequent recognition phase. For one, we intended to weaken the influence of one single and prepotently selected memory trace. For another, we motivated and enabled children to evaluate all four answer alternatives concerning their certainty before evaluating their final recognition choice by giving a confidence judgment. We compared monitoring discrimination and monitoring bias with a control condition whose task sequence did not include a pre-monitoring judgment. Contrary to our expectations, the pattern of results indicated that being instructed to pre-monitor did increase and not decrease overconfidence in young children. The present results will be discussed against the background of memory-metamemory interaction, confirmation bias, and methodological issues.
... Therefore, the accuracy of JOLs is of critical importance for being a successful learner. However, previous studies have shown that JOL accuracy is far from perfect (Nelson and Dunlosky 1991;Yang et al. 2023). Inaccurate JOLs may lead to inadequate learning of course materials in the case of stopping learning prematurely, and may also lead to poor learning efficiency when individuals unnecessarily expend extra efforts toward processing well-mastered materials. ...
Research has demonstrated that metacognition accuracy is far from perfect. The accuracy of judgments of learning (JOLs) is of critical importance in self-regulated learning. To explore what factors constrain JOL accuracy, the current study focused on mindfulness, which is intimately related to metacognition and anxiety. A total of 203 undergraduates (198 valid samples) were recruited to determine the relationships among five dimensions of dispositional mindfulness, test anxiety, and relative accuracy of JOLs. Results revealed that the interaction term for acting with awareness and test anxiety significantly predicted JOL accuracy. Further analyses indicated that for individuals with high test anxiety, but not for those with low test anxiety, acting with awareness positively predicted JOL accuracy. Considering that dispositional mindfulness is modifiable, these results help to inspire researchers to further explore whether mindfulness training can be used as a remedy to improve JOL accuracy.
... However, when making JOLs after a delay, participants do not have the target immediately available to them and have to retrieve it from long-term memory to make the judgment. Since participants also have to retrieve targets at test, the ease with which information is retrieved during the delayed JOL leads to more accurate judgments of subsequent memory (Nelson & Dunlosky, 1991). Accordingly, delayed ...
The generation effect refers to the finding that self-generated material (e.g. h_rse) is better remembered than material that is passively read (e.g. horse). Using a two-block mixed-list paradigm, we replicate previous research showing that the recall advantage for generated words in the first block is attenuated in the second block (Experiment 1), suggesting that participants change processing strategies for read words in the second block (i.e., generation-potentiated learning). To assess metacognitive awareness of this effect, we measured immediate versus delayed judgments of learning (Experiment 2), restudy decisions (Experiments 3A and 3B), and study duration (Experiments 4A and 4B). The results showed a dissociable pattern, such that immediate measures were influenced by ease of processing (i.e., read items were given higher judgments of learning, chosen to restudy less often, and studied for less time) while delayed measures were influenced by ease of retrieval (i.e., generated items were given higher JOLs, chosen to restudy less often, and studied for less time). However, additional study of read words in the delayed condition did not eliminate the generation advantage in the first block, suggesting a labor-in-vain. These findings suggest that control decisions are more effective when based on information diagnostic of subsequent memory (i.e., ease of retrieval), but that generation-potentiated learning may not occur until after the first test.
... There is some evidence from basic metamemory research indicating that the timing of metamnemonic assessments is important. The delayed-JOL effect describes the finding that JOLs are more predictive of accurate remembering when the solicitation is delayed after stimulus presentation, compared to when they are solicited immediately after presentation (Dunlosky & Nelson, 1992, 1994Nelson & Dunlosky, 1991). Some have argued that the act of providing a JOL invites retrieval practice, and that learners assign JOLs based on the success of that retrieval attempt (cf. ...
After a crime is committed, investigators may query witnesses about whether they believe they will be to identify the perpetrator. However, we know little about how such metacognitive judgments are related to performance on a subsequent lineup identification task. The extant research has found the strength of this relationship to be small or nonexistent, which conflicts with the large body of literature indicating a moderate relationship between predictions and performance on memory tasks. In Studies 1-3, we induce variation in encoding quality by having participants watch a mock crime video with either low, medium, or high exposure quality, and then assess their future lineup performance. Calibration analysis revealed that assessments of future lineup performance were predictive of identification accuracy. This relationship was driven primarily by poor performance following low assessments. Studies 4 and 5 showed that these predictions are not based on a witness's evaluation of their encoding experience, nor on a contemporaneous assessment of memory strength. These results reinforce the argument that variation in memory quality is needed to obtain reliable relationships between predictions and performance. An unexpected finding is that witnesses who made a prediction shortly after encoding evinced superior memory compared to those who made a prediction later.
... Metacognition needs to be prompted/ scaffolded. Nelson and Dunlosky (1991): judgements of learning for word pairs. ...
Most students fail to spontaneously adopt effective study behaviours in their self-regulated learning. This may in part be due to flawed metacognition, including misunderstandings of which techniques will best support longer-term retention and transfer of learning. Evidence-based techniques such as spacing out practice, variation and retrieval are especially misunderstood. In this session, I will explain why misconceptions arise among students, and discuss powerful ways of supporting them towards better metacognition and more effective study habits.
... Judgment of learning (JOL) is a predictive judgment about the performance of a learned item in subsequent tests (Nelson and Dunlosky 1991). As an important form of metacognitive monitoring, the accuracy of JOL plays an essential role in determining whether individuals can learn efficiently (Koriat et al. 2006;Metcalfe and Finn 2008;Metcalfe 2009). ...
Delayed judgment of learning (JOL) is a widely used metacognitive monitoring strategy that can also enhance learning outcomes. However, the potential benefits of delayed JOL on subsequent learning of new material, known as the forward effect of delayed JOL, and its stability and underlying mechanisms have yet to be fully explored. In this study, we investigated the forward effect of delayed JOL using previously unexamined word pair materials and explored the boundary conditions of this effect by manipulating the difficulty of the materials. We also examined this effect within the context of category learning. Our findings demonstrate that delayed JOL significantly enhanced the retention of new information (Experiment 1A), while the forward effect of the delayed JOL occurred only for material with a certain degree of difficulty rather than for easy material (Experiment 1B). These findings were extended and replicated using category learning (Experiment 2). These results suggest that delayed JOL can be used as a preparation strategy for subsequent learning, particularly when faced with challenging materials. Our study provides novel insights into the potential benefits and limitations of delayed JOL and contributes to our understanding of the underlying mechanisms that govern metacognitive monitoring and learning strategies.
... Interestingly, if judgments of learning are made immediately upon presentation of to-be-learned cuetarget pairs, and those pairs are still available to be read by the subject, many targets that will not be later remembered later are falsely evaluated as being learned. It seems that if the item is there, in front of the subject (or if it is still in short-term memory; Nelson & Dunlosky, 1993), the subject cannot resist thinking that it is known. This failure of metacognition, whereby one thinks one knows something that one does not know when it is physically present or is in working memory, does not occur (or, at least, not as badly) if the answers are not provided and physically present at the time of judgment (Dunlosky & Nelson, 1992). ...
Students regularly ask, “How can I do well in your course?” They are surprised when I provide a simple answer: Take advantage of the quizzes. Quizzes are not a silver bullet, but they improve students’ recollection of course information and, importantly to students, increase performance on exams. Pre-lecture reading quizzes encourage students to arrive prepared (pre-training), ongoing quizzes promote
regular studying (spacing), and review quizzes help students revisit material from previous topics (interleaving). Central to the present discussion, all of these types of quizzes require students to retrieve information to answer items, which improves performance on later exams (testing effect, retrieval practice). Still, questions remain about how to use quizzes most effectively. In particular, should we use
harder application quizzes or easier factual quizzes to help students do well in the course? That is to say, should we throw students in the deep end early in the learning process or not?
... The concept of perceived learning is defined as learners' monitoring judgments of their current level of learning. These judgments can be measured in many ways such as through Judgments of Learning (Nelson & Dunlosky, 1991), predictions of performance (van Loon et al., 2014), and feeling of knowing (FOK; Koriat, 1993). Within the realm of desirable difficulties, these monitoring judgments are usually made during encoding and before retrieval (e.g., when studying a learning task), but in the context of retrieval practice are also made post-retrieval. ...
Desirable difficulties are learning conditions that are often experienced as effortful, but have a positive effect on learning results and transfer of knowledge and skills (Bjork & Bjork, 2011; Bjork, 1994). Learners often do not appreciate the beneficial effects of desirable difficulties, and the negative experiences of high effort and perceived low learning make them resistant to engage in desirable difficulties (Biwer et al., 2020a). This ultimately limits learning outcomes and academic achievement. With the increasing emphasis on self-regulation in education, characterized by higher learner agency and abundant choices in what, when, and how to study, the field of educational psychology is in need of theoretical and empirically testable assumptions that improve self-regulation in desirably difficult learning conditions with the aim to foster self-regulation abilities, learning outcomes, and academic achievement. Here, we present a framework that describes how to support self-regulation of effort when engaging in desirable difficulties: the “Start and Stick to Desirable Difficulties (S2D2)” framework. The framework builds on the Effort Monitoring and Regulation model (de Bruin et al., 2020). The aim of this framework is (1) to describe evidence for the central role of perceived effort and perceived learning in (dis)engagement in desirable difficulties, and (2) to review evidence on, and provide an agenda for research to improve learners’ self-regulated use of desirable difficulties to help them start and persist when learning feels tough, but is actually effective.
... accuracy and the robustness of the item-level measurements. As a comparison to fMRI measures of encoding, we additionally ask participants to provide subjective "judgments of learning" (JOLs), a common behavioral measure related to encoding confidence and metacognition (76)(77)(78), for each studied word pair. Finally, participants are tested for their cued recall performance 3 d after the initial encoding, ensuring that we are measuring signals of durable memory encoding. ...
Over 40 y of accumulated research has detailed associations between neuroimaging signals measured during a memory encoding task and later memory performance, across a variety of brain regions, measurement tools, statistical approaches, and behavioral tasks. But the interpretation of these subsequent memory effects (SMEs) remains unclear: if the identified signals reflect cognitive and neural mechanisms of memory encoding, then the underlying neural activity must be causally related to future memory. However, almost all previous SME analyses do not control for potential confounders of this causal interpretation, such as serial position and item effects. We collect a large fMRI dataset and use an experimental design and analysis approach that allows us to statistically adjust for nearly all known exogenous confounding variables. We find that, using standard approaches without adjustment, we replicate several univariate and multivariate subsequent memory effects and are able to predict memory performance across people. However, we are unable to identify any signal that reliably predicts subsequent memory after adjusting for confounding variables, bringing into doubt the causal status of these effects. We apply the same approach to subjects' judgments of learning collected following an encoding period and show that these behavioral measures of mnemonic status do predict memory after adjustments, suggesting that it is possible to measure signals near the time of encoding that reflect causal mechanisms but that existing neuroimaging measures, at least in our data, may not have the precision and specificity to do so.
... In contrast, learners' memory performance in Experiment 2 exhibited improvements over their performance in Experiment 1 because learners covertly attempted to retrieve the targets by making delayed cue-only JOLs (Nelson & Dunlosky, 1991). In Experiment 2, even if the monitoring accuracy was not affected by the test without feedback, the delayed GJOL for the test without feedback was significantly greater than that for the test with feedback. ...
Feedback helps facilitate learning but impairs future prediction. Previous studies have revealed that learners appreciate feedback correcting misinformation less when they receive feedback for their mistakes than in tests without feedback. These studies have noted that judgment after feedback relies on memory for the past test (MPT). In contrast, interference-perseveration theory has indicated that immediate feedback following incorrect answers leads to proactive interference that impedes the acquisition of feedback information. In addition, the current study proposes that this proactive interference influences learners’ judgments when making future predictions. In Experiment 1, we instructed learners to judge their ability to predict feedback based on the global judgment of learning (GJOL), whereas in Experiment 2, we asked learners to delay the global judgment of learning (delayed GJOL) and thus not to base that judgment on the recent test. However, in both Experiments 1 and 2, learners’ predictions regarding their performance in the feedback condition to be lower than their actual memory performance and did not appreciate the benefits of such feedback. However, learners can restore their awareness of their actual memory performance and thus of the benefits of the feedback after taking a true final test. This finding indicates that learners overcome proactive interference because they might forget their mistakes when making postdictions after taking the final test and might thus subjectively be aware of the fact that the feedback they receive facilitates their learning. The general discussion section presents possible reasons for these findings and highlights the theoretical contributions made by this study.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
Memorability, or the likelihood that an image is later remembered, is an intrinsic stimulus property that is remarkably consistent across viewers. Despite this consistency in what people remember and forget, previous findings suggest a lack of consistency in what individuals subjectively believe to be memorable and forgettable. We aimed to improve the ability of participants to judge memorability using a feedback-based training paradigm containing face images (Experiment 1) or scene images (Experiment 2 and its replication and control experiments). Overall, participants were fairly accurate at categorizing the memorability of images. In response to the training, participants were able to improve their memorability judgments of scenes, but not faces. Those who used certain strategies to perform the task, namely relying on characteristic features of the scenes, showed greater learning. Although participants improved slightly over time, they never reached the level of ResMem, the leading DNN for estimating image memorability. These results suggest that with training, human participants can better their understanding of image memorability, but may be unable to access its full variance.
This study investigates when and how awareness of knowledge gaps (AKG) manifests by observing the problem-solving phase of the educational approach known as problem-solving followed by instruction (PS-I). By comprehensively exploring cognitive and meta-cognitive process of learners during this phase and categorizing students’ judgements of knowledge structure in relation to AKG, it strengthens the underlying mechanisms of PS-I. With sixteen university students as participants, this study quantitatively and qualitatively analyzes conversations that take place during problem-solving activities. In the analysis, the authors suggest a total of ten cognitive and metacognitive events that occur and six judgements of knowledge structure in relation to AKG. The findings indicate that students spend most of their time solving the problem and seldom evaluate their thoughts; few express awareness of a knowledge gap. The authors discuss the relationships between the judgements of knowledge structure and consider when—and to what extent—students perceive their knowledge gaps. Lastly, the authors bring four learning behaviors (i.e., rep-resenting and reflecting on knowledge; recognizing and specifying knowledge gaps) with possible instructional strategies to promote each learning behavior.
For decades, research on metacomprehension has demonstrated that many learners struggle to accurately discriminate their comprehension of texts. However, while reviews of experimental studies on relative metacomprehension accuracy have found average intra-individual correlations between predictions and performance of around .27 for adult readers, in some contexts even lower near-zero accuracy levels have been reported. One possible explanation for those strikingly low levels of accuracy is the high conceptual overlap between topics of the texts. To test this hypothesis, in the present work participants were randomly assigned to read one of two text sets that differed in their degree of conceptual overlap. Participants judged their understanding and completed an inference test for each topic. Across two studies, mean relative accuracy was found to match typical baseline levels for the low-overlap text sets and was significantly lower for the high-overlap text sets. Results suggest text similarity is an important factor impacting comprehension monitoring accuracy that may have contributed to the variable and sometimes inconsistent results reported in the metacomprehension literature.
Accurate monitoring is important, as it guides subsequent control of learning. With multimedia, learners often overestimate their learning. Retrieval practice provides insight into one’s understanding; still, it might not improve monitoring. In an online experiment with two groups, participants (N = 146) performed retrieval practice while studying multimedia instructions. We investigated whether monitoring prompts affect metacognitive accuracy, subsequent learning behavior, and performance: One group repeatedly received prompts, whereas a control group did not. After studying, both groups provided metacognitive judgments for a subsequent posttest. As expected, learners in the control group were overconfident about their learning; however, unexpectedly, prompted learners were also overconfident and did not differ from the control group. Nevertheless, prompted learners provided longer answers to retrieval tasks, which increased their posttest performance (indirect mediation). We observed no mediation for learning time or retrieval performance, although retrieval performance did relate to posttest performance. The results suggest that monitoring prompts do not improve metacognitive accuracy per se but might improve learning outcomes by controlling subsequent learning.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The Cambridge Handbook of Consciousness is the first of its kind in the field, and its appearance marks a unique time in the history of intellectual inquiry on the topic. After decades during which consciousness was considered beyond the scope of legitimate scientific investigation, consciousness re-emerged as a popular focus of research towards the end of the last century, and it has remained so for nearly 20 years. There are now so many different lines of investigation on consciousness that the time has come when the field may finally benefit from a book that pulls them together and, by juxtaposing them, provides a comprehensive survey of this exciting field. An authoritative desk reference, which will also be suitable as an advanced textbook.
The accuracy of students’ relative comprehension judgments when reading texts is typically rather low. This has been ascribed to students grounding their comprehension judgments on cues that are not diagnostic of their actual comprehension level. Asking students to complete causal diagrams—a diagramming scaffold—before judging comprehension has proved effective in providing them with more diagnostic cues and thereby fostered metacomprehension accuracy and self-regulated learning. However, there is still room for improvement. We investigated experimentally whether adding the instruction to students to self-assess their causal diagrams: (1) would lead to more accurate judgments than comprehension judgments, (2) would boost their utilization of diagnostic diagram cues by increasing the saliency of those cues, and (3) would enhance metacomprehension accuracy. Participants (N = 427 secondary students in The Netherlands) were randomly assigned to one of three conditions, namely (1) only diagram completion, (2) diagram completion plus diagram self-assessment, or a (3) filler task after reading (control). Self-assessments were more accurate than comprehension judgments, while both correlated strongly. However, no significant differences were found between diagramming conditions concerning diagram cue utilization and metacomprehension accuracy. Apparently, students self-assess their diagrams even without instruction to do so. Nonetheless, the effect of the diagramming scaffold for improving relative metacomprehension accuracy was replicated and extended to absolute metacomprehension accuracy.
Introduction
Metacognitive monitoring ability enables you to learn and solve problems more efficiently through appropriate strategies. At the same time, those who are high in monitoring ability are known to allocate more cognitive resources to the perception and control of negative emotions, as compared to those with low metacognitive ability. Therefore, while monitoring emotions may help reduce the negative emotion by enabling efficient control, it could also interrupt the use of an efficient strategy when problem-solving, as cognitive resources may be depleted.
Methods
To confirm this, we divided participants into groups with high and low monitoring abilities and manipulated emotions by presenting emotional videos. Subsequent to the manipulation, problem solving strategies were examined using items from the Cognitive Reflection Test (CRT).
Results
Results showed that those who were high in monitoring ability were shown to use more efficient problem-solving strategies than those who were lower in monitoring ability, but only in situations when positive or no emotions were manipulated. However, as hypothesized, when negative emotion was aroused, the CRT scores of high monitoring ability group were significantly lowered, decreasing to the same performance as those with low monitoring ability. We also found that metacognitive monitoring ability, when interacting with emotion, indirectly affected CRT scores, and that monitoring and control, when affected by emotion, were mediated in the process.
Discussion
These findings suggest a novel and complicated interaction between emotion and metacognition and warrant further research.
Although the study of metamemory monitoring originated in predictions for simple span tasks, the study of metacognition for working memory (WM) has been somewhat neglected in comparison with long-term memory. We aimed to fill this gap by exploring the ability to self-assess WM operations. Thirty-four participants performed 16 series of complex span tasks and rated their confidence in a verbal recall paradigm. We manipulated the cognitive load (CL) based on the TBRS model in order to analyze the role of attentional resources on both WM and metacognitive evaluations. As expected, we found that recall is affected by cognitive load and we found standard serial position effects. Interestingly, metacognitive evaluations followed the same pattern, and measures of metacognitive sensitivity suggest that participants are able to make item-by-item retrospective judgements reflective of their performance. We discuss how these results contribute to our understanding of metacognitive access to working memory operations.
Required 18 female undergraduates to memorize a list of 30 words (LT set) before participating in a recognition memory task. Test sessions were divided into pure and mixed trial blocks. During pure blocks, 1-4 words (ST set) were presented to S on each trial immediately before exposure of the test stimulus. The S made a positive response if the test stimulus was from the ST set and a negative response if it was a distractor. During mixed blocks the procedure was similar except that test stimuli were also drawn from the LT set; the S made a positive response if the test stimulus was from either the ST set or the LT set, and a negative response otherwise. Results indicate that in the mixed blocks reaction time to positive test stimuli from the LT set or to negative test stimuli did not depend on ST set size; for all other test stimuli, reaction time was an increasing function of ST set size. Findings are consistent with the view that Ss can search a long-term store and a short-term store simultaneously. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Proposes that verbal reports are data and that accounting for them, as well as for other kinds of data, requires explication of the mechanisms by which the reports are generated, and the ways in which they are sensitive to experimental factors (instructions, tasks, etc). Within the theoretical framework of human information processing, different types of processes underlying verbalization are discussed, and a model is presented of how Ss, in response to an instruction to think aloud, verbalize information that they are attending to in short-term memory (STM). Verbalizing information is shown to affect cognitive processes only if the instructions require verbalization of information that would not otherwise be attended to. From an analysis of what would be in STM at the time of report, the model predicts what could be reliably reported. The inaccurate reports found by other research are shown to result from requesting information that was never directly heeded, thus forcing Ss to infer rather than remember their mental processes. (112 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
We compared the predictions from several kinds of metamemory judgments (on the same set of items), both in terms of their predictive accuracy and in terms of the commonality of predictions. Undergraduates made judgments about the ease with which they could learn each item in a list (ease-of-learning judgments); then they learned every item, either to a minimal criterion of learning or with overlearning, and made judgments about how well they knew each item (judgments of knowing); finally, they returned 4 weeks later for a retention session and made feeling-of-knowing judgments on every time they could not recall, after which a recognition test assessed predictive accuracy. Ease-of-learning judgments had the least predictive accuracy. Surprisingly, however, the recognition of nonrecalled items was predicted equally well by judgments of knowing (made 4 weeks earlier) as by feeling-of-knowing judgments (made immediately prior to recognition). Moreover, those two kinds of judgments were only weakly correlated with each other, which implies that they do not tap memory in the same way.
This research explored the possibility that a metacognitive control process (namely, the allocation of self-paced study time) might be affected by the output from metacognitive monitoring processes (i.e., ease-of-learning and/or feeling-of-knowing judgments). In three experiments, university undergraduates received instructions that emphasized either accuracy of learning or speed of learning. The major findings were: (a) ease-of-learning judgments and feeling-of-knowing judgments are reliably related to study-time allocation, with more self-paced study time being allocated to the supposedly more difficult items; (b) even when instructed to master every item and when allowed unlimited study time to do so, people terminate study before learning is completed; and (c) large increases in self-paced study time can yield little or no increase in the subsequent likelihood of recall (the "labor-in-vain effect"). Implications are drawn for a model of the interplay between metacognitive monitoring processes and metacognitive control processes.
Discusses 8 quantitative measures of feeling-of-knowing accuracy that have been used in the literature. The 3 measures considered at length are J. T. Hart's difference score, L. A. Goodman and W. H. Kruskal's (1954) gamma correlation, and the phi correlation. Quantitative relations between these measures are reported, as are connections with some basic axioms and a probabilistic conception of feeling-of-knowing accuracy. The currently most popular measure, the Hart difference score, has serious shortcomings. The Goodman-Kruskal gamma seems to be best. The remaining measures are inappropriate for the available feeling-of-knowing data for a variety of reasons. Also discussed are the implications of these results for other situations in which ordered 2 × 2 tables are examined to determine the relationship between predictions and criterion performance. (60 ref)
The investigation is concerned with individual items instead of lists. "Forgetting over intervals measured in seconds was found. The course of retention after a single presentation was related to a statistical model. Forgetting was found to progress at differential rates dependent on the amount of controlled rehearsal of the stimulus. A portion of the improvement in recall with repetitions was assigned to serial learning within the item, but a second kind of learning was also found." (PsycINFO Database Record (c) 2006 APA, all rights reserved).
This chapter focuses on research program, providing a description of a theoretical framework that has evolved out of metamemory research, followed by a few remarks about the methodology. Research in metamemory is initiated by the paradoxical findings that people can accurately predict their subsequent likelihood of recognizing nonrecallable items and that they can quickly and accurately decide-on the basis of no more than a cursory search through memory-that they will not retrieve particular sought after items. Those findings lead to develop a methodology based on psychophysical methods that are used to empirically investigate people's feeling of knowing. The results of the experiments convinced that for dealing with only a part of a complex metacognitive system and to account adequately for feeling-of-knowing phenomena, a larger perspective was needed. This eventuated in the present theoretical framework that emphasizes the role of control and monitoring processes. The embedding of the feeling of knowing in a richer framework helped to dissipate the paradoxical nature of the feeling of knowing. The chapter discusses that today there are many capable, active investigators and a wealth of solid empirical findings.
First-, third-, fifth-grade children, and college students (ages 6, 8, 10, and 19 years) acquired a paired-associate list under the study-test procedure to a one error-less trial criterion. In addition, as each pair was presented the individual indicated whether he/she had that pair correct on the immediately preceding trial (postdiction responses). The data were interpreted in terms of a discrimination-utilization hypothesis which postulates that individuals discriminate their own correct and incorrect responses on a given trial and use this information for distributing processing effort on the subsequent trial. Analyses involving the accuracy of postdiction responses, the relation of postdiction accuracy to acquisition, and the consideration of the acquisition data in terms of a two-stage model led to the conclusion that older children and adults may use the discrimination-utilization strategy but younger children tend not to use it, probably because of both mediation and production deficiencies.
In a multiple-trial, paired associate procedure, subjects predicted “Yes” or “No” on each trial. Significant prediction accuracy was attributed (a) to knowledge of items correctly recalled on Trial N-1, leading to “Yes” predictions and correct recall on Trial N, and (b) to estimates of item difficulty. In Experiment 2, subjects served in triads. The Learner received 48 sentences in a four-trial, study-test paradigm, predicting his own recall on each trial for each sentence. The Listener predicted and heard the Learner's recall. The Observer predicted but did not hear the Learner's recall. The Learner had virtually no advantage in predicting his own recall compared to the Listener, with both performing better than the Observer. The results were related to “feeling-of-knowing” research, to self-responding as the basis of self-knowledge, and to the concept of privileged access.
The article reports four experiments that examine people's ability to predict the outcome of a future test of memory. Our thesis is that memory predictions are implicit judgments of how easily the item is processed while answering the predictive question. If items are processed easily because of factors that also cause memory to succeed, predictions are accurate; if the factors that cause easy processing are irrelevant for memory, predictions are less accurate. The experiments examine factors that influence the prediction taks and the memory test separately; these include item attributes, manner of processing, repetition, and similarity of processing between the prediction task and the memory test. Predictions are most accurate if the prediction task entails the same processes as the test, even if the predictive question is nominally irrelevant to the test; predictions are less accurate if the task and test have different entailments, even if the nominal question is specifically aimed at the test.
A comparison of current measures of the accuracy of feeling-of-knowing predictions
- T Nelson
- R Leonesio
Nelson, T,0, (1984). A comparison of current measures of the accuracy of feeling-of-knowing
predictions. Psychological Bulletin. 95, 109-133,
Nelson, T,0,, & Leonesio, R,J. (1988), Allocation of
self-paced study time and the "iabor-in-vain
effect," Journal of Experimental Psychology:
Learning, Memory, & Cognition, 14, 676-686.
Calibration of probabihties: The state ofthe art to Judgment under uncertainty: Heuristics and biases
- S Lichtenstein
- B Fischhoff
- L Phillips
Lichtenstein, S., Fischhoff, B., & Phillips, L,D.
(1982). Calibration of probabihties: The state
ofthe art to 1980, In D. Kahneman, P. Slovic.
& A, Tversky (Eds,). Judgment under uncertainty: Heuristics and biases (pp, 306-334).
(Mctamemory judgments
- G Bower
- P Winchesier
Bower, G,, & Winchesier. P. (1970). (Mctamemory
judgments,! Stanford University, unpublished
data (reported in Leonesio & Nelson, 1990).
Knowledge of one's own responding and the relation of such knowledge to learning
- G L Oisanz
- . G T Vesonder
- J F Voss
Oisanz, G.L,, Vesonder. G.T,, & Voss, J.F, fl978).
Knowledge of one's own responding and the
relation of such knowledge to learning. Journal of Experimental Child Psychology. 25,