ArticlePublisher preview available

Regarding Class Quizzes: a Meta-analytic Synthesis of Studies on the Relationship Between Frequent Low-Stakes Testing and Class Performance

Authors:
Lukas Sotola at Pace University
Lukas Sotola
Marcus Crede at Iowa State University
Marcus Crede
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

We present the results of a meta-analytic synthesis of the literature on the association between the use of frequent low-stakes quizzes in real classes and students’ academic performance in those classes. Data from 52 independent samples from real classes (N = 7864) suggests a moderate association of d = .42 between the use of quizzes and academic performance. Effects are even stronger in psychology classes (d = .47) and when quiz performance contributed to class grades (d = .51). We also find that performance on quizzes is strongly correlated with academic performance (k = 19, N = 3814, r = .57) such that quiz performance is relatively strongly predictive of later exam performance. We also found that the use of quizzes is associated with a large increase in the odds of passing a class (k = 5, N = 1004, odds ratio = 2.566).
Figures - available from: Educational Psychology Review
This content is subject to copyright. Terms and conditions apply.
A preview of this full-text is provided by Springer Nature.
Learn more
Content available from Educational Psychology Review
This content is subject to copyright. Terms and conditions apply.
META-ANALYSIS
Regarding Class Quizzes: a Meta-analytic Synthesis
of Studies on the Relationship Between Frequent
Low-Stakes Testing and Class Performance
Lukas K. Sotola
1
&Marcus Crede
1
#Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
We present the results of a meta-analytic synthesis of the literature on the association
between the use of frequent low-stakes quizzes in real classes and students’academic
performance in those classes. Data from 52 independent samples from real classes (N=
7864) suggests a moderate association of d= .42 between the use of quizzes and
academic performance. Effects are even stronger in psychology classes (d=.47) and
when quiz performance contributed to class grades (d= .51). We also find that perfor-
mance on quizzes is strongly correlated with academic performance (k=19, N=3814,
r= .57) such that quiz performance is relatively strongly predictive of later exam perfor-
mance. We also found that the use of quizzes is associated with a large increase in the
odds of passing a class (k=5, N= 1004, odds ratio = 2.566).
Keywords Testing effect .Test-potentiated new learning .Class quizzes
Introduction
Frequent low-stakes testing is a popular teaching tool. Many instructors use it to provide
students with feedback on their understanding of recently covered class material. This practice
is both popular among instructors and generally well-received by students (e.g., Narloch et al.
2006;LyleandCrawford2011) and is also aligned with the theoretical frameworks of learning
and memory that seek to explain both the testing effect (Karpicke and Roediger 2007) and test-
potentiated learning (Roediger et al. 2011; Wissman et al. 2011), as well as the substantial
body of empirical experimental work upon which these theories are based.
The testing effect refers to the finding that having people take tests on material of interest
tends to lead to improved memory for that material. There are numerous theoretical
https://doi.org/10.1007/s10648-020-09563-9
*Lukas K. Sotola
lksotola@iastate.edu
1
Department of Psychology, Iowa State University, Ames, IA 50011, USA
Published online: 14 August 2020
Educational Psychology Review (2021) 33:407–426
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... Quizzes are a common method of holding students accountable for pre-class preparation, including reading (Han & Klein, 2019), so students are likely familiar with the method. Quizzes may be helpful for learning due to the testing effect, in which repeated tests of information improve memory for that information perhaps due to active engagement with the content (Sotola & Crede, 2021). Indeed, the findings from meta-analyses indicated benefits of quizzes on course performance (Sotola & Crede, 2021;Yang et al., 2021). ...
... Quizzes may be helpful for learning due to the testing effect, in which repeated tests of information improve memory for that information perhaps due to active engagement with the content (Sotola & Crede, 2021). Indeed, the findings from meta-analyses indicated benefits of quizzes on course performance (Sotola & Crede, 2021;Yang et al., 2021). One study compared a course that had quizzes assigned on the textbook reading to a course in which reading was assigned without quizzes. ...
... Furthermore, most studies on social annotation have only been one-session (one-shot) studies and did not compare to another reading assignment (Sun et al., 2023). In addition, studies on the effectiveness of quizzes were compared to not having quizzes rather than another reading assignment (Sotola & Crede, 2021;Yang et al., 2021). Therefore, it appears that both social annotation and quizzes are likely better than no reading assignment, but it is uncertain whether social annotation or quizzes better support learning content from assigned readings. ...
Comparing Quizzes and Social Annotation for Pre-Class Reading Accountability
Article
Full-text available
  • Aug 2024
  • TEACH PSYCHOL
Background: Pre-class reading assignments are frequently used to encourage students to complete assigned textbook readings. Quizzes and social annotation are both pre-class reading assignments, but it is uncertain which is more effective for student learning and motivation. Objective: The purpose of this study was to compare quizzes and social annotation on student learning and motivation. Method: Undergraduate students (N = 58) in a child development course in a within-subjects counterbalanced design had quizzes for the textbook chapters covered in one unit exam and social annotation for a different exam. They were surveyed on their motivation at the end of the term. Results: There were no reliable differences in student learning based on exam scores. Students generally perceived social annotation as more motivating but were more likely to submit quizzes. Conclusion: Both quizzes and social annotation appear to have different benefits for students. There is not a clear answer on which one is more effective for student learning. Teaching Implications: Given the mixed results, a combination or choice of quizzes or social annotation as pre-class reading assignments may be optimal.
View
... Our analysis explored the interplay between the two conditions, learners' self-regulation, critical thinking, and quiz grades using a multiple regression model. The results revealed that treatment condition, self-regulation, and critical thinking were significant predictors of quiz grades (Table 4), suggesting a potential mechanism that self-regulation plays when considering the testing effect (Peng et al., 2019;Sotola & Crede, 2021). Notably, the interaction between the "Immediate Questioning" condition and self-regulation emerged as a significant factor, suggesting that the influence of embedded video questions on quiz grades varies based on learners' self-regulation abilities. ...
... In other words, learners in the "Immediate Questioning" condition who showed greater self-regulation tended to have significantly higher quiz grades. This pattern underscores the importance of considering learners' metacognitive strategies when examining the impact of instructional interventions online, highlighting the potential mechanism self-regulation plays in the testing effect (Peng et al., 2019;Sotola & Crede, 2021). Conversely, the interaction term between the two conditions and critical thinking was not significant (Table 5). ...
... Conceptually synchronized questions immediately embedded in the video player served as metacognitive reflective learning opportunities, empowering students to gauge their comprehension, identify areas for further exploration, and actively manage their learning progress (Delen et al., 2014;Wang et al., 2013;. One of the many benefits of the testing effect paradigm is acknowledging errors during low-stakes practice, allowing learners to self-regulate by reassessing initial understandings and fostering conceptual change (Richland et al., 2009;Iwamoto et al., 2017;Sotola & Crede, 2021). Enhancing students' metacognitive techniques like selfregulation can enrich skills applicable in various contexts, including other courses, workforce training, and time management (Barak et al., 2016;Fisher & Baird, 2005;Fries et al., 2020). ...
Immediate Versus Delayed Low-Stakes Questioning: Encouraging the Testing Effect Through Embedded Video Questions to Support Students’ Knowledge Outcomes, Self-Regulation, and Critical Thinking
Article
Full-text available
  • Jul 2024
In light of the educational challenges brought about by the COVID-19 pandemic, there is a growing need to bolster online science teaching and learning by incorporating evidence-based pedagogical principles of Learning Experience Design (LXD). As a response to this, we conducted a quasi-experimental, design-based research study involving nN = 183 undergraduate students enrolled across two online classes in an upper-division course on Ecology and Evolutionary Biology at a large R1 public university. The study extended over a period of 10 weeks, during which half of the students encountered low-stakes questions immediately embedded within the video player, while the remaining half received the same low-stakes questions after viewing all the instructional videos within the unit. Consequently, this study experimentally manipulated the timing of the questions across the two class conditions. These questions functioned as opportunities for low-stakes content practice and retention, designed to encourage learners to experience testing effect and augment the formation of their conceptual understanding. Across both conditions, we assessed potential differences in total weekly quiz grades, page views, and course participation among students who encountered embedded video questions. We also assessed students’ self-report engagement, self-regulation, and critical thinking. On average, the outcomes indicated that learners exposed to immediate low-stakes questioning exhibited notably superior summative quiz scores, increased page views, and enhanced participation in the course. Additionally, those who experienced immediate questioning demonstrated heightened levels of online engagement, self-regulation, and critical thinking. Moreover, our analysis delved into the intricate interplay between treatment conditions, learners’ self-regulation, critical thinking, and quiz grades through a multiple regression model. Notably, the interaction between those in the immediate questioning condition and self-regulation emerged as a significant factor, suggesting that the influence of immediate questioning on quiz grades varies based on learners’ self-regulation abilities. Collectively, these findings highlight the substantial positive effects of immediate questioning of online video lectures on both academic performance and cognitive skills within an online learning context. This discussion delves into the potential implications for institutions to continually refine their approach in order to effectively promote successful online science teaching and learning, drawing from the foundations of pedagogical learning experience design paradigms and the testing effect model.
View
... Online quizzes are utilised to assess students' comprehension of specific accounting concepts and to stimulate their interest and engagement. These quizzes can assist academics, even in large class sizes, by identifying areas where students might need further support (Enders et al., 2021;Sotola & Crede, 2021;Usagawa, 2018).Learning management systems such as Moodle provide discussion forums and facilitate quizzes (Sotola & Crede, 2021;Gamage et al., 2019), offering immediate or deferred feedback. Usagawa (2018) investigated the use of the Quizz app, which enabled multiplayer class activities-including quizzes-and concluded that students reported a positive impact on their learning experiences. ...
... Online quizzes are utilised to assess students' comprehension of specific accounting concepts and to stimulate their interest and engagement. These quizzes can assist academics, even in large class sizes, by identifying areas where students might need further support (Enders et al., 2021;Sotola & Crede, 2021;Usagawa, 2018).Learning management systems such as Moodle provide discussion forums and facilitate quizzes (Sotola & Crede, 2021;Gamage et al., 2019), offering immediate or deferred feedback. Usagawa (2018) investigated the use of the Quizz app, which enabled multiplayer class activities-including quizzes-and concluded that students reported a positive impact on their learning experiences. ...
Proceedings of the 40th AEAA Annual Conference on Reimagining Educational Assessment in the Age of Multiple Dimensions of Learning in a Global Society
Article
Full-text available
  • Dec 2024
The 40th AEAA Annual Conference Proceedings presents a collection of peer-reviewed articles that critically explore contemporary challenges and innovations in educational assessment. Under the theme "Reimagining Educational Assessment in the Age of Multiple Dimensions of Learning in a Global Society," these contributions examine transformative approaches to assessment within diverse educational contexts, with a strong focus on African and global perspectives. The twelve articles featured in this volume cover a range of pertinent topics, including frameworks for quality assurance in online assessments, strategies for enhancing learning outcomes through partnerships, and the role of formative assessment in self-regulated learning. Other discussions highlight emerging trends in school-based assessment, the integration of technology in assessment practices, and the evolving needs of learners with intellectual disabilities. Contributions also explore critical themes such as the authenticity of e-assessment policies, peer assessment in technical disciplines, and the alignment of national assessment standards with contemporary educational demands. Providing a rigorous analysis of current and emerging assessment methodologies, this volume offers valuable insights for educators, policymakers, and researchers. It serves as a foundational resource for enhancing assessment strategies that are responsive to evolving pedagogical landscapes, ensuring equitable, inclusive, and high-quality education.
View
... Online quizzes are utilised to assess students' comprehension of specific accounting concepts and to stimulate their interest and engagement. These quizzes can assist academics, even in large class sizes, by identifying areas where students might need further support (Enders et al., 2021;Sotola & Crede, 2021;Usagawa, 2018).Learning management systems such as Moodle provide discussion forums and facilitate quizzes (Sotola & Crede, 2021;Gamage et al., 2019), offering immediate or deferred feedback. Usagawa (2018) investigated the use of the Quizz app, which enabled multiplayer class activities-including quizzes-and concluded that students reported a positive impact on their learning experiences. ...
... Online quizzes are utilised to assess students' comprehension of specific accounting concepts and to stimulate their interest and engagement. These quizzes can assist academics, even in large class sizes, by identifying areas where students might need further support (Enders et al., 2021;Sotola & Crede, 2021;Usagawa, 2018).Learning management systems such as Moodle provide discussion forums and facilitate quizzes (Sotola & Crede, 2021;Gamage et al., 2019), offering immediate or deferred feedback. Usagawa (2018) investigated the use of the Quizz app, which enabled multiplayer class activities-including quizzes-and concluded that students reported a positive impact on their learning experiences. ...
Learning in Eclectic Formative Assessment Strategies: Experiences of Accounting Pre-service Teachers in one Higher Education Institution
Article
Full-text available
  • Dec 2024
One of the goals of Accounting education is to develop graduates capable of equipping learners with ana­lytical skills pertinent to analysing and understanding finan­cial information required for making economic decisions. Gaining and developing the knowledge and skills necessary for learning Accounting demands full engagement from both students and teachers throughout the learning pro­cess. This is primarily achieved through the application of wide-ranging formative assessment strategies that allow students to actively engage in learning while being chal­lenged to think critically. This paper, therefore, aims to un­derstand how the eclectic assessment practices of academ­ics in a School of Education in South Africa contributed to preparing pre-service teachers for teaching. It explores the learning experiences of fourth-year Accounting students through eclectic formative assessment strategies. A qualita­tive case study approach within an interpretive paradigm was employed to explore the experiences of thirty purpos­ively selected Accounting pre-service teachers. Framed by principled eclecticism, a thematic analysis of transcripts generated from semi-structured individual telephonic inter­views with thirty students and WhatsApp-based focused group interviews was conducted to elicit students' views on their eclectic formative assessment practices. Insights from this study will be helpful to academics who aim to produce future teach­ers with the requisite skills to train the future workforce of any country. Teacher education institutions and other higher education institutions would also stand to benefit from the out­comes, which have the potential to guide policy directions to enhance the quality of undergrad­uate education, which is their key mandate.
View
... Cognitive psychologists suggest that taking tests are impactful because the retrieval process helps to cement the knowledge by encoding a host of cues which help retrieval and reduce forgetting (Miller, 2011). The meta-analytic synthesis by Sotola & Crede (2021) extends these results from the laboratory to higher education classrooms and demonstrates the impact that quizzes have on academic performance. Their results also show that 'the use of quizzes is associated with a large increase in the odds of passing a class' (Sotola & Crede, 2021, p. 407). ...
... Reviews of the impact of tests on learning have also identified the positive impact that including feedback on test performance has (Day et al., 2017;Rowland, 2014;Sotola & Crede, 2021). Feedback enhances the retrieval practice because it enables learners to identify what they know and do not know and hence where to focus their study (Brown et al., 2014). ...
Improving student outcomes using automated feedback in a first-year economics class
Article
  • Sep 2024
The assessment literature advocates utilising formative assessment strategies to support and enhance student learning. This study uses online practice tests with inbuilt feedback, that students could repeatedly use, as a formative feedback strategy in a first-year introductory microeconomics module. The key question explored is whether repeated engagement with online practice tests had a statistically significant impact on students' subsequent performance on a summative assessment. Quantitative engagement and performance data (N = 223) were collected from a cohort of first-year students. An analysis of this data reveals that engaging with the online practice test multiple times has a statistically significant impact on final performance. Controlling for the mid-term grade, on average, each cycle of 'seeking and acting' increases grades in the final exam by 1.3% (p = 0.003). Hence, the data suggests that incorporating multiple opportunities to take an online practice test, which includes feedback, is an important feature of the strategy. A smaller sample of qualitative survey data (N = 131) reveals that these economics students hold traditional views of feedback with little awareness of their own role in seeking and acting on feedback. We argue that formative practice tests have the potential to support the development of student feedback literacy.
View
... Using effective learning strategies in schools is an essential factor to help improve outcomes for students. Research indicates that approaches such as retrieval and spaced practice are likely to be more effective in helping students achieve learning goals and can play an important role in the acquisition of core subject knowledge and in preparation for external science examinations (Adesope et al., 2017;Agarwal et al., 2021;Dunlosky et al., 2013;Karpicke & Aue, 2015;Roediger & Karpicke, 2006;Sotola & Crede, 2020). ...
An Evaluation of Secondary School Students’ Use and Understanding of Learning Strategies to Study and Revise for Science Examinations
Article
Full-text available
  • Jan 2025
There is currently no population-based data evaluating secondary school-aged students’ use, or understanding of, learning strategies to study/revise independently for science. There is also no research evaluating the effort students make towards independent science study and revision, nor how schools support students with study and/or revision strategies for science examinations. In this paper, we report data from a representative sample of 385 students (aged 14 to 15 years) from 29 secondary schools in the UK, using the Effective Revision and Study Strategies Questionnaire (ERaSSQ) survey. We conducted a cross-sectional survey using a multistage implicitly stratified sampling method. Our results show that the learning strategies most frequently used by students for independent science study and revision were making notes, repeatedly reading information, and highlighting or underlining information (i.e., lower utility learning strategies). Our findings also suggest many students do not have a complete understanding of the strategies that are known to have higher utility (i.e., retrieval and spaced practice). These results represent the first attempt to gather information using robust survey methods and are of interest to secondary school science teachers and education policymakers.
View
... In-class review questions are a form of ensuring students have done readings for a class and retained knowledge from previous classes. Such quizzes have been shown to improve students' scores on future assessments of the same material when the quizzes were administered a few days after reading the material [32]- [34]. This continual retrieval of course material helps with long term retention [35], and these types of quizzes can also help review previous material while providing guidance for a student if they have not mastered a topic and need additional help [36]. ...
View
... Nonetheless, a noteworthy shift in the dynamics of anxiety and stress occur when students engage in consistent preparation across all their classes, which shows a relationship between stress, anxiety and academic performance, so it is said that when there is a certain level of concern on the part of students, their academic success increases. For example, students who are tested constantly tend to perform better than those who are not tested as often [38]. By being constantly evaluated and taking tests more frequently, students generate greater and better study habits, which results in an improvement in performance, obtaining better results on the tests [39] [40]. ...
View
View
View
The impact of feedback during formative testing on study behaviour and performance of (bio)medical students: A randomised controlled study
Article
Full-text available
Background A potential concern of formative testing using web-based applications (“apps”) is provision of limited feedback. Adopting a randomised controlled trial in 463 first year (bio) medical students, we explored if providing immediate, detailed feedback during “app”-based formative testing can further improve study behaviour and study performance of (bio)medical students. Methods Students had access to a formative testing “app”, which involved 7 formative test modules throughout the 4-week course. In a randomised order, subjects received the “app” with (n = 231, intervention) or without (n = 232, control) detailed feedback during the formative test modules. Results No differences in app-use was found between groups (P = 0.15), whereas the intervention group more frequently reviewed information compared to controls (P = 0.007). Exam scores differed between non−/moderate−/intensive- users of the “app” (P < 0.001). No differences in exam scores were found between intervention (6.6 ± 1.1) versus control (6.6 ± 1.1, P = 0.18). Time spent studying was significantly higher compared to previous courses in moderate- and intensive-users (P = 0.006 and < 0.001, respectively), but not in non-users (P = 0.55). Time spent studying did not differ between groups (P > 0.05). Conclusions Providing detailed feedback did not further enhance the effect of a web-based application of formative testing on study behaviour or study performance in (bio)medical students, possibly because of a ceiling-effect.
View
Retrieval Potentiates New Learning: A Theoretical and Meta-Analytic Review
Article
Full-text available
  • Sep 2018
A growing body of research has shown that retrieval can enhance future learning of new materials. In the present report, we provide a comprehensive review of the literature on this finding, which we term test-potentiated new learning. Our primary objectives were to: 1) produce an integrative review of the existing theoretical explanations, 2) summarize the extant empirical data with a meta-analysis, 3) evaluate the existing accounts with the meta-analytic results, and 4) highlight areas that deserve further investigations. Here, we identified four non-exclusive classes of theoretical accounts, including resource accounts, metacognitive accounts, context accounts, and integration accounts. Our quantitative review of the literature showed that testing reliably potentiates the future learning of new materials by increasing correct recall or by reducing erroneous intrusions, and several factors have a powerful impact on whether testing potentiates or impairs new learning. Results of a meta-regression analysis provide considerable support for the integration account. Lastly, we discuss areas of under-investigation and possible directions for future research.
View
Quizzes Benefit Freshman and Sophomore Students More Than Junior and Senior Students in Introductory Psychology Classes With Noncumulative Exams
Article
Full-text available
  • Dec 2017
Many studies have examined quiz effectiveness on students’ information retention and exam performance. The present study furthers this area of research by being the first to my knowledge to test quiz effectiveness in introductory psychology classes with noncumulative exams and tests whether observed learning and performance differences relate to whether students are freshman/sophomores or junior/seniors. Data was obtained from six introductory psychology classes; I randomly selected three classes to include quizzes and three to not have quizzes. I measured students’ overall exam performance on three noncumulative exams and measured information retention using a general psychology knowledge test with questions distinct from all quiz and exam materials. Using bootstrapped regression techniques, I identified that freshman/sophomore students performed better on a general psychology knowledge test at the end of the semester in classes with quizzes. No similar benefit emerged for junior/senior students. Furthermore, there were no significant exam grade improvements regardless quiz inclusion in classes. This study is one of the first to my knowledge to consider whether quizzes have a differential effect on freshman/sophomore versus junior/senior students in introductory psychology classes. The results provide a foundation for important future research in this domain, especially considering that these results contrast with those obtained from studies in classes with cumulative exams. This study raises the possibility that quizzes interact with cumulative versus noncumulative exam format to influence student learning and performance.
View
A dual memory theory of the testing effect
Article
Full-text available
  • Jun 2017
  • PSYCHON B REV
A new theoretical framework for the testing effect-the finding that retrieval practice is usually more effective for learning than are other strategies-is proposed, the empirically supported tenet of which is that separate memories form as a consequence of study and test events. A simplest case quantitative model is derived from that framework for the case of cued recall. With no free parameters, that model predicts both proportion correct in the test condition and the magnitude of the testing effect across 10 experiments conducted in our laboratory, experiments that varied with respect to material type, retention interval, and performance in the restudy condition. The model also provides the first quantitative accounts of (a) the testing effect as a function of performance in the restudy condition, (b) the upper bound magnitude of the testing effect, (c) the effect of correct answer feedback, (d) the testing effect as a function of retention interval for the cases of feedback and no feedback, and (e) the effect of prior learning method on subsequent learning through testing. Candidate accounts of several other core phenomena in the literature, including test-potentiated learning, recognition versus cued recall training effects, cued versus free recall final test effects, and other select transfer effects, are also proposed. Future prospects and relations to other theories are discussed.
View
Rethinking the Use of Tests: A Meta-Analysis of Practice Testing
Article
Full-text available
  • Jun 2017
The testing effect is a well-known concept referring to gains in learning and retention that can occur when students take a practice test on studied material before taking a final test on the same material. Research demonstrates that students who take practice tests often outperform students in nontesting learning conditions such as restudying, practice, filler activities, or no presentation of the material. However, evidence-based meta-analysis is needed to develop a comprehensive understanding of the conditions under which practice tests enhance or inhibit learning. This meta-analysis fills this gap by examining the effects of practice tests versus nontesting learning conditions. Results reveal that practice tests are more beneficial for learning than restudying and all other comparison conditions. Mean effect sizes were moderated by the features of practice tests, participant and study characteristics, outcome constructs, and methodological features of the studies. Findings may guide the use of practice tests to advance student learning, and inform students, teachers, researchers, and policymakers. This article concludes with the theoretical and practical implications of the meta-analysis.
View
The Hype Cycle of Working Memory Training
Article
  • May 2019
Seventeen years and hundreds of studies after the first journal article on working memory training was published, evidence for the efficacy of working memory training is still wanting. Numerous studies show that individuals who repeatedly practice computerized working memory tasks improve on those tasks and closely related variants. Critically, although individual studies have shown improvements in untrained abilities and behaviors, systematic reviews of the broader literature show that studies producing large, positive findings are often those with the most methodological shortcomings. The current review discusses the past, present, and future status of working memory training, including consideration of factors that might influence working memory training and transfer efficacy.
View
Testing for funnel plot asymmetry of standardized mean differences
Article
  • Dec 2018
Publication bias and other forms of outcome reporting bias are critical threats to the validity of findings from research syntheses. A variety of methods have been proposed for detecting selective outcome reporting in a collection of effect size estimates, including several methods based on assessment of asymmetry of funnel plots, such as Egger's regression test, the rank correlation test, and the Trim‐and‐Fill test. Previous research has demonstated that Egger's regression test is mis‐calibrated when applied to log‐odds ratio effect size estimates, due to artifactual correlation between the effect size estimate and its standard error. This study examines similar problems that occur in meta‐analyses of the standardized mean difference, a ubiquitous effect size measure in educational and psychological research. In a simulation study of standardized mean difference effect sizes, we assess the Type I error rates of conventional tests of funnel plot asymmetry, as well as the likelihood ratio test from a three‐parameter selection model. Results demonstrate that the conventional tests have inflated Type I error due to correlation between the effect size estimate and its standard error, while tests based on either a simple modification to the conventional standard error formula or a variance‐stabilizing transformation both maintain close‐to‐nominal Type I error.
View
psychmeta : An R Package for Psychometric Meta-Analysis
Article
  • Sep 2018
  • APPL PSYCH MEAS
Over the past four decades, psychometric meta-analysis (PMA) has emerged a key way that psychological disciplines build cumulative scientific knowledge. Despite the importance and popularity of PMA, software implementing the method has tended to be closed-source, inflexible, limited in terms of the psychometric corrections available, cumbersome to use for complex analyses, and/or costly. To overcome these limitations, we created the psychmeta R package: a free, open-source, comprehensive program for PMA.
View
Testing potentiates new learning across a retention interval and a lag: A strategy change perspective
Article
  • May 2018
Practicing retrieval on previously studied materials can potentiate subsequent learning of new materials. In four experiments, we investigated the influence of retention interval and lag on this test-potentiated new learning (TPNL) effect. Participants studied four word lists and either practiced retrieval, restudied, or completed math problems following Lists 1–3. Memory performance on List 4 provided an estimate of new learning. In Experiments 1 and 2, participants were tested on List 4 after either a 1 min or 25 min retention interval. In Experiments 3 and 4, participants took at 25 min break before studying List 4. A TPNL effect was observed in all experiments. To gain insight into the mechanism that may underlie TPNL, we analyzed the extent to which participants organized their recall from list to list. Relative to restudy and math, testing led to superior semantic organization across lists. Our results support a strategy change account of TPNL.
View
To What Extent and Under Which Circumstances Are Growth Mind-Sets Important to Academic Achievement? Two Meta-Analyses
Article
  • Mar 2018
Mind-sets (aka implicit theories) are beliefs about the nature of human attributes (e.g., intelligence). The theory holds that individuals with growth mind-sets (beliefs that attributes are malleable with effort) enjoy many positive outcomes—including higher academic achievement—while their peers who have fixed mind-sets experience negative outcomes. Given this relationship, interventions designed to increase students’ growth mind-sets—thereby increasing their academic achievement—have been implemented in schools around the world. In our first meta-analysis (k = 273, N = 365,915), we examined the strength of the relationship between mind-set and academic achievement and potential moderating factors. In our second meta-analysis (k = 43, N = 57,155), we examined the effectiveness of mind-set interventions on academic achievement and potential moderating factors. Overall effects were weak for both meta-analyses. However, some results supported specific tenets of the theory, namely, that students with low socioeconomic status or who are academically at risk might benefit from mind-set interventions.
View