Figure - available from: Pediatric Radiology
This content is subject to copyright. Terms and conditions apply.
The Leitner system is a spaced repetition algorithm, which sorts flashcards into five piles according to how well the learner knows each diagnosis. Each correct answer advances the flashcard to the next, less frequent pile. Incorrect answers are sent back to the first pile. This system skews exposure toward unknown diagnoses

The Leitner system is a spaced repetition algorithm, which sorts flashcards into five piles according to how well the learner knows each diagnosis. Each correct answer advances the flashcard to the next, less frequent pile. Incorrect answers are sent back to the first pile. This system skews exposure toward unknown diagnoses

Source publication
Article
Full-text available
Applied memory research in the field of cognitive and educational psychology has generated a large body of data to support the use of spacing and testing to promote long-term or durable memory. Despite the consensus of this scientific community, most learners, including radiology residents, do not utilize these tools for learning new information. W...

Citations

... Qualitatively, participants reported that such revision was often superficially motivated by summative assessment deadlines, rather than for longer-term competency. This is consistent with existing studies that suggest students often adopt less effective shallow strategies for learning such as rereading, highlighting and cramming [23,24]. Therefore, interventions focusing on engaging medical students in ongoing revision and active learning provides clear benefits for student outcomes [25]. ...
... As a developed resource on an open source program, such an intervention would lower the major barrier to entry for students to engage in curriculum-specific pharmacology revision that is time-efficient and freely accessible [26]. Such a resource could encourage more effective revision involving principles such as retrieval practice and free recall [24], as well as the option to study with spaced-repetition scheduling that maximises time spent on challenging concepts to improve outcomes [13,27]. The digital nature of the flashcards allows for ease of editing, collaboration and access, thereby ensuring the sustainability of the resource in the long-term [14]. ...
... Notably, students perceived themselves engaging with the resource long-term and expressed an interest in similar resources developed for other medical course content, which potentially serves as a more accurate reflection of the overall perception of the resource potential. The prospect for longer-term engagement in the resource is particularly notable as the very nature of Anki ™ -based revision essentially ensures re-exposure to, and retention of, curriculum materials well into the future [24,31]. Based on our findings, the best strategy to implement flashcards in the medical curricula would be introduce them during preclinical years as an adjunct to other learning methods (lectures and tutorials), to not only support the memorisation and understanding of concepts as a prerequisite for higher order learning, but provide a structure for longterm revision. ...
Article
Full-text available
Background Pharmacology is a cornerstone of medical education as it underlies safe prescribing practices. However, medical students have reported unease regarding their perceived proficiency in medical pharmacology. Despite the significant impetus to improve student outcomes, there is little analysis available of the techniques used by medical students to learn, retain and apply medical pharmacology knowledge. Methods A mixed-methods, student-focused approach was conducted to design and evaluate specific resources developed to address gaps in pharmacology education. This methodology comprised an anonymised scoping survey, followed by semi-structured focus group interviews. We then developed a flashcard resource as an intervention to support long-term revision for academic and clinical success. This resource was released to a cohort of 100 graduate entry preclinical medical students who were invited at the end of year to evaluate the intervention via a subsequent anonymous survey. Results The scoping survey received 103 complete responses. Surveys and focus group interviews revealed that only 50% of students engage in ongoing revision. Amongst our cohort, we identified that the evidence-based technique of spaced-repetition was particularly well regarded. Hence, we developed and evaluated a bespoke resource utilising Anki™, an open-source, spaced-repetition flashcard program. A total of 1208 flashcards spanning 156 distinct classes of drugs with supplementary summary tables, diagrams and explanatory video and summary guides were created. Designed as a strategic revision tool to reinforce learning, evaluation showed students greatly appreciated the “comprehensive” and “well formatted” Anki™ resource that supported existing teaching modalities, with a global rating of 3.8 out of 5. Conclusions Strategic and personalised resources for medical pharmacology education that assist with in-semester revision and long-term retention are highly valued amongst students for examination preparation and preparedness for practice. Collectively, these results reflect a novel approach to identifying and addressing weaknesses in existing learning resources in a manner that is inclusive of, and acceptable to, medical students.
... Pacsbin (Orion Medical Technologies, Baltimore, MD) is a web-based radiology picture archiving and communication system (PACS) that provides learners with a familiar platform to scroll through CTs, simulating the way they engage with imaging studies in clinical practice and providing a potential vehicle for active learning (www.pacsbin.com). [25][26][27][28][29] While practice cases on various PACS platforms have supplemented existing curricula, [27][28][29] no prior study has used this technology to directly compare active and passive learning strategies. ...
... Pacsbin (Orion Medical Technologies, Baltimore, MD) is a web-based radiology picture archiving and communication system (PACS) that provides learners with a familiar platform to scroll through CTs, simulating the way they engage with imaging studies in clinical practice and providing a potential vehicle for active learning (www.pacsbin.com). [25][26][27][28][29] While practice cases on various PACS platforms have supplemented existing curricula, [27][28][29] no prior study has used this technology to directly compare active and passive learning strategies. ...
Article
Introduction: Head computed tomography (CT) interpretation is a vital skill for emergency physicians. Existing literature shows poor concordance between emergency physicians and radiologists in head CT interpretation. Prior studies have used passive learning methods to address this knowledge gap. We created an active learning curriculum for teaching head CT interpretation to emergency medicine (EM) residents and compared its effectiveness to a passive learning strategy. Methods: We conducted a prospective, randomized controlled study of EM residents at a single institution. Three educational sessions were delivered over a three-month period via video conference. The active learning cohort (ALC) scrolled through head CT teaching cases we designed on Pascbin, a web-based radiology picture archiving and communication system. The passive learning cohort (PLC) watched instructional videos that scrolled through the same cases. Both cohorts were given equal time to review the cases and ask an instructor questions. Residents took pre-intervention and post-intervention tests on head CT interpretation. We analyzed scores using paired and unpaired t-tests. Results: Forty-two residents took the pre-intervention test. Mean pre- and post-test scores for the ALC were 43.8% and 59.0% (P <0.001), and for the PLC were 41.7% and 45.3% (P = 0.29). The difference in ALC and PLC post-test scores was statistically significant (P = 0.009) with a large effect size (Cohen's d = 1.34). Conclusion: Our active learning head CT curriculum using Pacsbin showed superior learning outcomes when compared to a passive learning strategy and required no additional time or resources. This intervention offers a more effective and learner-centric method for implementing radiology curricula in EM residency programs.
... Qualitatively, participants reported that such revision was often super cially motivated by summative assessment deadlines, rather than for long-term competency. This is consistent with existing studies that suggest students often adopt less effective shallow strategies for learning such as rereading, highlighting and cramming (21,22). Therefore, interventions focusing on engaging medical students in structured revision possess and active learning provides clear bene ts for student outcomes (23). ...
... Firstly, as a developed resource on an open source program, such an intervention would lower the major barrier to entry for students to engage in curriculum-speci c pharmacology revision that is time-e cient and freely accessible (28). Secondly, such a resource could encourage more effective revision involving retrieval practice (22) and the option to study with spaced-repetition scheduling that maximises time spent on challenging concepts to improve outcomes (13,16). Finally, the digital nature of the ashcards allows for ease of editing, collaboration and access, thereby ensuring the sustainability of the resource in the long-term (14). ...
... Notably, students largely perceived themselves engaging with the resource long-term and expressed an interest in similar resources developed for other medical course content, which potentially serves as a more accurate re ection of the overall perception of the resource potential. The prospect for long-term engagement in the resource is particularly notable as the very nature of Anki™ -based revision essentially ensures re-exposure to, and retention of, curriculum materials well into the future (22,30). ...
Preprint
Full-text available
Background: Pharmacology is a cornerstone of medical education as it underlies safe prescribing practices. However, medical students have reported unease regarding their perceived proficiency in clinical pharmacology. Despite the significant impetus to improve student outcomes, there is little analysis available of the techniques used by medical students to learn, retain and apply pharmacology knowledge. Methods: A mixed methods, student-focused approach was conducted to design and refine specific resources developed to address gaps in pharmacology education. This methodology comprised an anonymised scoping survey, followed by structured focus group interviews. We developed a relevant and time efficient resource to support long-term revision for academic and clinical success. These resources were released to a cohort of 100 graduate preclinical medical students who were invited at the end of year to evaluate the intervention via a subsequent anonymous survey. Results: The scoping survey received 103 complete responses. Surveys and focus group interviews revealed that only 50% of students engage in ongoing revision. The analysis identified in-semester revision of pharmacology as a significant predictor of strategic and deep learning methods and improved quiz performance (a 5% higher score on average), compared to superficial learning methods. Amongst our cohort, we identified that the evidence-based technique of spaced-repetition was particularly well regarded. Hence, we developed and evaluated a bespoke resource utilising AnkiTM, an open-source, spaced-repetition flashcard program. A total of 1208 flashcards spanning 156 distinct classes of drugs with supplementary summary tables, diagrams and explanatory video and summary guides were created. Designed as a strategic revision tool to reinforce learning, evaluation showed students greatly appreciated the “comprehensive” and “well formatted” AnkiTM resource that supported existing teaching modalities, with a global rating of 3.8 out of 5. Conclusions: Strategic and personalised techniques for pharmacology learning that assist with in-semester revision and long-term retention are highly valued amongst students for examination preparation and preparedness for practice. Collectively, these results define a novel approach to identifying and addressing weaknesses in existing teaching methodologies in a manner that is inclusive of, and acceptable to, medical students.
... Spaced practice means that the contents to be learned are repeated after a certain time interval and are tested after a further retention interval [47,50]. Also in radiology teaching and surgical skill training, studies have shown that spaced procedures are beneficial compared to massed practice [51,52]. Future research should therefore specifically compare massed and spaced practice for learning how to diagnose OPTs to identify potentials for further improvements of students' accuracy. ...
Article
Full-text available
The interpretation of medical images is an error-prone process that may yield severe consequences for patients. In dental medicine panoramic radiography (OPT) is a frequently used diagnostic procedure. OPTs typically contain multiple, diverse anomalies within one image making the diagnostic process very demanding, rendering students’ development of visual expertise a complex task. Radiograph interpretation is typically taught through massed practice; however, it is not known how effective this approach is nor how it changes students’ visual inspection of radiographs. Therefore, this study investigated how massed practice–an instructional method that entails massed learning of one type of material–affects processing of OPTs and the development of diagnostic performance. From 2017 to 2018, 47 dental students in their first clinical semester diagnosed 10 OPTs before and after their regular massed practice training, which is embedded in their curriculum. The OPTs contained between 3 to 26 to-be-identified anomalies. During massed practice they diagnosed 100 dental radiographs without receiving corrective feedback. The authors recorded students’ eye movements and assessed the number of correctly identified and falsely marked low- and high prevalence anomalies before and after massed practice. Massed practice had a positive effect on detecting anomalies especially with low prevalence (p < .001). After massed practice students covered a larger proportion of the OPTs (p < .001), which was positively related to the detection of low-prevalence anomalies (p = .04). Students also focused longer, more frequently, and earlier on low-prevalence anomalies after massed practice (ps < .001). While massed practice improved visual expertise in dental students with limited prior knowledge, there is still substantial room for improvement. The results suggest integrating massed practice with more deliberate practice, where, for example, corrective feedback is provided, and support is adapted to students’ needs.
... Cases can be arranged by subspecialty, modality, or disease process depending on learner needs. Online platforms also interleave cases that users have previously answered incorrectly, increasing the likelihood that users address knowledge deficiencies [13]. ...
Article
The hype around artificial intelligence (AI) in radiology continues unabated, despite the fact that the exact role AI will play in future radiology practice remains undefined. Nevertheless, education of the radiologists of the future is ongoing and needs to account for the uncertainty of this new technology. Radiology residency training has evolved even before the recent advent of imaging AI. Yet radiology residents and fellows will likely one day experience the benefits of an AI-enabled clinical training. This will offer them a customized learning experience and the ability to analyze large quantities of data about their progress in residency, with substantially less manual effort than is currently required. Additionally, they will need to learn how to interact with AI tools in clinical practice, and more importantly, understand how to evaluate AI outputs in a critical fashion as yet another piece of information contributing to the interpretation of an imaging examination. Although the exact role AI will play in the future practice of radiology remains undefined, it will surely be integrated into the education of future radiologists.
... The decrease in duration of instruction presented a considerable obstacle to students' retaining the details of body structure. The retention of rote memorization has long been known to be strongly dependent upon the duration of study and spacing of instruction (Delaney, Godbole, Holden, & Chang, 2018;Dempster, 1988;Larsen, 2018;Morin et al., 2019). To adapt to the decreased duration of instruction (6 days for Lower Extremity) learning of muscle anatomy was reduced (only actions/innervations, not origins/insertions). ...
Article
Full-text available
The goal of this observational study was to develop effective approaches to introduce first year medical students to gross anatomy/embryology in a compressed time frame. Pedagogical reorganization of anatomy instruction in the regions of Lower Extremity and Head and Neck was based upon core clinical conditions taught in second‐year and USMLE Step 1 board review courses. These conditions were not presented as clinical problems, as many students had limited prior training in medical terminology, but focused upon clinical symptoms, allowing for direct correlation of structure and function. Instruction stressed vocabulary acquisition and was extended to prepare for laboratory dissections. Overall methodology was multi‐modal, including 'flipped' and traditional lectures, study of prosections/radiographs and small group laboratory review sessions. Content was significantly reduced: knowledge of muscle actions and innervations was required, not muscle origins and insertions. Performance was evaluated by criterion‐based written examinations that included a set of questions (34) asked repetitively over an 8 year period (n=606 students) and by regional practical exams. Mean scores in all areas were sustained or numerically improved, despite the compression of instruction duration. Analysis showed no significant differences based upon question format or instructional modality. Despite the high performance levels, students needing assistance in learning could be identified by score distributions. A survey of students indicated that these changes effectively decreased stress and facilitated review for the USMLE Step 1 Board examination. These results suggest that training in gross anatomy can be modified to a compressed duration by instruction in the context of clinical symptomatology. This article is protected by copyright. All rights reserved.
Article
    The American Board of Physical Medicine and Rehabilitation began administering the longitudinal assessment for physical medicine and rehabilitation for continuing certification in 2020. The longitudinal assessment for physical medicine and rehabilitation digitally delivers quarterly short assessments of content and repeats missed items to facilitate learning as well as serve as a summative certification assessment. With a goal of offering content relevant to an individual’s practice and learning needs, diplomates choose how to customize the domains or topic areas of their question content on an annual basis. This report describes the first year of experience with customization of longitudinal assessment for physical medicine and rehabilitation. The American Board of Physical Medicine and Rehabilitation diplomate customization data are grouped and compared in a variety of ways to ascertain whether there are differences in customization choices. While customization choices were similar across several domains, significant differences were seen when comparing groups with specific areas of practice or subspecialty certification. Smaller differences were also seen when comparing question domain allocation choice between sexes, age groups, and practice setting. The results from this first full year of experience confirm an alignment of this innovative assessment approach to individual physician practice, a significant step in improving the relevance of continuing certification overall for participating physicians.
    Article
    IntroductionElectronic flashcards allow repeated information exposure over time along with active recall. It is increasingly used for self-study by medical students but remains poorly implemented for graduate medical education. The primary goal of this study was to determine whether a flashcard system enhances preparation for the in-training examination in obstetrics and gynecology (ob-gyn) conducted by the Council on Resident Education in Obstetrics and Gynecology (CREOG).Methods Ob-gyn residents at Duke University were included in this study. A total of 883 electronic flashcards were created and distributed. CREOG scores and flashcard usage statistics, generated internally by interacting with the electronic flashcard system, were collected after the 2019 exam. The primary outcome was study aid usage and satisfaction. The secondary outcome was the impact of flashcard usage on CREOG exam scores.ResultsOf the 32 residents, 31 (97%) participated in this study. Eighteen (58%) residents used the study’s flashcards with a median of 276 flashcards studied over a median of 3.7 h. All of the flashcard users found the study aid helpful, and all would recommend them to another ob-gyn resident. Using the flashcards to study for the 2019 CREOG exam appeared to correlate with improvement in scores from 2018 to 2019, but did not achieve statistical significance after adjusting for post-graduate year (beta coefficient = 10.5; 95% confidence interval = − 0.60,21.7; p = 0.06).DiscussionThis flashcard resource was well received by ob-gyn residents for in-training examination preparation, though it was not significantly correlated with improvement in CREOG scores after adjusting for post-graduate year.
    Article
    Bloom's Taxonomy, an integral component of learning theory since its inception, describes cognitive skill levels in increasing complexity (Remember, Understand, Apply, Analyze, Evaluate, and Create). Considering Bloom's Taxonomy when writing learning objectives and lecture material, teaching residents at the workstation and creating multiple choice questions can increase an educator's effectiveness. The incorporation of higher Bloom levels aids in cultivating critical thinking skills vital to image interpretation and patient care, and becomes increasingly important as the radiologist's role evolves with the continued development of artificial intelligence. Following established tenets of multiple choice question writing, involving trainees in the question writing process, and incorporating audience response systems into lectures are all strategies in which higher Bloom level skills can be accomplished.