Figure 4 - available via license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Content may be subject to copyright.
Graph of the Dunning-Kruger effect on the confidence of medical students in their diagnostic ability.
Source publication
Background
Point-of-care ultrasound (POCUS) diagnosis, performed by a physician at the patient bedside, is routinely used in emergency medicine and critical care. Although training in ultrasonography has become part of the medical school curriculum, POCUS can be challenging for medical students. This study aimed to assess the effectiveness of a one...
Context in source publication
Similar publications
Point-of-care (POC) molecular diagnostics for clinical microbiology and virology has primarily focused on the detection of a single pathogen. More recently, it has transitioned into a comprehensive syndromic approach that employs multiplex capabilities, including the simultaneous detection of two or more pathogens. Multiplex POC tests provide highe...
Citations
... When examining students from different academic disciplines, students in chemistry and physics, medicine, dentistry, business administration, and management, as well as aviation or nursing are affected by the Dunning-Kruger effect [11,13,14,[24][25][26][27]. ...
... Even in different medical procedures like central venous catheterization, basic life support, ECG-recording and ultrasound [26,34,35], or complex clinical processes such as disease diagnosis, surgical procedures, or performance in anesthesiology and pathology residency training programs, a discrepancy between confidence and competence can be observed [17,[36][37][38][39]. ...
Introduction
The ability to self-assess is a crucial skill in identifying one’s own strengths and weaknesses and in coordinating self-directed learning. The Dunning-Kruger effect occurs when limited knowledge causes individuals to overestimate their competence and underestimate others’, leading to poor self-assessment and unrecognized incompetence. To serve as a foundation for developing strategies to improve self-assessment, the self-assessment abilities of first-semester students were assessed.
Methods
In the final weeks of the summer 2021, winter 2021/22, and summer 2022 semesters, the academic performance (oral anatomy exam) of first semester students was assessed (0–15 points). Before the exam results were announced, students were asked to self-assess their performance.
Results
Exam scores (M = 10.64, SD = 2.95) and self-assessed scores (M = 10.38, SD = 2.54) were comparable. The absolute difference between them, as a measure of self-assessment ability ranged from − 9 to + 9 points (M = -0.26, SD = 2.59). Among participants (N = 426), 18.5% assessed themselves accurately, 35.5% overestimated, and 46.0% underestimated their performance. The correlation between actual score and self-assessment was ρ = -0.590 (p < 0.001), reflecting the Dunning-Kruger effect. When separated by gender, correlation for females was ρ = -0.591 (p < 0.001), and for males ρ = -0.580 (p < 0.001).
Conclusions
Realistic self-assessment is a challenge for first-semester students. The data indicate that females tend to overestimate their performance while males underestimate theirs. A pronounced Dunning-Kruger effect is evident in both genders, with significant negative correlations between self-assessment and actual performance. There are several reasons for the occurrence of the Dunning-Kruger effect. Considering that the COVID-19 pandemic influenced learning environments, collaborative learning was significantly restricted. The lack of opportunities for comparison could potentially lead to unrealistic self-assessment.
... Furthermore, even if a brief educational intervention has been demonstrated to be effective in increasing proficiency on US [12], the lack of learning time as well as the paucity of skilled trainers are also reported limitations in literature [16,18]. As the use of US continues to increase, the need for early training also increases: numerous pieces of evidence report how starting upstream training from the medical school period would ease US application postgraduating [19][20][21]. ...
Background
The aim of this exploratory survey is to describe the current state of US (ultrasound) technique across different pediatric settings nationwide.
Methods
A questionnaire was emailed to all members of the Italian Society of Pediatrics, including pediatric residents. The survey was open from December 2021 to March 2022.
Results
There were 1098 respondents. Seven hundred and seven pediatricians (84.1%) reported any use of US, while 51 (44.3%) residents denied it. The majority of participants ( n = 956, 87.1%) reported to have a US machine available within the department, mostly cart-based ( n = 516, 66.9%) and provided from 1 to 5 years prior to the survey ( n = 330, 42.8%). Lung and neonatal cerebral regions were the most frequently scanned ( n = 289, 18.7% and n = 218, 14.1%, respectively). The suspicion of pneumonia or respiratory distress represented the main reasons for performing US in emergency room ( n = 390, 78% and n = 330, 66%, respectively). The majority of family pediatricians reported to scan lung and kidney/urinary tract regions ( n = 30, 16.9%, and n = 23,12.9%, respectively). Regarding US training, the majority of respondents ( n = 358, 34.6%) declared an experience-based education, with a deficient certification enabling the use of US in 71.6% ( n = 552) of cases. The most common barriers included the lack of a well-defined training program ( n = 627, 57.1%), unavailability of the US machine ( n = 196, 17.9%) and legal responsibility concern ( n = 175, 15.9%).
Conclusions
Despite the growing interest on pediatric US nationally, significant barriers still limit widespread adoption. These obstacles may be addressed through the dissemination of a specific US education plan and providing additional resources.
... Ultrasound allows to perform a real-time view of the structures inside human body, not obtainable with other conventional learning methods. Furthermore, the growing technological development made the ultrasound equipment more accessible both economically and dimensionally, up to portable ultrasound scanners that can be used at the patient's bed and therefore extremely easy to handle not only in the diagnostic but also in the teaching field [6,7]. ...
Purpose
Ultrasound is essential in the clinical practice of many medical specialties due to non-invasiveness, rapidity of examination, low costs and simplicity. Many specialized companies and universities pointed out its potential as a teaching tool for medical students. The aim of our study is to evaluate the impact of an ultrasound course on a sample of students attending the fourth, fifth and sixth year of the degree course in Medicine, highlighting changing in satisfaction and preparation. Another target is to verify the capability of a course on ultrasound to positively impact on participants knowledge and competences.
Methods
Students attending 6 training courses of Medicine held between 2017 and 2019 were recruited. Five trainings held during an Italian society of ultrasound in medicine and biology (SIUMB) congress, in a session dedicated to students, and one during an elective didactic activity (ADE) held in Chieti University. A questionnaire was given to the students before and after the course, in order to assess the impact of the course on the motivation and knowledge. Moreover, a test was also administered at the end of the theoretical part, with questions relating to the notions learned.
Results
There was an 81% of correct response to the learning questionnaire by calculating the mean of 5 SIUMB courses performed. The students are strongly motivated to continue learning ultrasound already from the beginning of the course, and this result remains unchanged in the questionnaire administered at the end. The interest of students towards this method is high, and they would ultrasound courses within the Medicine degree, even before participating in the training. It was evident how students positively assessed the course in relation to the acquisition of skills and knowledge, albeit with a tendency to acquire more knowledge rather than skills.
Conclusions
Our data support the usefulness of including ultrasound into the curriculum of medical students and on its use as a teaching tool. Students are highly motivated and perceive a significant improvement in both skills and knowledge following the proposed courses. Hands-on part is necessary in the training course on ultrasonography.
... Ultrasound has become an increasingly common tool in medical schools' curricula worldwide [1]. However, to date, only some comprehensive undergraduate ultrasound programs have been described, and those that exist have faced significant pedagogical challenges in their incorporation into the undergraduate [2]. This is due, in part, to the fact that medical students have been taught using existing or adapted courses, which were designed for practicing physicians or were inspired by programs established in other institutions, needing an educational context [3,4]. ...
... Among sixth-year students, studies have been carried out where it was shown that with a 2-week course, better results were obtained in the interpretation and use of these diagnostic elements compared to those who did not take it, it was also analyzed and it was obtained that the number of scans performed is not significant for learning, in another study a similar intervention was carried out, just with 1 day of training, obtaining similar results of achievement and satisfaction [2,20]. ...
Introduction. The use of didactic tools for teaching basic sciences in the medical career focuses on anatomical models, electrodiagnostic equipment, and simulation. Only some study programs incorporate images for teaching basic sciences; some of the reasons are the cost of the ultrasound equipment. However, many medical schools have the infrastructure to do so.
Materials and methods. We conducted a review of the scientific literature in the Scopus, Web of Science, and Google Academic databases, after which the researchers conducted discussion sessions to select the main ideas that would help build the educational proposal.
Results. Describe a proposal for curricular design for creating training programs and teacher training that allows maximizing the use of ultrasound as a teaching tool for the basic sciences of the medical career.
Conclusion. The best way to strengthen the teaching of medical sciences is through constant academic training, both in disciplinary content and in teaching. Only in this way can we face the great need to train doctors who are very aware of their social responsibility.
... A similar finding was made by Zawadka et al. who carried out pre and post course knowledge tests around a one-day point-of-care ultrasound course for medical students. They found that post course students were more confident in their test answers, for those questions where marks were higher, but also for those questions where marks did not increase [10]. Development of any POCUS educational resource requires consideration of how it may affect users' confidence in a way that could cause harm in clinical practice (e.g. if users become inappropriately confident, and this influences decision making by them or others), and our study is not exempt from this. ...
Background
Focused echocardiography is increasingly used in acute and emergency care, with point-of-care ultrasound integrated into several specialist training curricula (e.g. Emergency Medicine, Cardiology, Critical Care). Multiple accreditation pathways support development of this skill but there is scant empirical evidence to inform selection of teaching methods, accreditation requirements or quality assurance of education in focussed echocardiography. It has also been noted that access to in-person teaching can be a barrier to completing accreditation programmes, and that this may affect learners disproportionately depending on the location or nature of their institution. The purpose of the study was to determine whether serial image interpretation tasks as a distinct learning tool improved novice echocardiographers’ ability to accurately identify potentially life-threatening pathology from focused scans. We also aimed to describe the relationship between accuracy of reporting and participants’ confidence in those reports, and to assess users’ satisfaction with a learning pathway that could potentially be delivered remotely.
Methods
27 participants from a variety of healthcare roles completed a program of remote lectures and 2 in-person study days. During the program they undertook 4 ‘packets’ of 10 focused echocardiography reporting tasks (total = 40) based on images from a standardised dataset. Participants were randomized to view the scans in varying orders. Reporting accuracy was compared with consensus reports from a panel of expert echocardiographers, and participants self-reported confidence in their image interpretation and their satisfaction with the learning experience.
Results
There was a stepwise improvement in reporting accuracy with each set of images reported, from an average reporting score of 66% for the 1st packet to 78% for the 4th packet. Participants felt more confident in identifying common life-threatening pathologies as they reported more echocardiograms. The correlation between report accuracy and confidence in the report was weak and did not increase during the study (r s = 0.394 for the 1st packet, r s = 0.321 for the 4th packet). Attrition during the study related primarily to logistical issues. There were high levels of satisfaction amongst participants, with most reporting that they would use and / or recommend a similar teaching package to colleagues.
Conclusions
Healthcare professionals undertaking remote training with recorded lectures, followed by multiple reporting tasks were capable of interpreting focused echocardiograms. Reporting accuracy and confidence in identifying life-threatening pathology increased with the number of scans interpreted. The correlation between accuracy and confidence for any given report was weak (and this relationship should be explored further given the potential safety considerations). All components of this package could be delivered via distance learning to enhance the flexibility of echocardiography education.
... Algumas instituições introduziram a US em suas avaliações de exame clínico estruturado, 8,14,15 e alguns programas de treinamento avançado inseriram a avaliação de imagens na nota final. 16 Houve também universidades que utilizaram a autoavaliação 12,17,18 ou o desempenho nas simulações práticas 19 como método avaliativo. ...
O objetivo deste estudo foi analisar o histórico e os fatores associados ao ensino da ultrassonografia point-of-care na graduação de medicina. Trata-se de uma revisão sistemática da literatura, conduzida nas bases de dados PubMed®, Cochrane Library, Portal de Periódicos Capes e Biblioteca Virtual em Saúde, utilizando descritores controlados. Foram inclusos artigos que abordassem os benefícios e/ou entraves do uso da ultrassonografia point-of-care no currículo da graduação de medicina. Foram encontradas 959 referências, sendo que, destas, 42 foram incluídas na análise final; 64,3% eram estudos norte-americanos. Na maioria, os currículos foram implementados do primeiro ao quarto ano (28,6%), no primeiro e segundo ano (17,1%) ou no sexto ano (17,1%) da faculdade. Mostrou-se principalmente que a integração do ensino da ultrassonografia point-of-care na graduação de medicina é viável, já que a ultrassonografia é uma ferramenta de ensino valiosa e bem recebida pelos discentes. Além disso, esse ensino aumenta o conforto dos alunos e a capacidade de obter imagens de ultrassonografia precisas para fins diagnósticos e procedimentais. A incorporação da ultrassonografia point-of-care nos currículos e programas de ensino médico ainda está em desenvolvimento, mas os dados existentes já apontam que a inserção dessa ferramenta no processo de educação médica é possível e apreciada pelos discentes.
... Pediatric trainees are interested in obtaining POCUS training during residency and view POCUS as an important skill for their future careers. Our finding is consistent with other needs assessment studies indicating that pediatric residents increasingly demonstrate an interest in POCUS [18,[20][21][22][23]. Upstream of residency training, medical schools are also incorporating ultrasound training into undergraduate medical education [14,[24][25][26][27][28][29][30][31]. As this trend continues, future pediatric residents will likely have more familiarity with POCUS and likely seek training for its clinical applications. ...
Background
Prior studies showed that point-of-care ultrasound (POCUS) training is not commonly offered in pediatric residency. We assessed the need for a pediatric POCUS curriculum by evaluating pediatric trainees’ attitudes toward the use of POCUS and identifying barriers to training. We also aimed to evaluate the impact of a POCUS educational intervention on self-efficacy and behavior.
Methods
We conducted a cross-sectional survey of pediatric residents in a single large freestanding children’s hospital distributed via an institutional listserv and administered online. The survey included opinion-rating of statements regarding POCUS and barriers to training. We also offered a two-week POCUS course with online modules and hands-on scanning. Participating residents completed pre- and post-course knowledge assessments and follow-up surveys up to 12 months following the course to assess POCUS use and self-report confidence on POCUS indications, acquisition, interpretation, and clinical application.
Results
Forty-nine respondents were included in the survey representing all three pediatric levels with 16 specialty interest areas. Ninety-six percent of trainees reported that POCUS is an important skill in pediatrics. Ninety-two percent of trainees reported that residency programs should teach residents how to use POCUS. The most important perceived barriers to POCUS training were scheduling availability for POCUS rotations and lack of access to an ultrasound machine. Fourteen participants completed the pre- and post-course knowledge tests, with eight and six participants also completing the six- and 12-month follow-up surveys, respectively. Self-ratings of confidence were significantly improved post-intervention in indications (P = 0.007), image acquisition (P = 0.002), interpretation (P = 0.002), and clinical application (P = 0.004). This confidence improvement was sustained up to 6-12 months (P = 0.004-0.032). Participants also reported higher categorical POCUS use after course completion (P = 0.031).
Conclusions
Pediatric trainees perceive POCUS as an important skill, hold favorable opinions towards the use of POCUS, and support POCUS training within a pediatric residency. A POCUS course can improve resident POCUS knowledge, instill confidence, and motivate higher POCUS use. Further study is needed to evaluate POCUS applications in pediatric medicine to develop a standardized POCUS curriculum and establish a training guideline for pediatric residency.
... Point of care ultrasound (POCUS), for example, offers the opportunity to quickly assess cardiac contractility and look for a pericardial effusion. These have become core skills in some medical school classes 14 . Yet, finding a way to convey the increased information and skills required within the same amount of curricular time is challenging, particularly in scenarios with rare or limited case exposures. ...
Through this paper, we have discussed some ways in which medical education has changed in recent years, and ways we may expect it to change in the near future. These have included changes in the content and delivery of medical knowledge and information through the use of technology, changes to a competency-based educational model, and the use of new tools for learning. As examples, virtual- and augmented-reality learning programs facilitate learning for students by providing access to resources previously unavailable. These may enhance learning, particularly for those with specific needs and challenges. The use of gamification in the development of learning resources may improve engagement and mimic leisure activities in which many are already involved.
... POCUS has previously been incorporated into medical curricula through simulations (Shah et al., 2019), didactic instructions (Yamada et al., 2018;Shah et al., 2019;Coiffier et al., 2020), virtual instruction (Yamada et al., 2018), peer-to-peer teaching (Goodcoff et al., 2019), hands-on experiences (Yamada et al., 2018;Coiffier et al., 2020), during specific clinical rotations (Blackstock et al., 2015), taught with other subjects such as physical examination skills training (Butter et al., 2007;Dinh et al., 2015b), and longitudinally integrated throughout undergraduate medical curricula (Fu et al., 2016;Wilson et al., 2017;Celebi et al., 2019;Liu et al., 2019). The impact of brief POCUS interventions through multiple hands-on POCUS training sessions and one day, or even 1 h, workshops has also demonstrated favorable learning outcomes (Dinh et al., 2015a;Barrington et al., 2016;Krause et al., 2017;Ang et al., 2018;Zawadka et al., 2019). Although benefits have been shown through many approaches, incorporation of POCUS within undergraduate medical curricula requires careful consideration for effective integration. ...
... Such performance speaks to the value and impact of integrating brief clinical correlates into a saturated curriculum. This is supported by other works which demonstrated that brief POCUS educational interventions can have meaningful impact (Krause et al., 2017;Zawadka et al., 2019). ...
Despite its significant clinical use, there is no standardized point‐of‐care ultrasonography (POCUS) curriculum in undergraduate medical education. As Covid‐19 abruptly mandated the use of virtual education, instructors were challenged to incorporate and improve POCUS education within these new constraints. It was hypothesized that integrating POCUS into anatomy via brief video lessons and a subsequent interactive virtual lesson would lead to an objective understanding of POCUS concepts, improved understanding of the corresponding anatomy, and subjective improvement of student confidence with POCUS. A cross‐sectional descriptive study assessed first‐year medical students' perspectives and performance before and after the interventions (n = 161). The intervention was split into two parts: (1) three optional 10‐minute POCUS videos that reinforced anatomy concepts taught in the laboratory sessions, and (2) a subsequent two‐hour interactive virtual session reviewing POCUS and anatomy concepts. Students completed a knowledge and confidence assessment tool before and after the interactive session. Survey responses (n = 51) indicated that 94% of students felt the optional videos improved their understanding of POCUS and were educationally valuable. One half of medical students (50%) indicated that the demonstrations improved their anatomy understanding. Initial self‐reported confidence was low after the optional video lessons, despite an average score of 58% on the knowledge assessment (n = 130). However, confidence increased significantly along with an increase in score performance to 80% after the interactive session (n = 39, P < 0.01). Results suggest that the virtual integration pilot enhanced student learning of both anatomy and POCUS.
... POCUS curricula have been developed in medical schools across the country in recognition of the increasingly important role POCUS has across disciplines [8][9][10][11]. Undergraduate medical education (UME) POCUS curricula include one-day simulation labs, electives, longitudinal preclinical and clinical courses but these different approaches of POCUS education have had unclear association with medical student performance [11][12][13]. ...
Introduction
Point-of-care ultrasound (US) is used in clinical practice across many specialties. Ultrasound (US) curricula for medical students are increasingly common. Optimal timing, structure, and effect of ultrasound education during medical school remains poorly understood. This study aims to retrospectively determine the association between participation in a preclinical, longitudinal US curriculum and medical student academic performance.
Methods
All first-year medical students at a medical school in the Midwest region of the United States were offered a voluntary longitudinal US curriculum. Participants were selected by random lottery. The curriculum consisted of five three-hour hands on-sessions with matching asynchronous content covering anatomy and pathologic findings. Content was paired with organ system blocks in the standard first year curriculum at our medical school. Exam scores between the participating and non-participating students were compared to evaluate the objective impact of US education on performance in an existing curriculum. We hypothesized that there would be an association between participation in the curriculum and improved medical student performance. Secondary outcomes included shelf exam scores for the surgery, internal medicine, neurology clerkships and USMLE Step 1. A multivariable linear regression model was used to evaluate the association of US curriculum participation with student performance. Scores were adjusted for age, gender, MCAT percentile, and science or engineering degree.
Results
76 of 178 students applied to participate in the curriculum, of which 51 were accepted. US curriculum students were compared to non-participating students (n = 127) from the same class. The US curriculum students performed better in cardiovascular anatomy (mean score 92.1 vs. 88.7, p = 0.048 after adjustment for multiple comparisons). There were no significant differences in cumulative cardiovascular exam scores, or in anatomy and cumulative exam scores for the gastroenterology and neurology blocks. The effect of US curriculum participation on cardiovascular anatomy scores was estimated to be an improvement of 3.48 points (95% CI 0.78-6.18). No significant differences were observed for USMLE Step 1 or clerkship shelf exams. There were no significant differences in either preclinical, clerkship or Step 1 score for the 25 students who applied and were not accepted and the 102 who did not apply.
Conclusions
Participation in a preclinical longitudinal US curriculum was associated with improved exam performance in cardiovascular anatomy but not examination of other cardiovascular system concepts. Neither anatomy or comprehensive exam scores for neurology and gastrointestinal organ system blocks were improved.
