Measuring Self-assessment: Current State of the Art

Department of Surgery, University of Toronto, Toronto, Ontario, Canada.
Advances in Health Sciences Education (Impact Factor: 2.12). 02/2002; 7(1):63-80. DOI: 10.1023/A:1014585522084
Source: PubMed


The competent physician pursues lifelong learning through the recognition of deficiencies and the formulation of appropriate learning goals. Despite the accepted theoretical value of self-assessment, studies have consistently shown that the accuracy of self-assessment is poor. This paper examines the methodological issues that plague the measurement of self-assessment ability and presents several strategies that address these methodological problems within the current paradigm. In addition, the article proposes an alternative conceptualization of self-assessment and describes its associated methods. The conclusions of prior research in this domain must be re-examined in light of the common pitfalls encountered in the design of the studies and the analyses of the data. Future efforts to elucidate self-assessment phenomena need to consider the implications of this review.

Download full-text


Available from: Larry D Gruppen, Oct 10, 2015
25 Reads
  • Source
    • "The development of the self-assessment capacity is attracting a lot of attention in the academic world, given the importance of the student's involvement in the learning process, not only to improve his/her academic results, but also because this skill contributes to the student's professional development. However, the research carried out on self-assessment accuracy does not provide conclusive results, and it presents a lack of methodological rigor (Stefani, 1994 ; Ward et al., 2002 ). The present study has analyzed self-assessment accuracy in the university, after taking a series of methodological precautions recommended in the literature. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The paper’s main objective is to evaluate the self-assessment accuracy of university students. Specifically, the study analyzes the self-assessment of oral communication skills. It was carried out in a Firm Labor Organization course included in the Labor Relations and Human Resources Degree. The literature on self-assessment in Higher Education does not provide clear evidence about its accuracy, as a number of methodological problems have been detected. To reduce them, we have taken a number of precautions. Thus, a rubric was designed, and students were trained to use it. Several teachers and peers were introduced as referents, and a segmented analysis was conducted based on gender and the students’ level of competence from the teachers’ point of view. Results show that self-assessment accuracy is low. Moreover, regardless of the degree, men’s self-assessments are higher than women’s. Moreover, the findings suggest that the scoring rubric improves self-assessment accuracy when the speaker has good oral communication skills, but not when these skills are poor. These results lead us to propose the development of correction factors that can be adapted to any situation, thus allowing self-assessment to be used for summative purposes.
    Sustainable Learning in Higher Education, Edited by Marta Peris-Ortiz, José M. Merigó Lindahl, 01/2015: chapter 2: pages 21-35; Springer International Publishing., ISBN: 9783319108032
  • Source
    • "This belief is called perceived self-efficacy, and according to Bandura’s theory people with high self-efficacy are more likely to view difficult tasks as something to be mastered than something to be avoided. Research in medical education has shown however that students display poor capacity for self-assessment [44,45], and the transition into the clinical phase further reinforces students’ feelings of inadequacy [46]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Hospital clerkships are considered crucial for acquiring competencies such as diagnostic reasoning and clinical skill. The actual learning process in the hospital remains poorly understood. This study investigates how students learn clinical skills in workplaces and factors affecting this. Six focus group sessions with 32 students in Internal Medicine rotation (4-9 students per group; sessions 80-90 minutes). Verbatim transcripts were analysed by emerging themes and coded independently by three researchers followed by constant comparison and axial coding. Students report to learn the systematics of the physical examination, gain agility and become able to recognise pathological signs. The learning process combines working alongside others and working independently with increasing responsibility for patient care. Helpful behaviour includes making findings explicit through patient files or during observation, feedback by abnormal findings and taking initiative. Factors affecting the process negatively include lack of supervision, uncertainty about tasks and expectations, and social context such as hierarchy of learners and perceived learning environment. Although individual student experiences vary greatly between different hospitals, it seems that proactivity and participation are central drivers for learning. These results can improve the quality of existing programmes and help design new ways to learn physical examination skills.
    BMC Medical Education 03/2014; 14(1):61. DOI:10.1186/1472-6920-14-61 · 1.22 Impact Factor
  • Source
    • "We excluded papers measuring Kirkpatrick Levels 1 and 2 outcomes because they focus simply on learner reactions and learning measured in a simulated environment. In addition, given the abundant literature on self-assessment inaccuracy,17,18 papers reporting solely self-assessment data and considered a Level 1 (reaction) outcome, were excluded. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Simulation-based learning is increasingly used by healthcare professionals as a safe method to learn and practice non-technical skills, such as communication and leadership, required for effective crisis resource management (CRM). This systematic review was conducted to gain a better understanding of the impact of simulation-based CRM teaching on transfer of learning to the workplace and subsequent changes in patient outcomes. Studies on CRM, crisis management, crew resource management, teamwork, and simulation published up to September 2012 were searched in MEDLINE(®), EMBASE™, CINAHL, Cochrane Central Register of Controlled Trials, and ERIC. All studies that used simulation-based CRM teaching with outcomes measured at Kirkpatrick Level 3 (transfer of learning to the workplace) or 4 (patient outcome) were included. Studies measuring only learners' reactions or simple learning (Kirkpatrick Level 1 or 2, respectively) were excluded. Two authors independently reviewed all identified titles and abstracts for eligibility. Nine articles were identified as meeting the inclusion criteria. Four studies measured transfer of simulation-based CRM learning into the clinical setting (Kirkpatrick Level 3). In three of these studies, simulation-enhanced CRM training was found significantly more effective than no intervention or didactic teaching. Five studies measured patient outcomes (Kirkpatrick Level 4). Only one of these studies found that simulation-based CRM training made a clearly significant impact on patient mortality. Based on a small number of studies, this systematic review found that CRM skills learned at the simulation centre are transferred to clinical settings, and the acquired CRM skills may translate to improved patient outcomes, including a decrease in mortality.
    Canadian Anaesthetists? Society Journal 03/2014; 61(6). DOI:10.1007/s12630-014-0143-8 · 2.53 Impact Factor
Show more