Kevin W Eva's research while affiliated with University of British Columbia - Vancouver and other places

Publications (305)

Article
Purpose: The medical education continuum is interrupted by several transition periods that can adversely affect performance. Most of what has been learned about such periods focuses upon movement from one stage of training to another and movement from training to practice. Established physicians, however, experience transitions throughout their ca...
Article
Context: Given the widespread use of MMIs, their impact on the selection of candidates, and the considerable resources invested in preparing and administering them, it is essential to ensure their quality. Given the variety of station formats used and the degree to which that factor resides in the control of training programs, that we know so litt...
Article
Full-text available
Introduction Scholarship is a key activity in health professions education (HPE). When disseminating scholarly work, how one selects the journal to which they submit is often argued to be a key determinant of subsequent success. To draw more evidence-based recommendations in this regard, we surveyed successful scholars working in HPE regarding thei...
Chapter
Assessment of clinical competence continues to challenge the health professions, leading educators and researchers to seek new solutions to long existing and newly identified problems. Broadening definitions of competence, increasing complexities of practice, the valuing and devaluing of some features of assessment, newly identified limitations in...
Article
Seeing links others have missed? Our Editor‐in‐chief offers a challenge to health professional education scholars to draw Connections between Medical Education articles that offer new insights for the field.
Article
Phenomenon Near-peer interactions (NPIs) provide formal and informal mentorship that can allow medical students to share strategies for successful training. Such capacity to convey valuable advice, however, may depend on the similitude of experiences. Given that many factors can disrupt homogeneity, including curriculum renewal, we must better unde...
Article
Introduction: Effective continuing professional development (CPD) is critical for safe and effective health care. Recent shifts have called for a move away from didactic CPD, which often fails to affect practice, toward workplace learning such as clinical coaching. Unfortunately, coaching programs are complex, and adoption does not guarantee effec...
Article
Purpose: To optimize learning, health professional training programs need to achieve the right balance between depth of practice (gaining more experience with particular skills) and breadth of practice (spreading experience across an array of activities). Better understanding how training for a particular skill set is impacted by periods of focus...
Article
Want to know how COVID‐19 changes health professional education research priorities? Our Editor‐in‐chief argues that in the most important ways it shouldn't.
Article
Full-text available
Background Audit and feedback (A&F) interventions are one of the most common approaches for implementing evidence-based practices. A key barrier to more effective A&F interventions is the lack of a theory-guided approach to the accumulation of evidence. Recent interviews with theory experts identified 313 theory-informed hypotheses, spread across 3...
Article
Ever wondered why academics make things so difficult? Ajjawi and Eva explain with their introduction of the 2021 State of the Science issue and this year's theme of Solution‐ism.
Article
Full-text available
Introduction When medical education programs have difficulties recruiting or retaining clinical teachers, they often introduce incentives to help improve motivation. Previous research, however, has shown incentives can unfortunately have unintended consequences. When and why that is the case in the context of incentivizing clinical teachers remains...
Presentation
Accreditation aims to ensure all training programs meet agreed upon standards of quality. The process is complex, resource intensive, and costly. Its benefits are difficult to assess because contextual confounds obscure comparisons between systems that do and do not include accreditation. This study explores accreditation’s influence ‘within system...
Article
Introduction: Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring varia...
Article
Purpose: Written comments are gaining traction as robust sources of assessment data. Compared to the structure of numeric scales, what faculty choose to write is ad hoc, leading to idiosyncratic differences in what is recorded. This study offers exploration of what aspects of writing styles are determined by the faculty offering comment and what a...
Article
Purpose: Medical regulatory authorities are legally mandated to protect patients by monitoring the practice of medicine. While principally a matter of public safety, many pursue this mission by establishing quality improvement initiatives that prioritize professional development for all rather than identification of substandard performers. Engagin...
Article
Purpose: Accreditation aims to ensure all training programs meet agreed upon standards of quality. The process is complex, resource intensive, and costly. Its benefits are difficult to assess because contextual confounds obscure comparisons between systems that do and do not include accreditation. This study explores accreditation's influence "wit...
Article
Our Editor‐in‐chief shares the joy and pain he is feeling as we celebrate exceptional contributions to the field while struggling with world events.
Article
Purpose: Physician shortages and maldistribution, particularly within family medicine, have led many medical schools worldwide to create regional medical campuses (RMCs) for clerkship training. However, Canadian medical schools have developed a number of RMCs in which all years of training (i.e., a combined model that includes both preclerkship an...
Article
In response to the COVID‐19 pandemic, educators around the globe are scrambling to adapt in ways that facilitate the ongoing knowledge and skill development of the next generation of health professionals while social distancing and other measures are dramatically disrupting standard practices.
Article
As people around the globe grapple with the COVID‐19 pandemic it is difficult to know what to write in this space. Given publication lags, academic periodicals like Medical Education are a terrible place for “news” at the best of times. Our recently started release of “Accepted Articles” ensures this editorial will be made public in days rather tha...
Article
Full-text available
Safe and effective healthcare requires that new knowledge or skills, once learned, are incorporated into professional practice. However, this process is not always straightforward. Learning takes place in complex contexts, requiring practitioners to overcome various motivational, systemic, emotional, and social barriers to the application of knowle...
Article
Full-text available
Learning to take safe and effective action in complex settings rife with uncertainty is essential for patient safety and quality care. Doing so is not easy for trainees, as they often consider certainty to be a necessary precursor for action and subsequently struggle in these settings. Understanding how skillful clinicians work comfortably when unc...
Article
Full-text available
Many models of safe and effective clinical decision making in medical practice emphasize the importance of recognizing moments of uncertainty and seeking help accordingly. This is not always done effectively, but we know little about what cues prompt health professionals to call on resources beyond their own knowledge or skill set. Such information...
Article
Purpose: Using item analyses is an important quality monitoring strategy for written exams. Authors urge caution as statistics may be unstable with small cohorts, making application of guidelines potentially detrimental. Given the small cohorts common in health professions education, this study's aim was to determine the impact of cohort size on o...
Article
The authors describe influences associated with the incorporation of modern technologies into medical school admissions processes. Their purpose is not to critique or support specific technologies, but to prompt reflection upon the evolution that is afoot. Technology is now integral to the administration of multiple admissions tools, including the...
Article
Context The ability to self‐monitor one's performance in clinical settings is a critical determinant of safe and effective practice. Various studies have shown this form of self‐regulation to be more trustworthy than aggregate judgements (i.e. self‐assessments) of one's capacity in a given domain. However, little is known regarding what cues inform...
Article
PURPOSE: Medical education is dependent on clinicians and other faculty who volunteer time and expertise to teaching. Unfortunately, the literature reports increasing levels of dissatisfaction, burnout, and attrition. Incentivization provides an obvious intervention, but rewards must be implemented judiciously or risk unintended consequences. With...
Article
Clinician educators often experience distress caused by uncertainty regarding how effectively to participate in assessment practices in a way that supports both their programs and their students. Uncertainty is a common state for clinicians, particularly for those who see patients with early or ill-defined illness presentations. While clinicians of...
Chapter
The past three decades of research have seen substantial advances in medical education, much of it directly related to the application of sophisticated quantitative methods, particularly in the area of student assessment. This chapter begins with a commentary on the importance of precisely focusing one's research question, emphasising that while go...
Article
Full-text available
Information is generally more memorable after it is studied and tested than when it is only studied. One must be cautious to use this phenomenon strategically, however, due to uncertainty about whether testing improves memorability for only tested material, facilitates learning of related non-tested content, or inhibits memory of non-tested materia...
Article
Full-text available
Selection and recruitment into healthcare education and practice is a key area of interest for educators with significant developments in research, policy, and practice in recent years. This updated consensus statement, developed through a multi-stage process, examines future opportunities and challenges in selection and recruitment. There is both...
Article
In workplace-based assessment, research has suggested that aligning rating scales with how clinical supervisors naturally conceptualize trainee performance improves reliability and makes assessment more efficient. This study examined the generalizability of those findings by determining if construct alignment improves the reliability of competency-...
Article
en The Editor‐in‐chief bids a fond farewell to the stalwarts who have run the journal for more than a decade.
Article
The assessment of healthcare professionals is a critical determinant of patient safety and healthcare quality, playing both a gatekeeping function and a path toward performance improvement. Given the complexity of physician–patient interactions and the inadequacy of self-assessment for judging one's own strengths and weaknesses, medicine exemplifie...
Article
Purpose: There may be unintended consequences of broadening the competencies across which health professions trainees are assessed. This study was conducted to determine whether such broadening influences the formative guidance assessors provide to trainees and to test whether sequential collection of competency-specific assessment can overcome se...
Article
Full-text available
Introduction: Calls for enabling 'critical thinking' are ubiquitous in health professional education. However, there is little agreement in the literature or in practice as to what this term means and efforts to generate a universal definition have found limited traction. Moreover, the variability observed might suggest that multiplicity has value...
Article
Health professions the world over value various competencies in their practitioners that are not easily captured by academic measures of performance. As a result, many programs have begun using multiple mini-interviews (MMIs) to facilitate the selection of candidates who are most likely to demonstrate and further develop such qualities. In this twe...
Article
The Editor-in-chief outlines the vision for the 2018 State of the Science issue, which offers a 30 year retrospective on the influential Edinburgh Declaration.
Article
Full-text available
PURPOSE: The importance of confidence for learning and performance makes learners' perceptions of readiness for the next level of training valuable indicators of curricular success. The "Readiness for Clerkship" (RfC) and "Readiness for Residency" (RfR) surveys have been shown to provide reliable ratings of the relative effectiveness of various asp...
Article
PURPOSE: Health professionals are expected to routinely assess their weaknesses, set learning goals, and monitor their achievement. Unfortunately, it is well known that these professionals often struggle with effectively integrating external data and self-perceptions. To know how best to intervene, it is critical that the health professionals commu...
Article
Full-text available
Background: Asian medical students and doctors receive lower scores on average than their white counterparts in examinations in the UK and internationally (a phenomenon known as "differential attainment"). This could be due to examiner bias or to social, psychological or cultural influences on learning or performance. We investigated whether stude...
Article
Introduction: Although several studies have explored the relationship between learning and written tests, little is understood about how performance-based examinations influence learning. The purpose of this study was to explore how a formative objective structured clinical examination (OSCE) drives learning. Methods: We administered surveys to res...
Article
Full-text available
Validity is one of the most debated constructs in our field; debates abound about what is legitimate and what is not, and the word continues to be used in ways that are explicitly disavowed by current practice guidelines. The resultant tensions have not been well characterized, yet their existence suggests that different uses may maintain some valu...
Article
Full-text available
Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting 'idiosyncratic rater variance' is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical assessments h...
Article
The literature on multiple mini-interviews (MMIs) is replete with heterogeneous study results related to the constructs measured, correlations with other measures, and demographic relationships. Rather than view these results as contradictory, the authors ask, What if all of the results are correct? They point out that the MMI is not an assessment...
Article
Full-text available
Background Audit and feedback (A&F) is a common strategy for helping health providers to implement evidence into practice. Despite being extensively studied, health care A&F interventions remain variably effective, with overall effect sizes that have not improved since 2003. Contributing to this stagnation is the fact that most health care A&F inte...
Article
Purpose: In-training evaluation reports (ITERs) are ubiquitous in internal medicine (IM) residency. Written comments can provide a rich data source, yet are often overlooked. This study determined the reliability of using variable amounts of commentary to discriminate between residents. Method: ITER comments from two cohorts of PGY-1s in IM at t...
Article
Context: Interest is growing in the use of qualitative data for assessment. Written comments on residents' in-training evaluation reports (ITERs) can be reliably rank-ordered by faculty attendings, who are adept at interpreting these narratives. However, if residents do not interpret assessment comments in the same way, a valuable educational oppo...
Article
Purpose Training to become a physician is an emotionally laden experience. Research in cognitive psychology indicates that emotions can influence learning and performance, but the materials used in such research (e.g., word lists) rarely reflect the complexity of material presented in medical school. The present study examined whether emotions infl...
Article
Full-text available
Despite multifaceted attempts to "protect the public," including the implementation of various assessment practices designed to identify individuals at all stages of training and practice who underperform, profound deficiencies in quality and safety continue to plague the healthcare system. The purpose of this reflections paper is to cast a critica...
Article
This article describes the presentations and discussions at a conference co-convened by the Council on Medical Education of the American Medical Association (AMA) and by the American Board of Medical Specialties (ABMS). The conference focused on the ABMS Maintenance of Certification (MOC) Part III Examination. This article, reflecting the conferenc...
Article
Electronic practice data are increasingly being used to provide feedback to encourage practice improvement. However, evidence suggests that despite decades of experience, the effects of such interventions vary greatly and are not improving over time. Guidance on providing more effective feedback does exist, but it is distributed across a wide range...
Article
Objective: Receptiveness to interprofessional feedback, which is important for optimal collaboration, may be influenced by 'in-group or out-group' categorisation, as suggested by social identity theory. We used an experimental design to explore how nurses and resident physicians perceive feedback from people within and outside their own profession...
Article
Background: Audit and feedback (A&F) is a frequently used intervention aiming to support implementation of research evidence into clinical practice with positive, yet variable, effects. Our understanding of effective A&F has been limited by poor reporting and intervention heterogeneity. Our objective was to describe the extent of these issues. Me...
Article
Diagnostic reasoning has received substantial attention in the literature, yet what we mean by “diagnosis” may vary. Diagnosis can align with assignment of a “label,” where a constellation of signs, symptoms, and test results is unified into a solution at a single point in time. This “diagnostic labeling” conceptualization is embodied in our case-b...
Article
Theory: Assessment of clinical competence is a complex cognitive task with many mental demands often imposed on raters unintentionally. We were interested in whether this burden might contribute to well-described limitations in assessment judgments. In this study we examine the effect on indicators of rating quality of asking raters to (a) conside...
Article
Purpose: Interprofessional teamwork should include interprofessional feedback to optimize performance and collaboration. Social identity theory predicts that hierarchy and stereotypes may limit receptiveness to interprofessional feedback, but literature on this is sparse. This study explores perceptions among health professions students regarding...
Article
Purpose: To compare the relative utility of open-book examinations (OBEs) and closed-book examinations (CBEs) given the rapid expansion and accessibility of knowledge. Method: A systematic review of peer-reviewed articles retrieved from MEDLINE, ERIC, Embase, and PsycINFO (through June 2013). In 2013-2014, articles that met inclusion criteria we...
Article
Full-text available
Background: Health professions programs continue to search for meaningful and efficient ways to evaluate the quality of education they provide and support ongoing program improvement. Despite flaws inherent in self-assessment, recent research suggests that aggregated self-assessments reliably rank aspects of competence attained during preclerkship...
Article
Objectives: Easily palpable splenomegaly can be identified on physical examination, but it is difficult to detect lesser degrees of splenomegaly. Rapid bedside assessment can be conducted with hand-carried ultrasound. We performed this study to determine whether medical residents could reliably assess spleen size using hand-carried ultrasound afte...