Cees Van der Vleuten

• Managing Director at Maastricht University

Objective We expect medical students to be able to apply evidence-based medicine (EBM) skills in the context of the clinical care of patients. Previous assessments of this domain have primarily utilized decontextualized knowledge tests, which provide limited insights into students' understanding of EBM skills in the context of patient care. New per...

Context Four narrative profiles were previously developed as an evidence-informed framework for reflection and feedback on consultation performance in medical education. The profiles are grounded in four typologies mapped onto a conceptual framework, using the dimensions of doctor patient interaction (DPI) and medical expertise (ME) to classify ove...

Doctor-patient communication is a core competency in medical education, which requires learners to adapt their communication flexibly to each clinical encounter. Although conceptual learning models exist, information about how skilled communication develops over time is scant. This study aims to unpack this process of communication learning and to...

Purpose To create medical school service regions and examine national in-region graduate retention patterns across the medical education continuum and into professional practice as one approach to advancing social accountability in medical education. Methods Medical school service regions were created in Canada using publicly available data and ma...

Introduction: Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reason...

Introduction: We must ensure, through rigorous assessment that physicians have the evidence-based medicine (EBM) skills to identify and apply the best available information to their clinical work. However, there is limited guidance on how to assess EBM competency. With a better understanding of their current role in EBM education, Health Sciences L...

This study investigates the pivotal role of self-regulated learning in higher education and explores to what extent teachers’ conceptions of formative assessment influence the development of students’ self-regulated learning skills. Despite the recognised significance of self-regulated learning, many students in higher education lack effective self...

This article reports on findings of a qualitative study that investigated the difficulties teachers encounter while designing formative assessment plans and the strategies experienced teachers use to avoid those pitfalls. The pitfalls were identified through an analysis of formative assessment plans that searched for potential threats to alignment,...

Multiple Mini Interviews (MMIs) are sampling approaches that use multiple short stations to select prospective students for professional programmes. Each station uses different interview scenarios and raters to effectively assess candidates’ noncognitive skills. This study compared the performances of three sets of raters; academic, administrative...

High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S...

Introduction The shift toward an assessment for learning culture includes assessment quality criteria that emphasise the learning process, such as transparency and learning impact, in addition to the traditional validity and reliability criteria. In practice, the quality of the assessment depends on how the criteria are interpreted and applied. We...

Purpose: In health professions education (HPE) the effect of assessments on student motivation for learning and its consequences have been largely neglected. This is problematic because assessments can hamper motivation and psychological well-being. The research questions guiding this review were: How do assessments affect student motivation for l...

Most courses in higher education finish with one or more assessments which commonly all have to be passed. In these courses, student learning is commonly measured using conventional classroom tests, therefore test preparation is a common task for students. In higher education, compared with students’ prior education, they are faced with a more comp...

Purpose This study aims to report the design, development and evaluation of a digital quality assurance application aimed at improving and ensuring the quality of assessment programmes in higher education. Design/methodology/approach The application was developed using a design-based research (DBR) methodology. The application’s design was informe...

Purpose: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicin...

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of meas...

Objectives: To explore the key factors that influence professional identity construction in fourth year pharmacy students enrolled in a doctor of pharmacy program.Methods: A single site instrumental case study of current fourth year pharmacy students from the Leslie Dan Faculty of Pharmacy (LDFP), University of Toronto, was used. Thirteen students...

In the past few decades, health professions education programmes around the world have embraced the competency-based paradigm to guide the education and assessment of future healthcare workers [...]

Objective: Entrustable professional activities (EPAs) are used as clinical activities in postgraduate psychiatry training in Australasia. This study aimed to explore psychiatry trainees' perceptions of the impact of EPAs on their motivation and learning. Methods: A constructivist grounded theory approach was used to conceptualize the impact of E...

In programmatic assessment (PA), an arrangement of different assessment methods is deliberately designed across the entire curriculum, combined and planned to support both robust decision-making and student learning. In health sciences education, evidence about the merits and pitfalls of PA is emerging. Although there is consensus about the theoret...

Background Systematic reviews on simulation training effectiveness have pointed to the need to adhere to evidence-based instructional design (ID) guidelines. ID guidelines derive from sound cognitive theories and aim to optimize complex learning (integration of knowledge, skills, and attitudes) and learning transfer (application of acquired knowled...

Objectives At the very start of medical residency training, entrustment of clinical tasks may be a major challenge, on which current scientific knowledge is scarce. This study therefore aimed to gain insight into the process of supervisors’ decision making underlying the entrustment of clinical tasks at the start of one-on-one supervisor–trainee wo...

Phenomenon: Social accountability has become a universal component in medical education. However, medical schools have little guidance for operationalizing and applying this concept in practice. This study explored institutional practices and administrative perceptions of social accountability in medical education. Approach: An online survey was di...

Background Collaborative learning is a group learning approach in which positive social interdependence within a group is key to better learning performance and future attitudes toward team practice. Recent attempts to replace a face-to-face environment with an online one have been developed using information communication technology. However, this...

Introduction: Programmatic assessment is an approach to assessment aimed at optimizing the learning and decision function of assessment. It involves a set of key principles and ground rules that are important for its design and implementation. However, despite its intuitive appeal, its implementation remains a challenge. The purpose of this paper...

Background In medical residency, performance observations are considered an important strategy to monitor competence development, provide feedback and warrant patient safety. The aim of this study was to gain insight into whether and how supervisor-resident dyads build a working repertoire regarding the use of observations, and how they discuss and...

In the context of where the nature of the learning environment allows only minimum feedback, and less participation in learning, a uniform national examination seems to be a feasible way to standardize quality in health professions education. However, this kind of high-stakes national examination may represent the social hierarchy indicated in this...

The assessment of workplace learning by educators at the workplace is a complex and inherently social process, as the workplace is a participatory learning environment. We therefore propose seeing assessment as a process of judgment embedded in a community of practice and to this purpose use the philosophy of inferentialism to unravel the judgment...

Interprofessional education (IPE) often uses collaborative learning for better teamwork among multiple professions. Its theoretical background is social interdependence theory (SIT), and positive interdependence is key for successful collaborative learning. As there is little theoretical knowledge on how to optimize the social interdependence in IP...

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to an...

Associations between assessment and learning are widely studied and often organized around the notions of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). Although these notions are appealing in theory, the notions are unclear constructs to comprehend, as both their definitions and their practice are us...

Objective: To validate and refine typologies of consultation performance from previous research to identify learning needs associated with each typology. Methods: We performed a qualitative study in a General Practice Specialty Training programme, using a two-stage design. First, we selected four exemplars from 80 videotaped consultations of 7 f...

Introduction: Previous studies indicate that professional identity formation (PIF), the formation of a self-identity with the internalised values and norms of professionalism, may influence self-regulated learning (SRL). However, it remains unclear whether a PIF-oriented intervention can improve SRL in clinical education. The aim of this study was...

Optimize your assessment processes through Quality Assurance. This is a ground-breaking guide to ensuring quality assurance in the movement toward competency-based medical education With the increasing globalization of medical education comes the need for mutual recognition of quality and standards. Understanding Assessment in Medical Education thr...

Over the last decade, programmatic assessment has gained popularity in health professions education. This modern approach requires new ways of thinking about assessment. Programmatic assessment provides a framework to guide assessment practices, which allows health professions institutes to design their programs optimally for their (educational) co...

Introduction In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define...

Introduction Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of...

Educators use entrustment, a common framework in competency-based medical education, in multiple ways, including frontline assessment instruments, learner feedback tools, and group decision making within promotions or competence committees. Within these multiple contexts, entrustment decisions can vary in purpose (i.e., intended use), stakes (i.e.,...

Objectives The objectives of this scoping review were to (a) explore how pharmacists perceive their professional roles and identities and (b) describe factors impacting which professional roles or identities pharmacists embody in different pharmacy practice settings. Methods A scoping review using a deductive approach was undertaken for this study...

Background Problem-based learning (PBL) is classified as a collaborative learning approach, wherein students learn while contributing meaning to experiences and interactions with others. An important theoretical fundament of PBL is social interdependence theory (SIT) because positive social interdependence within a group has been found to be key to...

Introduction: With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores. Methods: During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242...

Background: Problem-based learning (PBL) is classified as a collaborative learning approach, wherein students learn while contributing meaning to experiences and interactions with others. An important theoretical fundament of PBL is social interdependence theory (SIT) because positive social interdependence within a group has been found to be key t...

Background Developing self-regulated learning in preclinical settings is important for future lifelong learning. Previous studies indicate professional identity formation, i.e., formation of self-identity with internalized values and norms of professionalism, might promote self-regulated learning. We designed a professional identity formation-orien...

Background Several studies have shown that conceptions of teachers on teaching and learning can influence the teaching practices and behavior in higher education. This association is also found in undergraduate medical education but not yet established in postgraduate medical setting. An instrument, Conceptions of Learning and Teaching (COLT) was d...

Background Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designe...

The way quality of assessment has been perceived and assured has changed considerably in the recent 5 decades. Originally, assessment was mainly seen as a measurement problem with the aim to tell people apart, the competent from the not competent. Logically, reproducibility or reliability and construct validity were seen as necessary and sufficient...

To decrease infusion pump administration errors, time-consuming training is often initiated. The aims of this study were twofold: to develop minimum competency requirements for programming and operation of infusion pumps and to develop and validate a test for nurses based on those requirements. The test was completed by 226 nurses between May and D...

Introduction Programmatic assessment was introduced as an approach to design assessment programmes with the aim to simultaneously optimize the decision-making and learning function of assessment. An integrative review was conducted to review and synthesize results from studies investigating programmatic assessment in health care professions educati...

Introduction: There has been a marked increase in institutional structures developed to support health professions education scholarship recently. These health professions education scholarship units (HPESUs) engage in a diverse range of activities. Previous work provided insight into factors that influence the functioning of such units, but data...

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived domina...

Purpose: Medical schools face growing pressures to produce stronger evidence of their social accountability, but measuring social accountability remains a global challenge. This narrative review aimed to identify and document common themes and indicators across large-scale social accountability frameworks to facilitate development of initial opera...

Introduction Recent assessment innovations have included the development of resident-sensitive quality measures (RSQMs), which are characterized by: 1) their importance to care quality, and 2) likelihood they are performed by residents. This study explored how clinical competency committee (CCC) members interpret, use, and prioritize RSQM data adde...

Objectives: This study aimed to determine the conceptions of teaching and learning of postgraduate medical teachers. Methods: We invited postgraduate clinical teachers to fill out COLT (Conceptions on Learning and Teaching) questionnaire, an 18-item instrument designed to measure the conceptions of faculty in undergraduate medical education, and...

Introduction: Teachers’ conceptions of learning and teaching (COLT) affect their teaching behaviour. The 18 item COLT instrument has been developed in the Netherlands and comprises three scales, ‘teacher centredness’, ‘appreciation of active learning’ and ‘orientation to professional practice’. Previously we found five teacher profiles. The aim of...

Background: Developing self-regulated learning (SRL) in a teacher-centered curriculum is challenging. Previous studies indicate professional identity formation (PIF), or the formation of self-identity with internalized values and norms of professionalism, might improve SRL．We designed a PIF-oriented education format for hybrid problem-based learnin...

Background: Developing self-regulated learning in preclinical settings is important for future lifelong learning. Previous studies indicate professional identity formation, i.e., formation of self-identity with internalized values and norms of professionalism, might promote self-regulated learning．We designed a professional identity formation-orien...

Background: Developing self-regulated learning in preclinical settings is important for future lifelong learning. Previous studies indicate professional identity formation, i.e., formation of self-identity with internalized values and norms of professionalism, might promote self-regulated learning．We designed a professional identity formation-orien...

The relationship between clinician teachers and their students is of major importance in medical education. However, there is little known about the effects on clinicians when conflict occurs with their students. What do clinicians perceive to be major causes of these conflicts? How do they react when and after conflict occurs? A phenomenological i...

Low-stakes assessments are theorised to stimulate and support self-regulated learning. They are feedback-, not decision-oriented, and should hold little consequences to a learner based on their performance. The use of low-stakes assessment as a learning opportunity requires an environment in which continuous improvement is encouraged. This may be h...

Background: Direct observation of clinical task performance plays a pivotal role in competency-based medical education. Although formal guidelines require supervisors to engage in direct observations, research demonstrates that trainees are infrequently observed. Supervisors may not only experience practical and socio-cultural barriers to direct o...

Objective: Evidence-based medicine practices of medical students in clinical scenarios are not well understood. Optimal foraging theory (OFT) is one framework that could be useful in breaking apart information-seeking patterns to determine effectiveness and efficiency of different methods of information seeking. The aims of this study were to use O...

Purpose: Resident-sensitive quality measures (RSQMs) are quality measures that are likely performed by an individual resident and are important to care quality for a given illness of interest. This study sought to explore how individual clinical competency committee (CCC) members interpret, use, and prioritize RSQMs alongside traditional assessmen...

Objectives: Despite widespread use of medical devices and their increasing complexity, their contribution to unintended injury caused by healthcare (adverse events, AEs) remains relatively understudied. The aim of this study was to gain insight in the incidence and types of AEs involving medical devices (AMDEs). Methods: Data from two patient re...

Purpose: This study explores the associations between resident-sensitive quality measures (RSQMs) and supervisor entrustment as well as between RSQMs and patient acuity and complexity for encounters in the pediatric emergency department (PED) in which residents are caring for patients. Method: Pediatric residents rotating through Cincinnati Chil...

Introduction: Programmatic assessment attempts to facilitate learning through individual assessments designed to be low-stakes and used only for high-stakes decisions when aggregated. In practice, low-stakes assessments have yet to reach their potential as catalysts for learning. Due to the powerful role teachers play in utilising assessment to su...

Background: Previous studies indicate professional identity formation (PIF), or formation of self-identity with internalized values and norms of professionalism, might influence self-regulated learning (SRL). However, it remains unclear whether or not PIF-oriented education can improve SRL in medical students. Methods: Rural community-based clinica...

Purpose A lack of quality measures aligned with residents’ work led to the development of resident-sensitive quality measures (RSQMs). This study sought to describe how often residents complete RSQMs, both individually and collectively, when they are implemented in the clinical environment. Method During academic year 2017–2018, categorical pediat...

Recent discussions have brought attention to the utility of contribution analysis for evaluating the effectiveness and outcomes of medical education programs, especially for complex initiatives such as competency-based medical education. Contribution analysis focuses on the extent to which different entities contribute to an outcome. Given that hea...

Arguably, constructive alignment has been the major challenge for assessment in the context of problem-based learning (PBL). PBL focuses on promoting abilities such as clinical reasoning, team skills and metacognition. PBL also aims to foster self-directed learning and deep learning as opposed to rote learning. This has incentivized researchers in...

Vocational education aims at preparing students for their future as professionals; thus, students are often required to undergo a form of on-the-job learning. Assessment of professional performance during on-the-job learning is not based on single observations, but on prolonged participation at the workplace culminating in judgement by a workplace...

Background: The cost of depression among residents is staggering as it extends into the quality and safety of patient care. Finding an explanation to resident depression by investigating the associated factors is therefore important. Study skills can be a possible factor, and a clear gap in the literature exists in this regard. We sought to invest...

Purpose: Internationally, health professions education scholarship units (HPESUs) are often developed to promote engagement in educational scholarship, yet little is known about how HPESUs change over time or what factors support their longevity. In hopes of helping HPESUs thrive, this study explored factors that shaped the evolution of 8 HPESUs o...

Introduction: Programmatic assessment (PA) is an approach to assessment aimed at optimizing learning which continues to gain educational momentum. However, the theoretical underpinnings of PA have not been clearly described. An explanation of the theoretical underpinnings of PA will allow educators to gain a better understanding of this approach an...

This AMEE guide provides a framework and practical strategies for teachers, learners and institutions to promote meaningful feedback conversations that emphasise performance improvement and professional growth. Recommended strategies are based on recent feedback research and literature, which emphasise the sociocultural nature of these complex inte...

Medical education research and development’s reason for existence is their contribution to producing better doctors. Arguably this is as. notion that nobody would disagree with. But, answering this question is not as straightforward as it may look. In this paper we describe six complexities that impact on such research and unfortunately contribute...

Purpose: Assessment in different languages should measure the same construct. However, item characteristics, such as item flaws and content, may favor one test-taker group over another. This is known as item bias. Although some studies have focused on item bias, little is known about item bias and its association with items characteristics. Theref...

Background Recent focus has been placed on the entrustment construct for assessing residents; specifically, how much supervision is needed to provide safe care. This method of assessment puts the patient at its core. However, whether entrustment decisions are related to quality care is not known. Objective Determine the association between entrust...

Purpose: Newer definitions of feedback emphasize learner engagement throughout the conversation, yet teacher and learner perceptions of each other's behaviors during feedback exchanges have been less well studied. This study explored perceptions of residents and faculty regarding effective behaviors and strategies during feedback conversations and...

Background: Community-based education (CBE) is strategically important to provide contextual learning for medical students. CBE is a priority for countries striving for better primary health care. However, the CBE literature provides little curriculum guidance to enhance undergraduate medical education with the primary health care context. We aim...

Objective: This paper addresses how communication skills can best be assessed. Since assessment and learning are strongly connected, the way communication skills are best learned is also described. Results: Communication skills are best learned in a longitudinal fashion with ample practice in an authentic setting. Confrontation of behavior initi...

Background Previous studies indicate that a teacher-centered context could hinder undergraduates from self-regulated learning (SRL), whereas a learner-centered context could promote SRL. However, SRL development between a teacher-centered and a learner-centered context has not directly compared in undergraduate settings. Also, it is still unclear h...

Assessment and evaluation of trainees’ clinical performance measures is needed to ensure safe, high-quality patient care. These measures also aid in the development of reflective, high-performing clinicians and hold graduate medical education (GME) accountable to the public. While clinical performance measures hold great potential, challenges of de...

Objective: To compare postpartum hemorrhage (PPH) patient outcomes before and after an in situ instructional design based PPH simulation attended by obstetrics and gynecology (OBGYN) residents. Methods: This uncontrolled before-and-after study was conducted in Recife, Brazil including all 1388 women delivering from June to August 2012 and all 1357...

Purpose: To inform graduate medical education (GME) outcomes at the individual resident level, this study sought a method for attributing care for individual patients to individual interns based on "footprints" in the electronic health record (EHR). Method: Primary interns caring for patients on an internal medicine inpatient service were record...

Abstract PURPOSE: To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance. METHOD: In October 2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessin...

Purpose: According to the principles of programmatic assessment, a valid high-stakes assessment of the students’ performance should amongst others, be based on a multiple data points, supposedly leading to saturation of information. Saturation of information is generated when a data point does not add important information to the assessor. In estab...

Background The educational environment is critical to learning and is determined by social interactions. Trainee satisfaction translates to career commitment, retention and a positive professional attitude as well as being an important factor in assessing the impact of the training program. This study aimed to validate the Scan of Postgraduate Educ...

Background: Depression among health professional students is a topic of concern. Often, it is associated with other unhealthy habits such as smoking. This study aimed to find the prevalence rate of depressive symptoms and smoking among applied medical science (AMS) students and their associations with each other. Methodology: In this descriptive-an...