ArticlePublisher preview availableLiterature Review

Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework

May 2021
Advances in Health Sciences Education 26(4):1-26

DOI:10.1007/s10459-020-10002-1

Authors:

Nyoli Valentine

ModMed

Steven J Durning

Uniformed Services University of the Health Sciences

Ernst Michael Shanahan

Flinders Medical Centre

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a ‘problem’ to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back and changing perspectives to focus on the fundamental underlying value of fairness in assessment may help re-set the traditional objective approach and provide a more relevant way to determine the appropriateness of subjective human judgements. Changing focus to look at what is ‘fair’ human judgement in assessment, rather than what is ‘objective’ human judgement in assessment allows for the embracing of many different perspectives, and the legitimising of human judgement in assessment. However, this requires addressing the question: what makes human judgements fair in health professions assessment? This is not a straightforward question with a single unambiguously ‘correct’ answer. In this hermeneutic literature review we aimed to produce a scholarly knowledge synthesis and understanding of the factors, definitions and key questions associated with fairness in human judgement in assessment and a resulting conceptual framework, with a view to informing ongoing further research. The complex construct of fair human judgement could be conceptualised through values (credibility, fitness for purpose, transparency and defensibility) which are upheld at an individual level by characteristics of fair human judgement (narrative, boundaries, expertise, agility and evidence) and at a systems level by procedures (procedural fairness, documentation, multiple opportunities, multiple assessors, validity evidence) which help translate fairness in human judgement from concepts into practical components.

The hermeneutic circle as a framework for the literature review (Boell and Cecez-Kecmanovic 2014)

…

Search strategies used in the literature review

…

Conceptual framework of fairness in human judgement in assessment. The values of fairness are supported by individual characteristics and system factors

…

Figures - available from: Advances in Health Sciences Education

This content is subject to copyright. Terms and conditions apply.

A preview of this full-text is provided by Springer Nature.

Learn more

Content available from Advances in Health Sciences Education

This content is subject to copyright. Terms and conditions apply.

Vol.:(0123456789)

Advances in Health Sciences Education (2021) 26:713–738

https://doi.org/10.1007/s10459-020-10002-1

1 3

REVIEW

Fairness inhuman judgement inassessment: ahermeneutic

literature review andconceptual framework

NyoliValentine1 · StevenDurning2· ErnstMichaelShanahan1· LambertSchuwirth1

Received: 13 May 2020 / Accepted: 19 October 2020 / Published online: 29 October 2020

Abstract

Human judgement is widely used in workplace-based assessment despite criticism that it

does not meet standards of objectivity. There is an ongoing push within the literature to

better embrace subjective human judgement in assessmentnot as a ‘problem’ to be cor-

rected psychometrically but as legitimate perceptions of performance. Taking a step back

and changing perspectives to focus on the fundamental underlying value of fairness in

assessment may help re-set the traditional objective approach and provide a more relevant

way to determine the appropriateness of subjective human judgements. Changing focus

to look at what is ‘fair’human judgement in assessment, rather than what is ‘objective’

human judgement in assessment allows for the embracing of many diﬀerent perspectives,

and the legitimising of human judgement in assessment. However, this requires addressing

the question: what makes human judgements fair in health professions assessment? This is

not a straightforward question with a single unambiguously ‘correct’ answer. In this herme-

neutic literature review we aimed to produce a scholarly knowledge synthesis and under-

standing of the factors, deﬁnitions and key questions associated with fairness in human

judgement in assessment and a resulting conceptual framework, with a view to informing

ongoing further research. The complex construct of fair human judgement could be con-

ceptualised through values (credibility, ﬁtness for purpose, transparency and defensibility)

which are upheld at an individual level by characteristics of fair human judgement (nar-

rative, boundaries, expertise, agility and evidence) and at a systems level by procedures

(procedural fairness, documentation, multiple opportunities, multiple assessors, validity

evidence) which help translate fairness in human judgement from concepts into practical

components.

Keywords Assessment· Fairness· Health professions education· Judgement· Subjective

* Nyoli Valentine

vale0046@ﬂinders.edu.au

1 Prideaux Health Professions Education, Flinders University, BedfordPark5042, SA, Australia

2 Center forHealth Professions Education, Uniformed Services University oftheHealth Sciences,

Bethesda, MD, USA

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Professional judgement: a social practice perspective on a multiple mini-interview for specialty training selection

Article

Full-text available

Jan 2025
BMC Med Educ

Background Interviewers’ judgements play a critical role in competency-based assessments for selection such as the multiple-mini-interview (MMI). Much of the published research focuses on the psychometrics of selection and the impact of rater subjectivity. Within the context of selecting for entry into specialty postgraduate training, we used an interpretivist and socio-constructivist approach to explore how and why interviewers make judgments in high stakes selection settings whilst taking part in an MMI. Methods We explored MMI interviewers’ work processes through an institutional observational approach, based on the notion that interviewers’ judgements are socially constructed and mediated by multiple factors. We gathered data through document analysis, and observations of interviewer training, candidate interactions with interviewers, and interviewer meetings. Interviews included informal encounters in a large selection centre. Data analysis balanced description and explicit interpretation of the meanings and functions of the interviewers’ actions and behaviours. Results Three themes were developed from the data showing how interviewers make professional judgements, specifically by; ‘Balancing the interplay of rules and agency,’ ‘Participating in moderation and shared meaning making; and ‘A culture of reflexivity and professional growth.’ Interviewers balanced the following of institutional rules with making judgment choices based on personal expertise and knowledge. They engaged in dialogue, moderation, and shared meaning with fellow interviewers which enabled their consideration of multiple perspectives of the candidate’s performance. Interviewers engaged in self-evaluation and reflection throughout, with professional learning and growth as primary care physicians and supervisors being an emergent outcome. Conclusion This study offers insights into the judgment-making processes of interviewers in high-stakes MMI contexts, highlighting the balance between structured protocols and personal expertise within a socially constructed framework. By linking MMI practices to the broader work-based assessment literature, we contribute to advancing the design and implementation of more valid and fair selection tools for postgraduate training. Additionally, the study underscores the dual benefit of MMIs—not only as a selection tool but also as a platform for interviewers’ professional growth. These insights offer practical implications for refining future MMI practices and improving the fairness of high-stakes selection processes.

Políticas agrarias, violencia y apuestas por la reconciliación en el campo colombiano

Article

Jul 2024

El estudio del proceso de paz en Colombia es un proceso complejo, no solo por la amplia diversidad de actores y contextos, sino porque su determinación puede ser explorada en las dimensiones histórica y presente. Además, las pesquisas comprometidas con la construcción de una praxis de paz frecuentemente se llevan a cabo en un plano reducido o no logran le necesaria integración entre los aspectos macrosociales y lo local. Por tanto, se realizó un estudio de revisión cualitativa, con enfoque hermenéutico y crítico, encaminado a profundizar en la relación entre políticas agrarias, violencia y paz, en el campo colombiano. Los resultados arrojaron cuatro temas fundamentales que favorecen el análisis histórico, la contextualización de las influencias y la comprensión de las contradicciones generadas por las divergencias entre los modelos de desarrollo concebidos desde arriba en su engranaje con lo local y patrimonial. Los hallazgos permitieron concluir que el proceso paz, aunque todavía distante de lo anticipado, presenta potencialidades para su construcción si las políticas agrarias parten del empoderamiento local-comunitario y concilian voluntades.

Gentrification as a field of study in environmental sciences

Article

Full-text available

Sep 2024

Gelber Rosas-Patiño

This article examines the interaction between gentrification and environmental sciences in Colombia, using a hermeneutic desk review approach to unravel how this urban phenomenon influences and is influenced by environmental factors. Through the hermeneutic circle method, adapted for a single researcher, a deep understanding of the existing literature is achieved and main lines of research are identified. Areas explored include green gentrification, climate change impacts, health and well-being effects, environmental justice, and associated public policies. The study highlights the need for policies that balance environmental improvement with social protection, and underlines the importance of including local communities in urban planning processes to prevent displacement and increase social equity. This interdisciplinary approach provides valuable insights to understand the complexity of gentrification and its multiple dimensions in an urban and environmental context

The social impacts of circular economy: disclosing epistemological stances and methodological practices

Article

Full-text available

Oct 2024
Environ Dev Sustain

The essence of circular economy is about closing the loop of material flows by reducing, reusing, recovering and recycling, ensuring the economic prosperity without any detriment for the environment. However, few information is available about the contribution of circular systems to the social dimension of sustainability (such as well-being, quality of life), how these aspects are evaluated and why they are deemed as important. The purpose of this study is to review those scientific studies that dealt with the social sustainability evaluations of circular systems to highlight new aspects: not only which are the most applied methodologies, which impact categories and indicators are mostly taken into account, but, above all, which is the most diffused epistemological position underlying the evaluation of social aspects of circularity. This last aspect is often disregarded but of utmost importance because it shapes the approaches and procedures choice, and legitimate and justify the insights provided: indeed, they can be the most diverse when dealing with social sciences, which are multiparadigmatic sciences. To the best of our knowledge, no studies have so far emerged that directly investigate the epistemological basis of social impacts within the circular economy. A systematic and hermeneutic review of the scientific literature has been led through a two steps method. The first step, dedicated to search and acquisition, consisted in a standardized replicable process called Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). In a second phase, a hermeneutic review has been conducted on the selected publications. Among the 42 studies reviewed, 7 papers were framed within the post-positivism paradigms, while 83,3% were based on interpretivist paradigms, showing how it is difficult to find consensus on standardized quantification methods for social impacts in circular economy studies. The most recurring qualitative methods relied on personal interpretations, perceptions or observations.

Legal framework and procedures of occupational health. A hermeneutic analysis

Article

Full-text available

Jul 2024

Establishing safety standards for regulating workers' health is one of the main components of a labor system. In order to optimize productive activities in a way that aligns with the well-being, rights, and duties of the worker, it is crucial to properly operationalize the legal framework, the procedures for its execution, and the establishment of clear responsibilities and guarantees. The study was aimed at a qualitative and hermeneutic review of the relevant literature, both nationally and internationally, in the Scielo, Redalyc, Dialnet, and Scopus databases. The study allowed for the mapping of the literature, a critical assessment of the texts, and the representation of the main categories according to the initially established themes. The results indicate that it is necessary to implement a safety plan in light of current regulations. In addition, this legal device must operate in conjunction with other cooperatives, the union with the academic sector, and the support of state health entities, thus establishing cooperative work that benefits all sectors in the spinoff style.

Ethical Considerations in Ai-Driven Facial Authentication for Online Testing and Examinations

Article

Full-text available

Aug 2024

Temitope Oluwafunmilayo Adetunji

Artificial intelligence (AI)-powered facial recognition technology has turn out to be more prevalent in online testing and exams, encouraging better security and swift authentication processes. Nevertheless, ethical matters are necessary to be carefully deliberated putting it to practice. The ethical concerns about the usage of AI-powered facial recognition in online assessments are explored in this paper. Via thorough research of its advantages and disadvantages, we elucidate the complicated relationship between technical progress and moral issues. Privacy concerns are judiciously considered, as well as the possibility of illegal surveillance and the biases existing in facial recognition software. Furthermore, we offer a set of ethical standards for online learning environments that safeguard user identities and regard privacy rights. We hope to add to the creation of moral outlines that uphold justice, accountability, and transparency in the use of facial recognition technology for virtual testing and exams by dealing with these moral problems and supporting accountable deployment.

"Praise in Public; Criticize in Private": Unwritable Assessment Comments and the Performance Information That Resists Being Written

Article

Aug 2024

Purpose Written assessment comments are needed to archive feedback and inform decisions. Regrettably, comments are often impoverished, leaving performance-relevant information undocumented. Research has focused on content and supervisor’s ability and motivation to write it but has not sufficiently examined how well the undocumented information lends itself to being written as comments. Because missing information threatens the validity of assessment processes, this study examined the performance information that resists being written. Method Two sequential data collection methods and multiple elicitation techniques were used to triangulate unwritten assessment comments. Between November 2022 and January 2023, physicians in Canada were recruited by email and social media to describe experiences with wanting to convey assessment information but feeling unable to express it in writing. Fifty supervisors shared examples via survey. From January to May 2023, a subset of 13 participants were then interviewed to further explain what information resisted being written and why it seemed impossible to express in writing and to write comments in response to a video prompt or for their own “unwritable” example. Constructivist grounded theory guided data collection and analysis. Results Not all performance-relevant information was equally writable. Information resisted being written as assessment comments when it would require an essay to be expressed in writing, belonged in a conversation and not in writing, or was potentially irrelevant and unverifiable. In particular, disclosing sensitive information discussed in a feedback conversation required extensive recoding to protect the learner and supervisor-learner relationship. Conclusions When written comment requests to document performance information are viewed as an act of disclosure, it becomes clear why supervisors may feel compelled to leave some comments unwritten. Although supervisors can be supported in writing better assessment comments, their failure to write invites a reexamination of expectations for documenting feedback and performance information as written comments on assessment forms.

Sistem Rekomendasi Al-Quran Berbasis Topik

Article

Full-text available

Apr 2024

Penelitian ini bertujuan untuk mengatasi tantangan kesehatan mental remaja Indonesia, yang merupakan salah satu hambatan dalam mencapai visi "Indonesia Emas" pada tahun 2045, di mana negara diharapkan memasuki masa bonus demografi dengan mayoritas penduduk di usia produktif. Data Kementerian Kesehatan (2023) menunjukkan bahwa 6.1% penduduk usia 15 tahun ke atas mengalami gangguan kesehatan mental, menandakan kebutuhan mendesak untuk intervensi yang efektif. Penelitian ini mengembangkan sistem rekomendasi ayat-ayat Al-Quran, menggunakan konsep similarity dan kategorisasi ayat (topik) berdasarkan Dewan Syariah Nasional MUI. Penelitian ini menyediakan rekomendasi yang akurat dan relevan untuk membantu penggunanya untuk mendapatkan solusi masalah kesehatan mental yang Islami. Evaluasi sistem menunjukkan bahwa mayoritas rekomendasi yang dihasilkan sesuai dengan pandangan para ahli, menandakan keefektifan sistem dalam menyediakan referensi yang jelas dan akurat. Penelitian ini tidak hanya memberikan wawasan baru dalam pengembangan solusi berbasis teks religius untuk isu kesehatan mental, tetapi juga berkontribusi pada upaya lebih luas dalam menghadapi tantangan sosial dan kesehatan yang dihadapi Indonesia.

Evaluating the Implementation of Quality Principles in Competence-Based Assessment at Private TVET Colleges: Perspectives, Differences, Influencing Factors, and Impacts on Student Competence

Article

Full-text available

Nov 2024

This study evaluates the implementation of quality principles in CBA at private TVET colleges in the Amhara Regional State. It employed a sequential explanatory mixed-methods approach, incorporating questionnaires, interviews, and document analysis. A questionnaire was distributed to 873 students and 189 teachers and assessors, including 47 items on a five-point Likert scale. Interviews involved 5 teachers, 4 assessors, 5 students, 2 deans, and 1 assessment center coordinator. The analysis included descriptive statistics such as proportions, means, and standard deviations, while inferential analysis utilized one-sample t-tests, Mann-Whitney U tests, and logistic regression. Qualitative data were analyzed through thematic and content analysis to gain a comprehensive understanding of CBA implementation. The findings indicated a significantly low implementation of CBA across all nine quality principles, with no substantial difference in perceptions between teachers and students. This overall low implementation adversely affected the quality of assessment and student competence. Key challenges identified included unethical interference of regulatory bodies and colleges, inadequate assessment literacy, unethical behavior among teachers and assessors, weak educational backgrounds, and dishonest conduct among students. The insufficient implementation of CBA principles in private TVET colleges complicates the relationship between students’ perceptions of implementation and their competence. Consequently, it is essential to provide professional support for educational assessment approaches to regulatory bodies, colleges, assessors, and teachers. Standardizing the assessments of IOCA, NOCA, and ROCA is critical, along with clearly distinguishing the roles of teachers, assessment tool developers, and assessors to enhance assessment quality.

Upholding ethical pillars in nursing academia

Article

Sep 2024
NURS ETHICS

William G Zic

This manuscript explores the philosophical implications of ethical principles such as honesty, integrity, fairness, reliability, and objectivity and their impact on professional nursing. By examining these values within Western society, the discussion highlights the importance of integrating these virtues into contemporary nursing education. Through a detailed analysis of each precept, the document underscores their potential to enhance the quality of education, improve interactions among faculty and staff, and achieve positive student outcomes. Ultimately, this treatise advocates for a balanced pedagogical approach in nursing that leverages these elements to foster a more compassionate world, where ethical connections in academia underpin our collective existence.

Toward Understanding Why Fairness Matters: The Influence of Mortality Salience on Reactions to Procedural Fairness

Article

Full-text available

Sep 2000

This article focuses on the question of why fairness matters to people. On the basis of fairness heuristic theory, the authors argue that people especially need fairness when they are uncertain about things that are important to them. Following terror management theory, the authors focus on a basic kind of human uncertainty: fear of death. Integrating these two theoretical frameworks, it is proposed that thinking about their mortality should make fairness a more important issue to people. The findings of three experiments support the authors’ line of reasoning: Asking participants to think about their mortality led to stronger fair process effects (positive effects of perceived procedural fairness on subsequent reactions) than not asking them to think about mortality. It is argued that these findings suggest that fairness especially matters to people when they are uncertain about fundamental aspects of human life such as human mortality.

When Do We Need Procedural Fairness? The Role of Trust in Authority

Article

Full-text available

Dec 1998

On the basis of fairness heuristic theory, the authors argue that when information about whether an authority can be trusted is not available, people will resolve the question of how they should interpret the decisions of the authority by relying on perceived procedural fairness. As a consequence, people who do not have information about authority's trustworthiness react more positively toward the outcomes of authority's decisions if the authority is using fair as opposed to unfair procedures. However, when people know that the authority either can or cannot be trusted, they are less in need of procedural fairness information, yielding less strong effects of procedural fairness on people's reactions. The findings of 2 experiments support the authors’ line of reasoning. It is concluded that people especially need procedural fairness when information about an authority's trustworthiness is lacking.

Identifying the narrative used by educators in articulating judgement of performance

Article

Full-text available

Mar 2019

Introduction Modern assessment in medical education is increasingly reliant on human judgement, as it is clear that quantitative scales have limitations in fully assessing registrars’ development of competence and providing them with meaningful feedback to assist learning. For this, possession of an expert vocabulary is essential. Aim This study aims to explore how medical education experts voice their subjective judgements about learners and to what extent they are using clear, information-rich terminology (high-level semantic qualifiers); and to gain a better understanding of the experts’ language used in these subjective judgements. Methods Six experienced medical educators from urban and rural environments were purposefully selected. Each educator reviewed a registrar clinical case analysis in a think out loud manner. The transcribed data were analyzed, codes were identified and ordered into themes. Analysis continued until saturation was reached. Results Five themes with subthemes emerged. The main themes were: (1) Demonstration of expertise; (2) Personal credibility; (3) Professional credibility; (4) Using a predefined structure and (5) Relevance. Discussion Analogous to what experienced clinicians do in clinical reasoning, experienced medical educators verbalize their judgements using high-level semantic qualifiers. In this study, we were able to unpack these. Although there may be individual variability in the exact words used, clear themes emerged. These findings can be used to develop a helpful shared narrative for educators in observation-based assessment. The provision of a rich, detailed narrative will also assist in providing clarity to registrar feedback with areas of weakness clearly articulated to improve learning and remediation.

The Power of Subjectivity in the Assessment of Medical Trainees

Article

Full-text available

Oct 2018
ACAD MED

Objectivity in the assessment of students and trainees has been a hallmark of quality since the introduction of multiple-choice items in the 1960s. In medical education, this has extended to the structured examination of clinical skills and workplace-based assessment. Competency-based medical education, a pervasive movement that started roughly around the turn of the century, similarly calls for rigorous, objective assessment to ensure that all medical trainees meet standards to assure quality of health care. At the same time, measures of objectivity, such as reliability, have consistently shown disappointing results. This raises questions about the extent to which objectivity in such assessments can be ensured. In fact, the legitimacy of “objective” assessment of individual trainees, particularly in the clinical workplace, may be questioned. Workplaces are highly dynamic and ratings by observers are inherently subjective, as they are based on expert judgment, and experts do not always agree—for good, idiosyncratic, reasons. Thus, efforts to “objectify” these assessments may be problematically distorting the assessment process itself. In addition, “competence” must meet standards, but it is also context dependent. Educators are now arriving at the insight that subjective expert judgments by medical professionals are not only unavoidable but actually should be embraced as the core of assessment of medical trainees. This paper elaborates on the case for subjectivity in assessment.

Managing tensions in assessment: moving beyond either-or thinking

Article

Full-text available

Oct 2018

Context In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional ‘fix‐the‐problem’ approaches (i.e. either–or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes. Methods Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its ‘both–and’ approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment. Discussion Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its ‘both–and’ approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.

Standing up for Subjectivity in the Assessment of Competencies

Article

Full-text available

Aug 2018

Thomas Rotthoff

Assessment, feedback and the alchemy of learning

Article

Full-text available

Aug 2018
MED EDUC

Context: Models of sound assessment practices increasingly emphasise assessment's formative role. As a result, assessment must not only support sound judgements about learner competence, but also generate meaningful feedback to guide learning. Reconciling the tension between assessment's focus on judgement and decision making and feedback's focus on growth and development represents a critical challenge for researchers and educators. Methods: We synthesise the literature related to this tension, framed around four trends in education research: (i) shifting perspectives on assessment; (ii) shifting perspectives on feedback; (iii) increasing attention on learners' perceptions of assessment and feedback, and (iv) increasing attention on the influence of culture on assessment and feedback. We describe factors that produce and sustain this tension. Results: The lines between assessment and feedback frequently blur in medical education. Models of programmatic assessment deliberately use the same data for both purposes: low-stakes individual data points are used formatively, but then are added together to support summative judgements. However, the translation of theory to practice is not straightforward. Efforts to embed meaningful feedback in programmes of learning face a multitude of threats. Learners may perceive assessment with formative intent as summative, restricting their engagement with it as feedback, and thus diminishing its learning value. A learning culture focused on assessment may limit learners' sense of safety to explore, to experiment, and sometimes to fail. Conclusions: Successfully blending assessment and feedback demands clarity of purpose, support for learners, and a system and organisational commitment to a culture of improvement rather than a culture of performance.

Biopsychosocial model of illnesses in primary care: A hermeneutic literature review

Article

Full-text available

May 2018

Biopsychosocial model is a useful worldview for primary care or family doctors. However, it is often considered as impractical or too complicated. The objective of this study is to review the implementation of the biopsychosocial model in clinical practice, and its contributions to clinical outcomes. Hermeneutic circle literature review was conducted to provide experiential learning in an attempt to understand biopscyhosocial model, first developed by George Engel. Literature search started with review articles in Medline and Scopus as search engines. Citations from previous articles, editorials, and research articles were identified and interpreted in the context of the knowledge derived from all identified relevant articles. The progress of biopsychosocial model has been slow, and primary care doctors do not implement biopsychosocial medicine in their practice, while biomedical thinking and approach are still the dominant model. Biopsychosocial research addressed chronic illnesses and functional disorders as conditions in need for biopsychosocial model implementation. As payment scheme, clinical guidelines and clinical performance indicators are biomedically oriented, there is no incentive for primary care doctors to adopt biopsychosocial model in their practice. Workload and lack of competence in primary care may hinder the implementation of biopsychosocial model. Biopsychosocial model helps primary care doctors to understand interactions among biological and psychosocial components of illnesses to improve the dyadic relationship between clinicians and their patients and multidisciplinary approaches in patient care. Biopsychosocial model potentially improves clinical outcomes for chronic diseases and functional illnesses seen in primary care.

In Pursuit of Honors: A Multi-Institutional Study of Students’ Perceptions of Clerkship Evaluation and Grading

Article

Jul 2019
ACAD MED

Purpose: To examine medical students' perceptions of the fairness and accuracy of core clerkship assessment, the clerkship learning environment, and their relationship to students' achievement. Method: Fourth-year medical students at 6 institutions completed a survey in 2018 assessing perceptions of the fairness and accuracy of clerkship evaluation and grading; the learning environment including clerkship goal structures (mastery- or performance-oriented); racial/ethnic stereotype threat; and student performance (honors earned). Factor analysis of five-point Likert items (1 = strongly disagree, 5 = strongly agree) provided scale scores of perceptions. Using multivariable regression, investigators examined predictors of honors earned. Qualitative content analysis of responses to an open-ended question yielded students' recommendations to improve clerkship grading. Results: Overall response rate was 71% (666/937). Students believed that being liked and particular supervisors most influenced final grades. Only 44.4% agreed that grading was fair. Students felt the clerkship learning environment promoted both mastery and performance avoidance behaviors (88.0%, 85.6%, respectively). Students from backgrounds underrepresented in medicine were more likely to experience stereotype threat vulnerability (55.7% vs 10.9%, P < .0005). Honors earned was positively associated with perceived accuracy of grading and interest in competitive specialties while negatively associated with stereotype threat. Students recommended strategies to improve clerkship grading: eliminating honors, training evaluators, and rewarding improvement on clerkships. Conclusions: Participants had concerns around the fairness and accuracy of clerkship evaluation and grading and potential bias. Students expressed a need to redefine the culture of assessment on core clerkships to create more favorable learning environments for all students.

What we measure … and what we should measure in medical education

Article

Sep 2018
MED EDUC

Context: As the practice of medicine evolves, the knowledge, skills and attitudes required to provide patient care will continue to change. These competency-based changes will necessitate the restructuring of assessment systems. High-quality assessment programmes are needed to fulfil health professions education's contract with society. Objectives: We discuss several issues that are important to consider when developing assessments in health professions education. We organise the discussion along the continuum of medical education, outlining the tension between what has been deemed important to measure and what should be measured. We also attempt to alleviate some of the apprehension associated with measuring evolving competencies by discussing how emerging technologies, including simulation and artificial intelligence, can play a role. Methods: We focus our thoughts on the assessment of competencies that, at least historically, have been difficult to measure. We highlight several assessment challenges, discuss some of the important issues concerning the validity of assessment scores, and argue that medical educators must do a better job of justifying their use of specific assessment strategies. Discussion: As in most professions, there are clear tensions in medicine in relation to what should be assessed, who should be responsible for administering assessment content, and how much evidence should be gathered to support the evaluation process. Although there have been advances in assessment practices, there is still room for improvement. From the student's, resident's and practising physician's perspectives, assessments need to be relevant. Knowledge is certainly required, but there are other qualities and attributes that are important, and perhaps far more important. Research efforts spent now on delineating what makes a good physician, and on aligning new and upcoming assessment tools with the relevant competencies, will ensure that assessment practices, whether aimed at establishing competence or at fostering learning, are effective with respect to their primary goal: to produce qualified physicians.

Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework

Abstract and Figures

Recommended publications

The pursuit of fairness in assessment: Looking beyond the objective

Making it fair: Learners’ and assessors’ perspectives of the attributes of fair judgement

Fairness in Assessment: Identifying a Complex Adaptive System

What Stops Fairness from Emerging in Assessment? The Forces on a Complex Adaptive System