ArticlePDF Available

Abstract

Medical education, the art and science behind medical learning and teaching, has progressed remarkably. Teaching and learning have become more scientific and rigorous, curricula are based on sound pedagogical principles, and Problem Based and other forms of active and self directed learning have become the mainstream. Teachers have progressed from the role of problem-identifier to that of the solution-provider. During the last three decades medical schools have been faced with a variety of challenges from society, patients, doctors and students. They have responded in several ways including the development of new curricula, the introduction of new learning situations, the introduction of the new methods of assessment and a realization of the importance of staff development. Many effective and interesting innovations have been forthcoming. The effective and efficient delivery of healthcare requires not only knowledge and technical skills but also analytical and communication skills, interdisciplinary care, counseling, evidence-and system-based care. This warrants our assessment systems to be comprehensive, sound and robust enough to assess the requisite attributes along with testing for essential knowledge and skills. Assessment is entering every phase of professional development. Assessment and evaluation are crucial steps in educational process. Before making a choice of assessment method, some important questions must be asked: what should be assessed?, why assess? For an assessment instrument one must also ask: is it valid? Is it reliable?, is it feasible? What is assessed and which methods are used will play a significant part in what is learnt. A wide range of assessment methods currently available include essay questions, patient management problems, modified essay questions (MEQs) checklists, OSCE, student projects, Constructed Response Questions (CRQs), MCQs, Critical reading papers, rating scales, extended matching items, tutor reports, portfolios, short case assessment and long case assessment, log book, trainer's report, audit, simulated patient surgeries, video assessment, simulators, self assessment, peer assessment and standardized patients. Assessment has a powerful positive steering effect on learning and the curriculum. It conveys what we value as important and acts as the most cogent motivator of student learning. Assessment is purpose driven. In planning and designing assessments, it is essential to recognize the stakes involved in it. The higher the stake, the greater the implications of the outcome of the assessment. The more sophisticated the assessment strategies, the more appropriate they become for feedback and learning. Measuring progress in acquiring core knowledge and competencies may be a problem if the exams are designed to measure multiple integrated abilities, such as factual knowledge, problem solving, analysis and synthesis of information. Students may advance in one ability and not in another. Therefore, progress tests that are designed to measure growth from the onset of learning until graduation should measure discrete abilities. 4 Mastery testing (criterion-reflected tests) requires that 100% of the items are measured correctly to determine whether students have attained a mastery level of achievements. In non-mastery testing attainment of 65% of a tested material is considered sufficient. Global rating scales are measurement tool for quantifying behaviors. Raters use the scale either by directly observing students or by recalling student performance. Raters judge a global domain of ability for example: clinical skills, problem solving, etc Self assessment (self regulation) is a vital aspect of the lifelong performance of physicians. Self monitoring requires that individuals are able not only to work independently but also to assess their own performance and progress. Every form of assessment can be used as a self assessment exercise as long as students are provided with 'gold standard' criteria for comparing their own performance against an external reliable measure. Self assessment approaches include: written exams (MCQs, True/False, Essay, MEQs, modified CRQs), performance exams (checklists, global rating, student logbook, portfolio, video, etc). Oral examination/Viva has poor content validity, higher inter-rater variability and inconsistency in marking. The instrument is prone to biases and is inherently unreliable. Long Essay Questions can be used for assessment of complex learning situations that can not be assessed by other means (writing skills, ability to present arguments succinctly). The Short Answer Question (SAQ) is an open ended, semi-structured question format. A structured predetermined marking scheme improves objectivity. The questions can incorporate clinical scenarios. A similar format is also known as Modified Essay Question (MEQ) or Constructed Response Question (CRQ). Equal or higher test reliabilities can be achieved with fewer SEQs as compared to true/false items. If a large amount of knowledge is required to be tested, MCQs should be used. SAQs have a better content coverage as compared to long essay question. Extended Matching Item is based on a single theme and has a long option list to avoid cueing. It can be used for the assessment of clinical scenarios with less cueing. It is a practical alternative to MCQ while maintaining objectivity and consistency. It can be used in both basic and clinical sciences. Key Feature Test is a clinical scenario-based paper and pencil test. A description of the problem is followed by a limited number of questions that focus on critical, challenging actions or decisions. It has higher content validity with proper blueprinting. Long Case involves use of a non-standardised real patient. Long case may provide a unique opportunity to test the physician's tasks and interaction with a real patient. It has poor content validity, is less reliable and lacks consistency. Reproducibility of the score is 0.39; meaning 39% of the variability of the score is due to actual performance of students (signal) and the remaining 61% of the variability is due to errors in measurement (noise) (Noricine,2002). In high stake summative assessment long case should be avoided. Short Case involves use of three to four non-standardised real patients with one to two examiners. It provides opportunity for assessment with real patients and allows greater sampling than single long cse. Objective Structured Clinical examination (OSCE) consists of multiple stations where each candidate is asked to perform a defined task such as taking a focused history or performing a focused clinical examination of a particular system. A standardized marking scheme specific for each case is used. It is an effective alternative to unstructured short cases.
3
Editorial
Assessment Methods in Medical Education
Medical education, the art and science behind medical learning and teaching, has
progressed remarkably. Teaching and learning have become more scientific and rigorous,
curricula are based on sound pedagogical principles, and Problem Based and other forms
of active and self directed learning have become the mainstream. Teachers have
progressed from the role of problem-identifier to that of the solution-provider.
During the last three decades medical schools have been faced with a variety of
challenges from society, patients, doctors and students. They have responded in several
ways including the development of new curricula, the introduction of new learning
situations, the introduction of the new methods of assessment and a realization of the
importance of staff development. Many effective and interesting innovations have been
forthcoming.
The effective and efficient delivery of healthcare requires not only knowledge and
technical skills but also analytical and communication skills, interdisciplinary care,
counseling, evidence- and system-based care. This warrants our assessment systems to
be comprehensive, sound and robust enough to assess the requisite attributes along with
testing for essential knowledge and skills.
Assessment is entering every phase of professional development. Assessment and
evaluation are crucial steps in educational process. Before making a choice of assessment
method, some important questions must be asked: what should be assessed?, why assess?
For an assessment instrument one must also ask: is it valid? Is it reliable?, is it feasible?
What is assessed and which methods are used will play a significant part in what is learnt.
A wide range of assessment methods currently available include essay questions, patient
management problems, modified essay questions (MEQs) checklists, OSCE, student
projects, Constructed Response Questions (CRQs), MCQs, Critical reading papers, rating
scales, extended matching items, tutor reports, portfolios, short case assessment and long
case assessment, log book, trainer’s report, audit, simulated patient surgeries, video
assessment, simulators, self assessment, peer assessment and standardized patients.
Assessment has a powerful positive steering effect on learning and the curriculum. It
conveys what we value as important and acts as the most cogent motivator of student
learning. Assessment is purpose driven. In planning and designing assessments, it is
essential to recognize the stakes involved in it. The higher the stake, the greater the
implications of the outcome of the assessment. The more sophisticated the assessment
strategies, the more appropriate they become for feedback and learning.
Measuring progress in acquiring core knowledge and competencies may be a problem if
the exams are designed to measure multiple integrated abilities, such as factual
knowledge, problem solving, analysis and synthesis of information. Students may
advance in one ability and not in another. Therefore, progress tests that are designed to
measure growth from the onset of learning until graduation should measure discrete
abilities.
4
Mastery testing (criterion-reflected tests) requires that 100% of the items are measured
correctly to determine whether students have attained a mastery level of achievements. In
non-mastery testing attainment of 65% of a tested material is considered sufficient.
Global rating scales are measurement tool for quantifying behaviors. Raters use the scale
either by directly observing students or by recalling student performance. Raters judge a
global domain of ability for example: clinical skills, problem solving, etc
Self assessment (self regulation) is a vital aspect of the lifelong performance of
physicians. Self monitoring requires that individuals are able not only to work
independently but also to assess their own performance and progress.
Every form of assessment can be used as a self assessment exercise as long as students
are provided with ‘gold standardcriteria for comparing their own performance against
an external reliable measure. Self assessment approaches include: written exams (MCQs,
True/False, Essay, MEQs, modified CRQs), performance exams (checklists, global
rating, student logbook, portfolio, video, etc).
Oral examination/Viva has poor content validity, higher inter-rater variability and
inconsistency in marking. The instrument is prone to biases and is inherently unreliable.
Long Essay Questions can be used for assessment of complex learning situations that can
not be assessed by other means (writing skills, ability to present arguments succinctly).
The Short Answer Question (SAQ) is an open ended, semi-structured question format. A
structured predetermined marking scheme improves objectivity. The questions can
incorporate clinical scenarios. A similar format is also known as Modified Essay
Question (MEQ) or Constructed Response Question (CRQ). Equal or higher test
reliabilities can be achieved with fewer SEQs as compared to true/false items. If a large
amount of knowledge is required to be tested, MCQs should be used. SAQs have a better
content coverage as compared to long essay question.
Extended Matching Item is based on a single theme and has a long option list to avoid
cueing. It can be used for the assessment of clinical scenarios with less cueing. It is a
practical alternative to MCQ while maintaining objectivity and consistency. It can be
used in both basic and clinical sciences.
Key Feature Test is a clinical scenario-based paper and pencil test. A description of the
problem is followed by a limited number of questions that focus on critical, challenging
actions or decisions. It has higher content validity with proper blueprinting.
Long Case involves use of a non-standardised real patient. Long case may provide a
unique opportunity to test the physician’s tasks and interaction with a real patient. It has
poor content validity, is less reliable and lacks consistency. Reproducibility of the score is
0.39; meaning 39% of the variability of the score is due to actual performance of students
(signal) and the remaining 61% of the variability is due to errors in measurement (noise)
(Noricine,2002). In high stake summative assessment long case should be avoided.
Short Case involves use of three to four non-standardised real patients with one to two
examiners. It provides opportunity for assessment with real patients and allows greater
sampling than single long cse.
Objective Structured Clinical examination (OSCE) consists of multiple stations where
each candidate is asked to perform a defined task such as taking a focused history or
performing a focused clinical examination of a particular system. A standardized marking
scheme specific for each case is used. It is an effective alternative to unstructured short
cases.
5
Mini-Clinical Evaluation Exercise (Mini-CEX) is a rating scale developed by American
Board of Internal Medicine to assess six core competencies of residents: medical
interviewing skills, physical examination skills, humanistic qualities/professionalism,
clinical judgment, counseling skills, organization and efficiency.
Direct Observation of Procedural Skills (DOPS) is a structured rating scale for assessing
and providing feedback on practical procedures. The competencies that are commonly
assessed include general knowledge about the procedure, informed consent, pre-
procedure preparation, analgesia, technical ability, aseptic technique, post-proicdure
management, and counseling and communication.
Clinical Work Sampling is an in-trainee evaluation method that addresses the issue of
system and rater biases by collecting data on observed behaviour at the same time of
actual performance and by using multiple observers and occasions.
Checklists are used to capture an observed behaviour or action oof a student. Generally
rating is by a five to seven point
360-Degree Evaluation/Multisource Assessment consists of measurement tools
completed by multiple individuals in a person’s sphere of influence. Assessment by
peers, other members of the clinical team, and patients can provide insight into trainees’
work habits, capacity for team work, and interpersonal sensitivity
In the Logbook students keep a record of the patients seen or procedures performed either
in a book or in a computer. It documents the range of patient care and learning experience
of students. Logbook is very useful in focusing students on important objectives that must
be fulfilled within a specified period of time (Blake, 2001).
Portfolio refers to a collection of one’s professional and personal goals, achievements,
and methods of achieving these goals. Portfolios demonstrate a trainees’ development
and technical capacity.
Skill based assessments are designed to measure the knowledge, skills, and judgment
required for competency in a given domain.
Test of clinical competence, which allows decisions to be made about medical
qualification and fitness to practice, must be designed with respect to key issues including
blueprinting, validity, reliability, and standard setting, as well as clarity about their
formative or summative function. MCQs, essays, and oral examinations could be used to
test factual recall and applied knowledge, but more sophisticated methods are needed to
assess clinical performance, including directly observed long and short cases, objective
structure clinical examinations, and the use of standardized patients.
The Objective Structure Clinical examination (OSCE) has been widely adopted as a tool
to assess students, or doctor’s competences in a range of subjects. It measures outcomes
and allows very specific feedback.
Other approaches to skill-based assessment include: traditional (Oral exam/viva, long
case); alternative formats (tackle the problems associated with traditional orals and long
cases by having examiners observe the candidates complete interaction with the patient,
training examiners to a structured assessment process, increasing the number of patient
problems. Traditional unstructured orals and long cases have largely been discontinued in
North America.
While selecting an assessment instrument it is necessary to know precisely what it is that
is to be measured. This should reflect course outcomes as different learning outcomes
6
require the use of different instruments. It is essential to use an instrument that is valid,
reliable and feasible (calculating the cost of the assessment, both in terms of resources
and time). Full variety of instruments will ensure that the results obtained are a true
reflection of the students’ performance.
Multiple sampling strategies as the accepted methods used in assessment in clinical
competency include OSCE, Short Answer Questions, mini-CEX (Mini Clinical
Evaluation Exerciser), Directly Observed Procedural Skills (DOPS), Clinical work
sampling (CWS), and 360-degree evaluation.
The assessment is an integral component of overall educational activities. Assessment
should be designed prospectively along with learning outcomes. It should be purpose
driven. Assessment methods must provide valid and usable data. Methods must yield
reliable and generalisable data.
Multiple assessment methods are necessary to capture all or most aspects of clinical
competency and any single method is not sufficient to do the job. For knowledge,
concepts, application of knowledge (‘Knows’ and ‘Knows How’ of Miller’s conceptual
pyramid for clinical competence) context-based MCQ, extended matching item and short
answer questions are appropriate. For ‘Shows How” multi-station OSCE is feasible. For
performance-based assessment (‘does’) mini-CEX, DOPS is appropriate. Alternatively
clinical work sampling and portfolio or log book may be used.
Standard setting involves judgment, reaching consensus, and expressing that consensus
as a single score on a test. Norm Referenced Scores are suitable for admission exercise
that requires selection of a predetermined number of candidates. Criterion Referenced
Standard (based on predefined test goals and standards in performance during an
examination where a certain level of knowledge or skill has been determined as required
for passing) is feasible for competency-based examination. Various approaches available
include test-centred approach (Agnoff’s method and its variations), examinee-centred
approach (borderline group method), and several other innovations. Blueprinting refers to
a process emphasizing that test content should be carefully planned against learning
objectives.
The purpose of assessment should direct the choice of instruments. Needs assessment is
the starting point of good assessment that identifies the current status of the students
before the commencement of the actual educational activities. Needs assessment is used
to determine the existing knowledge base, future needs, and priority areas that should be
addressed.
Student assessment is a comprehensive decision making process with many important
implications beyond the measure of students’ success. Student assessment is also related
to program evaluation. It provides important data to determine the program effectiveness,
improves the teaching program, and helps in developing educational concepts.
Good quality assessment not only satisfies the needs of accreditation but also contributes
to student’s learning. Assessment methods should match the competencies being learnt
and the teaching formats being used.
Competence is a habit of lifelong learning, is contextual (e.g. practice setting, the local
prevalence of disease, etc) and developmental (habits of mind and behaviour and
practical wisdom are gained through deliberate practice.
Further Reading
7
ACGME Outcome Project. Accreditaton Council for Graduate Medical Education &
American Board of Medical Specialist. Toolbox for assessment methods, version 1.1.
www.acgme.org/outcomes/assess/toolbox.pdf
Case S M & swanson D B. Constructing Written Test for the Basic & Clinical Sciences,
3
rd
. ed. (National Board of Medical Examiners) Philadelphia, PA, USA). 2002.
www.nbme.org/about/itemwriting.asp.
Day S C, Norcini J J, Diserens D, et al. The validity of the essay test of clinical
judgement. Acad Med. 1990;65(9):S39-40.
Epstein RM, and Hundert EM. Defining and assessing clinical competence, JAMA
2002;387:226-35.
Friedman Ben_David M. Standard setting in student assessment, AMEE education Guide
No: 18 (Association for Medical education in Europe, Dundee, UK), 2000.
Miller GE. The assessment of clinical skills/competencies/performance. Acad Med.
1990;65(9):S63-67.
Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of
multiple choice questions and patient management problem item formats in assessment of
clinical competence. Med Edu. 1985;19(3):238-47.
Norman G. Postgraduate assessment – reliability and validity. Trans J. Coll. Med. S. Afri.
2003;47:71-75.
Page G & Bordage G. & Allen T. Developing key feature problem and examination to
assess clinical decision making skills. Acad Med. 1995;70(3): 194-201.
Swanson DB. A measurement framework for performance based test. In: Hart IR, Harden
RM (eds.), Further Development in assessing Clinical Competence. Montreal Can-Heal.
1987.
Wass, Cees Van der Vleuten, John Shatzer, Roger Jones. Assessment of clinical
competence. The Lancet 2001;357:945-49.
Vleuten va der CPM. Validity of final examination in undergraduate medical traning.
BMJ 2000;321:1217-19.
Falchikov N, Boud D. Student Self-assement in higher education: a meta-analysis.
Review of Education Research 1989; S9:345-430.
Van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized
patients: state-of-the-art teaching and learning in medicine. 1990;22:58-76.
Syed Amin Tabish
FRC, FRCPE, FACP, FAMS, MD
Editor-in-Chief
... Various methods including didactic lectures, web-based teachings, reflection, seminars, small group discussions, etc can be used to teach professionalism. [9][10][11] Therefore this study was planned to teach AETCOM module 3.1 "Enumeration of professional qualities and role of the physician" followed by an assessment of the first-year MBBS students of 2019-2020 admission batch at a teaching hospital, Belagavi (Southern India). ...
... For the medical students, assessment methods whether in the form of MCQs, True/ False, short essay, modified essay questions, etc can be implemented as a form of exercise to assess their own performance against an external reliable measure. 9 Out of the 156 students, the majority of the students (27.56%) answered 8/10 questions correctly, which shows a good performance of the students. Another study done by Jagzape et al., found that 93.83% of the MBBS undergraduates perceived that the AETCOM modules were useful and relevant, 78.46% of students thought the modules has improved their communication skills and comprehensive development. ...
Article
Full-text available
Background: Academic proficiency along with clinical skills are essential assets for a medical practitioner. Therefore, Attitudes, Ethics, and Communication (AETCOM) module has been implemented by the Medical Council of India (MCI) as a structured program to train and develop professional skills among the first year MBBS students in medical colleges in India since 2019. Objective: To assess the knowledge and understanding of professional qualities in first-year medical students. Methods: This questionnaire-based study was conducted among 156 first-year MBBS students. The questionnaire was divided into three parts, held during the AETCOM Module Classes. Categorical variables were given in the form of frequency tables. The responses were depicted in the form of percentages. Results: In the process of evaluation, a significant number of students (99.36%) thought that the objectives of the session and the questions about the case scenario were interesting Majority of the students answered correctly for the multiple-choice question was Commitment to professional responsibilities don’t mean. (96.79%). About 12.82% of the students answered at least 5 questions incorrectly. Conclusion: Inclusion of MCI AETCOM module in medical student’s classes helped in learning and selfassessing professional skills and qualities. Keywords: Attitude, Knowledge, Medical Students, Professionalism, Self-Assessment.
... Exams developed to valuate many interspersed competence, for instance accurate intelligence, cogent, inquiry with integration of information can be problematic to measure the progress. Students may show progress in one trait so diverse abilities be measured through progress tests 1 . Educationists persuadeassessment, course objectives and intended outcomes be aligned with the feedback to improve competencies 2 . ...
... Judgment: required five public health experts selected randomly with modified Delphi technique. Experts were requested to rate individual item on a Likert scale from (1 as Strongly Disagree, 2 as Disagree, 3 as Neutral, 4 for Agree and finally 5 for Strongly Agree) as well as rating individual items on (Congruity, Accuracy, Integrity and Obscurity) on a four-point scale (1)(2)(3)(4) and advocate adjustments in wording, identify prolixity and recommend further items.Qualitative and quantitative expert's suggestions were assembled in relation to item's relevancy, accuracy, representativeness and comprehensiveness to gauge constitute; optionally delineated by these items to establish the content validity 2 & 3 . In relation to this phase, three Delphi rounds were conducted 22 . ...
Article
Full-text available
Background: Questionnaires are the most commonly used data collection methods in applied research for the assessment of inputs. It is a useful instrument if valid and reliable. Objectives; To establish, design, and appraise the reliability and validity tool for measuring knowledge with skills among undergraduate students. Methods: An observational study was conducted at Peshawar Medical College for six months through a three-stage process, after having the approval of the Institutional Review Board of Prime Foundation. A process initiated by slot regulation, component development, and questionnaire generation with judgment analysis of instrument by an expert panel of five public health specialists for relevance, representativeness, and transparency of each item based on the Likert rating scale. Validity and reliability were measured in the final steps. Suggestions put forward by the experts with item impact scores corresponded to face validity. Rewording, combination, and elimination resulted in the final 35-item instrument. Data were analyzed through SPSS Version-21 with computation of content validity ratio, content validity index, item content validity, scale validity, Kappa statistics, and Cronbach's alpha values. Results: Mean years of experience for the panelists was 14.2 years with S. D + 5.2 (n= 5). Excellent CVR, I CVI, S CVI, Percent Agreement, and Kappa statistics were calculated for the entire questionnaire as 1. The final 24-item knowledge section had 0.732 Inter-Class Correlation and acceptable Cronbach Alpha as 0.743, while the 11-item skill portion had 0.819 Inter-Class Correlation with good Cronbach Alpha 0.890. Conclusions: The findings support the face and content validity of the questionnaire.
... Summative assessments occur at the end of the learning process and, therefore, reflect the students' final level of achievement and performance (Al-Kadri, 2012). Many types of assessments are used in medical education: essay questions, multiple-choice questions (MCQs), objective structured clinical examinations (OSCEs), long cases, short cases, and others (Tabish, 2008). ...
Article
Full-text available
Exams should be valid, reliable, and discriminative. Multiple informative methods are used for exam analysis. Displaying analysis results numerically, however, may not be easily comprehended. Using graphical analysis tools could be better for the perception of analysis results. Two such methods were employed: standardized x-bar control charts with standard error of measurement as control limits and receiver operator characteristic curves. Exams of two medical classes were analyzed. For each exam, the mean, standard deviation, reliability, and standard error of measurement were calculated. The means were standardized and plotted against the reference lines of the control chart. The means were chosen as cut-off points to calculate sensitivity and specificity. The receiver operator characteristic curve was plotted and area under the curve determined. Standardized control charts allowed clear, simultaneous comparison of multiple exams. Calculating the control limits from the standard error of measurement created acceptable limits of variability in which the standard deviation and reliability were incorporated. The receiver operator characteristic curve graphically showed the discriminative power of the exam. Observations made with the graphical and classical methods were consistent. Using graphical methods to analyse exams could make their interpretation more accessible and the identification of exams that required further investigation easier.
... [19] Oral Examination Oral examination or Viva, this type of examination is susceptible to biases and is integrally unreliable with inconsistency in marking. [20,21] However, other reports suggested that the well-structured oral examination can reach virtuous reliability and identify the learner's capabilities in higher-order cognitive skills by welltrained examiners using a blueprint for briefed specific topic, and grading is on a marking template. [22] Online Assessments Due to COVID-19 pandemic, many medical schools all over the world have effectively changed their teaching and learning setting to emergency remote online learning, having the challenge to change the assessment strategies to examine preclinical and clinical students and completed an online summative and formative examinations. ...
Article
Full-text available
Anatomy Assesments
... 30 Problem based questions are for students to assess their knowledge and understanding level through which they can solve problem in a novel situation. 31 Current research found that percentage of problem based question had reduced dramatically over the last 10 years and this finding was concurrent to previous two studies conducted in Bangladesh although there is already identified need by pharmacology educators of the country to incorporate problem based questions in undergraduate written exams. 24,26,32 Regarding weightage of problem based questions, another point is significant difference (<0.00001) was observed among different universities in current study. ...
Article
Full-text available
Background: The key objective of pharmacology education is to make graduates competent enough to prescribe safely and effectively. There is worldwide speculation of inadequacy of pharmacology education to achieve the expected learning outcomes. Problem-based pharmacotherapy is considered as one of the crucial way to prepare future physicians as rational prescribers. As assessment drives learning priorities, adequate weightage on problem-based questions in undergraduate pharmacology examination might be helpful. Materials and methods: This descriptive cross-sectional study was conducted to analyze the undergraduate pharmacology written question papers (SAQ) of MBBS curriculum of 07 different universities (). Total 131 question papers were collected, and trend and weightage of problem-based questions were evaluated. Results: Problem-based questions have been reduced dramatically over the decade and there was significant difference (<0.00001) of weightage of problem-based questions among different universities of Bangladesh. Highest presence of problem-based question was observed in Gono Bishwabidyalay (GB), followed by Shahjalal University of Science and Technology (SUST) University of Rajshahi (RU) and Bangladesh University of Professionals (BUP) but there was not a single problem-based question in University of Dhaka (DU), University of Chittagong (CU) and University of Science and Technology (USTC) over 10 years period. Conclusion: Current study revealed negligible presence of problem-based questions in undergraduate pharmacology written question papers of Bangladesh over 10 years period.
... Therefore, before making a choice of assessment method, important questions should be asked by educators i.e. what should be assessed, why that particular aspect needs to assessed and what the best method of assessment should be to achieve the intended learning outcome/s. As such, education programmes should ensure that assessment approaches are comprehensive and robust enough to assess required attributes along with testing for essential knowledge and skills [7]. ...
Article
Full-text available
Background The training of optometrists aims to prepare practitioners with critical thinking skills who utilise their education and experience to solve clinical problems in real-life practice. Professional competencies should inform assessment, and as such, assessment methods for learning should encompass a wide range of approaches. The objective of this scoping review is therefore to map assessment approaches utilised within optometry education programmes globally. Methods This study is a scoping review based on the PRISMA methodology. The review will be guided by the following research question, “What are the assessment approaches that inform optometry training globally?”. This was validated by the Population-Concept-Context framework according to the methodology for Joanna Briggs Institution Scoping Reviews. Relevant peer-reviewed studies and grey literature conducted during the last 10 years will be identified from electronic databases including CINAHL, PubMed, PROquest and ERIC. The search strings using keywords such as “Optometry students and staff”, “Assessments” and “Optometry education” will be conducted using Boolean logic. An independent reviewer will conduct all title screening, two independent reviewers will conduct abstract and full article screening, followed by data extraction. Thereafter, a thematic analysis will be conducted. The Mixed Method Appraisal Tool version 2018 will be used for quality appraisal of mapped studies. Discussion The review will document evidence of assessment approaches utilised in optometry training globally. Considering the exit level competencies required in the basic job function of an optometrist, a coherence in assessment approaches and relevant rationale for these would be expected, if the accredited (regulated) training programmes follow a competency-based model.
Article
OBJECTIVE: To assess the undergraduate medical student's preference among different formats of written assessment. MATERIAL AND METHOD: Cross sectional study was conducted on Phase I and phase II undergraduate medical students with prior information and consent. Lecture was delivered on a topic for both phases followed by a test with same question in different formats and a questionnaire on student's feedback on these formats. RESULTS: Majority of students of both phases preferred Multiple choice questions over other three formats as it was easier to answer, time saving, enjoyable, can fetch more marks and free of evaluator bias. Other formats also had their advantages and drawbacks. CONCLUSION:Assessment of any format of written assessments can usually assess any level of cognitive domain with properly framed questions and model answers although students may prefer a particular one.
Article
Full-text available
Objective: To evaluate the preferred method of assessment from medical student's perspective. Study Design: Cross-sectional study Place and Duration of Study: This study was conducted at the Al-Tibri Medical College and Hospital, Isra University-Karachi Campus from March-2020 to October-2020. Materials and Methods: A valid questionnaire was adopted for the evaluation of the preferred method of assessment. Data was collected after the institutional ethical approval, and verbal consent was taken from the respondent. A total of 150 undergraduate students were included through a snow ball sampling method from the 2nd, 3rd, and 4th year of MBBS. Data were analyzed through SPSS version 21.0 and showed in the form of frequency and percentage. The chi-square test was applied to assess the qualitative data and level of significance was taken P =<0.05. Results: Out of 150 medical students, the mean age of 2nd-year students was 20.36± 4.00, 3rd-year students with 21.36 ± 0.95, and the mean age of 4th year was 22.87± 2.97. Male students' frequency was 75(50%), and female participants were 75(50%) of the total numbers of respondents. Numbers of students preferred MCQs a quality choice for assessment in comparison to essay questions. Maximum respondents preferred that MCQs give brief understanding and a wide range of course coverage, while essays enhance the written expression. Conclusion: Following the present study results concluded that MCQs are a more preferred assessment tool among the students of medical sciences compared to the essay question. MCQs cover more content areas in a limited period and assess the higher level of cognition; the level of competency depends on MCQ's quality structure.
Article
Full-text available
A little more than 10 years ago, the objective structured clinical examination (OSCE) was introduced. It includes several “stations,”; at which examinees perform a variety of clinical tasks. Although an OSCE may involve a range of testing methods, standardized patients (SPs), who are nonphysicians trained to play the role of a patient, are commonly used to assess clinical skills. This article provides a comprehensive review of large‐scale studies of the psychometric characteristics of SP‐based tests.Across studies, reliability analyses consistently indicate that the major source of measurement error is variation in examinee performance from station to station (termed content specificity in the medical‐problem‐solving literature). As a consequence, tests must include large numbers of stations to obtain a stable, reproducible assessment of examinee skills. Disagreements among raters observing examinee performance and differences between SPs playing the same patient role appear to have less effect on the precision of scores, as long as examinees are randomly assigned to raters and SPs.Results of validation studies (e.g., differences in group performance, correlations with other measures) are generally favorable, though not particularly informative. Future validation research should investigate the impact of station format, timing, and instructions on examinee performance; study the procedures used to translate examinee behavior into station and test scores; and work on rater and SP bias.Several recommendations are offered for improving SP‐based tests. These include (a) focusing on assessment of history taking, physical examination, and communication skills, with separately administered written tests used to measure diagnostic and management skills, (b) adoption of a mastery‐testing framework for score interpretation, and (c) development of improved procedures for setting pass‐fail standards. Use of generalizability theory in analyzing and reporting results of psychometric studies is also suggested.
Article
Full-text available
Despite a lack of face validity, there continues to be heavy reliance on objective paper-and-pencil measures of clinical competence. Among these measures, the most common item formats are patient management problems (PMPs) and three types of multiple choice questions (MCQs): one-best-answer (A-types); matching questions (M-types); and multiple true/false questions (X-types). The purpose of this study is to compare the reliability, validity and efficiency of these item formats with particular focus on whether MCQs and PMPs measure different aspects of clinical competence. Analyses revealed reliabilities of 0.72 or better for all item formats; the MCQ formats were most reliable. Similarly, efficiency analyses (reliability per unit of testing time) demonstrated the superiority of MCQs. Evidence for validity obtained through correlations of both programme directors' ratings and criterion group membership with item format scores also favoured MCQs. More important, however, is whether MCQs and PMPs measure the same or different aspects of clinical competence. Regression analyses of the scores on the validity measures (programme directors' ratings and criterion group membership) indicated that MCQs and PMPs seem to be measuring predominantly the same thing. MCQs contribute a small unique variance component over and above PMPs, while PMPs make the smallest unique contribution. As a whole, these results indicate that MCQs are more efficient, reliable and valid than PMPs.
Article
Full-text available
This article introduces the concept of a key feature and describes its function as the cornerstone of key-feature problems, a new problem format for the written assessment of clinical decision-making skills of medical trainees and practitioners. The rationale for using this problem format and the steps in problem and examination development--including issues of scoring and standard setting--are described. A key feature is defined as a critical step in the resolution of a clinical problem, and a key-feature problem consists of a clinical case scenario followed by questions that focus on only those critical steps. The questions can be presented to require examinees either to write in their responses or to select them from a list of options. For each question, examines can be instructed to supply or select whatever number of responses is appropriate to the clinical task being tested, and answer keys can comprise one or several responses. This problem format, with its focus on only the critical steps in problem resolution, and with its flexibility in question format and scoring keys, effectively addresses the psychometric considerations of content validity and test score reliability, and accommodates the complexity and configurations of actions often required in the resolution of clinical problems.
Article
Full-text available
Most medical schools, particularly those in the United Kingdom, have a final examination at the end of undergraduate medical training. Although the format of these examinations has been changed recently by the introduction of newer types of assessment such as objective structured clinical examinations, medical educators are still questioning their validity and worth. I believe that there is a strong case for better continuous assessment during undergraduate training and less reliance on final examinations. #### Summary points Even with modern forms of assessment, final examinations are of questionable reliability and validity They are of limited educational value to students because there is little opportunity for feedback and correction The effort spent on running final examinations would be better invested in improved continuous assessment during training Continuous assessment through “clinical work samples” is a promising new method of assessing medical students The final examination has at least two functions—an accountability or selective function and an educational one. First and foremost, the final exam should provide a guarantee to society that the training programme delivers competent doctors. It should be able to identify any students who are unfit to practise, so that they can be prevented from doing harm, and to license competent students who are ready for further practice and training. With regard to the educational function, proponents claim that the requirement to sit a comprehensive examination at the end of training means that students revise and recapitulate what they have learned throughout the course, a process which leads to a more integrated understanding of the knowledge and skills they have acquired. Let me examine these functions in greater detail. The outcome of the final examination should predict whether a student will be competent. The process should prevent false negative results and, in particular, false positive ones. In other words, the final examination should …
Article
Quantitative self-assessment studies that compared self- and teacher marks were subjected to a meta-analysis. Predictions stemming from the results of an earlier critical review of the literature (Boud & Falchikov, 1989) were tested, and salient variables were identified. Factors that seem to be important with regard to the closeness of correspondence between self- and teacher marks were found to include the following: the quality of design of the study (with better designed studies having closer correspondence between student and teacher than poorly designed ones); the level of the course of which the assessment was a part (with students in advanced courses appearing to be more accurate assessors than those in introductory courses); and the broad area of study (with studies within the area of science appearing to produce more accurate self-assessment generally than did those from other areas of study). Results of the analysis are discussed and differences signaled by the results of the three common metrics examined. The distinction between relative and absolute judgment of performance is drawn. It is recommended that researchers give attention to both good design and to adequate reporting of self-assessment studies.
Article
Tests of clinical competence, which allow decisions to be made about medical qualification and fitness to practise, must be designed with respect to key issues including blueprinting, validity, reliability, and standard setting, as well as clarity about their formative or summative function. Multiple choice questions, essays, and oral examinations could be used to test factual recall and applied knowledge, but more sophisticated methods are needed to assess clincial performance, including directly observed long and short cases, objective structured clinical examinations, and the use of standardised patients. The goal of assessment in medical education remains the development of reliable measurements of student performance which, as well as having predictive value for subsequent clinical competence, also have a formative, educational role.
Accreditaton Council for Graduate Medical Education & American Board of Medical Specialist. Toolbox for assessment methods, version 1
  • Acgme Outcome
  • Project
ACGME Outcome Project. Accreditaton Council for Graduate Medical Education & American Board of Medical Specialist. Toolbox for assessment methods, version 1.1. www.acgme.org/outcomes/assess/toolbox.pdf