Comparison of UMAT scores and GPA in prediction of performance in medical school: a national study.
ABSTRACT Medical schools continue to seek robust ways to select students with the greatest aptitude for medical education, training and practice. Tests of general cognition are used in combination with markers of prior academic achievement and other tools, although their predictive validity is unknown. This study compared the predictive validity of the Undergraduate Medicine and Health Sciences Admission Test (UMAT), the admission grade point average (GPA), and a combination of both, on outcomes in all years of two medical programmes.
Subjects were students (n = 1346) selected since 2003 using UMAT scores and attending either of New Zealand's two medical schools. Regression models incorporated demographic data, UMAT scores, admission GPA and performance on routine assessments.
Despite the different weightings of UMAT used in selection at the two institutions and minor variations in student demographics and programmes, results across institutions were similar. The net predictive power of admission GPA was highest for outcomes in Years 2 and 5 of the 6-year programme, accounting for 17-35% of the variance; UMAT score accounted for < 10%. The highest predictive power of the UMAT score was 9.9% for a Year 5 written examination. Combining UMAT score with admission GPA improved predictive power slightly across all outcomes. Neither UMAT score nor admission GPA predicted outcomes in the final trainee intern year well, although grading bands for this year were broad and numbers smaller.
The ability of the general cognitive test UMAT to predict outcomes in major assessments within medical programmes is relatively minor in comparison with that of the admission GPA, but the UMAT score adds a small amount of predictive power when it is used in combination with the GPA. However, UMAT scores may predict outcomes not studied here, which underscores the need for further validation studies in a range of settings.
- [show abstract] [hide abstract]
ABSTRACT: Admissions committees and researchers around the globe have used diligence and imagination to develop and implement various screening measures with the ultimate goal of predicting future clinical and professional performance. What works for predicting future job performance in the human resources world and in most of the academic world may not, however, work for the highly competitive world of medical school applicants. For the job of differentiating within the highly range-restricted pool of medical school aspirants, only the most reliable assessment tools need apply. The tools that have generally shown predictive validityin future performance include academic scores like grade point average, aptitude tests like the Medical College Admissions Test, and non-cognitive testing like the multiple mini-interview. The list of assessment tools that have not robustly met that mark is longer, including personal interview, personal statement, letters of reference, personality testing, emotional intelligence and (so far) situational judgment tests. When seen purely from the standpoint of predictive validity, the trends over time towards success or failure of these measures provide insight into future tool development.Advances in Health Sciences Education 04/2012; 14(5):759-775. · 2.06 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: History-taking is one of the most important clinical skills for the medical student to learn and remains the core component of a doctor's diagnostic 'toolkit'. Yet, it is one of the most difficult clinical skills to assess. Clinical assessment at a trainee intern level has typically focussed on examination skills, and case presentation, which are more easily measured. History-taking is assumed to be of an adequate standard on the basis of the case presentation rather than by direct observation. In this paper we discuss the importance of assessing the patient-doctor encounter directly through observation, in the context of the trainee intern long case examination. Despite changing assessment trends in medical education, these authors argue for the retention of the long case as an assessment tool for final year medical students on the basis of its high face validity and close resemblance to "real life" patient encounters. However, we believe addition of an observing examiner during the history-taking and physical examination augments the inherent value of the longcase and is recommended in order to increase the reliability of the assessment. Observation allows for direct assessment of the student-patient interaction and the hypothetico-deductive approach taken by the student to diagnosis. It provides opportunity to reconcile the multiple interactions occurring between the context and the construct (skills and knowledge) measured in an assessment. Importantly, at a trainee intern level it provides students with a final opportunity to receive feedback on their history taking and diagnostic skills, an integral part of all medical practice, prior to their graduation as junior doctors.The New Zealand medical journal 10/2008; 121(1282):51-7.
- [show abstract] [hide abstract]
ABSTRACT: Recently there has been much scrutiny of the medical school admissions process by universities, the General Medical Council and the public. Improved objectivity, fairness and effectiveness of selection procedures are desirable. The ultimate outcome sought is the graduation of competent doctors who reflect the values of and are in tune with the communities they serve. Applicants to the Scottish medical schools sat a battery of psychometric tests to measure cognitive ability, personality traits and moral/ethical reasoning (Personal Qualities Assessment, PQA). Analysis determined the potential impact of the latter variables, and those of educational background and socioeconomic class (assessed by residential 'deprivation category'), upon success in gaining a place to study medicine. Cognitive ability did not vary significantly as a function of gender or educational background, although there was a trend for it to be lower in individuals from more deprived backgrounds. Women as a group were more empathic, with a greater communitarian orientation, than men. There was no significant difference between individuals attending independent and state-funded schools in respect of any of the qualities measured by the PQA. Applicants from deprived backgrounds and those attending state-funded schools would not be disadvantaged by an admissions process based on the PQA. The incorporation of an assessment tool such as the PQA may have positive implications for widening access and the objective selection of suitable medical students, resulting in the training of doctors who are more representative of the community at large. A longterm follow-up of the professional careers of those medical students who completed the PQA will be undertaken.Medical Education 04/2005; 39(3):258-65. · 3.55 Impact Factor