No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Perceptions of national recruitment for orthodontic specialty registrars: 2012–2014
Abstract
Aim:
To assess the changes in interviewees' and interviewers' perceptions and experiences of national recruitment and a multi-station interview (MSI) for the selection of Orthodontic Specialty Registrars to UK training programmes from 2012 to 2014.
Design:
Questionnaire-based longitudinal survey.
Setting:
Interviews for selection of Orthodontic Specialty Registrars (StRs) held at London Deanery, London, UK.
Methods:
Interviewees and interviewers completed an anonymous questionnaire comprising of 17 and 26 questions, respectively.
Results:
Interviewees: The number, age (p = 0.29) and time since qualification (p = 0.90) increased slightly over the 3 years but these changes were not statistically significant. The proportion of females (p = 0.32) and those with a UK primary dental qualification (p = 0.52) varied slightly but the variation was not statistically significant. The proportion that had experience of the MSI format increased significantly over the 3 years (p < 0.01). More than 75% were positive about the organization, experience and fairness of the MSI interview format. Interviewers: The age (p = 0.54), time since being a consultant (p = 0.90), proportion of females (p = 0.43) and those favouring the MSI format (p = 0.29) varied slightly but this was not statistically significant. More than 75% were positive about the organization, experience and fairness of the MSI format. More than 90% of interviewers thought that the process selected the best candidates, was fair and that more than one assessor was required at each station.
Conclusions:
Interviewees were consistently very positive about the organization and fairness of the MSI format. Interviewers were consistently very positive about the selection of candidates, fairness and conduct of the MSIs.
Aim
To ascertain the perceptions and experiences of candidates of national recruitment and multi‐station interviews for the selection of oral surgery specialty trainees to UK during the first 3 years of being introduced.
Material and methods
A pre‐piloted questionnaire was administered in paper format to applicants immediately following completion of the final recruitment day. Applicants were asked six questions regarding appropriateness of each of the stations using a Likert scales (1 strongly disagree to 5 strongly agree).
Results
During a 3‐year period, feedback was favourable for those stations most likely to be associated with a career in oral surgery, which was in stark contrast to the critical appraisal station which consistently received feedback regarding the time allocated to it. Survey feedback each year was presented to the working group and helped to develop and inform future stations.
Conclusions
Candidates considered national recruitment to be both a fair and transparent process, which the majority supporting the move away from traditional CV‐focused interviews.
The design and implementation of an assessment centre in the South Yorkshire and South Humberside deanery for selecting doctors into postgraduate training in paediatric medicine is described. Eleven competency domains were identified in the job analysis. An assessment centre comprising of four exercises was implemented to assess candidates. There were modest relationships between candidates' performance on the various assessment centre exercises. Outcomes based on interview performance were related to, but not the same as, outcomes based on the combined results of the three other assessment centre exercises. Candidates perceived the assessment centre to be a fair selection method. It is concluded that an assessment centre approach to SHO recruitment is feasible and provides a greater breadth and depth of information about candidates than does a structured interview.
Appointments to the specialist registrar (SpR) grade depend almost entirely on performance at interview, yet standard panel interviews do not directly assess the competences required of a medical trainee. In this study, station interviews were used to select neurology SpRs. Eighteen candidates were assessed in three interviews, each involving three stations: a curriculum vitae (CV)-based interview, an interview with a simulated patient, and a discussion of scenarios based upon teaching, audit and research. Two or three assessors at each station ranked candidates independently before discussing the pooled rankings and reading written references. The CV-based interview rankings (resembling a traditional panel interview) correlated less well with the overall rankings (r=0.54) than did research (r=0.83), information giving (r=0.75), audit (r=0.70) or teaching presentation (r=0.59). Station interviews appear fairer (providing more time, more independent examiners, fresh starts at each station), although they require more planning and expense. Competency-based assessments should be more widely used in selecting medical trainees.
Multiple-Mini Interviews (MMIs) were used to assess professional attributes of candidates seeking admission to an occupational therapy professional entry-level master's program. Candidates and interviewers were invited to complete a questionnaire comprised of quantitative and open-ended questions following the MMIs. The MMIs were perceived to be fair, enjoyable, and capable of capturing professional attributes. Descriptive analysis of candidates' data revealed perceptions regarding logistics, interview station content, process, and interviewers. Interviewers commented on the positive and challenging aspects of the scenarios and the MMI process. Admissions committees need to consider several logistical, content, and process issues when designing and implementing MMIs as a selection tool.
Aim:
To assess interviewers' and interviewees' perceptions of the National Recruitment for Orthodontic Specialty Registrars in 2012.
Design:
Cross-sectional survey.
Methods:
Interviewers and interviewees completed an anonymous questionnaire comprising of 25 and 16 questions, respectively. Statistical analysis included descriptive statistics and frequency distributions.
Results:
All interviewees (83/83) and 88% (36/41) of interviewers completed the questionnaires. Of the interviewees, 61% were female; their mean age was 28·9 years (95% CI: 28·2-29·6). The mean time since bachelor of dental science (BDS) was 5·6 years (95% CI: 4·9-6·3) with 78% qualifying from a UK university. The interviewees preferred the multi-station interview (MSI) format, considered the questions easy to understand and thought that MSI was fairer than traditional interviews. Of the interviewers, 56% were male; their mean age was 45·5 years (95% CI: 43·0-48·0). The mean time that they had been a consultant was 11·4 years (95% CI: 8·7-13·1). The interviewers thought that the interviews were fair, tested an appropriate range of competences, selected the best candidates to be appointed and would appoint the same people if repeated.
Conclusions:
Interviewees were very positive about the organization and perceived fairness of the MSI format. Interviewers were positive about the selection of candidates, fairness and conduct of the MSI format.
Significant demographic, legal, and educational developments during the last ten years have led medical schools to review critically their selection procedures. A critical component of this review is the selection interview, since it is an integral part of most admission processes; however, some question its value. Interviews serve four purposes: information gathering, decision making, verification of application data, and recruitment. The first and last of these merit special attention. The interview enables an admission committee to gather information about a candidate that would be difficult or impossible to obtain by any other means yet is readily evaluated in an interview. Given the recent decline in numbers of applicants to and interest in medical school, many schools are paying closer attention to the interview as a powerful recruiting tool. Interviews can be unstructured, semistructured, or structured. Structuring involves analyzing what makes a medical student successful, standardizing the questions for all applicants, providing sample answers for evaluating responses, and using panel interviews (several interviewers simultaneously with one applicant). Reliability and validity of results increase with the degree of structuring. Studies of interviewers show that they are often biased in terms of the rating tendencies (for instance, leniency or severity) and in terms of an applicant's sex, race, appearance, similarity to the interviewer, and contrast to other applicants). Training interviewers may reduce such bias. Admission committees should weigh the purposes of interviewing differently for various types of candidates, develop structured or semistructured interviews focusing on nonacademic criteria, and train the interviewers.
Interviews are commonly used to measure noncognitive traits of medical school applicants. The present study investigated the influence of knowledge of applicants' cognitive abilities on interviewers' ratings of noncognitive traits.
Academic and demographic predictors of interview ratings of applicants' noncognitive traits were examined at the Medical University of South Carolina College of Medicine during two years: 1992, when applicants' Medical College Admission Test (MCAT) total scores and undergraduate grade-point averages (GPAs) were available to interviewers; and 1993, when MCAT and GPA data were not available. In 1992, 226 applicants met study criteria (i.e., they received ratings from three interviewers in addition to having MCAT and GPA data on file); in 1993, 245 applicants met the criteria. Step-wise regression analyses were conducted to measure the influences of seven independent variables on applicants' interview ratings. Two-way analyses of variances and t-tests were used to determine the effects of gender of applicants and interviewers. Cronbach's alpha coefficients were used as measures of interviewers' reliability.
GPA was the best predictor for both years but accounted for double the amount of variance in interview ratings in 1992 (15.7%) compared with 1993 (7.4%). The reliability coefficients for the interviewers were .496 for 1992 and .473 for 1993.
If the goal of the medical school admission interview is to assess noncognitive traits independently from academic skills, the authors recommend that MCAT and GPA data not be available to interviewers during interviews. The authors also found that gender and race influenced interview ratings in accordance with affirmative-action goals. Finally, the authors found that interview scores were only moderately reliable across different interviewers. They discuss ways to increase their reliability.
To assess the consistency of ratings assigned by health sciences faculty members relative to community members during an innovative admissions protocol called the Multiple Mini-Interview (MMI).
A nine-station MMI was created and 54 candidates to an undergraduate MD program participated in the exercise in Spring 2003. Three stations were staffed with a pair of faculty members, three with a pair of community members, and three with one member of each group. Raters completed a four-item evaluation form. All participants completed post-MMI questionnaires. Generalizability Theory was used to examine the consistency of the ratings provided within each of these three subgroups.
The overall test reliability was found to be .78 and a Decision Study suggested that admissions committees should distribute their resources by increasing the number of interviews to which candidates are exposed rather than increasing the number of interviewers within each interview. Divergence of ratings was greater within the pairing of community member to faculty member and least for pairings of community members. Participants responded positively to the MMI.
The MMI provides a reliable protocol for assessing the personal qualities of candidates by accounting for context specificity with a multiple sampling approach. Increasing the heterogeneity of interviewers may increase the heterogeneity of the accepted group of candidates. Further work will determine the extent to which different groups of raters provide equally valid (albeit different) judgments.
To assess candidates' and interviewers' perceptions of the use of a multiple mini-interview (MMI) for selection of senior house officers (SHOs) to a UK regional paediatric training programme.
Both candidates and interviewers completed anonymous questionnaires (comprising 16 and 25 questions, respectively). Demographic data were recorded for both groups. Data were analysed by frequencies; using Mann-Whitney and Kruskall-Wallis tests for comparisons; and Cronbach's alpha for internal consistency within the data.
Both candidates and interviewers were positive about the fairness of the MMI (mean scores of 4.0 and 4.4, respectively). The majority of candidates (83%) had not been to this type of interview before. Gender, age and previous experience of MMIs did not account for differences in candidate responses (P > 0.05). A total of 86% of candidates were international medical graduates who preferred the format more than UK graduates did (P = 0.01). Interviewers were mainly experienced consultants who agreed that the multi-station format was better than the traditional interview (mean score 4.8) and represented a reliable process (mean score 4.4). Interviewers were concerned about the range of competencies covered and the subsequent performance of candidates in post (mean scores 3.6 and 3.2, respectively).
Both candidates and interviewers agreed that the MMI format was reliable, fair and asked appropriate, easy-to-understand questions. In high-stakes interviews such as for specialty training in Modernising Medical Careers programmes, it is vital that all concerned have confidence in the selection process.