The Electronic Residency Application Service Application Can Predict Accreditation Council for Graduate Medical Education Competency-Based Surgical Resident Performance

Department of Surgery, Harbor-UCLA Medical Center, Torrance, California 90509, USA.
Journal of Surgical Education (Impact Factor: 1.39). 11/2010; 67(6):444-8. DOI: 10.1016/j.jsurg.2010.05.002
Source: PubMed

ABSTRACT Program directors often struggle to determine which factors in the Electronic Residency Application Service (ERAS) application are important in the residency selection process. With the establishment of the Accreditation Council for Graduate Medical Education (ACGME) competencies, it would be important to know whether information available in the ERAS application can predict subsequent competency-based performance of general surgery residents.
This study is a retrospective correlation of data points found in the ERAS application with core competency-based clinical rotation evaluations. ACGME competency-based evaluations as well as technical skills assessment from all rotations during residency were collected. The overall competency score was defined as an average of all 6 competencies and technical skills.
A total of77 residents from two (one university and one community based university-affiliate) general surgery residency programs were included in the analysis. Receiving honors for many of the third year clerkships and AOA membership were associated with a number of the individual competencies. USMLE scores were predictive only of Medical Knowledge (p = 0.004). Factors associated with higher overall competency were female gender (p = 0.02), AOA (p = 0.06), overall number of honors received (p = 0.04), and honors in Ob/Gyn (p = 0.03) and Pediatrics (p = 0.05). Multivariable analysis showed honors in Ob/Gyn, female gender, older age, and total number of honors to be predictive of a number of individual core competencies. USMLE scores were only predictive of Medical Knowledge.
The ERAS application is useful for predicting subsequent competency based performance in surgical residents. Receiving honors in the surgery clerkship, which has traditionally carried weight when evaluating a potential surgery resident, may not be as strong a predictor of future success.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The purpose of our study was to determine the predictive impact of individual academic measures for the matriculation of senior medical students into a general surgery residency. Academic records were evaluated for third-year medical students (n = 781) at a single institution between 2004 and 2011. Cohorts were defined by student matriculation into either a general surgery residency program (n = 58) or a non-general surgery residency program (n = 723). Multivariate logistic regression was performed to evaluate independently significant academic measures. Clinical evaluation raw scores were predictive of general surgery matriculation (P = .014). In addition, multivariate modeling showed lower United States Medical Licensing Examination Step 1 scores to be independently associated with matriculation into general surgery (P = .007). Superior clinical aptitude is independently associated with general surgical matriculation. This is in contrast to the negative correlation United States Medical Licensing Examination Step 1 scores have on general surgery matriculation. Recognizing this, surgical clerkship directors can offer opportunities for continued surgical education to students showing high clinical aptitude, increasing their likelihood of surgical matriculation.
    American journal of surgery 10/2013; DOI:10.1016/j.amjsurg.2013.07.037 · 2.41 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: An analytics-driven process for improving the efficiency of initial candidate screening by pharmacy residency programs is described. In an initiative to streamline pharmacy resident selection at Johns Hopkins Hospital, retrospective analyses of materials submitted by prospective residents during two application periods (n = 277) were conducted to identify the applicant characteristics most strongly associated with the ultimate extension of an invitation to interview. Multiple two-member teams of pharmacist reviewers independently scored each application on 13 variables, with the resultant item scores tallied to derive rank sum scores. Univariate and multivariate logistic regression analyses were performed to identify the factors most important in distinguishing candidates invited for an interview from those not invited. Univariate analysis indicated that all 13 evaluated applicant characteristics correlated with the likelihood of an interview invitation, but some were relatively less determinative; these factors were excluded from a final multivariate logistic regression model containing only the 7 factors most strongly predictive of an invitation to interview: grade point average, pharmacy work experience, professional association involvement, rotation experiences, presentations, publications, and skills and certifications. The final model was found to be highly explanatory (r(2) = 0.66) of variances in interview-invitation decisions and has been adopted as a guide to future initial screening of candidate applications. By analyzing the relative importance of specific residency applicant characteristics and focusing on those deemed most useful in determining which candidates are invited for interviews, a large teaching institution streamlined preliminary application screening while maintaining an equitable candidate selection process.
    American journal of health-system pharmacy: AJHP: official journal of the American Society of Health-System Pharmacists 10/2013; 70(19):1670-1675. DOI:10.2146/ajhp120769 · 2.21 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective For the past 15 years at our institution’s general surgery residency program, 3 of the senior residents have been chosen to be awarded either (1) Best Resident in Research, (2) Best Resident in Teaching, or (3) Best Resident Overall. Considering that these awards serve as data representing outstanding performance as surgical residents, the objective of this study was to determine the association between receiving one of these awards and objective measures of performance. Methods Individual files were reviewed for the 103 residents who graduated from our institution’s general surgery program from 1994 to 2010. These data were studied as a whole, and then divided into an award-winning group and a non–award winning group and subsequently compared across several objective parameters, including The United States Medical Licensing Examination (USMLE) scores, American Board of Surgery In-Training Examination (ABSITE) scores, first-time American Board of Surgery Certifying and Qualifying Examination pass rates, Alpha Omega Alpha membership status, and number of research years, using a logistic regression model. Results Overall, 103 residents completed their general surgery residency training at our institution from 1994 to 2010, and of these residents, 16 (16%) received the Best Resident in Research award, 15 (16%) received the Best Resident in Teaching award, and 17 (17%) received the Best Resident Overall award in their final years of training. Compared with those who did not receive an award, a hypothesis-based one-tailed test revealed that award winners had a significantly lower median USMLE Step 1 scores (p = 0.04) and marginally lower median USMLE Step 2 scores (p = 0.05). Alpha Omega Alpha membership status, median ABSITE percent correct overall, first-time American Board of Surgery examination pass rates, and number of research years during residency were not significantly different between the 2 groups. Conclusion Many factors contribute to success during general surgery residency. Our study showed that higher USMLE and ABSITE scores were not associated with receiving top awards in final years of training at one institution over 15 years.
    Journal of Surgical Education 01/2013; 71(2). DOI:10.1016/j.jsurg.2013.07.012 · 1.39 Impact Factor