The Electronic Residency Application Service Application Can Predict Accreditation Council for Graduate Medical Education Competency-Based Surgical Resident Performance

Department of Surgery, Harbor-UCLA Medical Center, Torrance, California 90509, USA.
Journal of Surgical Education (Impact Factor: 1.38). 11/2010; 67(6):444-8. DOI: 10.1016/j.jsurg.2010.05.002
Source: PubMed


Program directors often struggle to determine which factors in the Electronic Residency Application Service (ERAS) application are important in the residency selection process. With the establishment of the Accreditation Council for Graduate Medical Education (ACGME) competencies, it would be important to know whether information available in the ERAS application can predict subsequent competency-based performance of general surgery residents.
This study is a retrospective correlation of data points found in the ERAS application with core competency-based clinical rotation evaluations. ACGME competency-based evaluations as well as technical skills assessment from all rotations during residency were collected. The overall competency score was defined as an average of all 6 competencies and technical skills.
A total of77 residents from two (one university and one community based university-affiliate) general surgery residency programs were included in the analysis. Receiving honors for many of the third year clerkships and AOA membership were associated with a number of the individual competencies. USMLE scores were predictive only of Medical Knowledge (p = 0.004). Factors associated with higher overall competency were female gender (p = 0.02), AOA (p = 0.06), overall number of honors received (p = 0.04), and honors in Ob/Gyn (p = 0.03) and Pediatrics (p = 0.05). Multivariable analysis showed honors in Ob/Gyn, female gender, older age, and total number of honors to be predictive of a number of individual core competencies. USMLE scores were only predictive of Medical Knowledge.
The ERAS application is useful for predicting subsequent competency based performance in surgical residents. Receiving honors in the surgery clerkship, which has traditionally carried weight when evaluating a potential surgery resident, may not be as strong a predictor of future success.

1 Follower
9 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: To evaluate the information available about otolaryngology residency applicants for factors that may predict future success as an otolaryngologist. Retrospective review of residency applications; survey of resident graduates and otolaryngology clinical faculty. Otolaryngology residency program. Otolaryngology program graduates from 2001 to 2010 and current clinical faculty from Barnes-Jewish Hospital/Washington University School of Medicine. Overall ratings of the otolaryngology graduates by clinical faculty (on a 5-point scale) were compared with the resident application attributes that might predict success. The application factors studied are United States Medical Licensing Examination part 1 score, Alpha Omega Alpha Honor Medical Society election, medical school grades, letter of recommendation, rank of the medical school, extracurricular activities, residency interview, experience with acting intern, and extracurricular activities. Forty-six graduates were included in the study. The overall faculty rating of the residents showed good interrater reliability. The objective factors, letters of recommendation, experience as an acting intern, and musical excellence showed no correlation with higher faculty rating. Rank of the medical school and faculty interview weakly correlated with faculty rating. Having excelled in a team sport correlated with higher faculty rating. Many of the application factors typically used during otolaryngology residency candidate selection may not be predictive of future capabilities as a clinician. Prior excellence in a team sport may suggest continued success in the health care team.
    No preview · Article · Aug 2012 · Archives of otolaryngology--head & neck surgery
  • [Show abstract] [Hide abstract]
    ABSTRACT: Selection of surgical residents is a difficult task, and program directors are interested in identifying the best candidates. Among the qualities being sought after is the ability to acquire surgical knowledge, and eventually do well on their board examinations. During the interview process, many programs use results from the United States Medical Licensing Exam (USMLE) to identify residents they think will do well academically. The purpose of this study was to evaluate a different method of identifying such residents, through the use of a surgery-specific written exam (SSWE). A retrospective review of residents in our program between 2004 and 2012 was done. A 50-question SSWE was designed and administered to candidates on the day of their interview. Scores on the SSWE and the USMLE were compared with results on the American Board of Surgery In-Training Exam (ABSITE). Correlation coefficients were calculated and compared. Community based General Surgery residency program. Resident applicants. Forty-three residents had scores available from the SSWE, USMLE Part 1 (USMLE-1), and Part 2 (USMLE-2). There were ABSITE scores available for 38 in postgraduate year (PGY) 1. USMLE-1 had a statistically significant correlation (r = 0.327, p = 0.045) with the ABSITE score in PGY-1 (ABSITE-1), while with USMLE-2 had slightly less correlation (r = 0.314, p = 0.055) with ABSITE-1. However, the SSWE had a much stronger correlation (r = 0.656, p < 0.001) than either of them. An SSWE is a good method to identify residents who will later do well on the ABSITE. It is a better method than using the more general USMLE. Since the ABSITE has been shown to correlate with performance on board examinations, residency programs interested in identifying candidates that will do well on their board examinations, should consider incorporating an SSWE into their application process.
    No preview · Article · Nov 2012 · Journal of Surgical Education
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective For the past 15 years at our institution’s general surgery residency program, 3 of the senior residents have been chosen to be awarded either (1) Best Resident in Research, (2) Best Resident in Teaching, or (3) Best Resident Overall. Considering that these awards serve as data representing outstanding performance as surgical residents, the objective of this study was to determine the association between receiving one of these awards and objective measures of performance. Methods Individual files were reviewed for the 103 residents who graduated from our institution’s general surgery program from 1994 to 2010. These data were studied as a whole, and then divided into an award-winning group and a non–award winning group and subsequently compared across several objective parameters, including The United States Medical Licensing Examination (USMLE) scores, American Board of Surgery In-Training Examination (ABSITE) scores, first-time American Board of Surgery Certifying and Qualifying Examination pass rates, Alpha Omega Alpha membership status, and number of research years, using a logistic regression model. Results Overall, 103 residents completed their general surgery residency training at our institution from 1994 to 2010, and of these residents, 16 (16%) received the Best Resident in Research award, 15 (16%) received the Best Resident in Teaching award, and 17 (17%) received the Best Resident Overall award in their final years of training. Compared with those who did not receive an award, a hypothesis-based one-tailed test revealed that award winners had a significantly lower median USMLE Step 1 scores (p = 0.04) and marginally lower median USMLE Step 2 scores (p = 0.05). Alpha Omega Alpha membership status, median ABSITE percent correct overall, first-time American Board of Surgery examination pass rates, and number of research years during residency were not significantly different between the 2 groups. Conclusion Many factors contribute to success during general surgery residency. Our study showed that higher USMLE and ABSITE scores were not associated with receiving top awards in final years of training at one institution over 15 years.
    No preview · Article · Jan 2013 · Journal of Surgical Education
Show more