Content uploaded by Heather Yeo
Author content
All content in this area was uploaded by Heather Yeo on Mar 07, 2022
Content may be subject to copyright.
Association of Demographic and Program Factors With American Board
of Surgery Qualifying and Certifying Examinations Pass Rates
Heather L. Yeo, MD, MHS, MBA, MS; Patrick T. Dolan, MD,MS; Jialin Mao, MD, MS; Julie A. Sosa, MD, MA
IMPORTANCE American Board of Surgery board certification requires passing both a written
qualifying examination and an oral certifying examination. No studies have been conducted
assessing the effect of sociodemographic variables on board passage rates.
OBJECTIVE To evaluate if trainee sociodemographic factors are associated with board
passage rates.
DESIGN, SETTING, AND PARTICIPANTS This national and multi-institutional prospective
observational cohort study of 1048 categorical general surgery trainees starting in
2007-2008 were surveyed. Data collection began in June 2007, follow-up was completed
on December 31, 2016, and analysis began September 2018.
MAIN OUTCOMES AND MEASURES Survey responses were linked to American Board of Surgery
board passage data.
RESULTS Of 662 examinees who had complete survey and follow-up data, 443 (65%) were
men and 459 (69%) were white, with an overall board passage rate of 87% (n = 578). In a
multinomial regression model, trainees of Hispanic ethnicity were more likely to not attempt
the examinations (vs passed both) than non-Hispanic trainees (odds ratio [OR], 4.7; 95% CI,
1.5-14). Compared with examinees who were married with children during internship,
examinees who were married without children (OR, 0.3; 95% CI, 0.1-0.8) or were single (OR,
0.4; 95% CI, 0.2-0.9) were less likely to fail the examinations. Logistic regression showed
white examinees compared with nonwhite examinees (black individuals, Asian individuals,
and individuals of other races) (OR, 1.8; 95% CI, 1.03-3.0) and examinees who performed
better on their first American Board of Surgery In-Training Examination (OR, 1.03; 95% CI,
1.02-1.05) were more likely to pass the qualifying examination on the first try. White
examinees compared with nonwhite examinees (OR, 1.8; 95% CI, 1.1-2.8), non-Hispanic
compared with Hispanic examinees (OR, 2.4; 95% CI, 1.2-4.7), and single women compared
with women who were married with children during internship (OR, 10.3; 95% CI, 2.1-51)
were more likely to pass the certifying examination on the first try.
CONCLUSIONS AND RELEVANCE Resident race, ethnicity, sex, and family status at internship
were observed to be associated with board passage rates. There are multiple possible
explanations for these worrisome observations that need to be explored. Tracking
demographics of trainees to help understand passage rates based on demographics will
be important. The American Board of Surgery already has begun addressing the potential
for unconscious bias among board examiners by increasing diversity and adding implicit
bias training.
JAMA Surg. 2020;155(1):22-30. doi:10.1001/jamasurg.2019.4081
Published online October 16, 2019.
Invited Commentary page 30
Author Audio Interview
Supplemental content
Author Affiliations: Department of
Surgery, NewYork-Presbyterian/Weill
Cornell Medicine, New York (Yeo,
Dolan); Department of Healthcare
Policy and Research,
NewYork-Presbyterian/Weill Cornell
Medicine, New York (Yeo, Mao);
Department of Surgery, University
of California, San Francisco,
San Francisco (Sosa).
Corresponding Author: Heather L.
Yeo,MD, MHS, MBA, MS, Department
of Surgery, NewYork-Presbyterian/
Weill Cornell Medical Center,
525 E 68th St, PO Box 172,
New York, NY 10065
(hey9002@med.cornell.edu).
Research
JAMA Surgery | Original Investigation
22 (Reprinted) jamasurgery.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
The certification process for general surgeons in North
America requires passing 2 examinations adminis-
tered by the American Board of Surgery (ABS). The
qualifying examination is a multiple choice examination
that residents are eligible to take after their fourth year of
clinical training. After passing the qualifying examination
and graduating from an accredited US or Canadian residency
program, examinees are then eligible for the certifying
examination. The certifying examination is an in-person
examination that is split into 3 consecutive 30-minute ses-
sions using case-based scenarios. An average of 1400 exam-
inees took the qualifying and certifying examinations each
year over the last 5 years, with an average pass rate of 85%
and 79%, respectively.
1
Candidates are board eligible for 7
years after graduating residency, and candidates who pass
both examinations are board certified.
Board certification is not mandatory by law to practice sur-
gery and is distinct from required state licensing; however,
many institutions will only consider hiring job applicants if they
are board certified or board eligible.
2
Board certification among
graduates also has important consequences for residency
programs. The Accreditation Council for Graduate Medical
Education requires 65% of a residency program’s graduates
over a 5-year period to pass both examinations on their first
attempt for the residency to maintain accreditation.
3
Several
studies have been published attempting to identify predic-
tive factors of passing the board examinations and evaluating
interventions to improve pass rates.
3-14
These smaller studies
have focused on the effect of mock oral examinations, prior
standardized test performance, and training factors (eg, case
volume, fellowship training). Previously, there have been few
sociodemographic data on board examinees, and currently it
is unknown if there are any differences in board examination
passage rates based on resident race, ethnicity, family status,
or residency program factors.
This study is a follow-up of a national prospective cohort
study of all categorical general surgery interns entering gen-
eral surgery residency training in the 2007-2008 academic
year. Interns in this class were surveyed about sociodemo-
graphic information, medical school experience, and expec-
tations of life as a surgeon. We then linked these data with ABS
data on board examination passage rates. This is the first study
using data from a national sample of US general surgery train-
ees to analyze the likelihood of passing the board examina-
tions based on a variety of sociodemographic and program vari-
ables, to our knowledge.
Methods
All general surgery interns in the entering class of 2007-2008
who participated in the National Study of Expectations and
Attitudes of Residentsin Surgery and who completed our sur-
vey were included.
15
The National Study of Expectations and
Attitudes of Residents in Surgery study has been well de-
scribed previously in the literature.
15-20
Interns who partici-
pated in this survey were followed up for 8 years, using link-
able data provided by the ABS. This study was initially approved
by the Yale School of Medicine institutional review board and
was then transferred and reapproved by the Weill Cornell
Medicine institutional review board.
16
Completion of the
survey constituted implied consent, and the survey included
information that respondents were goingto be followed up and
may be recontacted. Data collection began in June 2007, and
analysis began September 2018.
Development of the survey was based on qualitative in-
terviews of general surgery residents who left their training
programs, as well as prior literature on attrition and collabo-
ration with the ABS.
15
Survey questions gathered informa-
tion about intern demographics, expectations for residency
training, reasons why interns chose specific residency pro-
grams, and expectations of life as an attending surgeon.
We performed a secondary analysis of the data we col-
lected as part of the National Study of Expectations and
Attitudes of Residents in Surgery cohort; our primary study
end point was the pass rate for ABS-administered qualifying
and certifying examinations. We explored what sociodemo-
graphic and program factors, if any, were associated with pass-
ing these examinations compared with either failing or mak-
ing no attempt to take them. Secondary end points included
what sociodemographic and program factors, if any, were as-
sociated with passing these examinations on the first at-
tempt. For this study, only those interns who completed the
sociodemographic questionnaire and finished training were
included. Residents who finished training in 2016 also were
excluded as these residents were not followed up long enough
to obtain reliable board passage data.
Cohort characteristics were examined. Univariate analy-
ses comparing sociodemographic and program factors among
those who (1) did not attempt the board examinations,
(2) passed both examinations, or (3) failed either examina-
tion were performed. Differences across groups were as-
sessed using χ
2
tests. Univariate analyses also were done com-
paring sociodemographic and program factors between those
who passed the qualifying and certifying examinations on the
first try vs those who did not. Differences between groups were
assessed using χ
2
tests.
Key Points
Question Is there an association between sociodemographic
factors and American Board of Surgery board examination
passage rates?
Findings In this study of 662 individuals, trainees of Hispanic
ethnicity were more likely to not attempt either of the board
examinations (qualifying and certifying examinations), and
examinees who were married with children were more likely to fail
the examinations. White examinees were more likely to pass the
qualifying examination on the first attempt; white examinees,
non-Hispanic ethnicity examinees, and single women compared
with women with children during internship were more likely to
pass the certifying examination on the first attempt.
Meaning In a national sample of trainees, we observed adverse
impact based on sociodemographic factors on passing the board
certification examinations that needs further exploration.
Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates Original Investigation Research
jamasurgery.com (Reprinted) JAMA Surgery January 2020 Volume155, Number 1 23
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
A multinomial regression analysis was performed to
ascertain the likelihood of either making no attempt at the
boards compared with board passage as well as failing the
boards compared with board passage as predicted by
interns’ answers to sociodemographic and program ques-
tions. This model included individual intern characteristics,
including sex, race, ethnicity, family status (married with
children, married without children, unmarried with partner,
and single), graduation from a foreign medical school,
residency finish year (2012, 2013-2015), and first ABS
In-Training Examination (ABSITE) performance as well as
training program characteristics, including type (academic,
community, military), size (more than 5 graduating chiefs vs
less), and geographic location (Northeast, South, Midwest,
West). Logistic regression analyses were performed, includ-
ing the same independent variables described above to
ascertain the likelihood of passing the qualifying and certify-
ing examinations on the first attempt (vs not) by sociodemo-
graphic and program variables. Interactions between partici-
pant characteristics and program size and type were
examined and tested for significance. With respect to pass-
ing the certifying examination on the first attempt, the inter-
action between sex and family status was tested to be sig-
nificant (P= .03). Therefore, the likelihood of passing the
certifying examination on the first attempt by family status
was further broken down into 2 subgroups by sex. All statis-
tical tests were 2-sided; statistical significance was defined
asaPvalue less than .05. The statistical analysis was per-
formed using SAS version 9.3 (SAS Institute Inc).
Results
Of 1048 categorical general surgery interns who started their
training in 2007,836 (80%) had linkable survey data. Of these,
672 (64%) finished residency and therefore were eligible to take
the board examinations. We excluded those who had missing
demographic information and those who finished residency
in 2016, as they did not have sufficient follow-up, leaving a
final cohort of 662 (63%). This cohort was 65% (n = 433) male
and 69% (n = 459) white. Overall board passage rate was 87%
(n=578)(Table 1).
Univariate analysis of passing both qualifying and certi-
fying examinations showed no significant differences based
on sex, race, ethnicity, family status during internship, medi-
cal school location, residency program location, program size,
or program type. There was a difference in first ABSITE per-
formance between the no attempt, passed both, and failed
either groups (mean 55th vs 58th vs 44th percentile; F
2
= 7.86;
P< .001). There was also a difference in passage rates based
on the year trainees completed residency, with a 90.1% (374
of 415) passage rate in trainees who finished in 2012 com-
pared with an 82.6% (204 of 247) passage rate in those who
finished in 2013-2015 (P=.02)(Table 2).
A multinomial regression analysis assessing the likeli-
hood of making no attempt or failing either examination vs
passing both showed that graduated trainees of Hispanic eth-
nicity were more likely to not attempt either examinationcom-
pared with those of non-Hispanic ethnicity (odds ratio [OR],
4.66; 95% CI, 1.51-14.4). Compared with examinees who were
married with children during internship, examinees who were
married without children (OR, 0.3; 95% CI, 0.11-0.78) or were
single (OR, 0.36; 95% CI, 0.16-0.85) were less likely to fail the
examinations. Examinees who performed better on their first
ABSITE examination were less likely to fail the board exami-
nations (OR, 0.98; 95% CI, 0.96-0.99) (Table 3).
With respect to passing the qualifying examination on the
first try, univariate analysis showed those who performed bet-
ter on the first ABSITE they took (mean 60th vs 39th percen-
tile; t= −7.29; P< .001) passed the examination on the first try.
Table 4 provides a full list of statistically significant univari-
ate associations. In a logistic regression analysis, white exam-
inees were more likely to pass the qualifying examination on
Table 1. Characteristics of the Cohort
Characteristic No. (%)
Total No. 662
Men 433 (65.4)
Race
White 459 (69.3)
Black 33 (5.0)
Asian 113 (17.1)
Other 57 (8.6)
Ethnicity
Non-Hispanic 609 (92.0)
Hispanic 53 (8.0)
Family status
Married with children 79 (11.9)
Married without children 169 (25.5)
Unmarried with partner 174 (26.3)
Single 240 (36.3)
Went to medical school in the United States or Canada 568 (85.8)
Program location
Northeast 191 (28.9)
South 235 (35.5)
Midwest 153 (23.1)
West 83 (12.5)
Large program size (>5 graduating chiefs) 166 (25.1)
Program type
Academic 396 (59.8)
Community 247 (37.3)
Military 19 (2.9)
Residency finish year
2012 415 (62.7)
2013-2015 247 (37.3)
First-year ABSITE percentile, mean (SD) 56.7 (25.8)
Board passage status
No attempt
a
27 (4.1)
Passed both 578 (87.3)
Failed either 57 (8.6)
Abbreviation: ABSITE, American Board of Surgery In-Training Examination.
a
Either no attempt at the qualifying examination or passed the qualifying
examination and made no attempt at the certifying examination.
Research Original Investigation Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates
24 JAMA Surgery January 2020 Volume 155, Number 1 (Reprinted) jamasurgery.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
the first try compared with nonwhite examinees (black indi-
viduals, Asian individuals, and individuals of other races) (OR,
1.75; 95% CI, 1.03-2.95) and examinees who performed better
on their first ABSITE were more likely to pass the qualifying
examination on the first try (OR, 1.03; 95% CI, 1.02-1.05)
(Table 4).
With respect to passing the certifying examination on
the first try, univariate analysis showed a higher percentage
of white compared with nonwhite examinees (367 [85%] vs
137 [75%]; χ
2
1
=8.09; P= .004), non-Hispanic compared with
Hispanic ethnicity examinees (471 [83%] vs 33 [67%];
χ
2
1
=7.31; P= .007), and single women compared with
women who were unmarried with a partner, married with no
children, or married with children during internship (84
[90%] vs 52 [75%] vs 34 [76%] vs 5 [56%]; P= .01) passed the
certifying examination on the first try. On univariate analy-
sis, there was no difference in first-year ABSITE performance
between those who passed the certifying examination on
their first attempt compared with those who did not (mean
54th vs 59th percentile; t=−1.69;P= .09). In a logistic
regression analysis, white examinees compared with non-
white examinees (OR, 1.76; 95% CI, 1.11-2.79), non-Hispanic
examinees compared with Hispanic examinees (OR, 2.35;
95% CI, 1.18-4.67), and single women compared with women
who were married with children at the time of their intern-
ship (OR, 10.26; 95% CI, 2.08-50.63) were more likely to pass
the certifying examination on the first try. Notably family
status during internship had no effect on the likelihood of
male examinees passing the certifying examination on their
first try (Table 5). A sensitivity analysis was performed in
which we transformed ABSITE score from a continuous into
a categorical variable, dividing the percentile scores into ter-
tiles. This had no effect on the results in any of our regres-
sion models (eTables 1-5 in the Supplement).
Table 2. UnivariateAnalysis of Passing the Qualifying and Certifying Examinations
by Demographic and Program Variables Usingχ
2
Tes ts
Characteristic
No. (%)
PValue
No Attempt
(n = 27)
Passed Both
(n = 578)
Failed Either
(n = 57)
Sex
Men 21 (4.8) 377 (87.1) 35 (8.1)
.33
Women 6 (2.6) 201 (87.8) 22 (9.6)
Race
White 19 (4.1) 406 (88.5) 34 (7.4)
.25
Nonwhite
a
8 (3.9) 172 (84.7) 23 (11.3)
Ethnicity
Non-Hispanic 22 (3.6) 535 (87.8) 52 (8.5)
.11
Hispanic 5 (9.4) 43 (81.1) 5 (9.4)
Family status
Married with children 2 (2.5) 65 (82.3) 12 (15.2)
.12
Married without children 5 (3.0) 155 (91.7) 9 (5.3)
Unmarried with partner 6 (3.4) 151 (86.8) 17 (9.8)
Single 14 (5.8) 207 (86.3) 19 (7.9)
Medical school location
United States/Canada 25 (4.4) 493 (86.8) 50 (8.8)
.52
Others 2 (2.1) 85 (90.4) 7 (7.4)
Program location
Northeast 6 (3.1) 163 (85.3) 22 (11.5)
.42
South 10 (4.3) 206 (87.7) 19 (8.1)
Midwest 9 (5.9) 132 (86.3) 12 (7.8)
West 2 (2.4) 77 (92.8) 4 (4.8)
Program size
Small 16 (3.2) 435 (87.7) 45 (9.1)
.13
Large (>5 graduating chiefs) 11 (6.6) 143 (86.1) 12 (7.2)
Program type
Academic 21 (5.3) 339 (85.6) 36 (9.1)
.30
Community 5 (2.0) 223 (90.3) 19 (7.7)
Military 1 (5.3) 16 (84.2) 2 (10.5)
Residency finish year
2012 13 (3.1) 374 (90.1) 28 (6.7)
.02
2013-2015 14 (5.7) 204 (82.6) 29 (11.7)
First-year ABSITE percentile, mean (SD) 55.0 (25.9) 58.1 (25.2) 44.1 (28.0) <.001
Abbreviation: ABSITE, American
Board of Surgery In-Training
Examination.
a
The nonwhite category includes
black, Asian, and other races.
Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates Original Investigation Research
jamasurgery.com (Reprinted) JAMA Surgery January 2020 Volume155, Number 1 25
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
Discussion
To our knowledge, this is the first study to describe the asso-
ciation of a variety of sociodemographic and program vari-
ables with the likelihood of passing the qualifying and certi-
fying examinations for ABS certification. We observed
concerning associations between examinee race, ethnicity,
sex, and family status during internship on rates of taking the
boards and board passage.
Based on these data, there appears to be an association
between several sociodemographic factors including race,
ethnicity, parental status, and duration of training, and at-
tempting the ABS qualifying and certifying examinations, as
well as passing the examinations on the first attempt. Some
of our findings were not statistically significant on univariate
analysis and became so after adjustment in our multivariable
regression models, likely owing to some degree of confound-
ing between sociodemographic variables that were only con-
trolled for in a multivariate analysis. A 2012 study showed that
medical school graduates entering general surgery training who
were underrepresented racial minorities were nearly twice as
likely to not be board certified than graduates who were white.
21
A limitation of the previous study is that the reason for not
being board certified was not ascertained. It is possible that the
disparity they observed was due to other factors besides pass-
ing the examinations, such as failure to complete residency
training or a purposeful choice by the graduate to forego board
certification. Either way this is concerning, as there is already
underrepresentation in surgery. Our data are the first to show
sociodemographic disparities in attempting and passing the
ABS examinations to become board certified.
The associations observed between race, ethnicity, and
board passage were stronger in the first passagerate of the cer-
tifying examination compared with the qualifying examina-
tion. It is important to note the 2 examinations are designed
to test different qualities in the examinee, with the qualify-
ing examination more testing applied knowledge and the cer-
tifying examination testing judgment and effective commu-
nication, building on the knowledge component tested on the
qualifying examination. However, it is equally important to
note the qualifying examination is a written multiple choice
examination, whereas the certifying examination is con-
ducted in-person, and it is impossible to blind examiners to
their perception of the race and ethnicity of the examinee. This
makes the administration and grading of the examination sus-
ceptible to implicit bias. Furthermore, these examinations are
taken in series; an examinee cannot sit for the certifying ex-
amination until they have passed the qualifying examina-
tion, essentially filtering the pool of examinees to only those
who have the fund of knowledge to succeed on the certifying
examination.
There were performance differences observed in the quali-
fying examination, with a higher percentage of white com-
pared with nonwhite examinees and a higher percentage of
non-Hispanic compared with Hispanic examinees who passed
the examination on the first attempt, by 4.5% and 4.2%, re-
spectively. However, in a pool of ostensibly more qualified can-
didates now taking an in-person examination, this difference
in first-time pass rates more than doubled to 9.7% between
white and nonwhite examinees and nearly quadrupled to 15.6%
between non-Hispanic and Hispanic examinees. Although we
cannot know for certain, with such a stark difference in first-
time pass rates between written and in-person examinations,
albeit designed to test different things, we cannot ignore the
possibility that implicit bias is playing a role in how examin-
ees are graded on their performance during the certifying ex-
amination. However, there are other possible explanations for
this finding besides implicit bias on the part of the examin-
ers. Prior data have shown that residents who identified as an
Table 3. Multinomial RegressionAnalysis of Passing the Qualifying
and Certifying Examinations by Demographic and Program Variables
Characteristic
Odds Ratio (95% CI)
No Attempt
vs Passed Both
a
Failed Either
vs Passed Both
b
Sex
Men 1 [Reference] 1 [Reference]
Women 0.38 (0.14-1.02) 0.98 (0.52-1.82)
Race
Nonwhite
c
1 [Reference] 1 [Reference]
White 1.08 (0.44-2.65) 0.57 (0.31-1.05)
Ethnicity
Non-Hispanic 1 [Reference] 1 [Reference]
Hispanic 4.66 (1.51-14.4) 1.11 (0.39-3.14)
Family status
Married with children 1 [Reference] 1 [Reference]
Married without children 1.13 (0.20-6.48) 0.30 (0.11-0.78)
Unmarried with partner 1.59 (0.28-8.99) 0.51 (0.21-1.23)
Single 2.49 (0.50-12.44) 0.36 (0.16-0.85)
Medical school location
United States/Canada 1 [Reference] 1 [Reference]
Others 0.55 (0.12-2.63) 1.00 (0.39-2.53)
Program location
Northeast 1 [Reference] 1 [Reference]
South 0.76 (0.25-2.35) 0.54 (0.26-1.13)
Midwest 1.41 (0.46-4.36) 0.53 (0.24-1.20)
West 0.47 (0.09-2.50) 0.30 (0.10-0.96)
Program size
Small 1 [Reference] 1 [Reference]
Large (>5 graduating chiefs) 1.86 (0.76-4.56) 0.80 (0.38-1.69)
Program type
Academic 1 [Reference] 1 [Reference]
Community 0.53 (0.17-1.64) 0.66 (0.33-1.32)
Military 1.55 (0.16-14.64) 0.69 (0.13-3.65)
Residency finish year
2012 1 [Reference] 1 [Reference]
2013-2015 1.89 (0.82-4.39) 2.15 (1.17-3.97)
First-year ABSITE percentile
1-Percentile increase 0.99 (0.98-1.01) 0.98 (0.96-0.99)
Abbreviation: ABSITE, American Board of Surgery In-Training Examination.
a
No attempt vs passed both are the odds of making no attempt.
b
Failed either vs passed both are the odds of failing either.
c
The nonwhite category includes black, Asian, and other races.
Research Original Investigation Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates
26 JAMA Surgery January 2020 Volume 155, Number 1 (Reprinted) jamasurgery.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
underrepresented minority were less likely to feel they could
count on their resident peers for help and that they were less
likely to feel like they fit in at their programs.
22
In conjunc-
tion with other data that suggest mock oral examinations lead
to modest improvement in certifying examination perfor-
mance and that residency program administration of mock oral
examinations heavily depends on the participation of senior
residents, it is plausible that underrepresented minority resi-
dents are not receiving adequate program-levelpreparation for
the certifying examination.
14,23
Regardless of the potential causes of our observation, we
believe it is important to highlight these associations and iden-
tify them as problematic. The impact of these findings cannot
be understated, especially as we are addressing the likely re-
lated problem of our failure to retain underrepresented minori-
ties in academic surgery.
24,25
These findings also have signifi-
cant consequences for residency programs, as first passage rates
of program graduates are being used as a metric to maintain
Accreditation Council for Graduate Medical Education accredi-
tation. Adverse impact on pass rates for underrepresented
minorities on their first attempt at the board examinations
could lead to a disastrous feedback loop in which residency
programs either lose accreditation or become less willing to
accept minorities into their programs, subsequently leading
to even less representation of these groups in surgery.
To address this issue, we believeall board examiners should
have implicit bias training and that the pool of examiners
should more closely resemble the pool of examinees with re-
spect to sociodemographic factors. The ABS has already taken
steps to do this. However, there are limitations to implicit bias
training. There are data that show training has some success
at changing individual beliefs, but a meta-analysis suggests it
may not be effective at improving institutional-level inequi-
ties. Furthermore, some research has shown implicit bias train-
ing may reinforce biases.
26
This is why we believe that track-
ing examinee sociodemographic factors in a prospective
Table 4.Univariate Analysis of Passing the Qualifying Examination (QE) on the First Try
a
Characteristic
Passed QE on the First Try, No. (%) Passed QE on First Try
vs Not,
OR (95% CI)
b
No (n = 87) Yes (n = 575) PValue
Sex
Men 52 (12.0) 381 (88.0)
.24
1 [Reference]
Women 35 (15.3) 194 (84.7) 1.07 (0.64-1.81)
Race
Nonwhite
c
33 (16.3) 170 (83.7)
.11
1 [Reference]
White 54 (11.8) 405 (88.2) 1.75 (1.03-2.95)
Ethnicity
Hispanic 9 (17.0) 44 (83.0)
.39
1 [Reference]
Non-Hispanic 78 (12.8) 531 (87.2) 1.23 (0.53-2.85)
Family status
Married with children 12 (15.2) 67 (84.8)
.33
1 [Reference]
Married without children 16 (9.5) 153 (90.5) 1.71 (0.72-4.08)
Unmarried with partner 22 (12.6) 152 (87.4) 1.46 (0.62-3.42)
Single 37 (15.4) 203 (84.6) 1.29 (0.59-2.83)
Medical school location
United States/Canada 81 (14.3) 487 (85.7)
.04
1 [Reference]
Others 6 (6.4) 88 (93.6) 1.80 (0.70-4.63)
Program location
Northeast 25 (13.1) 166 (86.9)
.92
1 [Reference]
South 32 (13.6) 203 (86.4) 1.30 (0.67-2.51)
Midwest 21 (13.7) 132 (86.3) 1.26 (0.62-2.55)
West 9 (10.8) 74 (89.2) 1.49 (0.62-3.54)
Program size
Small 66 (13.3) 430 (86.7)
.83
1 [Reference]
Large (>5 graduating chiefs) 21 (12.7) 145 (87.3) 0.90 (0.48-1.66)
Program type
Academic 54 (13.6) 342 (86.4)
.46
1 [Reference]
Community 29 (11.7) 218 (88.3) 1.06 (0.59-1.91)
Military 4 (21.1) 15 (79.0) 0.83 (0.23-3.03)
Residency finish year
2012 45 (10.8) 370 (89.2)
.02
1 [Reference]
2013-2015 42 (17.0) 205 (83.0) 0.55 (0.33-0.91)
First-year ABSITE percentile, mean (SD) 38.7 (24.2) 59.5 (24.9) <.001 1.03 (1.02-1.05)
Abbreviations: ABSITE, American
Board of Surgery In-Training
Examination; OR, odds ratio.
a
χ
2
Tests comparingdemographic
and program variables are reported
as Pvalues.
b
Logistic regression analysis by
demographic and program variables
reported as OR (95% CI).
c
The nonwhite category includes
black, Asian, and other races.
Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates Original Investigation Research
jamasurgery.com (Reprinted) JAMA Surgery January 2020 Volume155, Number 1 27
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
manner is important to assess how our interventions are im-
pacting inequities in examination scores. The surgical com-
munity cannot begin to address and fix this problem if we do
not know what passage rates are.
We observed that sex and marital status at the time of in-
ternship also had a significant association with whether an in-
dividual passed the board examinations. Examinees who were
married with children were more likely to fail either exami-
nation than their married but childless and single counter-
parts. Women who weresingle during internship were 10 times
more likely than women with children to pass the certifying
examination on their first try. The association between hav-
ing children and first-time certifying examination passage was
not observed in men, and implicit bias on the part of examin-
ers is an unlikely cause. This observation could highlight a
potential consequence of a broader sociologic phenomenon
that although married women with children are working more,
they are still expected to fulfill their traditional gender role of
primary caregiver to their children.
27
According to a survey of
US households between 2013 and 2017 conducted by the US
Bureau of Labor Statistics, women with children younger than
6 years spent an average of 1.1 hours per day of direct physical
care to children compared with 26 minutes for men.
28
Al-
though this is a survey of the general population, there is no
reason to believe the results would be different in a subgroup
analysis of surgeons and surgeons in training. These statis-
tics likely translate to a substantial advantage in examination
preparation for men regardless of childbearing statusand single
or childless women compared with women with children. This
is a significant societal problem, with consequences that go
Table 5.Univariate Analysis of Passing the Certifying Examination (CE) on the First Try
a
Characteristic
Passed CE on the First Try, No. (%) Passed CE on First Try
vs Not,
OR (95% CI)
b
No (n = 113) Yes (n = 504) PValue
Race
Nonwhite
c
46 (25.1) 137 (74.9)
.004
1 [Reference]
White 67 (15.4) 367 (84.6) 1.76 (1.11-2.79)
Ethnicity
Hispanic 16 (32.7) 33 (67.3)
.007
1 [Reference]
Non-Hispanic 97 (17.1) 471 (82.9) 2.35 (1.18-4.67)
Family status
Among men
Married with children 12 (19.4) 50 (80.7)
.81
1 [Reference]
Married without children 19 (16.2) 98 (83.8) 1.17 (0.51-2.67)
Unmarried with partner 15 (16.1) 78 (83.9) 1.45 (0.60-3.51)
Single 26 (20.2) 103 (79.8) 1.10 (0.49-2.46)
Among women
Married with children 4 (44.4) 5 (55.6)
.01
1 [Reference]
Married without children 11 (24.4) 34 (75.6) 2.83 (0.58-13.92)
Unmarried with partner 17 (24.6) 52 (75.4) 3.60 (0.76-17.04)
Single 9 (9.7) 84 (90.3) 10.26 (2.08-50.63)
Medical school location
United States/Canada 97 (18.4) 429 (81.6)
.84
1 [Reference]
Others 16 (17.6) 75 (82.4) 1.42 (0.73-2.77)
Program location
Northeast 42 (23.7) 135 (76.3)
.05
1 [Reference]
South 28 (12.9) 189 (87.1) 2.25 (1.25-4.04)
Midwest 27 (18.9) 116 (81.1) 1.44 (0.79-2.60)
West 16 (20.0) 64 (80.0) 1.38 (0.68-2.79)
Program size
Small 87 (18.8) 375 (81.2)
.57
1 [Reference]
Large (>5 graduating chiefs) 26 (16.8) 129 (83.2) 1.02 (0.59-1.77)
Program type
Academic 67 (18.2) 302 (81.8)
.14
1 [Reference]
Community 46 (19.8) 186 (80.2) 1.21 (0.73-2.02)
Military 0 (0) 16 (100) NA
Residency finish year
2012 80 (20.1) 318 (79.9)
.12
1 [Reference]
2013-2015 33 (15.1) 186 (84.9) 1.46 (0.88-2.41)
First-year ABSITE percentile, mean (SD) 54.3 (26.3) 58.7 (25.2) .09 1.01 (1.00-1.02)
Abbreviations: ABSITE, American
Board of Surgery In-Training
Examination; NA, not applicable;
OR, odds ratio.
a
χ
2
Tests comparingdemographic
and program variables are reported
as Pvalues.
b
Logistic regression analysis by
demographic and program variables
reported as OR (95% CI).
c
The nonwhite category includes
black, Asian, and other races.
Research Original Investigation Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates
28 JAMA Surgery January 2020 Volume 155, Number 1 (Reprinted) jamasurgery.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
beyond board passage. Female surgeons are more likely to leave
residency training later in their training, less likely to reach se-
nior academic leadership positions, and earn on average
$40 000 per year less than men.
29,30
Although the larger so-
cietal issue of traditional gender family roles is difficult to
address, the surgical community should strive to changeby pro-
viding equal opportunities for both men and women to suc-
ceed. Improved pay parity and increased flexibility in sched-
uling would be large strides in empowering professional
women both in the workplace and at home, potentially lead-
ing to improved gender diversity in academic surgery.
Strengths and Limitations
The main strength of our study is that we used a large na-
tional cohort of surgery residents from many institutions to
obtain our results, which are likely generalizable. A limita-
tion of our study is that study participants were not reas-
sessed during the study period, and answers to their sociode-
mographic questions (particularly their family status) may have
changed from the time they were interns to the time they took
their board examinations. Also, we could not ascertain howex-
aminees prepared for examinations or their performance dur-
ing residency and on prior standardized examinations, all of
which could be potentially confounding variables. However,
other studies cited throughout this article have assessed the
potential effect of these other variables. Our aim specifically
was to assess differences in demographic and program vari-
ables, as this has not been done previously, and the ABS does
not keep track of these data. A statistical limitation of our study
is that our multivariable model contains more variables than
recommended given the few outcomes we studied, limitingits
power. However, we did try to limit the number of variables
to those factors we thought would be most important.
Conclusions
We found in a national cohort of general surgery trainees that
race, ethnicity, sex, and family status during internship were
significantly associated with attempting and passing the ABS
certification examinations overall and on the first attempt.
Every effort should be made to retain underrepresented mi-
norities and women in surgery, as there is currently a signifi-
cant lack of representation of these groups within academic
surgery.The adverse impact observed for these groups in pass-
ing board examinations is a potential contributing factor to this
lack of diversity. These findings have significant potential im-
pact on training programs, as the board passage rate on exam-
inees’ first attempts has become a metric to maintain pro-
gram accreditation. It is important to highlight that we do not
know why the observed differences exist, and implicit bias
by examiners is not the only potential explanation. Going for-
ward, it is imperative to track these variables and outcomes
to ensure equal and fair treatment of the examinees and to ad-
dress the potential for implicit bias among examiners by ad-
ministering training and amplifying diversity among the ex-
aminer pool. Our finding that women who were single during
internship were 10 times more likely to pass the certifying ex-
amination on the first try compared with women who were
married with children during internship, a finding not ob-
served in men, likely reflects larger societal issues around tra-
ditional gender roles in families. Therefore, it will likely be more
difficult to address at an institutional level. It is incumbent
upon us to view training and examination preparation through
this lens, and it should be of paramount importance to en-
sure all trainees have adequate time and resources to prepare
for the ABS examinations.
ARTICLE INFORMATION
Accepted for Publication: July 21, 2019.
Published Online: October 16, 2019.
doi:10.1001/jamasurg.2019.4081
Author Contributions: Drs Yeo and Sosa had full
access to all of the data in the study and take
responsibility for the integrity of the data and the
accuracy of the data analysis.
Concept and design: Yeo,Mao, Sosa.
Acquisition, analysis, or interpretation of data:
All authors.
Drafting of the manuscript: Yeo,Dolan.
Critical revision of the manuscript for important
intellectual content: Yeo, Mao, Sosa.
Statistical analysis: Mao.
Obtained funding: Yeo.
Administrative, technical, or material support:
Yeo,Dolan.
Supervision: Yeo, Dolan, Sosa.
Conflict of Interest Disclosures: Dr Yeo serves on a
medical advisory board for SurvivorNet. Dr Dolan
reports a grant from the National Institutes of
Health Agency for Healthcare Research and Quality
outside the submitted work. Dr Sosa is a member of
the data monitoring committee of the Medullary
Thyroid Carcinoma Consortium Registry supported
by Novo Nordisk, GlaxoSmithKline, AstraZeneca,
and Eli Lilly and Company.No other disclosure s
were reported.
Funding/Support: The original cohort study was
partially supported by the Robert Wood Johnson
Foundation.
Role of the Funder/Sponsor:The funder had
no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
REFERENCES
1. The American Board of Surgery. General surgery
exam pass rates. http://www.absurgery.org/default.
jsp?statgeneral. Accessed September 6, 2019.
2. Freed GL, Dunham KM, Singer D. Use of board
certification and recertification in hospital
privileging: policies for general surgeons, surgical
specialists, and nonsurgical subspecialists. Arch Surg.
2009;144(8):746-752. doi:10.1001/archsurg.
2009.28
3. Fingeret AL, Arnell T, McNelis J, Statter M,
Dresner L, Widmann W. Sequential participation in
a multi-institutional mock oral examination is
associated with improved American Board of
Surgery certifying examination first-time pass rate.
J Surg Educ. 2016;73(6):e95-e103. doi:10.1016/j.
jsurg.2016.06.016
4. Rowland-Morin PA, Coe NP,Greenburg AG, et al.
The effect of improving communication
competency on the certifying examination of the
American Board of Surgery. Am J Surg. 2002;183
(6):655-658. doi:10.1016/S0002-9610(02)
00861-9
5. Sako EY, Petrusa ER Jr,Paukert JL. Factors
influencing outcome of the American Board of
Surgery Certifying examination: an observational
study.J Surg Res. 2002;105(2):75-80. doi:10.1006/
jsre.2002.6371
6. Guzman E, Babakhani A, Maker VK. Improving
outcomes on the ABS Certifying Examination: can
monthly mock orals do it? J Surg Educ. 2008;65(6):
441-444. doi:10.1016/j.jsurg.2008.04.005
7. Shellito JL, Osland JS, Helmer SD, Chang FC.
American Board of Surgery examinations: can we
identify surgery residency applicants and residents
who will pass the examinations on the first
attempt? Am J Surg. 2010;199(2):216-222.
doi:10.1016/j.amjsurg.2009.03.006
8. de Virgilio C, Yaghoubian A, Kaji A, et al.
Predicting performance on the American Board of
Surgery qualifying and certifying examinations:
a multi-institutional study.Arch Surg. 2010;145(9):
852-856. doi:10.1001/archsurg.2010.177
Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates Original Investigation Research
jamasurgery.com (Reprinted) JAMA Surgery January 2020 Volume155, Number 1 29
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
9. Rowland PA, Trus TL, Lang NP, et al. The
certifying examination of the American Board of
Surgery: the effect of improving communication
and professional competency: twenty-year results.
J Surg Educ. 2012;69(1):118-125. doi:10.1016/
j.jsurg.2011.09.012
10. Malangoni MA, Jones AT, Rubright J,Biester
TW, Buyske J,Lewis FR Jr. Delay in taking the
American Board of Surgery qualifying examination
affects examination performance. Surgery. 2012;
152(4):738-743. doi:10.1016/j.surg.2012.07.001
11. Maker VK, Zahedi MM, Villines D, Maker AV. Can
we predict which residents are going to pass/fail
the oral boards? J Surg Educ. 2012;69(6):705-713.
doi:10.1016/j.jsurg.2012.08.009
12. Jones AT, Biester TW, Buyske J, Lewis FR,
Malangoni MA. Using the American Board of
Surgery In-Training Examination to predict board
certification: a cautionary study. J Surg Educ.2014;
71(6):e144-e148. doi:10.1016/j.jsurg.2014.04.004
13. Jones AT, Biester TW, Lewis FR Jr, Malangoni
MA. Review courses for the American Board of
Surgery certifying examination do not provide an
advantage. Surgery. 2015;158(4):890-896. doi:10.
1016/j.surg.2015.04.044
14. Kimbrough MK, Thrush CR, Smeds MR, Cobos
RJ, Harris TJ, Bentley FR. National landscape of
general surgery mock oral examination practices:
survey of residency program directors. J Surg Educ.
2018;75(6):e54-e60.doi:10.1016/j.jsurg.2018.07.012
15. Yeo H, Viola K, Berg D, et al. Attitudes, training
experiences, and professional expectations of US
general surgery residents: a national survey. JAMA.
2009;302(12):1301-1308. doi:10.1001/jama.2009.
1386
16. Abelson JS, Sosa JA, Symer MM, et al.
Association of expectations of training with
attrition in general surgery residents. JAMA Surg.
2018;153(8):712-717. doi:10.1001/jamasurg.2018.0611
17. Sullivan MC, Yeo H, Roman SA, et al. Surgical
residency and attrition: defining the individual and
programmatic factors predictive of trainee losses.
J Am Coll Surg. 2013;216(3):461-471. doi:10.1016/
j.jamcollsurg.2012.11.005
18. Symer MM, Abelson JS, Gade L, Mao J, Sosa JA,
Yeo HL. Association between American Board of
Surgery in-training examination score and attrition
from general surgery residency. Surgery. 2018;164
(2):206-211. doi:10.1016/j.surg.2018.03.016
19. Symer MM, Abelson JS, Yeo HL, Sosa JA,
Rosenthal MZ. The surgical personality: does
surgery resident motivation predict attrition? JAm
Coll Surg. 2018;226(5):777-783. doi:10.1016/
j.jamcollsurg.2018.02.007
20. Yeo HL, Abelson JS, Mao J, et al. Who makes it
to the end? a novel predictive model for identifying
surgical residents at risk for attrition. Ann Surg.
2017;266(3):499-507. doi:10.1097/SLA.
0000000000002384
21. Andriole DA, Jeffe DB. Certification by the
American Board of Surgery among US medical
school graduates. J Am Coll Surg. 2012;214(5):806-
815. doi:10.1016/j.jamcollsurg.2012.01.049
22. Wong RL, Sullivan MC, Yeo HL, Roman SA, Bell
RH Jr, Sosa JA. Race and surgical residency: results
from a national survey of 4339 US general surgery
residents. Ann Surg. 2013;257(4):782-787. doi:10.
1097/SLA.0b013e318269d2d0
23. Pennell C, McCulloch P. The effectiveness of
public simulated oral examinations in preparation
for the American Board of Surgery Certifying
Examination: a systematic review.J Surg Educ.
2015;72(5):1026-1031. doi:10.1016/j.jsurg.2015.03.
018
24. Abelson JS, Symer MM, Yeo HL, et al. Surgical
time out: our counts are still short on racial diversity
in academic surgery.Am J Surg. 2018;215(4):542-548.
doi:10.1016/j.amjsurg.2017.06.028
25. Abelson JS, Wong NZ, Symer M, Eckenrode G,
Watkins A, Yeo HL. Racial and ethnic disparities in
promotion and retention of academic surgeons. Am
J Surg. 2018;216(4):678-682. doi:10.1016/
j.amjsurg.2018.07.020
26. Pritlove C, Juando-Prats C, Ala-Leppilampi K,
Parsons JA. The good, the bad, and the ugly of
implicit bias. Lancet. 2019;393(10171):502-504.
doi:10.1016/S0140-6736(18)32267-0
27. Fry R. Young adult households are earning
more than most older Americans did at the same
age. Pew Research Center. December 11, 2018.
http://www.pewresearch.org/fact-tank/2018/12/11/
young-adult-households-are-earning-more-than-
most-older-americans-did-at-the-same-age/.
Accessed September 6, 2019.
28. American Time Use Survey: 2018 results [news
release]. Washington, DC: US Bureau of Labor
Statistics; June 19, 2019. https://www.bls.gov/
news.release/pdf/atus.pdf. Accessed September 6,
2019.
29. Yeo HL, Abelson JS, Symer MM, et al.
Association of time to attrition in surgical residency
with individual resident and programmatic factors.
JAMA Surg. 2018;153(6):511-517. doi:10.1001/
jamasurg.2017.6202
30. Hoops HE, Brasel KJ, Dewey E, et al. Analysis
of gender-based differences in surgery faculty
compensation, promotion, and retention:
establishing equity.Ann Surg. 2018;268(3):479-487.
doi:10.1097/SLA.0000000000002920
Invited Commentary
Diversity and Inclusion—One Size Does Not Fit All
Keon YoungPark, MD, PhD; Scott R. Chaiet, MD, MBA; Caprice C. Greenberg, MD, MPH
In her groundbreaking 1989 paper, Kimberlé Crenshaw
introduced the term intersectionality and defined the inter-
sectional challenges for women of color as “greater than
the sum of racism and
sexism.”
1(p140)
She proposed
that a single identity, such
as sex, race, or ethnicity,
cannot accurately define the experiences of a group of
people and attempts to do so will particularly marginalize
those who are at the intersection of overlapping identities.
The concept has been expanded to include age, sexual ori-
entation, gender identity, and socioeconomic class, among
other identities.
In surgery, we have begun to have important conversa-
tions about diversity and inclusion; however, these conversa-
tions seem to focus narrowly on the issues and experiences
of particular groups defined by sex or race. This approach
suggests that we can neatly classify individuals into groups
based on observable and quantifiable characteristics; how-
ever, Crenshaw’s work
1
suggests that policies based on this
approach might be an oversimplification with potentially
unintended consequences.
In this issue of JAMA Surgery, Yeo et al
2
examine the as-
sociation between sociodemographic factors and attempts at
and success rates in passing the American Board of Surgery ex-
aminations. Responses from 662 general surgery resident phy-
sicians across the United States, while not particularly sur-
prising, were nevertheless concerning. Hispanic resident
physicians were more likely to forgo attemptsat board exami-
nation, and nonwhite and Hispanic resident physicians were
less likely to pass the examination on the first attempt.
Married resident physicians with children were more
likely to fail at least 1 of the examinations compared with
married resident physicians without children or single resi-
dent physicians, but this analysis did not stratify by sex.
When sex was considered, married female resident physi-
cians with children were less likely to pass the certifying
examination on their first try than other female resident
physicians, but no such difference was observed for male
participants. In fact, an earlier study by Yeo and colleagues
3
Related article page 22
Research Original Investigation Association of Demographic and Program Factors With ABS Qualifying and Certifying Examinations Pass Rates
30 JAMA Surgery January 2020 Volume 155, Number 1 (Reprinted) jamasurgery.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 03/07/2022
