Access to this full-text is provided by Wiley.
Content available from Journal of the American College of Emergency Physicians Open
This content is subject to copyright. Terms and conditions apply.
Received: 9 May 2022 Revised: 7 July 2022 Accepted: 14 July 2022
DOI: 10.1002/emp2.12797
ORIGINAL RESEARCH
Physician Wellness
Reliability and validity support for an abbreviated Copenhagen
burnout inventory using exploratory and confirmatory factor
analysis
Melissa A. Barton MD1Michelle D. Lall MD, MHS2Mary M. Johnston PhD1
Dave W. Lu MD, MSCI3Lewis S. Nelson MD4Karl Y. Bilimoria MD, MS5
Earl J. Reisdorff MD1
1American Board of Emergency Medicine, East
Lansing, Michigan, USA
2Department of Emergency Medicine, Emory
University, Atlanta, Georgia, USA
3Department of Emergency Medicine,
University of Washington, Seattle,
Washington, USA
4Department of Emergency Medicine, Rutgers
New Jersey Medical School, Newark, New
Jersey, USA
5Department of Surgery, Northwestern
University, Chicago, Illinois, USA
Correspondence
Earl J. Reisdorff,American Board of Emergency
Medicine, East Lansing, MI 48823, USA.
Email: ereisdorff@abem.org
Funding and support:ByJACEP Open policy, all
authors are required to disclose any and all
commercial, financial, and other relationships
in any way related to the subject of this article
as per ICMJE conflict of interest guidelines
(see https://www.icmje.org).The authors have
stated that no such relationships exist.
Abstract
Objective: The Copenhagen Burnout Inventory (CBI) is an open-access, valid, and
reliable instrument measuring burnout that includes 19 items distributed across the
following 3 domains (factors): personal burnout, work burnout, and patient burnout.
The primary objective of this study was to determine the validity and reliability of an
abbreviated CBI to assess burnout in emergency medicine residents.
Methods: This cross-sectional study used data from the CBI that followed the 2021
American Board of Emergency Medicine In-training Examination. Exploratory factor
analysis (EFA) was followed by confirmatory factor analysis (CFA).
Results: Of the 8491 eligible residents, 7225 (85.1%) completed the survey; the EFA
cohort included 3613 residents and the CFA cohort included 3612 residents. EFA
showed 2 eigenvalues ≥1, an internal factor and an external factor. There were 6
CBI items that contributed to the 2 factors. The first factor was related to personal
burnout and work-related burnout and the second factor was related to working with
patients. There were 4 CBI items that contributed to the internal factor and 2 CBI items
that contributed to the external factor. Using the abbreviated CBI, the incidence of a
resident having 1 or both types of burnout was 34.1%.
Conclusions: This study provides validity evidence and reliability support for the use
of a 6-item, 2-factor abbreviated CBI. A shorter, reliable, valid, and publicly accessible
burnout inventory provides numerous advantages for burnout research in emergency
medicine.
KEYWORDS
burnout measurement, Copenhagen Burnout Index, reliability, residents, validity
Supervising Editor: Catherine Marco, MD.
This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any
medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
© 2022 The Authors. JACEP Ope n publishedby Wiley Periodicals LLC on behalf of American College of Emergency Physicians.
JACEP Ope n 2022;3:e12797. wileyonlinelibrary.com/journal/emp2 1of10
https://doi.org/10.1002/emp2.12797
2of10 BARTON ET AL
1INTRODUCTION
1.1 Background
Burnout among physicians is associated with numerous negative rami-
fications, including medical error,1,2 poor job satisfaction,3,4 decreased
professional fulfillment,5increased alcohol and drug use,6–8 and
increased depression and suicidal ideation.9The prevalence of burnout
among emergency medicine residents varies widely, depending on the
methods used and definitions of burnout.10 Using criteria applied to
an abbreviated Maslach Burnout Inventory, the prevalence is 28% dur-
ing the first-year of an emergency medicine residency and increases
to more than 40% in the final year of training.11 Another recent
study reported that the overall incidence was 30% (Lu DW, Zhan T,
Bilimoria KY, et al unpublished data, 2021).12 Determining the preva-
lence of burnout is complicated by the varied definitions that are
applied to this enigmatic syndrome,12 as well as the arduousness
and expense in using some burnout inventories. Identifying burnout
requires the application of a valid and reliable measurement instru-
TAB L E 1 Copenhagen Burnout Inventory (CBI)
Personal burnout
1 How often do you feel tired?a
2 How often are you physically exhausted?a
3 How often are you emotionally exhausted?a
4 How often do you think: “I can’t take it anymore”?a
5 How often do you feel worn out?a
6 How often do you feel weak and susceptible to illness?a
Work burnout
7Is your work emotionally exhausting?b
8Do you feel burned out because of your work?b
9Does your work frustrate you?b
10 Do you feel worn out at the end of the working day?a
11 Are you exhausted in the morning at the thought of another day
at work?a
12 Do you feel that every working hour is tiring for you?a
13 Do you have enough energy for family and friends during leisure
time?a,c
Patient burnout
14 Do you find it hard to work with patients?b
15 Do you find it frustrating to work with patients?b
16 Does it drain your energy to work with patients?b
17 Do you feel that you give more than you get back when you work
with patients?b
18 Are you tired of working with patients?a
19 Do you sometimes wonder how long you will be able to continue
working with patients?a
a5-point rating scale: never/almost never, seldom, sometimes, often, always.
b5-point rating scale: to a very low degree, to a low degree, somewhat, to a
high degree, to a very high degree.
cReverse scored.
The Bottom Line
This study found that the Abbreviated Copenhagen Burnout
Inventory, a 6-item, 2-factor abbreviated instrument, is a
reliable, valid, and publicly accessible burnout inventory.
ment. There are a limited number of burnout inventories used to
assess physicians, some of which are proprietary and require the
payment of fees for use. For large studies, these fees can be cost
prohibitive.
The Copenhagen Burnout Inventory (CBI) is an open-access instru-
ment that involves 19 items distributed across the following 3
domains (factors): personal burnout, work burnout, and patient
burnout (Table 1). The CBI has been used to assess varied types
of health care personnel in several countries, amassing substan-
tial validity evidence.13–16 The CBI has been applied extensively to
physicians.17–36 Despite widespread international use in measuring
burnout among physicians, the CBI has been used infrequently to mea-
sure burnout among emergency physicians in the United States.37–38
1.2 Importance
Burnout is a frequently reported problem within emergency
medicine.10–11 Conducting additional research on emergency physi-
cian burnout would provide an opportunity to better characterize
the root causes of burnout and to explore more system-based inter-
ventions that could benefit the specialty. More frequent, longitudinal
assessments would be easier to conduct using a shorter inventory.
Demonstrating validity and reliability evidence related to emergency
medicine would provide greater confidence in the application of the
CBI for emergency physicians. Finally, an open source and abbreviated
CBI would facilitate the ease with which the CBI could be used.
1.3 Goals of this investigation
The primary objective of this study was to determine the validity
and reliability of an abbreviated CBI to assess burnout in emergency
medicine residents using factor analysis.
2METHODS
2.1 Study design and setting
This was a cross-sectional study using data from the optional post-
examination survey on the American Board of Emergency Medicine
(ABEM) In-training Examination (ITE). The post-ITE survey has been
used for more than 20 years and gathers information about the
BARTON ET AL 3of10
examination experience. The 2021 ITE survey also included the 19-
item CBI. The ITE was administered from February 23 to March 5, 2021
to residents in Accreditation Council of Graduate Medical Education
(ACGME)-accredited emergency medicine residency programs.
2.2 Selection of participants
Every resident who completed the ABEM ITE was invited to volun-
tarily complete the post-examination survey. All residents in United
States categorical ACGME-accredited emergency medicine residency
programs were included in the study; physicians in combined training
programs and international programs were excluded. Only the results
from physicians who completed the CBI were included for analysis. This
study was deemed exempt by the Emory Institutional Review Board
(Emory University, Atlanta, GA).
2.3 Interventions
There were no interventions.
2.4 Measurements
This study used the results of the 19-item CBI (Table 1). The CBI
is divided into the following 3 sections: personal burnout (6 items);
work-related burnout (7 items); and client (patient)-related burnout (6
items). The CBI uses a 5-unit Likert scale that varies depending on the
item. There are 2 different scales; one scale is based on frequency of
occurrence and the other is based on the intensity of a feeling. The
scales apply to different items in the inventory (Table 1). All responses
were self-reported. Residents were instructed to answer all questions
based on the academic year at the time of the survey (July 2020 to
March 2021). Survey responses were sent to a secure server at ABEM.
All CBI measurements were deidentified and segregated from the ITE
performance data, as well as the other survey responses.
2.5 Outcomes
The primary outcomes were identification of unique measurement
factors, as well as identification of the specific CBI items that con-
tributed to measuring the identified factors. The final outcome was to
determine rating thresholds for the various factors at which burnout
was likely. Rating thresholds used an anticipated frequency between
30% and 40% because of prior studies using items from the Maslach
Burnout Inventory and nearly identical resident cohorts (Lu DW, Zhan
T, Bilimoria KY, et al unpublished data, 2021).11 There were sev-
eral intermediary results needed to determine the primary outcomes.
These intermediary outcomes included interitem correlations, deter-
minants of factorability,and several data points needed to derive a final
inventory.
2.6 Analyses
The survey responses were randomized into data sets of nearly identi-
cal sizes. An ABEM staff psychometrician performed the analyses.
Exploratory factor analysis (EFA) was performed on the first data
set, as the investigators made no a priori assumptions about the
existence or number of factors but did assume that if multiple fac-
tors existed, those factors would be related. Before EFA, the data’s
amenability to factor analysis was examined using the Kaiser-Meyer-
Olkin (KMO) Measure of Sampling Accuracy. KMO values >0.60
are considered amenable to factoring. In addition, Bartlett’s Test
of Sphericity was performed. Bartlett’s Test of Sphericity detects
redundancy between the variables. A significant Bartlett’s value also
indicates that observed data can be factored.
EFA was performed using direct oblimin rotation. Direct oblimin
rotation was used because the various dimensions of burnout were
assumed to be related based on prior research.39 Multiple meth-
ods were used to determine the appropriate number of factors to
extract, including a scree plot of the eigenvalues, Horn’s parallel anal-
ysis, and Velicer’s minimum average partial (MAP) procedure.40,41 A
scree plot of the eigenvalues is useful in visually determining the num-
ber of factors to retain. Typically, factors with eigenvalues of ≥1are
retained. Horn’s parallel analysis also determines the number of fac-
tors to retain in EFA. Briefly, this method compares eigenvalues from
the observed data to the 50th and 95th percentile eigenvalues from
an empirical sampling distribution that is randomly generated from
matrices with the same structure as the observed data. In short, the
number of observed eigenvalues greater than the average or 95th per-
centile of simulated eigenvalues indicates how many factors should be
extracted. Velicer’s MAP also determines the number of components
to be retained in EFA, focusing on the magnitude of variance within a
correlation matrix.
Once the factors were identified for extraction, as well as the CBI
items that contributed to any of the identified factors, the second
response cohort underwent confirmatory factor analysis (CFA). The
CFA used the Satorra–Bentler (SB) robust scaling method. Model fit
used the following 4 distinct methods: (1) SB-scaled chi-square (χ2SB);
(2) standardized root mean square residual (SRMR); (3) SB-scaled root
mean square error of approximation (RMSEA); and (4) the SB-scaled
comparative fit index (CFISB). These indices are used to determine
whether the derived model fits the data. The following criteria were
used to assess model fit: ≤0.08 for SRMR, ≤0.08 for RMSEA, and ≥0.90
for CFI.42,43 After the CFA, the factors were evaluated to ensure that
they accounted for a sufficient amount of the variance in the responses
(≥10% of the variance within the abbreviated CBI). Reliability for the
EFA and CFA initial cohorts used Cronbach’s alpha. Reliability for the
inventory resulting from CFA was calculated as coefficient omega.
The rating thresholds for any identified factors that defined burnout
were reviewed; the goals were similar frequencies for various factors
and a total measured frequency of burnout similar to prior levels. SAS
9.4 was the primary software platform for descriptive analysis (SAS
Institute Inc., 2021). Mplus 8.6 was used to estimate all factor analysis
models (Muthén & Muthén, 1998–2021).
4of10 BARTON ET AL
FIGURE 1 Eigenvalue screen plot
3RESULTS
3.1 Characteristics of study subjects
The 2021 ABEM ITE was administered to 8863 residents, of whom
8491 were residents in categorical US ACGME-accredited emer-
gency medicine residencies. There were 7225 emergency medicine
residents who completed the survey, for a response rate of 85.1%;
1266 residents (14.9%) did not complete the survey. The EFA
cohort included 3613 residents and the CFA cohort included 3612
residents.
3.2 Main results
For internal reliability, Cronbach’s alpha for the original 19-item CBI
was 0.94 for both samples. The KMO Measure of Sampling Accuracy
was 0.96, which was well above the threshold for factorability (0.60).
Bartlett’s Test of Sphericity yielded a χ2=54,649 (171 df;P<0.001),
further confirming factorability. Interitem correlations and descrip-
tive statistics were calculated for all items in the CBI (Table 2). Using
this matrix of correlation coefficients, EFA was performed and eigen-
values were calculated. Of the 19 resulting eigenvalues, only 2 were
above the 1.0 threshold for inclusion in an abbreviated CBI model
(Table 3). Horn’s parallel analysis (Table 3) also found that only 2 fac-
tors had eigenvalues that exceeded the parallel analysis results at both
the 50th and 95th percentiles (10.33 and 2.08). A scree plot visually
confirmed the identification of the 2 factors above the 1.0 level thresh-
old (Figure 1). Velicer’s MAP also indicated that 2 factors should be
extracted as the smallest average squared partial correlation (0.016)
occurred with the second factor.
Two factors were extracted and rotated to a final solution using
direct oblimin rotation. Because the factors were assumed to be
moderately related, the delta parameter was fixed equal to 0. Items
with pattern coefficients ≥0.40 were considered salient and retained.
There were 6 CBI items that had pattern coefficients that were ≥0.40
(Table 4).
The resulting model included 2 factors composed of 6 items. The fac-
tors were named to best characterize the items that contributed to the
factors. The first factor (the “internal factor”) was related to feelings of
burnout that were personal and work-related; the second factor (the
“externalfactor”) was related to working with patients. Specifically, CBI
items 1, 2, 8, and 10 loaded onto the internal factor and items 14 and 15
loaded onto the external factor.
The pattern coefficients (Table 4) reflect the partial correlation
between an item and the factor, controlling for all other factors,
whereas a structure coefficient reflects the item’s zero-order corre-
lation with the factor. For example, the pattern coefficient for item 1
and the internal factor is 0.88 and the structure coefficient is 0.84.
Therefore, the correlation between item 1 and the internal factor is
0.88, controlling for all other factors, whereas the zero-order correla-
tion between item 1 and the internal factor is 0.84. As hypothesized,
the 2 resultant factors from the EFA were moderately correlated with
one another (r=0.50). Before rotation, the internal factor accounted
for 60.9% of the common variance and the external factor accounted
for 19.0% of the common variance, combining to account for 79.8% of
the total variation that was obtained by all 19 items.
After examining the EFA results, a CFA model was estimated using
the second cohort sample. The results of the 6-item, 2-factor CFA
model revealed the 2-factor model fit the data: χ2SB (8 df)=557.77
(P<0.01), SRMR =0.047, RMSEA =0.138, and CFISB =0.95.
With the exception of RMSEA, the fit indices displayed good fit,
BARTON ET AL 5of10
TAB L E 2 Inter-item correlations: descriptive statistics and interitem correlations of the 19-item Copenhagen Burnout Inventory, for exploratory factor analysis sample (n =3613) and
confirmatory factor analysis sample (n =3612)
12345678910111213141516171819MeanSD
1 — 0.72 0.66 0.48 0.69 0.41 0.57 0.59 0.48 0.64 0.58 0.50 −0.29 0.33 0.33 0.35 0.32 0.32 0.32 3.54 0.81
20.72 —0.71 0.55 0.68 0.52 0.57 0.62 0.50 0.62 0.59 0.55 −0.35 0.37 0.35 0.38 0.31 0.36 0.35 3.07 0.89
3 0.66 0.71 — 0.65 0.73 0.49 0.66 0.69 0.57 0.60 0.62 0.58 −0.36 0.42 0.40 0.43 0.37 0.41 0.40 3.07 0.94
40.47 0.54 0.64 —0.61 0.54 0.53 0.63 0.57 0.48 0.58 0.61 −0.39 0.45 0.43 0.45 0.35 0.47 0.48 1.94 0.97
5 0.69 0.67 0.72 0.60 — 0.53 0.62 0.69 0.58 0.66 0.64 0.59 −0.37 0.42 0.42 0.44 0.37 0.43 0.42 3.06 0.97
60.42 0.50 0.52 0.55 0.54 —0.43 0.50 0.45 0.41 0.47 0.50 −0.33 0.35 0.33 0.37 0.28 0.37 0.38 1.97 0.93
7 0.57 0.57 0.67 0.53 0.62 0.44 — 0.73 0.66 0.67 0.61 0.58 −0.29 0.48 0.48 0.50 0.43 0.47 0.46 3.15 0.98
80.60 0.61 0.69 0.65 0.70 0.50 0.73 —0.75 0.66 0.71 0.68 −0.41 0.54 0.51 0.54 0.44 0.55 0.54 2.77 1.07
9 0.50 0.50 0.60 0.58 0.60 0.45 0.66 0.76 — 0.58 0.63 0.65 −0.32 0.58 0.58 0.58 0.47 0.59 0.56 2.62 1.03
10 0.62 0.61 0.60 0.47 0.65 0.41 0.65 0.64 0.57 —0.66 0.60 −0.30 0.43 0.43 0.46 0.39 0.44 0.41 3.44 0.92
11 0.58 0.58 0.60 0.58 0.65 0.48 0.61 0.70 0.64 0.66 — 0.73 −0.36 0.49 0.48 0.51 0.40 0.51 0.49 2.83 1.04
12 0.49 0.54 0.57 0.61 0.59 0.50 0.59 0.68 0.66 0.60 0.75 —−0.39 0.54 0.52 0.55 0.41 0.57 0.53 2.24 1.00
13 −0.32 −0.34 −0.37 −0.37 −0.38 −0.33 −0.29 −0.39 −0.31 −0.28 −0.35 −0.38 — −0.25 −0.22 −0.26 −0.20 −0.28 −0.28 3.49 0.91
14 0.33 0.36 0.42 0.46 0.42 0.37 0.48 0.53 0.58 0.43 0.51 0.57 −0.23 —0.83 0.81 0.57 0.73 0.63 2.04 0.85
15 0.33 0.34 0.41 0.44 0.43 0.34 0.47 0.52 0.60 0.44 0.50 0.54 −0.22 0.83 — 0.82 0.61 0.73 0.63 2.14 0.88
16 0.36 0.38 0.46 0.46 0.47 0.38 0.51 0.55 0.59 0.46 0.53 0.58 −0.25 0.78 0.81 —0.61 0.75 0.64 2.13 0.92
17 0.33 0.30 0.37 0.32 0.40 0.29 0.43 0.45 0.49 0.41 0.42 0.42 −0.20 0.57 0.61 0.62 — 0.56 0.51 2.64 1.10
18 0.33 0.34 0.41 0.46 0.44 0.36 0.45 0.55 0.59 0.42 0.52 0.58 −0.23 0.73 0.74 0.75 0.58 —0.77 1.92 0.90
19 0.34 0.36 0.42 0.48 0.44 0.38 0.47 0.55 0.58 0.43 0.52 0.58 −0.25 0.63 0.64 0.65 0.53 0.76 — 2.00 1.03
Mean 3.56 3.09 3.06 1.97 3.07 2.00 3.16 2.79 2.65 3.44 2.82 2.25 3.50 2.04 2.13 2.13 2.65 1.95 2.01
SD 0.83 0.89 0.95 0.99 1.00 0.95 0.97 1.09 1.04 0.91 1.06 1.00 0.90 0.86 0.89 0.91 1.10 0.92 1.04
Note: Descriptive statistics for exploratory factor analysis (EFA) sample are presented at the bottom of the table; interitem correlations are on the lower half of the diagonal. Descriptive statistics for confirmatory
factor analysis (CFA) samples are presented on the rightside of the t able;interitem correlations are on the upper half of the diagonal. EFA sample, n =3612; CFA sample, n =3612.
6of10 BARTON ET AL
TAB L E 3 Unrotated eigenvalues and Horn’s parallel analysis
results
Parallel analysis results
No. of
factors Eigenvalue
50th
percentile
95th
percentile
1 10.33 1.13 1.15
22.08 1.10 1.12
3 0.89 1.09 1.10
40.69 1.07 1.08
5 0.65 1.06 1.07
60.52 1.05 1.06
7 0.50 1.03 1.04
80.47 1.02 1.03
9 0.40 1.01 1.02
10 0.35 1.00 1.01
11 0.33 0.99 0.99
12 0.29 0.98 0.98
13 0.26 0.96 0.97
14 0.23 0.95 0.96
15 0.23 0.94 0.95
16 0.22 0.93 0.94
17 0.20 0.92 0.93
18 0.19 0.90 0.91
19 0.16 0.88 0.89
TAB L E 4 Pattern (structure) coefficients for the 6-item, 2-factor
exploratory factor analysis solution
Item Factor 1 internal factor Factor 2 external factor
10.88 (0.84) −0.08 (0.36)
20.86 (0.84) −0.05 (0.38)
80.61 (0.75) 0.27 (0.38)
10 0.68 (0.75) 0.14 (0.48)
14 0.01 (0.46) 0.90 (0.91)
15 −0.00 (0.46) 0.91 (0.91)
Note: Bolded coefficients reflect salient loading. n =3613.
indicating that the 2-factor model reproduced the observed rela-
tionships well. For CFA, the unstandardized pattern coefficients are
interpreted as unstandardized regression coefficients, whereas the
standardized pattern coefficients are interpreted as standardized
regression coefficients (Table 5). Accordingly, the standardized pattern
coefficients can be squared to yield an R2value, which indicates the
proportion of an item’s variance explained by the factor. Both factors
accounted for at least 60% of the variance relative to the CBI items
in their factor group. For example, the internal factor accounted for
65.1% of the variance in item 1, whereas the external factor accounted
for 84.7% of the variance in item 14. Overall, the internal factor
accounted for 63.9% of the total variance within its items, and the
external factor accounted for 83.0% of the total variance within its
items. The 2 factors from the CFA were positively correlated with
each other (r=0.56). The magnitude of the correlation indicates that
although the factors are related, they maintain a degree of distinction.
The reliability of the 2 factors using coefficient omega for the unstan-
dardized parameter estimates was 0.88 for the internal factor and 0.91
for the external factor.
To determine quantitative thresholds for burnout, the 2-factor
model was compared to a large study of emergency medicine res-
idents using the Maslach Burnout Inventory.11 Given the 2-factor
model, each factor had to be regarded as an independent indicator of
burnout. Therefore, each factor required an individual threshold. For
the internal factor, there were 4 CBI items using a rating range of 1
(never/almost never, to a very low degree) to 5 (always, to a very high
degree), which proved a possible score range of 4–20. The mean rating
was 12.8 (SD 3.2). At a threshold of 16 or higher, the incidence of inter-
nally caused burnout was 19.7% (Table 6). For the external factor,there
were 2 CBI items using the same Likert scoring with a possible range
of 2–10. The mean rating was 4.2 (SD 1.7). At a threshold of 6 or higher,
the incidence of externally caused burnout was 24.4%. Given that some
residents had both types of burnout (10.1% had both), the overall inci-
dence of a resident having 1 or both types of burnout was 34.1%, which
is very similar to prior reports using validated burnout inventories for
emergency medicine residents.11 In addition, the incidence of burnout
increased as training progressed. Emergency medicine first-year res-
idents had a 29.3% incidence that increased to 37.5% for emergency
medicine third- and fourth-year residents (Table 6)
Given the identification of 2 unique factors (internal and exter-
nal), the CBI should be used in such a way as to identify the type of
burnout a resident might have (eg, internal, external, both). By iden-
tifying burnout type in this way, greater investigation into cause and
treatment can occur.
For comparison, the personal, work, and patient burnout ratings
based on the 19-item CBI were calculated for comparison. The inci-
dence of personal, work, and patient burnout was 19.5%, 7.3%, and
18.1%, respectively. The overall incidence of a resident having at least
1 of the 3 original types of burnout was 30.2%.
4 LIMITATIONS
As with most surveys, the results were self-reported and surveys such
as this can be prone to social desirability bias. However, such bias is
unlikely to affect the interitem correlations that created the 6-item
model. Still, this bias could have contributed to a lower rate of burnout.
The survey was administered when many emergency medicine res-
idents were likely under considerable stress caring for patients during
the COVID-19 pandemic. This situational stress could have increased
burnout more than in other times pre-pandemic. However, the pur-
pose of this study was not to determine the prevalence of burnout
but rather to determine the interitem correlations and potential item
redundancy that could be used to create an abbreviated inventory.
Nonetheless, the findings are consistent with other burnout findings
BARTON ET AL 7of10
TAB L E 5 Unstandardized (standardized) parameter estimates for 6-item, 2-factor confirmatory factor analysis solution
Factor Inventory items Pattern coefficients Error variance R2
Factor 1 (internal factor) 1 0.66 (0.81) 0.23 (0.35) 0.65
2 0.72 (0.81) 0.27 (0.34) 0.66
8 0.85 (0.79) 0.43 (0.38) 0.62
10 0.74 (0.80) 0.31 (0.37) 0.63
Factor 2 (external factor) 14 0.78 (0.92) 0.11 (0.15) 0.85
15 0.80 (0.90) 0.14 (0.19) 0.82
Note: All unstandardized parameter estimates were statistically significant (P<0.01). n =3612.
TAB L E 6 Burnout incidence measured by abbreviated Copenhagen Burnout Inventory
Emergency
medicine level
No burnout,
n
Internal burnout
only, n
External burnout
only, n
Both burnout
types, n
%Any
burnout
1(n=1216) 860 134 134 88 29.3
2(n=1128) 730 104 178 116 35.3
3/4 (n =1205) 792 112 205 159 37.5
Tota l ( n =3612) 2382 350 517 363 34.1
conducted during the COVID-19 pandemic (Lu DW, Zhan T, Bilimoria
KY, et al unpublished data, 2021).
The survey was administered after the ITE. A resident’s self-
perception of performance could affect their survey responses. Specif-
ically, if a resident found the ITE to be difficult, they could have
greater feelings of burnout. In prior ABEM ITE surveys, no nega-
tive bias was detected. Moreover, the Dunning-Kruger effect in other
fields suggests that test-takers tend to overrate their relative test
performances.
The model is not statistically perfect. For the CFA, the RMSEA did
not demonstrate excellent fit, despite the other statistical analyses
demonstrating good fit. The RMSEA was 0.14, when ideally it should
have been <0.08. The RMSEA is a measure of absolute fit and is sen-
sitive to misspecified factor loadings. The likely source of this result is
that item 8, which loaded on the internal factor, also partially loaded
onto the external factor, albeit somewhat weakly. The other items (1,
2, 10, 14, and 15) tended to have large pattern and structure coeffi-
cients for a single factor, which were accompanied by small pattern
and structure coefficients for the other factor. Nonetheless, given the
strength of the other tests for fit, the RMSEA did not, by itself, negate
the 2-factor model.
The abbreviated CBI might slightly overestimate the incidence of
burnout. The 19-item CBI found an overall incidence of burnout to
be 30.2%, whereas the abbreviated CBI calculated an incidence of
34.1%. Of note, the abbreviated CBI more closely approximately prior
estimates and estimates using the Maslach Burnout Inventory.
This model did not attempt to independently establish the inci-
dence of burnout de novo for the survey respondents. Rather, the
2-factor model was compared with prior studies that used a similar
survey methodology and similar cohort (Lu DW, Zhan T, Bilimoria KY,
et al unpublished data, 2021).11 The proposed scoring rubric requires
prospective application to determine reliability and provide additional
validity evidence.
The abbreviated CBI should not be assumed to generalize to emer-
gency physicians who have been in practice for a substantial length of
time. Likewise, the abbreviated CBI should not be assumed to general-
ize to other specialties. Although there is substantial validity evidence
for using the CBI in other medical specialties, using the abbreviated CBI
requires further validity investigation. The abbreviated model requires
further prospective factor analysis using the abbreviated CBI. This
analysis of the ABEM 2022 ITE post-examination survey is planned.
5DISCUSSION
This study is the first to use factor analysis to assess the psychomet-
ric properties of the CBI in emergency medicine residents. In addition,
this is the first study designed to create an abbreviated CBI. The
results of this study are important in that they demonstrate substan-
tial reliability and validity evidence to support the ongoing use of an
abbreviated form of the CBI, as well as use of the CBI for emergency
physicians. This study successfully identified a 2-factor, 6-item inven-
tory that can assess burnout risk in emergency medicine residents
(Table 7). Two findings provide substantial validity evidence for the
derived model. First, the overall frequency of 34% is similar to stud-
ies using items from the Maslach Burnout Inventory involving residents
taking the ABEM ITE (Lu DW, Zhan T, Bilimoria KY, et al unpublished
data, 2021).11 Second, the prevalence of burnout increased as resi-
dents progressed through training, consistent with prior studies (Lu
DW, Zhan T, Bilimoria KY, et al unpublished data, 2021).11
8of10 BARTON ET AL
TAB L E 7 Abbreviated Copenhagen Burnout Inventory (CBI) items
1. How often do you feel tired?a
2. How often are you physically exhausted?a
8. Do you feel burned out because of your work?b
10. Do you feel worn out at the end of the working day?a
14. Do you find it hard to work with patients?b
15. Do you find it frustrating to work with patients?b
Note: Internal factor is determined from items 1, 2, 8, and 10. External factor
is determined from items 14 and 15.
a5-point rating scale: never/almost never, seldom, sometimes, often, always.
b5-point rating scale: to a very low degree, to a low degree, somewhat, to a
high degree, to a very high degree.
Although other investigators have applied EFA and CFA to define
the psychometric properties of the CBI, those efforts were not directed
toward identifying the essential items within the inventory that could
be used for an abbreviated format. Prior work tended to use fac-
tor analysis to provide validity evidence for the CBI construct of the
following 3 assessment categories of burnout: personal, work, and
patient. For example, Todorovic et al used a Serbian version of the
CBI to determine whether it could confidently assess burnout among
Serbian medical students.44 The study applied EFA to the CBI and
confirmed the presence of the 3 aforementioned subcategories that
demonstrated a high degree of correlation.
Javanshir et al also evaluated the psychometric properties of the
CBI to gather validity support for the use of an Iranian version of the
instrument in assessing a diverse group of workers, including health
care staff.45 The results from EFA and CFA similarly provided support
for construct validity for the 3-factor CBI construct. Internal reliability
and test-retest reliability were also high.
A study of pharmacists used a slightly shorter inventory by eliminat-
ing 2 items, but that modification was not based on factor analysis.46
This shortening was a pre hoc decision based on item validity. CFA
of the amended CBI still provided reliability and validity evidence for
the CBI. Although CFA supported the use of this 17-item inventory, no
further reduction in inventory items was attempted.
Not all studies supported a 3-factor model for the CBI. In a study of
Iranian nurses, EFA identified 4 factors that were supported by subse-
quent CFA.47 Of note, our study found 2 factors (internal and external).
A Brazilian-Portuguese version of the CBI used to assess Brazilian
health care workers was also found to have a 2-factor pattern48 that
was similar to our study. Specifically, that study found that 1 factor
was a combination of items from personal burnout and work-related
burnout; the other factor included items from patient burnout. How-
ever, that study did not aim to shorten the inventory; rather, its
primary purpose was to validate the Brazilian-Portuguese version of
the CBI.
Physician burnout is a major area of emphasis within emergency
medicine. The Quadruple Aim approach is not only patient-centric
but addresses improved physician experience as well. The notion is
that without improving the clinical experience of the physician, it will
be more difficult to improve the patient care experience, improve
the health of a population, and reduce per capita health care costs.
One key to improving the physician experience is reducing burnout.
A seminal step to reducing burnout is to measure it reliably and
accurately. Although there are available burnout inventories, many
have limitations. For example, the Maslach Burnout Inventory and the
Mayo Well-Being Index have substantial costs for use. The Stanford
Professional Fulfillment Index is relatively new and has 16 items.
The advantages of the CBI are its widespread use geographically
and among varied health care professionals. Given the complexity of
burnout, identifying the type of burnout (internal vs external) likely
has an advantage for creating solutions. Another practical advantage
is that the CBI is free. This study provides validity and reliability evi-
dence for the use of the abbreviated format that provides greater ease
of use. Finally, the ease of use and the open-access of the abbreviated
CBI make the inventory a viable instrument for program directors to
monitor burnout among residents.
Our study provides both construct validity evidence and reliability
support for the use of a 6-item, 2-factor abbreviated CBI. A shorter,
reliable, valid, and publicly accessible burnout inventory provides
numerous advantages for burnout research in emergency medicine.
An additional prospective study using CFA is underway to provide
additional validity evidence for the abbreviated CBI.
ACKNOWLEDGMENT
The authors wish to acknowledge Ms Frances Spring for her assistance
with the preparation and submission of this manuscript. There is no
direct funding for this study.
CONFLICTS OF INTEREST
Melissa A. Barton, Mary M. Johnston, and Earl J. Reisdorff are
employed by the American Board of Emergency Medicine. Lewis S. Nel-
son is a former member of the Board of Directors of the American
Board of Emergency Medicine.
AUTHOR CONTRIBUTIONS
Melissa A. Barton: Manuscript review. Michelle D. Lall: Manuscript
review, institutional review board review. Mary M. Johnston: Study
design, data analysis, supervised the conduct of the trial and data
collection, managed the data, including quality control, provided sta-
tistical advice on study design, and analyzed the data. Dave W. Lu:
Manuscript review. Lewis S. Nelson: Manuscript review. Karl Y. Bil-
imoria: Manuscript review. Earl J. Reisdorff: Conceived of the study,
manuscript draft, manuscript review,institutional review board review,
and takes responsibility for the paper as a whole.
ORCID
Earl J. Reisdorff MD https://orcid.org/0000-0003-3553-446X
REFERENCES
1. Menon NK, Shanafelt TD, Sinsky CA, et al. Association of physician
burnout with suicidal ideation and medical errors. JAMA Netw O pen.
2020;3:e2028780.
2. Al-Ghunaim TA, Johnson J, Biyani CS, Alshahrani KM, Dunning
A, O’Connor DB. Surgeon burnout, impact on patient safety and
BARTON ET AL 9of10
professionalism: a systematic review and meta-analysis. Am J Surg.
2022;223(1 Pt A):228-238.
3. West CP, Dyrbye LN, Satele DV, Shanafelt TD. Colleagues meeting
to promote and sustain satisfaction (COMPASS) groups for physi-
cian well-being: a randomized clinical trial randomized controlled trial.
Mayo Clin Proc. 2021;96:2606-2614.
4. Nguyen J, Liu A, McKenney M, Liu H, Ang D, Elkbuli A. Impacts
and challenges of the COVID-19 pandemic on emergency medicine
physicians in the United States. Am J Emerg Med. 2021;48:38-47.
5. Lazarides AL, Belay ES, Anastasio AT, Cook CE, Anakwenze OA.Physi-
cian burnout and professional satisfaction in orthopedic surgeons
during the COVID-19 pandemic. Work . 2021;69(1):15-22.
6. Rath KS, Huffman LB, Phillips GS, Carpenter KM, Fowler JM. Burnout
and associated factors among members of the society of gynecologic
oncology. Am J Obstet Gynecol. 2015;213(6):e1-e9.
7. Oreskovich MR, Shanafelt T, Dyrbye LN, et al. The prevalence
of substance use disorders in American physicians. Am J Addict.
2015;24(1):30-38.
8. Shanafelt TD, Balch CM, Dyrbye L, et al. Special report: suicidal
ideation among American surgeons. Arch Surg. 2011;146(1):54-62.
9. Oreskovich MR, Kaups KL, Balch CM, et al. Prevalence of alcohol use
disorders among American surgeons. Arch Surg. 2012;147(2):168-174.
10. Lin M, Battaglioli N, Melamed M, Mott SE, Chung AS, Robinson DW.
High prevalence of burnout among US emergency medicine residents:
results from the 2017 National Emergency Medicine Wellness Survey.
Ann Emerg Med. 2019;74(5):682-690.
11. Vanyo LZ, Goyal DG, Dhaliwal RS, et al. Emergency medicine
resident burnout and examination performance. AEM Educ Train.
2020;5(3):e10527.
12. Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of burnout among
physicians: a systematic review. JAMA. 2018;320(11):1131-1150.
13. Agbobli YA, Konu YR, Gbeasor-Komlanvi FA, et al. Prevalence and fac-
tors associated with burnout among healthcare workers during the
COVID-19 pandemic in Togo, June 2021 [published online ahead of
print February 22, 2022]. Arch Environ Occup Health. 22;1-10. doi:10.
1080/19338244.2022.2042172
14. Alameddine M, Bou-Karroum K, Hijazi MA. A national study on the
resilience of community pharmacists in Lebanon: a cross-sectional
survey.J Pharm Policy Pract. 2022;15(1):8.
15. Arora S, Knight A. Questionnaire survey of burnout amongst dentists
in Singapore. Int Dent J. 2022;72(2):161-168.
16. Creedy DK, Sidebotham M, Gamble J, Pallant J, Fenwick J. Prevalence
of burnout, depression, anxiety and stress in Australian midwives:
a cross-sectional survey. BMC Pregnancy Childbirth. 2017;17(1):13.
doi:10.1186/s12884-016-1212-5
17. Kristensen TB, Hallas MK, Høgsted R, Groenvold M, Sjøgren P, Marsaa
K. Burnout in physicians: a survey of the Danish society for palliative
medicine [published online ahead of print June 29, 2021]. BMJ Sup-
port Palliat Care. Online ahead of print. doi:10.1136/bmjspcare-2021-
003237
18. Cheli S, Zagonel V, Oliani C, Blasi L, Fioretto L. Is burnout affecting life
satisfaction in oncologists? The moderating role of family concerns in
an Italian sample. Psychooncology. 2021;30(3):385-391.
19. Degraeve A, Lejeune S, Muilwijk T, et al. When residents work less,
they feel better: lessons learned from an unprecedent context of
lockdown. Prog Urol. 2020;30(16):1060-1066.
20. El Hachem C, Atallah E. Burnout prevalence and associated factors in a
sample of Lebanese residents [published online ahead of print Decem-
ber 3, 2021]. Encephale. Online ahead of print. doi:10.1016/j.encep.
2021.08.013
21. Hussain M, Amjad MB, Ahsan J, Minhas SO. Implementation of
National Institute of Health Guidelines and other factors contributing
to work-related burnout in Covid isolation ward and ICU physicians. J
Ayub Med Coll Abbottabad. 2021;33(2):283-288.
22. Jacobsen FM, Jensen CFS, Schmidt MLK, et al. Burnout among urolo-
gists from Denmark and Michigan. Urology. 2021;147:68-73.
23. Jaulin F, Nguyen DP, Marty F, et al. Perceived stress, anxiety
and depressive symptoms among anaesthesia and intensive care
residents: a French national survey. Anaesth Crit Care Pain Med.
2021;40(3):100830.
24. Kurzthaler I, Kemmler G, Holzner B, Hofer A. Physician’s burnout
and the COVID-19 pandemic-a nationwide cross-sectional study in
Austria. Front Psychiatry. 2021;12:784131.
25. Kwan KYH, Chan LWY, Cheng PW, Leung GKK, Lau CS. Burnout
and well-being in young doctors in Hong Kong: a territory-wide
cross-sectional survey.Hong Kong Med J.2021;27(5):330-337.
26. Lapa T, Carvalho S, Viana J, Ferreira PL, Pinto-Gouveia J, Cabete
AB. Development and evaluation of a global burnout index derived
from the use of the Copenhagen Burnout Inventory in Portuguese
physicians. Acta Med Port. 2018;31(10):534-541.
27. Møller CM, Clausen T, Aust B, Eiberg JP. A cross-sectional national
study of burnout and psychosocial work environment in vascular
surgery in Denmark. JVascSurg. 2022;75(5):1750-1759.
28. Ng APP, Chin WY, Wan EYF, Chen J, Lau CS. Prevalenceand severity of
burnout in Hong Kong doctors up to 20 years post-graduation: a cross-
sectional study. BMJ Open. 2020;10(10):e040178.
29. Ö˘
gütlü H, McNicholas F, Türkçapar H. Stress and burnout in psy-
chiatrists in Turkey during COVID-19 pandemic. Psychiatr Danub.
2021;33(2):225-230.
30. Papaefstathiou E, Tsounis A, Malliarou M, Sarafis P. Translation and
validation of the Copenhagen Burnout Inventory amongst Greek
doctors. Health Psychol Res. 2019;7(1):7678.
31. Pooja V, Khan A, Patil J, Chaudhari B, Chaudhury S, Saldanha D.
Burnout and resilience in doctors in clinical and preclinical depart-
ments in a tertiary care teaching and dedicated COVID-19 hospital. Ind
Psychiatry J. 2021;30(Suppl 1):S69-S74.
32. Nuss P, Tessier C, Masson M, et al. Factors associated with a higher
score of burnout in a population of 860 French psychiatrists. Front
Psychiatry. 2020;11:371.
33. Salloum NL, Copley PC, Mancuso-Marcello M, Emelifeonwu J,
Kaliaperumal C. Burnout amongst neurosurgical trainees in the UK
and Ireland. Acta Neurochir (Wien). 2021;163(9):2383-2389.
34. St Onge JE, Allespach H, Diaz Y, et al. Burnout: exploring the differ-
ences between U.S. and international medical graduates. BMC Med
Educ. 2022;22(1):69.
35. Thrush CR, Guise JB, Gathright MM, et al. A one-year institutional view
of resident physician burnout. Acad Psychiatry. 2019;43(4):361-368.
36. Werdecker L, Esch T. Burnout, satisfaction and happiness among Ger-
man general practitioners (GPs): a cross-sectional survey on health
resources and stressors. PLoS One. 2021;16(6):e0253447.
37. Byrd J, Knowles H, Moore S, et al. Synergistic effects of emergency
physician empathy and burnout on patient satisfaction: a prospective
observational study. Emerg Med J. 2021;38(4):290-296.
38. Wolfshohl JA, Bradley K, Bell C, et al. Association between empathy
and burnout among emergency medicine physicians. J Clin Med Res.
2019;11(7):532-538.
39. Rocha FLR, de Jesus LC, Marziale MHP, Henriques SH, Marôco J,
Campos JADB. Burnout syndrome in university professors and aca-
demic staff members: psychometric properties of the Copenhagen
Burnout Inventory-Brazilian version. Psicol Reflex Crit. 2020;33(1):11.
doi:10.1186/s41155-020-00151-y
40. Velicer WF, Eaton CA, Fava JL. Construct explication through factor
or component analysis: a review and evaluation of alternative proce-
dures for determining the number of factors or components. In: Goffin
RD, Helmes E, eds. Problems and Solutions in Human Assessment. Kluwer;
2000:41-71.
41. Velicer WF. Determining the number of components from the matrix
of partial correlations. Psychometrika. 1976;31:321-327.
10 of 10 BARTON ET AL
42. Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociol
Methods Res. 1993;21:230-258.
43. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance struc-
ture analysis: conventional criteria versus new alternatives. Struct Equ
Modeling. 1999;6(1):1-55.
44. Todorovic J, Terzic-Supic Z, Divjak J, et al. Validation of the study
burnout inventory and the Copenhagen burnout inventory for
the use among medical students. Int J Occup Med Environ Health.
2021;34(6):737-745.
45. Javanshir E, Dianat I, Asghari-Jafarabadi M. Psychometric properties
of the Iranian version of the Copenhagen burnout inventory. Health
Promot Perspect. 2019;9(2):137-142.
46. Fadare OO, Andreski M, Witry MJ. Validation of the Copenhagen
Burnout Inventory in pharmacists. Innov Pharm. 2021;12(2). 10.
24926/iip.v12i2.3699
47. Mahmoudi S, Atashzadeh-Shoorideh F, Rassouli M, Moslemi A,
Pishgooie AH, Azimi H. Translation and psychometric properties of the
Copenhagen burnout inventory in Iranian nurses. Iran J Nurs Midwifery
Res. 2017;22:117-122.
48. Moser CM, Tietbohl-Santos B, Arenas DL, et al. Psychometric proper-
ties of the Brazilian Portuguese version of the Copenhagen Burnout
Inventory (CBI) in healthcare professionals [published online ahead of
print October 27, 2021]. Trends Psychiatry Psychother. Online ahead of
print. doi:10.47626/2237-6089-2021-0362
How to cite this article: Barton MA, Lall MD, Johnston MM,
et al. Reliability and validity support for an abbreviated
Copenhagen burnout inventory using exploratory and
confirmatory factor analysis. JACEP Open. 2022;3:e12797.
https://doi.org/10.1002/emp2.12797
Available via license: CC BY-NC-ND
Content may be subject to copyright.