Validity evidence for a French version of the Stigma of Occupational Stress Scale for Doctors (SOSS-D)

Full text

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
L’Encéphale
xxx
(xxxx)
xxx–xxx
Disponible
en
ligne
sur
ScienceDirect
www.sciencedirect.com
Research
article
Validity
evidence
for
a
French
version
of
the
Stigma
of
Occupational
Stress
Scale
for
Doctors
(SOSS-D)
Preuve
de
validité
d’une
version
franc¸
aise
de
l’échelle
«
Stigma
of
Occupational
Stress
Scale
for
Doctors
(SOSS-D)
»
Nadia
M.
Bajwaa,∗,
Sohie
Favrea,
Thomas
Pernegera,
Melissa
Dominicé
Daoa,
Marie-Claude
Audetatb,
Mathieu
R.
Nendazb,
Noëlle
Junod
Perrona,
Hélène
Richard-Lepouriela
aDépartement
de
l’enfant
et
de
l’adolescent,
Geneva
University
Hospital,
rue
Willy-Donzé,
6,
1211
Geneva,
Switzerland
bFaculty
of
Medicine,
University
of
Geneva,
1211
Geneva,
Switzerland
a
r
t
i
c
l
e
i
n
f
o
Article
history:
Received
16
October
2023
Accepted
10
October
2024
Available
online
xxx
Keywords:
Stigma
Burnout
Stress
Graduate
medical
education
Physician
health
a
b
s
t
r
a
c
t
Objectives.
–
Physicians
demonstrate
low
rates
of
help
seeking
for
psychiatric
disorders,
occupational
stress,
and
burnout
due
to
perceived
stigma.
The
Stigma
of
Occupational
Stress
Scale
for
Doctors
(SOSS-
D)
is
a
brief
standardized
tool
designed
to
measure
stigma
(personal,
perceived
other,
and
perceived
structural)
in
physicians.
The
aim
of
this
study
was
to
gather
validity
evidence
for
a
French
version
of
the
SOSS-D.
Methods.
–
The
SOSS-D
was
translated
into
French
and
piloted
with
12
physicians.
The
scale
was
admin-
istered
to
physicians
at
the
Geneva
University
Hospitals.
We
computed
descriptive
statistics
and
internal
consistency
coefficients.
Construct
validity
was
analyzed
using
exploratory
(EFA)
and
confirmatory
factor
(CFA)
analyses.
Results.
–
In
total,
323
physicians
participated
in
the
survey.
The
internal
consistency
coefficient
for
the
French
SOSS-D
was
0.72,
0.55
for
the
personal
stigma
subscale,
0.66
for
the
perceived
other
stigma
subscale,
and
0.65
for
the
perceived
structural
stigma
subscale.
CFA
indicated
a
marginal
fit.
EFA
revealed
three
factors:
personal,
perceived
other,
and
perceived
structural
stigma.
Discussion.
–
Our
findings
support
the
hypothesis
that
stigma
is
a
multi-dimensional
construct.
However,
the
French
version
of
the
SOSS-D
scale
did
demonstrate
some
differences
when
compared
to
its
English
version.
Identifying
stigmatization
beliefs
among
physicians
may
allow
for
increased
identification
of
physicians
at
risk,
and
increased
communication
concerning
preventive
actions.
Moreover,
being
aware
of
and
fighting
stigma
can
reduce
barriers
to
help
seeking
and
increased
access
to
care
resources
for
burnt-out
physicians.
Conclusion.
–
The
results
provide
evidence
of
the
validity
and
reliability
of
the
French
version
of
the
SOSS-D
indicating
its
suitability
for
use
in
a
French-speaking
physician
population.
©
2024
L’Enc´
ephale,
Paris.
This
is
an
open
access
article
under
the
CC
BY-NC-ND
license
(http://
creativecommons.org/licenses/by-nc-nd/4.0/).
Mots
clés
:
Stigmatisation
Épuisement
professionnel
Stress
Formation
post-graduée
Santé
des
médecins
r
é
s
u
m
é
Objectifs.
–
Les
médecins
sont
peu
nombreux
à
demander
de
l’aide
pour
des
maladies
psychiatriques,
le
stress
professionnel
et
l’épuisement
en
raison
de
la
stigmatisation
qu’ils
ressentent.
L’échelle
SOSS-
D
(Stigma
of
Occupational
Stress
Scale
for
Doctors)
est
un
bref
outil
standardisé
conc¸
u
pour
mesurer
la
stigmatisation
(personnelle,
perc¸
ue
des
autres
et
structurelle)
chez
les
médecins.
L’objectif
de
cette
étude
était
de
recueillir
des
preuves
de
la
validité
d’une
version
franc¸
aise
du
SOSS-D.
∗Corresponding
author.
E-mail
address:
nadia.bajwa@hcuge.ch
(N.M.
Bajwa).
https://doi.org/10.1016/j.encep.2024.10.002
0013-7006/©
2024
L’Enc´
ephale,
Paris.
This
is
an
open
access
article
under
the
CC
BY-NC-ND
license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Please
cite
this
article
as:
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.,
Validity
evidence
for
a
French
version
of
the
Stigma
of
Occupational
Stress
Scale
for
Doctors
(SOSS-D),
Encéphale,
https://doi.org/10.1016/j.encep.2024.10.002

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
Méthodes.
–
Le
SOSS-D
a
été
traduit
en
franc¸
ais
et
testé
auprès
de
12
médecins.
L’échelle
a
été
administrée
à
des
médecins
des
Hôpitaux
Universitaires
de
Genève.
Nous
avons
calculé
des
statistiques
descriptives
et
des
coefficients
de
cohérence
interne.
La
validité
a
été
analysée
à
l’aide
d’analyses
factorielles
exploratoires
(EFA)
et
confirmatoires
(CFA).
Résultats.
–
Au
total,
323
médecins
ont
participé
à
l’enquête.
Le
coefficient
de
cohérence
interne
du
SOSS-D
franc¸
ais
était
de
0,72,
0,55
pour
la
sous-échelle
de
stigmatisation
personnelle,
0,66
pour
la
sous-échelle
de
stigmatisation
perc¸
ue
des
autres
et
0,65
pour
la
sous-échelle
de
stigmatisation
structurelle
perc¸
ue.
Le
CFA
a
indiqué
une
adéquation
marginale.
L’EFA
a
révélé
trois
facteurs
:
stigmate
personnel,
stigmate
perc¸
u
comme
autre
et
stigmate
perc¸
u
comme
structurel.
Discussion.
–
Nos
résultats
soutiennent
l’hypothèse
que
la
stigmatisation
est
un
concept
multidimension-
nel.
Cependant,
la
version
franc¸
aise
de
l’échelle
SOSS-D
a
montré
quelques
différences
par
rapport
à
la
version
anglaise.
L’identification
des
croyances
de
stigmatisation
chez
les
médecins
peut
permettre
de
mieux
identifier
les
médecins
à
risque
et
d’améliorer
la
communication
concernant
les
actions
préven-
tives.
En
outre,
le
fait
d’être
conscient
de
la
stigmatisation
et
de
la
combattre
peut
réduire
les
obstacles
à
la
recherche
d’aide
et
améliorer
l’accès
aux
ressources
de
soins
pour
les
médecins
épuisés.
Conclusions.
–
Les
résultats
fournissent
des
preuves
de
la
validité
et
de
la
fiabilité
de
la
version
franc¸
aise
du
SOSS-D,
indiquant
qu’elle
peut
être
utilisée
dans
une
population
de
médecins
francophones.
©
2024
L’Enc´
ephale,
Paris.
Cet
article
est
publi´
e
en
Open
Access
sous
licence
CC
BY-NC-ND
(http://
creativecommons.org/licenses/by-nc-nd/4.0/).
1.
Introduction
Physicians
experience
high
levels
of
chronic
occupational
stress,
which
may
lead
to
maladaptive
physiological
and
psychological
reactions
that
can
occur
when
heavy
workplace
demands
are
com-
bined
with
low
work
decision
latitude
[1].
This
psychological
strain
can
lead
to
burnout,
which
is
characterized,
by
emotional
exhaus-
tion,
depersonalization,
and
a
lack
of
personal
accomplishment
[2].
Burnout
is
defined
as
“a
state
of
chronic
fatigue,
depression
or
frustration
induced
by
a
devotion
to
a
cause,
lifestyle,
or
rela-
tionship,
that
fails
to
produce
the
expected
rewards
and
ultimately
leads
to
decreased
accomplishment
at
work”
[3].
Among
physi-
cians,
burnout
is
becoming
more
prevalent
since
25%
to
65%
of
physicians
suffer
from
burnout
and
rates
are
twice
as
high
as
in
other
professions
[4].
Among
Swiss
doctors,
more
than
one
third
(35.2%)
suffer
from
moderate
or
high
levels
of
burn
out
[5].
Physicians
may
be
more
prone
to
experience
burnout
due
to
several
factors.
On
an
individual
level,
irregular
schedules,
long
work
hours,
and
stressful
work
situations
have
been
shown
to
promote
burnout
[6,7].
Physicians
may
experience
an
imbalance
between
their
work
and
their
private
life
as
conflicts
between
pro-
fessional
and
personal
choices
are
usually
resolved
in
favor
of
the
professional
life
[8].
These
difficulties
may
lead
to
fragile
social
real-
ities
such
as
marital
or
financial
difficulties
or
isolation
[9].
Several
studies
have
shown
that
in
addition
to
various
sociodemographic
characteristics,
some
personality
traits
of
physicians
are
risk
factors
for
burnout.
Physicians
who
demonstrate
a
high
degree
of
perfec-
tionism,
neuroticism,
integrity,
and
empathy
suffer
the
most
from
burnout
[10,11].
At
an
institutional
level,
the
health
care
system
is
often
seen
as
inequitable,
focused
on
financial
profitability
and
academic
prestige,
and
dehumanized
[10,12,13].
Burnout
in
medical
doctors
has
serious
consequences
for
both
individuals
and
institutions.
Physician
suicide
rates
are
up
to
six
times
higher
than
among
the
general
population
[9,14].
Physi-
cians
suffering
from
burnout,
particularly
medical
students
and
residents,
are
also
more
likely
to
develop
depression,
substance
dependence
[15,16],
or
cardiovascular
disease
[9].
Another
con-
sequence
of
burnout
in
medical
doctors
is
the
decrease
in
the
quality
of
care
provided
to
their
patients,
mainly
through
the
alter-
ation
of
three
dimensions:
professionalism,
expertise,
and
health
promotion
[17,18].
Concerning
professionalism,
physicians
expe-
riencing
burnout
show
decreased
empathy
and
attention
towards
their
patients
[19,20].
In
terms
of
expertise,
one
study
established
a
significant
association
between
high
burnout
and
poor
medical
knowledge
among
16,000
internal
medicine
residents
[21].
Regard-
ing
health
promotion,
several
studies
have
shown
that
doctors
and
medical
students
suffering
from
burnout
are
also
less
likely
to
have
an
altruistic
view
of
their
responsibility
to
society
[4,22].
Finally,
burnout
may
lead
to
increased
absenteeism
[23],
early
departures
[24,25]
and
also
an
increased
desire
to
change
profession
[23,26].
Medical
turnover
in
relation
to
job
dissatisfaction
increases
the
costs
of
recruiting
and
retaining
doctors
[25]
and
decreases
the
cohesion
in
the
institution
[27].
Physician
representations
about
burnout
may
moderate
whether
or
not
they
seek
treatment
for
burnout
[28].
In
traditional
medical
culture,
perceptions
that
doctors
should
be
“invincible”,
and
that
work
should
always
come
before
personal
needs
are
still
deeply
grounded
[29,30].
In
one
study,
more
than
10%
of
physicians
felt
that
asking
for
help
for
an
emotional
or
psychological
problem
was
a
sign
of
weakness
or
inadequacy
[31].
Half
of
medical
doctors
surveyed
felt
that
supervisors
and
colleagues
had
negative
attitudes
about
mental
health
and
25%
preferred
not
to
seek
help
for
fear
of
a
lack
of
confidentiality
[31].
This
phenomenon
of
stigmatization
is
defined
as
“identification
and
marking
an
undesirable
charac-
teristic
in
a
way
that
narrows
a
person’s
social
identity
to
that
characteristic”
[32].
Stigmatization
can
be
divided
into
four
sub-
categories:
public,
perceived,
self-stigmatization,
and
structural
[33,34].
Personal
or
public
stigmatization
involves
having
beliefs
or
attitudes
that
exclude
someone
from
a
community.
Perceived
stigma
is
characterized
by
the
perceptions
of
beliefs
or
attitudes
that
others
hold
towards
himself
or
herself.
In
self-stigmatization,
a
person
integrates
negative
attitudes
and
rejection
towards
him-
self
or
herself.
Finally,
structural
stigma
describes
the
institutional
practices
that
often
unintentionally
reinforce
the
stigmatization
of
a
person.
Stigmatization
has
been
shown
to
have
adverse
conse-
quences
such
as
delayed
help
seeking
behaviors
[35,36].
Moreover,
stigmatization
may
create
an
obstacle
for
burnt
out
doctors
to
return
to
work
[37,38].
However,
there
is
little
data
available
con-
cerning
stigmatization
of
physicians
suffering
from
burnout
and
the
resulting
consequences
in
terms
of
recognition,
treatment,
and
professional
support
by
peers.
The
Stigma
of
occupational
stress
and
burnout
in
medical
doc-
tors
scale
(SOSS-D)
[39]
is
a
standardized
brief
measure
of
stigma
related
to
occupational
stress
and
burnout
among
physicians.
The
scale
measures
stigma
towards
stress
and
burnout
broadly
and
is
constructed
to
cover
stigma
in
its
facets
of
personal
stigma,
per-
ceived
other
stigma,
and
perceived
structural
stigma.
The
scale
is
not
intended
to
measure
aspects
of
self-stigmatization.
The
aim
2

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
of
this
study
was
to
translate
and
gather
validity
evidence
for
the
French
version
of
the
Stigma
of
occupational
stress
scale
for
doc-
tors
(SOSS-D).
This
study
is
part
of
a
larger
study
that
investigated
the
association
of
personal
factors,
stigma
of
occupational
stress
and
burnout,
and
burn
out
among
medical
doctors
from
different
fields.
2.
Methods
2.1.
Questionnaire
The
SOSS-D
is
an
11-item
self-report
scale
that
measures
three
dimensions
of
stigmatization:
personal
stigma,
perceived
other
stigma,
and
perceived
structural
stigma
(Clough,
Ireland,
&
March,
2019).
The
scale
consists
of
a
seven-point
Likert
scale
ranging
from
“strongly
disagree”
to
“strongly
agree”.
The
scale
was
first
created
and
tested
with
Australian
physicians
in
2017.
Three
items
com-
pose
the
“personal
stigma”
subscale
(1,
7,
11),
and
three
items
the
“perceived
other
stigma”
subscale
(2,
4,
10).
Five
items
compose
the
“perceived
structural
stigma”
subscale
(3*,
5,
6,
8,
9*),
and
two
items
(*)
are
intended
to
be
reversed
scored
(Table
1).
2.2.
Translation
procedure
for
the
SOSS-D
The
English
version
of
the
SOSS-D
was
first
translated
into
French
by
a
professional
translator,
a
bilingual
medical
doctor,
and
a
bilingual
psychologist.
The
research
team
then
performed
back
translations
to
ensure
the
content
and
conceptual
equiva-
lence
of
the
translation
[40].
The
first
version
of
the
French
SOSS-D
was
reviewed
by
a
panel
of
seven
physicians
from
the
Institute
of
Primary
Care
Medicine
(IMPR)
and
a
psychologist
at
the
Geneva
University
Hospitals.
After
review,
consensus
was
achieved
for
the
French
version
of
the
SOSS-D.
The
French
version
of
the
SOSS-D
was
piloted
with
12
physicians
during
a
3-month
period.
The
12
physicians
represented
different
levels
of
hierarchy
(residents,
fel-
lows,
and
attendings)
and
four
different
departments
(Pediatrics,
Obstetrics
and
Gynecology,
Family
Medicine,
Internal
Medicine,
and
Psychiatry)
at
the
Geneva
University
Hospitals.
The
comments
of
these
12
physicians
were
then
reviewed
by
the
panel
and
only
minor
changes
were
made
for
the
final
French
version
of
the
SOSS-D
(Online
supplement).
2.3.
Participants
and
setting
The
study
was
conducted
in
an
academic
hospital
setting,
the
Geneva
University
Hospitals.
Nine
hundred
and
sixty-one
residents,
fellows,
and
attending
in
Psychiatry,
Internal
Medicine,
Family
Medicine,
Pediatrics,
and
Obstetrics
and
Gynecology
were
invited
to
participate
in
the
survey
via
an
email
solicitation
to
fill
out
the
electronic
survey.
We
initially
estimated
a
response
rate
of
50%
with
a
minimum
of
200
responses
needed
to
be
able
to
conduct
the
analyses
[41].
Non-responders
were
resolicited
two
times
over
a
three-month
period.
All
survey
responses
were
anonymous,
and
participants
indicated
their
consent
to
participate
in
the
study
at
the
beginning
of
the
survey.
Participation
was
voluntary
and
there
was
no
compensation
for
completing
the
questionnaire.
2.4.
Statistical
analysis
Demographic
data
(means
and
standard
deviation)
including
gender,
age,
marital
status,
department,
position,
and
years
of
experience
were
collected
for
each
participant.
Mean
scores
and
standard
deviation
(SD)
were
calculated
for
the
three
subscales
of
the
SOSS-D.
To
examine
the
internal
consistency
of
the
scale,
we
used
Cronbach
alphas
and
compared
them
with
previously
published
coefficients
of
the
original
version.
To
examine
possible
alternative
structures,
we
performed
an
exploratory
factor
analysis
(EFA)
using
principal
axis
factoring
(PAF)
and
an
oblique
rotation
(direct
oblimin)
of
the
11
items.
We
retained
factors
that
had
an
eigenvalue
>
1.
We
obtained
the
Kaiser-Meyer-Olkin
(KMO)
mea-
sure
of
sampling
adequacy
(>
0.6
is
recommended)
and
the
Bartlett
test
of
sphericity.
To
verify
that
the
internal
structure
of
the
SOSS-D
conformed
to
theory,
we
performed
a
confirmatory
factor
analy-
sis
(CFA)
of
the
11
items
using
Maximum
Likelihood
with
Missing
Values.
Fit
indices
included
the
root
mean
square
error
of
approxi-
mation
(RMSEA),
the
comparative
fit
index
(CFI),
and
the
coefficient
of
determination
(CD).
We
performed
the
EFA
with
SPSS
26
for
Macintosh
(IBM
Corp.,
Armonk,
New
York)
and
used
StataSE
16
for
Macintosh
(Stata
Corp,
College
Station,
Texas)
for
the
descriptive
statistics,
Cronbach’s
alpha,
and
the
CFA.
2.5.
Ethics
All
methods
were
performed
in
accordance
with
the
Declaration
of
Helsinki.
The
study
protocol
was
received
ethical
approval
by
the
Geneva
Cantonal
Ethical
Commission
Commission
(November
5th,
2018.
Ethical
Approval
Reference
Number:
2018-00597).
Written
informed
consent
for
the
research
was
obtained
from
all
participants.
At
the
end
of
the
survey,
mental
health
resources
for
health
professionals
(staff
health
service,
support
network
for
physicians)
were
made
available
to
participants.
3.
Results
3.1.
Participants
Three
hundred
and
twenty-three
physicians
participated
in
the
survey
(36%
response
rate).
Of
the
308
(32%)
physicians
who
com-
pleted
the
demographic
portion
of
the
survey,
38%
(117)
were
men
and
62%
(191)
were
women.
A
majority
(236,
77%)
was
in
a
relationship
and
23%
(72)
were
single.
The
mean
age
was
38.6
years
(SD
=
9.2).
40%
(123)
were
residents,
36.8%
(112)
were
fel-
lows,
and
23%
(70)
were
attending.
Participants
had
been
practicing
for
12.5
(SD
=
9.4)
years
on
average.
The
distribution
among
the
five
departments
was:
23%
(72)
psychiatrists,
29%
(88)
internal
medicine
physicians,
21%
(66)
family
medicine
physicians,
22%
(67)
pediatricians,
and
5%
(15)
obstetricians
and
gynecologists.
The
participation
rate
was
similar
between
the
departments:
psychiatry
(31%),
internal
medicine
(33.5%),
family
medicine
(32.5%),
and
pediatrics
(34.2%),
except
for
obstetrics
and
gynecol-
ogy,
who
had
a
lower
participation
rate
(22.4%).
3.2.
Internal
consistency
Scores
on
the
three
subscales
were
as
follows:
perceived
struc-
tural
stigma
mean
=
4.49
(SD
=
0.99),
personal
stigma
mean
=
2.58
(SD
=
1.18),
and
perceived
other
stigma
mean
=
3.78
(SD
=
1.22).
Cronbach’s
alpha
overall
for
the
11
item
SOSS-D
was
0.72,
0.65
for
the
perceived
structural
stigma
subscale,
0.55
for
the
personal
stigma
subscale,
and
0.66
for
the
perceived
other
stigma
subscale.
3.3.
Exploratory
factor
analysis
The
KMO
measure
of
sampling
adequacy
was
acceptable
at
0.72
and
the
Bartlett
test
of
sphericity
was
significant
(P
<
0.001)
indi-
cating
that
the
sample
was
suitable
for
factor
analysis.
Items
1
and
8
exhibited
low
communalities
but
were
left
in
the
scale
for
the
analysis
for
comparison
to
the
original
version
of
the
SOSS-D.
The
EFA
identified
three
factors
with
eigenvalues
>
1
that
explained
55%
of
total
variance.
Intercorrelations
were
moderate
between
the
factors
with
r
=
0.21
between
personal
and
perceived
structural
stigma,
r
=
0.38
between
perceived
other
and
perceived
structural
3

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
Table
1
Principal
axis
factor
loadings
for
11
SOSS-D
items.
Item
number Item
stem Factor
Dimension
1
2
3
3aA
doctor
who
is
experiencing
stress
or
burnout
would
be
treated
fairly
compared
to
any
other
doctor
−0.007
−0.019
0.032
Factor
1:
perceived
structural
stigma
5
Where
I
work,
a
doctor
who
is
experiencing
occupational
stress
or
burnout
would
probably
be
better
off
not
telling
anyone
0.698
0.090
0.014
6
A
doctor
who
reports
experiencing
occupational
stress
or
burnout
is
more
likely
to
experience
discrimination
or
prejudice
0.396
−0.017
0.206
8
A
doctor
who
is
experiencing
occupational
stress
or
burnout
would
be
considered
as
less
eligible
for
career
progression
0.061
0.179
0.212
9aWhere
I
work,
any
doctor
who
is
experiencing
occupational
stress
or
burnout
would
be
given
understanding
and
support
0.913
−0.022
−0.152
1
A
doctor
who
experiences
occupational
stress
or
burnout
is
somehow
less
capable
than
a
doctor
who
does
not
experience
those
difficulties
0.005
0.300
0.129
Factor
2:
personal
stigma
7
I
would
try
to
distance
myself
from
a
doctor
who
is
experiencing
occupational
stress
or
burnout
0.113
0.706
−0.112
11
I
would
have
reservations
about
working
with
a
doctor
who
is
experiencing
difficulties
with
occupational
stress
or
burnout
−0.107
0.890
−0.068
2
Most
doctors
would
consider
the
experience
of
occupational
stress
or
burnout
as
a
sign
that
the
doctor
is
not
“right”
for
the
profession
−0.078
−0.005
0.736
Factor
3:
perceived
other
stigma
4
Most
doctors
would
consider
a
doctor
who
is
experiencing
occupational
stress
or
burnout
as
too
sensitive
or
weak
0.140
−0.016
0.678
10
Most
doctors
would
agree
that
occupational
stress
or
burnout
is
a
state
of
mind
that
one
should
snap
out
of
0.448
−0.004
0.288
Extraction
method:
principal
axis
factoring,
rotation
method:
oblique
(direct
oblimin)
with
Kaiser
normalization,
the
rotation
converged
in
eight
iterations.
aReverse
scored.
stigma,
and
r
=
0.24
between
personal
and
perceived
other
stigma
supporting
the
choice
to
use
an
oblique
rotation.
Three
items
loaded
to
different
factors
than
in
the
original
instrument.
Item
3
“would
be
treated
fairly”
and
item
8
“would
be
considered
as
less
eligible
for
career
progression”
demonstrated
low
loadings
to
all
factors,
while
item
10
“burnout
is
a
state
of
mind”
loaded
to
structural
stigma
rather
than
to
perceived
other
stigma.
Factor
loadings
for
the
11
items
can
be
found
in
Table
1.
3.4.
Confirmatory
factor
analysis
Confirmatory
factor
analysis
was
then
performed
based
on
the
same
sample
based
on
the
original
structure
of
the
scale.
Goodness
of
fit
statistics
showed
that
the
RMSEA
was
0.10
and
indicated
a
marginal
fit.
The
CFI
=
0.818
also
indicated
an
acceptable
fit
and
the
coefficient
of
determination
was
very
close
to
one
at
0.981.
Results
from
the
model
are
shown
in
Fig.
1.
The
analysis
was
conducted
using
maximum
likelihood
with
missing
values.
Observed
variables
are
indicated
in
the
rectangles
and
the
latent
variables
are
indicated
in
the
ellipses.
The
model
demonstrates
acceptable
fit
with
a
RMSEA
of
0.10
and
a
CFI
of
0.818.
4.
Discussion
This
study
provides
validity
evidence
for
the
French
version
of
the
SOSS-D.
Like
the
original
study,
we
identified
three
domains
related
to
stigma:
personal
stigma,
perceived
other
stigma,
and
per-
ceived
structural
stigma.
Our
findings
support
the
hypothesis
that
stigma
is
a
multi-dimensional
construct.
However,
the
French
ver-
sion
of
the
SOSS-D
scale
did
demonstrate
some
differences
when
compared
to
the
English
version
of
the
SOSS-D.
We
found
that
some
items
loaded
to
different
factors
in
the
French
version.
For
example,
item
3
“would
be
treated
fairly”
and
item
8
“would
be
considered
as
less
eligible
for
career
progression”
had
low
loadings
to
all
three
factors,
while
item
10
“burnout
is
a
state
of
mind”
loaded
to
struc-
tural
stigma
rather
than
to
perceived
other
stigma.
Also,
items
1
and
8
demonstrated
low
communalities
but
were
kept
in
the
anal-
ysis
in
order
to
render
a
comparison
with
the
original
scale.
These
differences
may
be
due
to
the
translation
of
the
items.
For
example,
in
item
3,
“would
be
treated
fairly”,
it
may
have
been
ambiguous
as
to
who
would
treat
fairly
the
physician
suffering
from
burnout.
When
comparing
the
similarity
of
covariance
of
the
latent
variables
between
the
two
instruments,
we
found
that
55%
of
the
variance
was
explained
by
the
three
factors
in
the
French
version
while
the
English
version
of
the
SOSS-D
reported
63.4%
variance
for
the
three
factors
[39].
We
also
observed
differences
in
the
internal
con-
sistency
of
the
instrument
compared
to
the
original
version.
The
Cronbach
alphas
for
each
subscale
were
lower
than
those
reported
in
the
English
version.
In
interpreting
the
differences
between
the
results
of
the
fac-
tor
analyses,
one
should
bear
in
mind
that
the
original
instrument
included
16
items
at
the
outset,
from
which
5
items
were
removed
4

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
Fig.
1.
Confirmatory
Factor
Analysis
for
the
SOSS-D.
in
order
to
achieve
a
clean
factor
structure
–
2
items
due
to
low
communalities
and
3
items
due
to
cross-loadings
[39].
Thus,
their
proposed
factor
structure
is
potentially
over-fitted
to
the
devel-
opment
sample.
It
is
not
certain
that
it
could
be
fully
reproduced
in
another
sample,
even
in
the
English
language.
Nevertheless,
the
original
dimensions
make
sense
from
a
content-validity
standpoint
and
the
observed
discrepancies
do
not
justify
a
modification
of
the
instrument’s
structure
in
the
French
version.
The
three
types
of
stigma
in
the
SOSS-D
are
defined
as
per-
sonal
stigma
that
is
a
person’s
stigmatizing
attitudes
about
other
people,
perceived
other
stigma
that
is
a
person’s
beliefs
about
the
stigmatizing
attitudes
that
other
people
hold,
and
perceived
struc-
tural
stigma
that
represents
policies
and
practices
that
restrict
the
opportunities
or
well-being
of
the
stigmatized
person
[39].
In
our
study
as
in
the
original
version,
the
lowest
endorsements
of
stigma
were
for
the
personal
stigma
subscale
and
participants
scored
highest
on
perceived
structural
stigma.
Perceived
structural
stigma
reflects
perceptions
of
stigmatization
of
burnout
within
the
context
of
broader
structural
influences
in
workplace
while
high
perceived
other
stigma
may
symbolize
the
implicit
culture
at
our
institution
where
there
may
be
a
contradiction
between
delivered
and
perceived
messages.
The
medical
culture
may
foster
a
message
that
there
is
no
room
for
mistakes
in
medicine
nor
space
for
flaws
when
physicians
are
concerned
[42].
Significant
cultural
changes
are
needed
to
nuance
physician
representations
about
burnout.
The
medical
community
needs
to
look
inward
to
identify
those
cultural
factors
that
may
contribute
to
perceptions
of
stigmatiza-
tion
regarding
burnout.
For
example,
a
heroic
conception
of
stress
arising
from
work
as
a
doctor
is
a
major
contributor
to
the
stigmati-
zation
of
burn
out
[43].
Moreover,
role
models
may
have
a
negative
impact
on
burnout.
Indeed,
senior
physicians
may
perceive
that
the
signs
and
symptoms
of
burnout
are
normal
within
the
medical
cul-
ture
and
perpetuate
stigma
around
those
that
cannot
handle
the
stress
[31,44].
Residents
may,
in
turn,
conceal
burnout
from
their
attendings
due
to
fear
of
the
negative
repercussions
resulting
from
disclosure
and
help
seeking
behaviors
[31,44].
Changes
in
beliefs
can
have
a
greater
impact
if
they
are
driven
by
those
in
leadership
positions
[43].
Furthermore,
this
improvement
is
even
more
mean-
ingful
if
senior
physicians
reveal
that
they
themselves
have
lived
through
shame,
fear,
and
burnout
[45].
A
recent
study
reported
that
exposure
to
senior
physicians
who
openly
shared
their
lived
experi-
ences
with
mental
illness,
including
burnout,
had
a
positive
impact
on
medical
students’
attitudes
about
individuals
with
mental
ill-
ness
[42].
In
a
secure
institutional
environment,
without
fear
of
the
consequences
on
their
career,
the
physicians’
self-disclosure
of
burnout
seems
to
be
an
effective
way
to
change
false
beliefs
about
this
condition.
A
scale-assessing
stigma
of
burnout
in
medical
doctors
is
an
innovative
and
essential
tool.
Identifying
stigmatization
beliefs
among
physicians
may
allow
for
increased
identification
of
physi-
cians
at
risk,
and
increased
communication
concerning
preventive
actions.
Moreover,
being
aware
of
and
fighting
stigma
can
reduce
barriers
to
help
seeking,
and
increased
access
to
care
resources
for
burnt-out
physicians
[43].
Stigmatization
is
also
important
to
con-
sider
improving
return
to
work
conditions
for
physicians
suffering
from
burnout.
Many
physicians
may
internalize
the
perceived
neg-
ative
responses
of
colleagues
and
others
to
their
illness
and
it
may
contribute
to
the
difficulties
faced
by
physicians
in
returning
to
work
[37].
Moreover,
some
authors
suggest
that
shame
might
actu-
ally
prolong
sick
leave
[46].
In
contrast,
social
support
and
positive
feedback
from
supervisors
facilitates
a
full
return
to
work
[47].
Strengths
of
this
study
include
the
rigorous
process
of
trans-
lation
of
the
original
scale
and
the
validity
evidence
collected
to
demonstrate
the
utility
of
the
scale.
To
our
knowledge,
this
is
the
first
study
that
describes
the
development
of
a
measure
of
stigma
and
reports
the
perceptions
of
stigma
in
a
French-speaking
physi-
cian
population.
By
demonstrating
construct
validity
for
the
scale
and
the
three
subscales,
it
is
possible
to
determine
the
primary
source
of
physician
stigma
and
to
potentially
use
this
information
to
provide
meaningful
feedback
and
to
identify
which
sources
of
stigma
need
to
be
most
urgently
addressed.
Limitations
include
that
this
study
was
conducted
in
only
one
institution
with
a
pre-
dominantly
female
sample
indicating
the
need
to
repeat
this
study
in
other
more
diverse
settings.
The
response
rate
(36%)
and
the
participation
rate
(32%)
were
low,
possibly
due
to
the
study’s
sen-
sitive
nature,
but
were
comparable
to
that
of
a
study
with
a
similar
methodology
[44].
While
the
sample
size
was
large
enough
to
con-
duct
the
validity
analyses,
we
would
also
like
to
see
future
studies
with
larger
sample
sizes.
This
study
is
also
limited
by
two
of
the
items
having
low
communalities
(items
1
and
8).
If
the
low
com-
munalities
persist
in
future
studies,
removal
of
the
items
may
be
necessary
or
there
may
exist
another
factor
that
could
be
identifi-
able
if
other
items
are
added
to
the
scale
[48].
5.
Conclusions
The
French
version
of
the
SOSS-D
assesses
the
stigma
of
occupa-
tional
stress
and
burnout
in
French-speaking
medical
doctors.
This
5

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
scale
provides
a
time-efficient
tool
that
can
raise
awareness
and
contribute
to
prevention
by
identifying
stigmatizing
attitudes
and
perceptions
of
stress
and
burnout
in
medical
doctors
to
support
help
seeking
in
this
population.
Ethics
approval
and
consent
to
participate
This
study
received
ethical
approval
by
the
ethics
commission
from
the
Geneva
Cantonal
Ethical
Commission
the
5th
of
November
2018.
Project
number
2018-00597.
Availability
of
data
and
materials
The
datasets
used
and/or
analysed
during
the
current
study
are
available
from
the
corresponding
author
on
reasonable
request.
Authors’
contributions
NMB
and
HRL
contributed
to
the
conception,
design,
adminis-
tration,
and
analysis
of
the
study
as
well
as
the
drafting
and
revision
of
the
manuscript.
SF
and
NJP
contributed
to
the
conception,
design,
and
administration
of
the
study
as
well
as
the
drafting
and
revision
of
the
manuscript.
MCA,
MDD,
and
MRN
contributed
to
the
con-
ception,
design,
and
administration
of
the
study
as
well
as
revision
of
the
manuscript.
TP
contributed
to
the
design
and
analysis
of
the
study
as
well
as
the
revision
of
the
manuscript.
All
authors
read
and
approved
the
final
manuscript.
Contribution
Nadia
M.
Bajwa,
MD,
MHPE,
PhD
is
the
Residency
Program
Director
in
the
Department
of
General
Pediatrics
at
the
Children’s
Hospital,
Geneva
University
Hospitals,
and
Faculty
Member
at
the
Unit
of
Development
and
Research
in
Medical
Education
(UDREM),
Faculty
of
Medicine,
University
of
Geneva,
Geneva,
Switzerland.
ORCID:
http://orcid.org/0000-0002-1445-4594.
Sophie
Favre
is
a
psychologist
in
the
Mood
Disorders
Unit
at
the
Geneva
University
Hospitals,
Geneva,
Switzerland.
Thomas
Perneger,
MD,
PhD,
is
Full
Professor
and
Head
of
the
Division
of
Clinical
Epidemiology
Division
at
the
Geneva
University
Hospitals,
Geneva,
Switzerland.
ORCID:
http://orcid.org/0000-0001-5667-0968.
Melissa
Dominicé
Dao
MD,
MSc,
is
Attending
physician,
Director
of
Post-graduate
Training
at
the
Division
of
Pri-
mary
Care
Medicine,
Department
of
Primary
Care
Medicine,
Geneva
University
Hospitals,
Geneva,
Switzerland.
ORCID:
https://orcid.org/0000-0002-0839-277X.
Marie-Claude
Audétat,
M.Ps,
MA(e),
Ph.D,
is
Associate
Professor
in
the
Unit
of
Development
and
Research
in
Medical
Educa-
tion
(UDREM),
Faculty
of
Medicine,
University
of
Geneva,
Geneva,
Switzerland.
Mathieu
R.
Nendaz,
MD,
MHPE,
is
a
Full
Professor
and
the
Director
of
the
Unit
of
Development
and
Research
in
Med-
ical
Education
(UDREM),
Faculty
of
Medicine,
University
of
Geneva,
and
Attending
Physician
in
the
Division
of
General
Inter-
nal
Medicine,
Geneva
University
Hospitals,
Geneva,
Switzerland.
ORCID:
http://orcid.org/0000-0003-3795-3254.
Noëlle
Junod
Perron,
MD,
PhD
is
the
Coordinator
of
the
Insti-
tute
of
Primary
Care,
Geneva
University
Hospitals,
and
Faculty
Member
at
the
Unit
of
Development
and
Research
in
Medical
Edu-
cation
(UDREM),
Faculty
of
Medicine,
University
of
Geneva,
Geneva,
Switzerland.
ORCID:
http://orcid.org/0000-0002-9124-8663.
Hélène
Richard-Lepouriel,
MD
is
Head
of
the
Mood
Disorders
unit
in
the
Psychiatric
specialties
service,
Department
of
Psychiatry,
Geneva
University
Hospitals,
Geneva,
Switzerland.
Funding
This
study
was
supported
by
the
Private
Foundation
of
the
Geneva
University
Hospitals.
Disclosure
of
interest
The
authors
declare
that
they
have
no
competing
interest.
Acknowledgments
We
would
like
to
thank
the
participants
of
our
study
for
their
collaboration.
We
thank
the
“Fondation
Privée
des
HUG”
for
their
support.
Online
supplement.
Supplementary
data
Supplementary
data
associated
with
this
article
can
be
found,
in
the
online
version,
at
https://doi.org/10.1016/j.encep.2024.10.002.
References
[1]
Karasek
J,
Robert
A.
JJob
demands,
job
decision
latitude,
and
mental
strain:
implications
for
job
redesign.
Adm
Sci
Q
1979:285–308.
[2]
Maslach
C.
Burnout:
the
cost
of
caring.
Los
Altos:
Ishk;
2003.
[3]
Veyssier-Belot
C.
Burnout
syndrome
among
physicians.
Rev
Med
Interne
2015;36(4):233–6.
[4]
Shanafelt,
Boone
S,
Tan
L,
et
al.
Burnout
and
satisfaction
with
work-life
balance
among
US
physicians
relative
to
the
general
US
population.
Arch
Intern
Med
2012;172(18):1377.
[5]
Arigoni
F,
Bovier
P,
Sappino
A.
Trend
of
burnout
among
Swiss
doctors.
Swiss
Med
Wkly
2010;140:w13070.
[6]
Gelfand
DV.
Effect
of
the
80-hour
workweek
on
resident
burnout.
Arch
Surg
2004;139(9):933.
[7]
Ishak
WW,
Lederer
S,
Mandili
C,
et
al.
Burnout
during
residency
training:
a
literature
review.
J
Grad
Med
Educ
2009;1(2):236–42.
[8]
Seidler
A,
Thinschmidt
M,
Deckert
S,
et
al.
The
role
of
psychosocial
working
con-
ditions
on
burnout
and
its
core
component
emotional
exhaustion
–
a
systematic
review.
J
Occup
Med
Toxicol
2014;9(1):10.
[9]
Wallace
JE,
Lemaire
JB,
Ghali
WA.
Physician
wellness:
a
missing
quality
indica-
tor.
Lancet
2009;374(9702):1714–21.
[10]
Cole
TR,
Carlin
N.
The
suffering
of
physicians.
Lancet
2009;374(9699):1414–5.
[11]
McManus
IC,
Keeling
A,
Paice
E.
Stress,
burnout
and
doctors’
attitudes
to
work
are
determined
by
personality
and
learning
style:
a
twelve
year
longitudinal
study
of
UK
medical
graduates.
BMC
Med
2004;2:29.
[12]
Gabbard
GO.
Medicine
and
its
discontents.
Mayo
Clin
Proc
2013;88(12):1347–9.
[13]
Tijdink
JK,
Vergouwen
AC,
Smulders
YM.
Publication
pressure
and
burn
out
among
Dutch
medical
professors:
a
nationwide
survey.
PLoS
One
2013;8(9):e73381.
[14]
Schernhammer
E.
Taking
their
own
lives
–
the
high
rate
of
physician
suicide.
N
Engl
J
Med
2005;352(24):2473–6.
[15]
Goebert
D,
Thompson
D,
Takeshita
J,
et
al.
Depressive
symptoms
in
medical
students
and
residents:
a
multischool
study.
Acad
Med
2009;84(2):236–41.
[16]
Lebensohn
P,
Dodds
S,
Benn
R,
et
al.
Resident
wellness
behaviors:
relationship
to
stress,
depression,
and
burnout.
Fam
Med
2013;45(8):541–9.
[17]
Dewa
CS,
Loong
D,
Bonato
S,
et
al.
The
relationship
between
physician
burnout
and
quality
of
healthcare
in
terms
of
safety
and
acceptability:
a
systematic
review.
BMJ
Open
2017;7(6):e015141.
[18]
Tawfik
DS,
Scheid
A,
Profit
J,
et
al.
Evidence
relating
health
care
provider
burnout
and
quality
of
care.
Ann
Intern
Med
2019;171(8):555.
[19]
Back
AL,
Deignan
PF,
Potter
PA.
Compassion,
compassion
fatigue,
and
burnout:
key
insights
for
oncology
professionals.
Am
Soc
Clin
Oncol
Educ
Book
2014;34:e454–9.
[20]
Gleichgerrcht
E,
Decety
J.
Empathy
in
clinical
practice:
how
individual
dis-
positions,
gender,
and
experience
moderate
empathic
concern,
burnout,
and
emotional
distress
in
physicians.
PLoS
One
2013;8(4):e61526.
[21]
West
CP,
Shanafelt
TD,
Kolars
JC.
Quality
of
life,
burnout,
educational
debt,
and
medical
knowledge
among
internal
medicine
residents.
JAMA
2011;306(9):952–60.
[22]
Dyrbye
L,
Massie
FS,
Eacker
A,
et
al.
Relationship
between
burnout
and
professional
conduct
and
attitudes
among
US
medical
students.
JAMA
2010;304(11):1173.
[23]
Soler
JK,
Yaman
H,
Esteva
M,
et
al.
Burnout
in
European
family
doctors:
the
EGPRN
study.
Fam
Pract
2008;25(4):245–65.
[24]
Shanafelt,
Sloan
J,
Satele
D,
et
al.
Why
do
surgeons
consider
leaving
practice?
J
Am
Coll
Surg
2011;212(3):421–2.
6

ARTICLE IN PRESS
G Model
ENCEP-1807;
No.
of
Pages
7
N.M.
Bajwa,
S.
Favre,
T.
Perneger
et
al.
L’Encéphale
xxx
(xxxx)
xxx–xxx
[25]
West
CP,
Dyrbye
LN,
Satele
DV,
et
al.
Concurrent
validity
of
single-item
mea-
sures
of
emotional
exhaustion
and
depersonalization
in
burnout
assessment.
J
Gen
Intern
Med
2012;27(11):1445–52.
[26]
Zhang
Y,
Feng
X.
The
relationship
between
job
satisfaction,
burnout,
and
turnover
intention
among
physicians
from
urban
state-owned
medical
insti-
tutions
in
Hubei,
China:
a
cross-sectional
study.
BMC
Health
Serv
Res
2011;11:235.
[27]
Leape
LL,
Fromson
JA.
Problem
doctors:
is
there
a
system-level
solution?
Ann
Intern
Med
2006;144(2):107–15.
[28]
Givens
JL,
Tjia
J.
Depressed
medical
students’
use
of
mental
health
services
and
barriers
to
use.
Acad
Med
2002;77(9):918–21.
[29]
Harvey
S,
Laird
B,
Henderson
M,
et
al.
The
mental
health
of
health
care
pro-
fessionals:
a
review
for
the
Department
of
Health.
London:
National
Clinical
Assessment
Service;
2009.
[30]
Dyrbye
L,
West
CP,
Satele
D,
et
al.
Work/Home
conflict
and
burnout
among
aca-
demic
internal
medicine
physicians.
Arch
Intern
Med
2011;171(13):1207–9.
[31]
Dyrbye
L,
Eacker
A,
Durning
S,
et
al.
The
impact
of
stigma
and
personal
expe-
riences
on
the
help
seeking
behaviors
of
medical
students
with
burnout.
Acad
Med
2015;90:961–9.
[32]
Chen
J,
Courtwright
A.
Stigmatization.
In:
Ten
Have
H,
editor.
Encyclopedia
of
global
bioethics.
Cham:
Springer
International
Publishing;
2016.
p.
2706–12.
[33]
Corrigan
PW,
Watson
AC.
The
paradox
of
self-stigma
and
mental
illness.
Clin
Psychol
Sci
Pract
2002;9(1):35–53.
[34]
Sartorius
N.
Stigma
and
mental
health.
Lancet
2007;370(9590):810–1.
[35]
Dattilio
FM.
The
self-care
of
psychologists
and
mental
health
professionals:
a
review
and
practitioner
guide.
Aust
Psychol
2015;50(6):393–9.
[36]
Schnyder
N,
Panczak
R,
Groth
N,
et
al.
Association
between
mental
health-
related
stigma
and
active
help
seeking:
systematic
review
and
meta-analysis.
Br
J
Psychiatry
2017;210(4):261–8.
[37]
Henderson
M,
Brooks
SK,
del
Busso
L,
et
al.
Shame!
Self-stigmatisation
as
an
obstacle
to
sick
doctors
returning
to
work:
a
qualitative
study.
BMJ
Open
2012;2(5):e001776.
[38]
Brooks
SK,
Del
Busso
L,
Chalder
T,
et
al.
‘You
feel
you’ve
been
bad,
not
ill’:
Sick
doctors’
experiences
of
interactions
with
the
General
Medical
Council.
BMJ
Open
2014;4(7):e005537.
[39]
Clough,
Ireland
MJ,
March
S.
Development
of
the
SOSS-D:
a
scale
to
measure
stigma
of
occupational
stress
and
burnout
in
medical
doctors.
J
Ment
Health
2019;28(1):26–33.
[40]
Helmich
E,
Cristancho
S,
Diachun
L,
et
al.
‘How
would
you
call
this
in
English?’
Being
reflective
about
translations
in
international,
cross-cultural
qualitative
research.
Perspect
Med
Educ
2017;6(2):127–32.
[41]
Rouquette
A,
Falissard
B.
Sample
size
requirements
for
the
internal
validation
of
psychiatric
scales.
Int
J
Methods
Psychiatr
Res
2011;20:235–49.
[42]
Martin
A,
Chilton
J,
Gothelf
D,
et
al.
Physician
self-disclosure
of
lived
experi-
ence
improves
mental
health
attitudes
among
medical
students:
a
randomized
study.
J
Med
Educ
Curricular
Dev
2020;7,
2382120519889352.
[43]
Clough,
March
S,
Leane
S,
et
al.
What
prevents
doctors
from
seeking
help
for
stress
and
burnout?
A
mixed-methods
investigation
among
metropolitan
and
regional-based
australian
doctors.
J
Clin
Psychol
2019;75(3):418–32.
[44]
Williford
ML,
Scarlet
S,
Meyers
MO,
et
al.
Multiple-institution
comparison
of
resident
and
faculty
perceptions
of
burnout
and
depression
during
surgical
training.
JAMA
Surg
2018;153(8):705–11.
[45]
Atherley
A,
Meeuwissen
SNE.
Time
for
change:
overcoming
perpetual
feelings
of
inadequacy
and
silenced
struggles
in
medicine.
Med
Educ
2020;54(2):92–4.
[46]
Knapstad
M,
Øverland
S,
Henderson
M,
et
al.
Shame
among
long-term
sickness
absentees:
correlates
and
impact
on
subsequent
sickness
absence.
Scandn
J
Public
Health
2014;42(1):96–103.
[47]
Pijpker
R,
Vaandrager
L,
Veen
EJ,
et
al.
Combined
interventions
to
reduce
burnout
complaints
and
promote
return
to
work:
a
systematic
review
of
effec-
tiveness
and
mediators
of
change.
Int
J
Environ
Res
Public
Health
2019;17(1)
https://www.mdpi.com/about/announcements/784.
[48]
Costello
AB,
Osborne
J.
Best
practices
in
exploratory
factor
analysis:
four
rec-
ommendations
for
getting
the
most
from
your
analysis.
Pract
Assess
Res
Eval
2005;10(1):7.
7