Content uploaded by Colleen Y. Colbert
Author content
All content in this area was uploaded by Colleen Y. Colbert on Nov 25, 2022
Content may be subject to copyright.
568 Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Teaching residents how to break bad news: piloting a
resident- led curriculum and feedback task force as a
proof- of- conceptstudy
Joseph Sleiman ,1 David J Savage ,1,2 Benjamin Switzer,1,3
Colleen Y Colbert ,4,5 Cory Chevalier,5,6 Kathleen Neuendorf,5,6 David Harris5,6
Original research
To cite: Sleiman J, Savage DJ,
Switzer B, etal.
BMJ Simul Technol Enhanc Learn
2021;7:568–574.
►Additional supplemental
material is published online
only. To view, please visit the
journal online (http:// dx. doi.
org/ 10. 1136/ bmjstel- 2021-
000897).
For numbered affiliations see
end of article.
Correspondence to
Dr Joseph Sleiman, Internal
Medicine Residency Program,
Cleveland Clinic, Cleveland, OH
44195, USA; joseph. sleiman.
22@ gmail. com
Poster presented at ACGME
Back to Bedside Consortium
Meeting, Chicago, Illinois, USA,
August 2019 (J Sleiman, DJ
Savage, B Switzer, C Chevalier
and D Harris, Improving
Residents’ Skills for Leading
Bad News Conversations
with Patients at the Bedside);
ACGME Back to Bedside
Consortium Meeting, October
2020 (virtual conference) (J
Sleiman, DJ Savage, B Switzer,
C Chevalier and D Harris, The
BBN Task Force: A Trainee-
Led Communication Skills
Training Program for Teaching
Breaking Bad News); AMA GME
Innovations Summit, October
2020 (virtual conference) (DJ
Savage, J Sleiman, B Switzer,
C Chevalier and D Harris, The
BBN Task Force: A Trainee- Led
Communication Skills Training
Program for Teaching Breaking
Bad News).
Accepted 12 June 2021
Published Online First 22 June
2021
© Author(s) (or their
employer(s)) 2021. No
commercial re- use. See rights
and permissions. Published
by BMJ.
ABSTRACT
Background Breaking bad news (BBN) is a critically
important skill set for residents. Limited formal
supervision and unpredictable timing of bad news
delivery serve as barriers to the exchange of meaningful
feedback.
Purpose of study The goal of this educational
innovation was to improve internal medicine residents’
communication skills during challenging BBN encounters.
A formal BBN training programme and innovative on-
demand task force were part of this two- phase project.
Study design Internal medicine residents at a large
academic medical centre participated in an interactive
workshop focused on BBN. Workshop survey results
served as a needs assessment for the development of
a novel resident- led BBN task force. The task force was
created to provide observations at the bedside and
feedback after BBN encounters. Training of task force
members incorporated video triggers and a feedback
checklist. Inter- rater reliability was analysed prior
to field testing, which provided data on real- world
implementation challenges.
Results 148 residents were trained during the 2- hour
communications skills workshop. Based on survey results,
73% (108 of 148) of the residents indicated enhanced
confidence in BBN after participation. Field testing of
the task force on a hospital ward revealed potential
workflow barriers for residents requesting observations
and prompted troubleshooting. Solutions were
implemented based on field testing results.
Conclusions A trainee- led BBN task force and
communication skills workshop is offered as an
innovative model for improving residents’ interpersonal
and communication skills in BBN. We believe the model
is both sustainable and reproducible. Lessons learnt are
offered to aid in implementation in other settings.
PURPOSE OF THE PROOF-OF-CONCEPT STUDY
Effective communication is a core skill in clinical
practice and postgraduate training.1 Gaps in physi-
cians’ communication skills are often exposed
during difficult conversations.2 These gaps can
impact patient outcomes, with poor performance
leading to patient mistrust and decreased physician
satisfaction.3 4 The Accreditation Council for Grad-
uate Medical Education (ACGME) underscored
the importance of interpersonal and communica-
tion skills (ICS) when it included ICS among the
six core competencies in 1999.5 6 Communication
skills training for residents has been associated
with decreased patient anxiety and depression7
and increased hope8 when delivering bad news. A
systematic review showed that rigorous communi-
cation skills training can improve trainee- reported
confidence in delivering bad news as well as
observer- rated skills assessment.9
Direct observation of patient–physician commu-
nication, considered necessary within competency-
based educational frameworks,10 remains
underdeveloped during residency training11 and is
often difficult to implement.12 Specific challenges
include limited staff physician time, inflexible
schedules, unpredictable opportunities to deliver
bad news and variable abilities of staff physicians
to provide meaningful feedback.13–19 To address
educational barriers and increase opportunities
for residents to request direct observation, we
created a two- phase programme for teaching the
skills involved in breaking bad news (BBN): an
interactive workshop on BBN and a resident- led
‘task force’. The goals of this innovation were to
provide training on key communication skills and
to create a group of dedicated clinician- educators
(ie, task force) trained to observe BBN encounters
and provide feedback on- demand. In this paper, we
describe our educational innovation, including the
BBN task force, a proof- of- concept project.
Conceptual framework
Adult learning theory provided a conceptual frame-
work to guide development of the on- demand BBN
task force. According to adult learning theory,20
adult learners are thought to be self- directed and
intrinsically motivated to improve performance
when tasks are authentic and reflect real- world
demands. The focus on formative assessment and
a collegial relationship with task force observers is
also rooted in competency- based education.11
METHODS
Timeline and participants
This prospective project was implemented from
2019 to 2020 at Cleveland Clinic in Cleveland,
Ohio. Internal medicine (IM) residents (postgrad-
uate year, or PGY, 1–3, N=148) participated in the
BBN workshop. The initial task force consisted of
nine volunteers including senior residents, palliative
medicine fellows and palliative medicine faculty
members. Any IM resident on an inpatient primary
service could request a bedside observation.
569
Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
Context
All IM residents have clinical experiences with death and dying
due to the nature of clinical care (ie, usually high- acuity patients).
In addition, IM residents at our institution have two additional
teaching experiences related to this topic: Foundations of
Resident Assessment, Mentorship and Emotional Intelligence
(FRAME), which are monthly sessions dedicated to engaging
residents in ethical topics surrounding the life of a doctor.21
Death and dying is part of the FRAME curriculum, and DH
codirects this session. Second, during the first- year retreat, resi-
dents participate in a session dedicated to this topic as well. They
discuss their experiences with a psychologist. While all residents
engaged in both teaching sessions described above prior to the
project’s start, no resident had standardised patient experiences
focusing on communication skills during residency prior to this
project.
Measurement instruments
Pre- workshop and post- workshop surveys were created with
input from CYC, a social scientist with survey design experience.
The pre- workshop survey measured demographics and residents’
self- assessed skills in BBN. The post- workshop survey included
the same questions along with 10 items assessing the workshop
(online supplemental file 1).
Description of interventions
Communication skills workshop
This 2- hour, mandatory workshop introduced basic communica-
tion skills with a focus on using a ‘warning shot’ and delivering
bad news in a single sentence followed by silence and empathy.
The majority of the time was spent in small group skills practice.
DH taught and facilitated each of these sessions using a lesson
plan developed by our team (online supplemental file 2), which
integrated best practices in bad news delivery and physician
communication skills.22 Residents’ self- reported skills in BBN
were surveyed pre- workshop and post- workshop.
BBN task force formation and member training
The BBN task force was created in the fall of 2019 with support
from the ACGME’s Back to Bedside grant.23 Nine volunteers
formed the initial task force. Task force members participated in
three, 2- hour training sessions on BBN to patients and how to
provide meaningful feedback. Volunteers were taught best prac-
tices from adult learning theory and principles from small group
facilitation, which were applied to feedback on direct observa-
tions. During the final session, task force members participated
in frame- of- reference training using video triggers featuring bad
news delivery. For each video trigger, participants used a modi-
fied version of a previously developed checklist24 to rate the
quality of the conversation (online supplemental file 3). Between
video triggers, group discussions were used to reconcile differ-
ences in ratings among participants.
Bedside observations
The following protocol was developed for bedside observations:
►One task force member is ‘on- call’ to provide bedside obser-
vations during regular business hours.
►Residents are encouraged to schedule an observation with
the ‘on- call’ observer before an expected BBN encounter.
Of note, during the education of residents on the process of
scheduling an observation, some reluctance was expressed
in terms of being observed by a coresident, but not by a
palliative medicine fellow or an attending. Residents were
allowed to decline observations if they were not willing to be
observed by a known observer, especially if it was a resident.
►The task force member first discusses the case with the resi-
dent. Due to the on- demand style of requesting BBN obser-
vations, there is a chance of familiarity between the observer
and the resident during patient encounters. However, the
pre- planning of the observation includes the observer
informing the participant about what to expect in terms of
observation, that the observer’s purpose is to provide posi-
tive and constructive feedback, and that the observation
will not be shared with the IM programme for any compe-
tency assessment, in accordance with medical education
guidelines.12
►Prior to initiating the observation, patients are given an
institutional review board (IRB)- approved information sheet
by the observer (not the resident) to inform them that the
observer’s purpose is to assess the resident’s communica-
tion skills with patients. The observer explains that they are
not part of the patient care team and will not personally
participate in the conversation. Patients are given freedom
to decline. Patients were specifically not made aware that
this is a BBN type of conversation, as that would impact the
introductory piece of BBN for the resident; this action was
IRB- approved. The observer then observes the encounter
and provides verbal feedback to the resident.
►Formative assessments are performed via a modified check-
list24 and open- ended questions (online supplemental file 3).
Field testing
A quality assurance assessment of task force initiation (ie, field
testing) for a bedside observation was carried out in February
2020, prior to COVID-19 social distancing. Field testing allows
for the assessment of innovations, protocols, processes and
surveys under realistic conditions,25 26 with adjustments made
prior to formal implementation or launch. Field testing is critical
in assessing innovations prior to a significant investment of time
or expense.
Planned analysis
Descriptive statistics were run on demographic data. Survey
results were analysed via a two- sided paired t- test. The signif-
icance level was set at 0.05. For task force training, inter- rater
reliability was calculated for the first and second video using
average pairwise percentage agreement and Fleiss’ kappa for
nominal categories. An intraclass correlation coefficient (ICC)
was calculated to assess agreement among ratings for quantita-
tive data.
RESULTS
Communication skills workshop
A total of 148 PGY1–3 residents participated in the communica-
tion skills workshop. Sixty- five (44%) residents were female and
108 (72.9%) completed the feedback surveys. Prior to the work-
shop, 21 of 108 (19.4%) residents endorsed being either ‘not
skilled at all’ or marginally skilled when asked ‘How skilled are
you in communicating bad news to a family member or patient
about their illness?’ Post- workshop, only 5 of 108 (4.6 %) resi-
dents endorsed being marginally skilled in bad news delivery,
and the remainder of the residents said they were ‘somewhat
skilled’ or better. The mean score improved from 3.06 (2.25–
3.88) to 3.44 (2.79–4.08) (p=0.0004). When asked ‘How
helpful was this training to your overall communications skills
development as a resident?’, 50.9% of the residents answered
570 Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
‘very helpful’ and the average response was 4.35 (3.55–5). When
asked ‘How likely is it that this training will affect your commu-
nication of bad news to patients?’, 63 of 108 (59.4%) residents
endorsed ‘very likely’, with an average response of 4.43 (3.63–5)
(figure 1).
In narrative comments, residents were enthusiastic about the
classroom- based simulations (Box 1). Participants reported that
the training they received was applicable to their current prac-
tice of medicine and would improve future patient encounters.
They also expressed willingness to undergo a refresher training
in future years.
BBN task force recruitment and training
Seven of nine volunteers completed the 6- hour task force training
and expressed familiarity and confidence in using the bedside
scoring instrument. For nominal data, pairwise percentage
agreement revealed that participants were not consistent in their
ratings for some questions (eg, communication questions) while
very consistent in others (eg, ‘Discusses plan only after patient
asks to or gives permission to discuss next steps’). This was also
reflected by a low Fleiss’ kappa, which indicated poor inter- rater
reliability across raters on both videos (table 1). ICC, calculated
to assess reliability among ratings (table 2), showed that there
was poor correlation with video trigger 1 (ICC=0.12), but
moderate or good correlation with video trigger 2 (ICC=0.65).
Field testing
Due to COVID-19 social distancing restrictions, one bedside
observation was conducted during field testing of the task force
concept. The observed resident indicated that it was very easy to
request a task force observation, that he/she was likely to request
an observation in the future and that the task force was easy to
contact. The resident was ‘somewhat comfortable’ with having
an observer in the room and rated the likelihood of observation
improving future BBN practice as 4 out of 5 on a Likert scale.
In discussions with residents about the task force concept at
the end of the communication skills workshop and during subse-
quent announcements for the project, we did not encounter
reluctance about being observed by palliative medicine fellows or
attendings. Despite making residents aware that peer observers
are trained in proper observation and feedback techniques,
faculty/fellow task force observations were preferred over peers
due to perceptions that feedback would be of higher quality
(ie, objective and constructive). While our field testing did not
provide enough data to generalise about patient comfort with the
task force model, the patient involved in the first field test was
comfortable with the observer in the room, as they were used
to multiple providers during regular patient doctor encounters.
Subsequent task force requests
Task force paging and face- to- face observations were paused
due to social distancing requirements related to the COVID-19
pandemic.
DISCUSSION
The aim of this innovation project was to facilitate the transfer of
communication skills from the classroom to real patient encoun-
ters. We specifically focused on BBN, which physicians consis-
tently rate as an extremely important but infrequently taught
skill.27 This resident- led project included an interactive work-
shop to teach residents fundamental BBN skills and the creation
of an innovative task force for direct observation of real patient
encounters at the bedside. Our overall goal was for the task force
to be a trainee- led, sustainable model for teaching effective BBN
skills. By integrating task force observations into the clinical
workflow of trainees, we hoped to promote sustainability.
The communication skills workshop proved successful
as a primer for residents in BBN, but the bigger impact was
increasing resident confidence in requesting observations for
real patient conversations, as well as building rapport with their
future observers. Similarly, the curriculum creates an opportu-
nity to publicise the task force’s on- demand function and to get
a sense of residents’ willingness to participate. Among narrative
comments received from residents, a divide was sensed between
wanting to participate voluntarily and feeling the observations
should be mandatory (Box 1). This helped us strategise several
approaches to increase residents’ willingness to participate (see
the Generalisability section).
Although resource- intensive at 6 hours of total training time,
the formal task force training programme was crucial to ensuring
that residents were offered consistent, high- quality feedback
during bedside observations. While the use of video triggers
during frame- of- reference training was effective in improving
inter- rater reliability, inter- rater reliability remained inconsistent
during some elements of the video assessment. This parallels some
studies that have explored how raters’ differing assessments of
observed performances contribute to measurement error;28 other
studies have shown that rater training does not always improve
inter- rater reliability or accuracy of assessment.29 30 Given that
it is normal for individuals to employ different communication
approaches—and that the purpose of task force observations
was formative, low- stakes assessment—consistency across raters
did not carry the same weight as it would have for summative,
high- stakes assessments. That said, we provided open response
formats to gather raters’ comments and better understand the
reasoning processes which went into rater scoring. Discussions
during rater training regarding verbal and non- verbal feedback
to trainees will help us to improve future iterations of task force
training.
Our approach to task force training is in line with recommen-
dations from an international group of researchers.30 Although
they offer three differing perspectives on rater cognition to better
understand variability in assessment (eg, ‘assessor as trainable’,
‘assessor as fallible’ and ‘assessor as meaningful idiosyncratic’),
they all agree that faculty development is needed to enhance the
observation and assessment skills of raters.30 In addition, they
urge immediate attention to the deficiency in real- world practice
of observation- based assessments of undergraduate and post-
graduate medical trainees, which currently limits further under-
standing within this area of research.30
There have been many prior educational interventions
to improve BBN ability using didactic courses,31 simula-
tion,24 32 group discussions,33 online assessment34 and instruc-
tional videos.35 However, it was our use of a rigorously trained
task force of clinicians available for performing observations of
real patient encounters and then providing feedback that truly
made this intervention novel and potentially generalisable to
other institutions and training programmes. This field test expe-
rience and the lessons learnt from it will enable our residency
programme and others to offer meaningful communication skills
observations once the present COVID-19 pandemic subsides.
Limitations
This educational innovation was carried out at a single
institution where we had strong buy- in from the residency
programme’s leadership. The results may not be generalisable
571
Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
Figure 1 Pre- workshop and post- workshop survey results from the interactive BBN skills session that was attended by all categorical internal
medicine residents (n=108). BBN, breaking bad news.
PGY: postgraduate year
572 Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
to other contexts. In addition, it is possible that residents may
have had prior experiences with standardised patients related
to BBN training during medical school or on the clinical wards;
this may have impacted survey responses during communication
skills training. In addition, due to social distancing requirements
during the COVID-19 pandemic, we were unable to study the
effects of task force implementation longitudinally within our
residency programme.
Lessons learnt
Residents’ willingness to request observations was the biggest
barrier to maximising the benefit of this project and studying
its impact. Residents expressed reluctance about requesting BBN
observations due to perceived stress while being observed and
the disruption to their daily workflow. During field testing, we
identified additional challenges to full task force implementa-
tion. Based on resident feedback after the field test, we adjusted
aspects of the task force model to mitigate barriers to implemen-
tation (table 3).
►Pre- implementation challenges: During the implementation
process, we learnt about the need to convey our mission to
programme leadership in order to garner support. We also
learnt that residents truly need to appreciate the benefit of
this project in order to engage, given their prioritisation of
other medical opportunities that may relate better to their
career interests. Communicating with resident leaders and
coordinating with inpatient service attendings proved key
to approaching residents and gaining maximal participation.
►Dissemination: We presented at the monthly programme
town hall meetings to inform them about the ease of
requesting an observation and the low- stakes nature of the
observation process. Our residency programme was very
supportive of this project, which helped with launching the
interactive classroom curriculum and advertising the project.
►Resident encouragement: At monthly programme town hall
sessions, we advertised the BBN initiative and shared posi-
tive testimonials from residents by recognising a ‘Participant
of the Month’. All participants received a gift card for their
first observation and were eligible for educational credit
via a mini- CEX (mini- Clinical Evaluation Exercise) for
‘Conducting a Family Meeting’, which became part of their
training portfolios.
►Workflow integration: We simplified the process for
requesting observations. Prior to halting observations for
COVID-19, we contacted staff physicians for teaching teams
weekly to encourage them to request observations.
Sustainability and generalisability
Sustainability
This train- the- trainer model will create a cycle of trainee- led
sustainability and continued buy- in from residency programme
Box 1 Narrative comments from residents while
assessing the BBN inclass curriculum (question 10 on the
questionnaire)
►“I thought this was an excellent session. Despite being a
PGY3 that has had many difficult conversations and broken
lots of bad news, I still felt that my skills were improved by
this course.”
►“This was an excellent and helpful course. It will change my
practice for delivering bad news. Thank you!”
►“I wish I had it last year.”
►“I think a video simulation before the last exercise might
have helped put everyone in the zone for acting out and
getting the best experience.”
►“Really appreciate the tips and tricks, different phrasing, and
practice.”
►“This framework for delivering bad news is easy to remember
and helpful. Essential to every internist. Would definitely
recommend it.”
►“I think there should be more refresher courses throughout
the same PGY year because this topic is pretty relevant to our
training.”
►“Great session. Extremely important for patient care. Hope
this can lead to residents getting more chances to lead family
meetings and break bad news on inpatient services.”
►“If it was not mandatory, I would likely not attend, because
there is little free time, so I think we should make it
mandatory.”
BBN: breaking bad news; PGY: postgraduate year.
Table 1 Comparison of rater scores on yes/no question items after
observing standardised BBN videos during the task force training
Variable
Video 1 Video 2
% agree Kappa % agree Kappa
BBN
Q1 71 100
Q2 52 43
Q3 100 43
Q4 100 43
Q5 52 43
Q6 71 100
Q7 71 43
Q8 43 71
Q9 100 100
Overall 74 0.46 65 0.14
Communication
Q1 52 100
Q2 43 71
Q3 43 71
Q4 71 100
Q5 52 71
Overall 52 0.03 83 −0.09
BBN, breaking bad news.
Table 2 Comparison of rater scores on Likert- scaled question items
after observing standardised BBN videos during the task force training
Video 1 Video 2
Mean SD Range Mean SD Range
Performance
Q1 0.5 0.5 0–1 0.6 0.5 0–1
Q2 1.0 1.0 0–3 0.8 0.6 0–2
Q3 0.9 0.7 0–2 0.6 0.5 0–1
Q4 1.4 1.4 0–4 0.6 0.5 0–1
Q5 1.0 0.9 0–3 1.2 0.9 0–3
Q6 1.4 1.4 0–4 0.6 0.6 0–2
Overall 1.0 1.1 0–4 0.7 0.7 0–3
ICC 0.12 0.65
BBN, breaking bad news; ICC, intraclass correlation coefficient.
573
Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
leadership and chief residents. Specifically, it will require yearly
handoffs to more junior trainees to ensure sustainability over
time. We recommend designating a faculty member to assist in
the recruitment and training of new task force members. Ulti-
mately, we believe this programme will become a sustainable
element of our IM residency programme.
Generalisability
We believe that this model is applicable to other crucial commu-
nication skills and training programmes. A surgical residency, for
example, could use task force observations to improve resident
skills in taking surgical consent or describing an intraoperative
surgical complication to a family. Our intervention was imple-
mented at one of the largest residency programmes in the USA,
and the model may be easier to implement in a programme with
fewer trainees. When sharing this model with other programmes,
it is important to note the narrative comments received from
residents who said they felt it is better to make the curriculum
and observations mandatory (Box 1). From what we experienced,
residents willing to participate in either task force volunteering or
on- demand observations appeared to be internally motivated to
learn more about the BBN topic, or it matched their future career
focus. For residents who are less willing to participate, options
to enhance buy- in include making the observations a mandatory
minimal requirement for a graduation milestone related to the
ACGME’s ‘Interpersonal and Communication Skills’ competency,
or creating a give- and- take system whereby there is a reward for
participation (curriculum vitae title, mini- CEX accomplishment,
‘Participant of the Month’ award or monetary incentive).
CONCLUSIONS
Our resident- led BBN curriculum enhanced opportunities for
our trainees to learn critical communication skills and to receive
actionable feedback from real patient encounters. Develop-
ment of the BBN task force allowed our programme to provide
advanced training to task force members, preparing them to
observe difficult encounters, and provide feedback in a stan-
dardised way. The results of this proof- of- concept project indi-
cate that residents viewed the classroom curriculum favourably
and that our task force members could be standardised in their
approach to observation. This model, while narrowly focusing on
BBN skills building with IM residents, could easily be expanded
to other bedside communication skills and/or to other medical
or surgical specialties.
Author affiliations
1Internal Medicine Residency Program, Cleveland Clinic, Cleveland, Ohio, USA
2Department of Medicine, University of California San Diego Health System, San
Diego, California, USA
3Hematology/Oncology Fellowship Program, University at Buffalo, Buffalo, New York,
USA
4Center for Educational Resources, Education Institute, Cleveland Clinic, Cleveland,
Ohio, USA
5Cleveland Clinic Lerner College of Medicine, Case Western Reserve University,
Cleveland, Ohio, USA
6Department of Palliative Medicine and Supportive Care, Cleveland Clinic, Cleveland,
Ohio, USA
Twitter Joseph Sleiman @JosephHabibi_MD, David J Savage @DocDavidSavage,
Benjamin Switzer @BenSwitzerDO, Cory Chevalier @CoryChevalier, Kathleen
Neuendorf @KtNeuendorf and David Harris @DaveHarrisMD
Table 3 Summary of proposed project adjustments that were adopted after field testing in preparation for task force launching
Project obstacle Adopted solution
Initial difficulty with dissemination of project purpose and protocol for
requesting the task force.
►Advertise the task force during academic noon conferences and town hall.
►Creation of e- posters that simplify the request process into three- step model.
Residents’ reluctance to be assessed and evaluated by a co- resident. ►Allowed option to be observed only by fellow and/or palliative attending.
►Emphasising anonymity of the observation and feedback that would not be part of the programme
evaluation of the resident unless they request a mini- CEX evaluation.
Sustaining number of weekly requests. ►Creation of monthly ‘BBN resident of the month’ during town hall meeting to encourage others to
participate.
►Small monitory incentives allocated from the ACGME fund for residents who participate in the
observation.
COVID-19 pandemic limited the number of physicians in a room with a
patient per hospital protocol.
►Investigating virtual platforms that allow for proper observation of BBN by task force members.
►Limiting the activity to one single unit in the hospital during the pandemic.
Limited availability of a task member when requested. ►Organisation of a weekly ‘call schedule’ based on observers’ academic rotation.
►Increasing the number of trained task members if needed.
Reluctance by inpatient rotation attending to allow residents to perform
such requests.
►Weekly emails to the attendings on services with residents to explain how this task force does not
interfere with patient care.
►Open forum during quaternary faculty meetings to answer concerns that may arise.
Reluctance of the patient to participate in observation. ►IRB- approved script given to patient to explain the role of the observer.
►Confirmation by the caring resident that the observer has no role in the patient’s care and is only
available to give feedback to the resident’s communication skills.
ACGME, Accreditation Council for Graduate Medical Education; BBN, breaking bad news; IRB, institutional review board; mini- CEX, mini- Clinical Evaluation Exercise.
What is already known on this subject
►Breaking bad news (BBN) skills are critical to professional
development and patient care and should be formally taught
to residents.
►There are many examples of didactic skills training
programmes to teach effective BBN skills, but few are led
by trainees and none offers direct bedside observation and
feedback of real patient encounters.
What this study adds
►This study reports a trainee- led initiative to teach BBN skills
to residents using an interactive skills workshop and direct
bedside observations conducted by a task force.
►We report a proof of concept for providing standardised
direct bedside observation with feedback using an innovative
trainee- led task force.
574 Sleiman J, etal. BMJ Simul Technol Enhanc Learn 2021;7:568–574. doi:10.1136/bmjstel-2021-000897
Original research
Acknowledgements This project was prepared with support from the
Accreditation Council for Graduate Medical Education (ACGME) as part of the
Back to Bedside Initiative. We would also like to thank all of the physicians who
participated in the task force, including Bryce Montané, Kevin Harris, Ahed Makhoul,
Tiffany Onger and Sina Najafi.
Contributors JS, DJS and DH conceived of the project, wrote the grant proposal,
wrote the IRB, implemented the project, oversaw data collection, wrote the initial
manuscript and revised the manuscript. CYC contributed to IRB, aided in data
analysis and revised the manuscript. BS, KN and CC contributed to grant proposal,
assisted with project implementation and revised the manuscript.
Funding This study was supported by the Accreditation Council for Graduate
Medical Education (ACGME) (Back to Bedside Grant 2019-2021).
Disclaimer The content reflects the views of the authors who are the grant
recipients and does not purport to reflect the views of the ACGME or any member of
the Back to Bedside Initiative.
Competing interests None declared.
Ethics approval This project was approved by the Cleveland Clinic Institutional
Review Board after expedited review.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the
article or uploaded as supplementary information. A summary of all data for this
project was included with our submission. We are happy to share the raw data for
the didactic courses with any interested party.
ORCID iDs
JosephSleiman http:// orcid. org/ 0000- 0002- 7390- 5949
David JSavage http:// orcid. org/ 0000- 0003- 4690- 5115
Colleen YColbert http:// orcid. org/ 0000- 0002- 2608- 7218
REFERENCES
1 Alston C, Paget L, Halvorson GC, etal. Communicating with patients on health care
evidence. NAM Perspect 2012;2.
2 Bylund CL, Banerjee SC, Bialer PA, etal. A rigorous evaluation of an institutionally-
based communication skills program for post- graduate oncology trainees. Patient
Educ Couns 2018;101:1924–33.
3 Orlander JD, Fincke BG, Hermanns D, etal. Medical residents’ first clearly remembered
experiences of giving bad news. J Gen Intern Med 2002;17:825–40.
4 Tulsky JA, Beach MC, Butow PN, etal. A research agenda for communication between
health care professionals and patients living with serious illness. JAMA Intern Med
2017;177:1361.
5 Holmboe ES, Edgar L, McLean S. The milestone Guidebook. ACGME, 2020: 41.
6 Benson BJ. Domain of competence: interpersonal and communication skills. Acad
Pediatr 2014;14:S55–65.
7 Curtis JR, Back AL, Ford DW, etal. Effect of communication skills training for residents
and nurse practitioners on quality of communication with patients with serious illness:
a randomized trial. JAMA 2013;310:2271–81.
8 Smith TJ, Dow LA, Virago E, etal. Giving honest information to patients with advanced
cancer maintains hope. Oncology 2010;24:521–5.
9 Johnson J, Panagioti M. Interventions to improve the breaking of bad or difficult news
by physicians, medical students, and Interns/Residents: a systematic review and meta-
analysis. Acad Med 2018;93:1400–12.
10 Carraccio C, Wolfsthal SD, Englander R, etal. Shifting paradigms: from Flexner to
competencies. Acad Med 2002;77:361–7.
11 Holmboe ES, Sherbino J, Long DM, etal. The role of assessment in competency- based
medical education. Med Teach 2010;32:676–82.
12 Kogan JR, Hatala R, Hauer KE, etal. Guidelines: The do’s, don’ts and don’t knows
of direct observation of clinical skills in medical education. Perspect Med Educ
2017;6:286–305.
13 Witteles RM, Verghese A. Accreditation Council for graduate medical education
(ACGME) Milestones- Time for a Revolt? JAMA Intern Med 2016;176:1599.
14 Madan R, Conn D, Dubo E, etal. The enablers and barriers to the use of direct
observation of trainee clinical skills by supervising faculty in a psychiatry residency
program. Can J Psychiatry 2012;57:269–72.
15 Carraccio C, Englander R, Van Melle E, etal. Advancing competency- based medical
education: a charter for clinician- educators. Acad Med 2016;91:645–9.
16 Watling C. Cognition, culture, and credibility: deconstructing feedback in medical
education. Perspect Med Educ 2014;3:124–8.
17 Dath D, Iobst W. The importance of faculty development in the transition to
competency- based medical education. Med Teach 2010;32:683–6.
18 Watling C. The uneasy alliance of assessment and feedback. Perspect Med Educ
2016;5:262–4.
19 Frank JR, Snell LS, Cate OT, etal. Competency- Based medical education: theory to
practice. Med Teach 2010;32:638–45.
20 Knowles MS, Holton EF, Swanson RA. The adult learner : the definitive classic in
adult education and human resource development, 2015. Available: https:// public.
ebookcentral. proquest. com/ choice/ publicfullrecord. aspx? p= 1883897
21 Brateanu A, Switzer B, Scott SC, etal. Higher Grit scores associated with less burnout
in a cohort of internal medicine residents. Am J Med Sci 2020;360:357–62.
22 Curtis JR, Engelberg RA, Wenrich MD, etal. Studying communication about end- of- life
care during the ICU family conference: development of a framework. J Crit Care
2002;17:147–60.
23 Hipp DM, Rialon KL, Nevel K, etal. "Back to Bedside": Residents’ and Fellows’
Perspectives on Finding Meaning in Work. J Grad Med Educ 2017;9:269–73.
24 Vermylen JH, Wood GJ, Cohen ER, etal. Development of a simulation- based mastery
learning curriculum for breaking bad news. J Pain Symptom Manage 2019;57:682–7.
25 Livingston A. The condition of education 2002; 2002.
26 Deitz D, Dowell RN, Madigan EA, etal. OASIS- C: development, testing, and release.
An overview for home healthcare clinicians, administrators, and policy makers. Home
Healthc Nurse 2010;28:353–62.
27 Monden KR, Gentry L, Cox TR. Delivering bad news to patients. Proc 2016;29:101–2.
28 Gingerich A, Ramlo SE, van der Vleuten CPM, etal. Inter- rater variability as mutual
disagreement: identifying raters’ divergent points of view. Adv in Health Sci Educ
2017;22:819–38.
29 Cook DA, Dupras DM, Beckman TJ, etal. Effect of rater training on reliability and
accuracy of mini- CEX scores: a randomized, controlled trial. J Gen Intern Med
2009;24:74–9.
30 Gingerich A, Kogan J, Yeates P, etal. Seeing the ’black box’ differently: assessor
cognition from three research perspectives. Med Educ 2014;48:1055–68.
31 Ungar L, Alperin M, Amiel GE, etal. Breaking bad news: structured training for family
medicine residents. Patient Educ Couns 2002;48:63–8.
32 Karkowsky CE, Landsberger EJ, Bernstein PS, etal. Breaking Bad News in obstetrics:
a randomized trial of simulation followed by debriefing or lecture. J Matern Neonatal
Med 2016;29:3717–23.
33 Bradley CT, Webb TP, Schmitz CC, etal. Structured teaching versus experiential
learning of palliative care for surgical residents. Am J Surg 2010;200:542–7.
34 Rat A- C, Ricci L, Guillemin F, etal. Development of a web- based formative self-
assessment tool for physicians to practice breaking bad news (BRADNET). JMIR Med
Educ 2018;4:e17.
35 Silva DH. A competency- based communication skills workshop series for pediatric
residents. Bol Asoc Med P R 2008;100:8–12.
