Content uploaded by Marjel van Dam
Author content
All content in this area was uploaded by Marjel van Dam on Aug 18, 2021
Content may be subject to copyright.
Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Supervision, Interprofessional Collaboration,
and Patient Safety in Intensive Care Units
during the COVID-19 Pandemic
Marije P. Hennus
1
, John Q. Young
2
, Martina Hennessy
3
, Karen A. Friedman
2
, Bas de Vries
1
, Reinier G. Hoff
1
,
Enda O’Connor
4
, Aileen Patterson
3
, Gerard Curley
5
, Krima Thakker
2
, Marjel van Dam
1
, Diederik van Dijk
1
,
Wilton A. van Klei
1
, and Olle ten Cate
1
1
University Medical Center Utrecht, Utrecht, the Netherlands;
2
Donald and Barbara Zucker School of Medicine at
Hofstra/Northwell, New York, New York;
3
School of Medicine, Trinity College Dublin, Dublin, Ireland;
4
Department of
Anesthesia and Critical Care, St. James’s Hospital, Dublin, Ireland; and
5
Department of Anesthesia and Critical Care,
Beaumont Hospital, Dublin, Ireland
ORCID IDs: 0000-0003-1508-0456 (M.P.H.); 0000-0003-2219-5657 (J.Q.Y.); 0000-0002-2153-5288 (M.H.); 0000-0003-
1980-1839 (K.A.F.); 0000-0002-7432-7087 (R.G.H.); 0000-0003-4315-0917 (A.P.); 0000-0003-0271-195X (G.C.); 0000-0001-
9802-7516 (M.v.D.); 0000-0002-3592-4671 (D.v.D.); 0000-0003-2665-1672 (W.A.v.K.); 0000-0002-6379-8780 (O.t.C.).
ABSTRACT
Background: To meet coronavirus disease (COVID-19) demands in the spring of
2020, many intensive care (IC) units (ICUs) required help of redeployed personnel
working outside their regular scope of practice, causing an expansion and change of
staffing ratios.
Objective: How did this composite alternative ICU workforce experience supervision,
interprofessional collaboration, and quality and safety of care under the unprecedented
clinical circumstances at the height of the first pandemic wave as lived experiences
uniquely captured during the first peak of the pandemic?
Methods: An international, cross-sectional survey was conducted among physicians,
nurses, and allied personnel deployed or redeployed to ICUs in Utrecht, New York,
and Dublin from April to May of 2020. Data were analyzed separately for the three
sites. Quantitative data were treated for descriptive statistics; qualitative data were ana-
lyzed thematically and combined for general interpretations.
Results: On the basis of 234, 83, and 34 responses (response rates of 68%,48%, and
41%in Utrecht, New York, and Dublin, respectively), we found that the amount of
supervision and the quality and safety of care were perceived as being lower than usual
(Received in original form December 2, 2020; accepted in final form April 22, 2021)
This article is open access and distributed under the terms of the Creative Commons Attribution
Non-Commercial No Derivatives License 4.0. For commercial usage and reprints, please e-mail
Diane Gern.
Copyright © 2021 by the American Thoracic Society
DOI: 10.34197/ats-scholar.2020-0165OC
| Hennus, Young, Hennessy, et al.: LEAP Study 1
ORIGINAL RESEARCH
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
but still acceptable. The working atmosphere was overwhelmingly felt to be
collaborative and supportive. Where IC-certified nurse–to–patient ratios had decreased
most (Utrecht), nurses voiced criticism about supervision and quality of care. Continuity
within the work environment, team composition, and informal (“curbside”) consulta-
tions were critical mediators of success.
Conclusion: In the exceptional circumstances encountered during the COVID-19
pandemic, many ICUs were managed by a composite workforce of IC-certified and
redeployed personnel. Although supervision is critical for safe care, supervisory roles
were not clearly related to the amount of prior ICU experience. Vital for satisfaction
with the quality of care was the span of control for those who assumed supervisory roles
(i.e., the ratio of certified to noncertified personnel). Stable teams that matched less
experienced personnel with more experienced personnel; a strong, interprofessional,
collaborative atmosphere; a robust culture of informal consultation; and judicious,
more flexible use of rules and regulations proved to be essential.
Keywords:
supervision; quality and safety of care; interprofessional collaboration; COVID-19;
intensive care
The coronavirus disease (COVID-19)
pandemic has affected populations
differently across times and geographic
locations. On May 10, 2020, the World
Health Organization reported over 3.9
million cases and 274,000 deaths
worldwide. On that day, the Netherlands,
the Unites States, and Ireland reported
311, 244, and 304 deaths per million
inhabitants, respectively, excluding
undocumented cases (1). These countries
were among the many that had been
forced to adapt healthcare systems rapidly
in response to the pandemic in April.
Major bottlenecks included shortages of
intensive care (IC) unit (ICU) beds,
materials, and qualified personnel (2, 3).
Typically, qualification for practice
requires proper licensing, specialty
certification, and hospital credentialing or
Author Contributions: M.P.H. is the guarantor of the content of the manuscript, including the data
and analysis. M.P.H. and O.t.C. conceived the study, designed the questionnaire, and
collaboratively wrote the first and final versions of the manuscript. M.P.H., O.t.C., J.Q.Y., M.H.,
K.A.F., and A.P. organized the survey administration; collected and analyzed data from the
Utrecht (M.P.H., O.t.C.), New York (J.Q.Y., K.A.F.), and Dublin (M.H., A.P.) samples; and critically
reviewed all versions of the manuscript for important critical contributions, in writing and in
multiple video conferences. B.d.V., R.G.H., and M.v.D. contributed to the Utrecht questionnaire and
organized data collection in the clinical workplace. D.v.D. and W.A.v.K. reviewed the questionnaire
and the final report and enabled participation of all Utrecht personnel. K.T. analyzed New York
data. E.O’C. and G.C. enabled participation of all personnel in Dublin. All authors reviewed
multiple versions of the manuscript and approved the final version to be published.
Correspondence and requests for reprints should be addressed to Marije P. Hennus, M.D., Ph.D.,
M.Sc., University Medical Center Utrecht, Post Office Box 85090, 3508 AB Utrecht, the Netherlands
E-mail: m.p.hennus@umcutrecht.nl.
This article has a data supplement, which is accessible from this issue’s table of contents at
www.atsjournals.org.
ORIGINAL RESEARCH
2Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
participation in a relevant training
program with sufficient supervision. In
regions experiencing a surge of patients
requiring hospitalization, and particularly
ICU admission, the COVID-19 pandemic
disrupted the usual criteria for the scope
of practice (4, 5). Rules of permission and
privileging to perform specialized medical
and nursing tasks became malleable.
Numerous healthcare workers, including
primary care physicians, ambulatory
center–based medical specialists (e.g., der-
matologists, rheumatologists), surgeons,
psychiatrists, pediatricians, pediatric IC
nurses, retired healthcare workers, and
final-year medical students graduating
early, were mandated or volunteered
themselves to care for patients in the
ICUs (6, 7). ICUs can be dynamic, com-
plex, and highly stressful work environ-
ments, where three features are critical for
success: adequate supervision, interprofes-
sional collaboration, and patient safety (8).
“Supervision,”defined as the provision of
guidance and support in learning and
working effectively in health care by
observing and directing the execution of
tasks or activities, and ensuring that these
are done correctly and safely, from a
position of being in charge (9), is a
responsibility that should not be taken
lightly. A supervisor must be able to make
ad hoc entrustment decisions for those
working under their responsibility and
gauge supervisees’capacities. Providing
supervision or being supervised forms an
integral part of training and daily practice
for most physicians. Most IC-certified
physicians (attending physicians, fellows)
have experience in providing supervision
of non–IC-certified physicians (residents,
students). That does not hold true for
nursing. Most registered nurses have been
supervised during the years in training but
are generally less accustomed to providing
supervision. That role is usually reserved
for a limited number of registered nurses
specifically trained for and officially
appointed to supervisory positions. During
the extraordinary circumstances of the
COVID-19 pandemic, however, most
IC-certified nurses were asked to supervise
redeployed, non–IC-certified personnel, in
addition to providing regular patient care
for their own patients.
Building trust in the capacities of
redeployed workers, during times such as
the COVID-19 crisis, requires familiarity
with their skill levels combined with at
least some experience of working
together. Supervision is intimately related
to the concept of “entrustment.”Deci-
sions to trust learners with tasks in health
care now often pertain to “entrustable
professional activities”(EPAs) (10). Super-
vision for these EPAs—units of profes-
sional practice that can be entrusted to a
trainee once he or she has demonstrated
adequate competence—has been framed
as five levels of supervision that are
applied according to what is adequate for
a learner (see Table E1 in the data supple-
ment),which is determined on the basis of
a learner’s stage of development (4,
10–12). In regular training circumstances,
the progression of trainees in EPA-based
programs is determined by clinical com-
petency committees on the basis of the
reported experiences with the learner, as
collected in a portfolio and in other data
sources, to arrive at valid, summative
entrustment decisions to advance learners
and formally decrease required supervi-
sion levels for EPAs (13). The COVID-19
pandemic precluded regular processes and
evoked the question of how the new mix
of experienced personnel and unexper-
ienced learners navigated acceptable
demands and provisions of supervision.
An adequate ratio of supervisors to
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 3
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
coworkers, both for physicians and nurs-
ing staff, is key to not only quality of care
but also the safety of both the patient and
the learner.
“Interprofessional collaboration,”another
key element, describes the active and
ongoing partnership among professionals
from diverse backgrounds who have
distinctive professional cultures, who possibly
represent different organizations or sectors,
and who work together to provide services
for the benefit of patients (14). Evidence
suggests that effective interprofessional
collaboration results in improved outcomes
for ICU patients (15).
“Patient safety,”a priority in health care
since the 1980s (16, 17), not only requires
avoidance of errors but also relies on an
intelligent environment and a cooperative
culture that facilitates the safe delivery of
acute care (18).
To meet COVID-19 demands in the
spring of 2020, ICUs employed several
strategies to increase capacity. In particu-
lar, they decreased nursing staffing ratios
and brought new staff into the ICU. Ade-
quate supervision, targeted for personnel
with varying degrees of preparedness, is
key to maintaining a basic level of care
quality and safety. The purpose of this
study was to understand the lived experi-
ences of delivering care during the height
of the first pandemic peak. Specifically, we
wanted to understand 1) the perceived
quality of supervision; 2) the perceived
quality of patient care provision, as
reported by physicians and nurses of vari-
ous backgrounds and amounts of experi-
ence; and 3) how interprofessional
collaboration was perceived, with the pur-
pose of generating recommendations for
similar situations in the future. We aimed
to report quantitative data summarizing
supervision practices and qualitative data
describing the lived experience of working
in new team structures during the pan-
demic. As this was not a local crisis in one
hospital or country, we sought collabora-
tion among three institutions in similar
circumstances in different countries with
the aim of drawing generalized
conclusions.
METHODS
This international cross-sectional survey
was conducted between April 16, 2020,
and May 15, 2020, among healthcare pro-
fessionals deployed at ICUs at the Univer-
sity Medical Center (UMC) Utrecht, the
Netherlands; two of Northwell Health’s
teaching hospitals (Long Island Jewish
Medical Center and North Shore Univer-
sity Hospital) in New York, New York,
United States; and two Dublin hospitals
(St. James’s Hospital, affiliated with Trin-
ity College Dublin, and Beaumont Hospi-
tal, affiliated with the Royal College of
Surgeons Ireland). Ethical approval was
waived by the medical ethical review
board of UMC Utrecht (No. 20-216/C);
granted by the institutional review board
at Northwell Health; and approved by the
National COVID Research Ethics Com-
mittee in Dublin.
Settings and Participants
Utrecht
.Between April 16 and April 23,
data were collected from three ICUs (two
of which were newly created after March
18) at UMC Utrecht, a 1,042-bed hospital
including 24 ICU beds. As COVID-19
admissions increased, the ICU-bed capac-
ity was expanded to 80, superseding other
hospital functionalities. The ICUs were
run by a limited number of certified IC
personnel and supplemented with physi-
cians and nurses recruited from neighbor-
ing health professions, resulting in an
IC-certified nurse–to–patient ratio of
(rounded) 1:3 (normally 1:1). A mixture of
ORIGINAL RESEARCH
4Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
voluntary and mandated redeployed pro-
fessionals received 1 day of IC-specific
prior training. This training included lec-
tures on COVID-19, lectures on admitting
patients to an ICU and ICU logistics, a
guided ICU tour, an introduction to Peer
Support, and a practical skills carousel
(donning personal protection equipment,
prone positioning, collecting blood sam-
ples from arterial lines, etc.). On the basis
of the week schedule, 34 IC physicians
and 24 non-IC physicians of various spe-
cialties (including residents) as well as 172
IC-certified and 112 non–IC-certified
nurses (including regular ward and scrub
nurses, anesthesiology technicians, physi-
cian assistants, and pharmacy assistants)
were eligible for inclusion (Table 1). Par-
ticipation was voluntary.
New York
.Between April 16 and May 5,
data were collected at Northwell Health, a
24-hospital system in the New York met-
ropolitan region. Two of its teaching hos-
pitals participated in this study, Long
Island Jewish Medical Center and North
Shore University Hospital. Combined, the
two hospitals typically have 1,256 beds,
Table 1. Surveyed population and respondents
Surveyed Respondents Response* (%) Volunteered
†
(%)
Utrecht respondents
Physicians, IC-certified 34 14 41.2 25.0
Physicians, not IC-certified 24 20 83.3 40.0
Nurses, IC-certified 172 79 45.9 43.1
Nurses, not IC-certified
‡
112 121 108.0 47.2
Total 342 234 68.4 43.8
New York respondents
Residents, not IC-certified 52 35 67.3 28.6
Fellows, not IC-certified 25 14 56.0 42.9
Attendings, not IC-certified 114 44 38.6 47.7
Total 191 93 48.7 39.8
Dublin respondents
Physician, IC-certified 25 13 52.0 —
Nurses, IC-certified
§
118
12
38.9
—
Nurses, not IC-certified
§
34 —
Total 143 59 41.2 —
Total respondents —386 ——
Definition of abbreviation: IC = intensive care.
*Response rates are approximations based on the best available data.
†
In Dublin, people were rostered according to skill level, with an option to opt out in liaison with the site
manager or with a derogation.
‡
Not all scheduled non–IC-certified, trained nurses in Utrecht were properly registered.
§
In Dublin, IC-certified and non–IC-certified nurses could not be differentiated for eligibility
to participate.
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 5
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
including 145 ICU beds. From March 16,
2020, the total number of patients with
COVID-19 in the ICUs increased, requir-
ing a peak ICU-bed capacity of 333
(130%increase). During this time, 52 resi-
dents, 25 fellows, and 114 attending physi-
cians were redeployed to the ICUs. All of
those who were redeployed did not typi-
cally practice in either setting. Redeploy-
ment was a mixture of voluntary and
mandated redeployment. All 191 rede-
ployed residents, fellows, and attending
physicians were invited to participate in
the survey (Table 1).
Dublin
.Data were collected between May
8 and May 15 at the two Dublin
university teaching hospitals that
participated in the study, St. James’s
Hospital and Beaumont Hospital. These
have a combined acute inpatient capacity
of 1,340 beds and had a pre–COVID-19
ICU-bed capacity of 56 beds (31 at St.
James’s and 25 at Beaumont). From
March 13 to May 15, 2020, the number
of patients with COVID-19 increased,
requiring a peak capacity of 171%and
192%, respectively. Additional ICU beds
were created by using alternative critical
care areas, such as cardiac care units, neu-
rologic ICUs, cardiothoracic units, and
operating theater recovery suites across
both hospitals. On the basis of the week
schedule, 23 IC consultant physicians and
trainees and 120 nurses (IC-certified
nurses plus nurses redeployed from regular
wards, specialized units, or operating thea-
ters) were invited to participate in the sur-
vey (Table 1).
Instrument
This study was conceived and performed
during the first peak of the COVID-19
pandemic in the spring of 2020. Within
the constraints placed by UMC Utrecht’s
Medical Ethics Committee and Outbreak
Management Team, a concise, one-page
questionnaire was created. After a litera-
ture search that did not yield a similar
useful survey, the questionnaire was devel-
oped using six steps of a recommended
seven-step process for designing high-
quality questionnaires (19); the last step of
pilot testing was not included because of
time constraints. The survey was initially
designed by a pediatric intensivist educa-
tor (M.P.H.) who worked together with an
experienced health profession education
researcher (O.t.C.), and it was critically
reviewed by an anesthesiology educator
and an intensivist educator (R.G.H. and
M.v.D). The resulting survey was reviewed
and adapted by the Dutch author team.
This version was pretested by using think-
aloud cognitive interviewing with four
qualified pediatric ICU nurses, two pediat-
ric ICU residents, and one ICU physician,
who led minor textual revisions. Finally,
the questionnaire was translated into
English, reviewed, and amended for use in
New York and Dublin (J.Q.Y. and M.H.).
The New York site decided to add an
item on comparison with baseline care
after the Utrecht and Dublin surveys had
been consolidated. See Figure E1 in the
data supplement for questionnaire items
from the Utrecht version of the survey.
The survey was conceived in Dutch and
translated, and sites (J.Q.Y. and M.H.)
made adaptations to meet local needs and
distributed the surveys either on paper
(Utrecht and Dublin) or online (the
research platform Research Electronic
Data Capture [National Institutes of
Health] was used in New York). The
questionnaire included items on demo-
graphics, the voluntary versus mandated
nature of the work, and the perceived
amount and quality of supervision, the
perceived quality of care, and the percep-
tion of interprofessional collaboration as
assessed by using 5-point Likert scales
ORIGINAL RESEARCH
6Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
(20). For supervision, we asked the follow-
ing questions: 1)“How often (1 = ‘never’
to 5 = ‘almost always’) have you worked at
each of five levels of supervision (i.e.,
received this supervision) in the week
before the survey?”and 2)“How did you
evaluate the supervision provided
(1 = ‘poor’to 5 = ‘excellent’)?”Similarly,
participants were asked to evaluate the
perceived quality of both care and inter-
professional collaboration (1 = “very poor”
to 5 = “very good”and 1 = “insufficient”
to 5 = “excellent,”respectively). Open-
space boxes were added to provide “tops”
(“What went well?”), “tips”(“What could
be improved?”), and additional comments.
Ethical Considerations and Data
Management Plan
All participants received information and
actively gave written consent to
participate. All data were collected
anonymously, saved on local network
drives in secure locations, and only shared
among the core author team. In Utrecht,
paper surveys were used for logistical
reasons. Participants were to deposit their
completed surveys in one of the secured,
closed-off boxes strategically placed in the
ICUs. These boxes were emptied on mul-
tiple days by one of the authors (M.P.H.).
After being scanned and saved to local
network drives in secure locations for data
entry, paper surveys were destroyed to
ensure anonymity. Both other sites used
electronic surveys.
Data Analysis
For quantitative analysis, frequencies,
means, and variances (standard deviation
and 95%confidence interval) were
calculated by using Microsoft Excel
(2016). As an observational study, no tests
of significance were performed. Data were
broken down by groups and/or clinical
settings differently by location. Open-text-
box (qualitative) data were thematically
analyzed by using the established six-step
process of thematic analysis as described
by Kiger and Varpio (21). In short, com-
ments were independently categorized by
multiple researchers at each location
(Utrecht: M.P.H. and O.t.C., New York:
J.Q.Y. and K.A.F.; Dublin: M.H. and
A.P.). Recurrent comments were identified
with initial codes and led to general
themes, which were all tabulated by using
Microsoft Excel (2016). Differences were
resolved by discussion at each location.
Next, the themes were compared among
the locations and merged into a limited
number of broad themes.
RESULTS
Guided by the survey questions and on
the basis of the qualitative data, we
structured the results into five themes:
supervision experienced; perceived quality
and safety of care; collaboration,
communication, and atmosphere;
scheduling and team composition; and
organization and facilities. The latter three
theme results are based on qualitative
data only.
Population and Response
Utrecht had 234 respondents (response
rate of 68%), including physicians and
nurses, most of whom (66%) were not
IC-certified. New York had 93 physician
respondents (response rate of 49%),
including attending physicians, fellows,
and resident physicians, none of whom
were IC-certified. Dublin had 59 respond-
ents (response rate of 45%), including only
trained or trainee IC physicians and a mix
of IC-certified and non–IC-certified
nurses. A minority across the settings
(34–44%) had volunteered to work on a
COVID-19 unit; others were asked or
scheduled. (Table 1)
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 7
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Table 2. Percentage of respondents acting often to always (score 4 or 5) under a supervision level of 1–5
Utrecht New York Dublin
Medical Nurse Resident Fellow Attending Medical Nurse
Not
IC-certified
(n=20)
IC-certified
(n=14)
Not
IC-certified
(n=121)
IC-certified
(n=79)
Not
IC-certified
(n=35)
Not
IC-certified
(n=14)
Not
IC-certified
(n=44)
IC-certified
(n=12)
Not
IC-certified
(n=34)
IC-certified
(n=13)
Level 1: to only observe new tasks, not
perform them (observe)
20.0 0.0 24.4 4.2 50.0 0.0 18.4 23.1 15.2 36.4*
Level 2: to perform new tasks with a
more experienced colleague
physically present (direct
supervision)
30.0 21.4 42.5 32.9 46.4 0.0 10.5 30.8 42.4 27.3
Level 3: to perform new tasks with a
more experienced colleague
available but not physically present
(indirect supervision)
45.0 42.9 44.5 52.9 67.8
†
30.8 36.8 38.5 30.3 54.5
Level 4: to perform new tasks without
supervision (no supervision)
35.0 50.0 26.3 60.9 17.9 46.2 39.5 30.8 15.2 50
Level 5: to act as a supervisor for less
experienced colleagues (supervise)
15.8 71.4 9.5
‡
71.1 17.9 46.2 55.3 61.5 12.5 54.5
In the past week, how often was the
amount of supervision lower than
needed?
————1.8 0.0 1.9 —— —
In the past week, how did you
experience the supervision you
received?
§
4.1 (0.7–0.3) 3.7 (1.0–0.5) 3.7 (0.8–0.2) 3.2 (1.1–0.3) 4.0 (1.0–0.3) 3.3 (1.3–0.8) 4.2 (0.9–0.3) 4.5 (1.0–0.5) 3.7 (1.2–0.4) 4 (1.2–0.7)
Definition of abbreviation: IC = intensive care.
A score 4–5 means “In the past week, I have often or always experienced this level of supervision”(level 1, 2, 3, 4, or 5).
*One of three examples on how to interpret the table: 36.4%(n= 4) of the 13 Dublin IC-certified nurses never performed tasks (as they were likely superfluous).
†
One of three examples on how to interpret the table: 67.8%of the 35 New York residents stated that they often or always performed tasks under indirect supervision.
‡
One of three examples on how to interpret the table: 9.5%of the non–IC-certified Utrecht nurses stated that they often or always provided supervision (as they were likely needed).
§
1 = insufficient; 2 = mediocre; 3 = sufficient; 4 = good; 5 = excellent. Data are means (95%confidence intervals).
ORIGINAL RESEARCH
8Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Survey Results: Quantitative
Supervision
.The condensed results of the
questions related to supervision are
summarized in Table 2. Generally, much
of the work was performed under
“indirect supervision”(i.e., a supervisor
was not present but was quickly available
if needed) for all categories of personnel,
even among those with IC certification.
Direct supervision (i.e., a supervisor being
physically present) was common among
non–IC-certified personnel, with the
exception of redeployed fellows and
attending physicians who operated more
independently (New York). Most non–IC-
certified personnel (except fellows, New
York) reported regular instances of
“observing only,”and some IC-certified
personnel reported this as well (Dublin).
All IC-certified personnel and redeployed
fellows and attending physicians regularly
acted in a supervisory role themselves.
The perceived quality of supervision was,
in general, reported to be “good”across
all categories of personnel. However,
closer analysis revealed that IC-certified
personnel in Utrecht, nurses in particular,
were less satisfied with the supervision
provided. More than 15%of physicians
and 20%of nurses in Utrecht found
supervision to be “mediocre”or “poor.”
These respondents included both
individuals who received and individuals
who provided supervision. Emerging
supervisory roles were not always clearly
linked to prior IC experience. In the
largest sample (Utrecht), 86 respondents
qualified themselves as “not IC-certified or
experienced.”Nevertheless, 30 of them
(35%) indicated that they regularly, often,
or always acted in a supervisory role.
Conversely, 79 qualified themselves as
“IC-certified or experienced,”but 23
(29%) of them indicated that they
regularly, often, or always worked under
direct supervision.
Quality of care
.At all three sites, the quality
of care provided was perceived as ranging
from being acceptable to being good (with
means of 3.3–4.3 on a 5-point scale across
all groups) (see Table 3). New York
respondents were also asked to compare
the quality of care with baseline standards
and judged that less favorably (with means
of 2.4, 2.4, and 2.5 for residents, fellows,
and attending physicians, respectively) (see
Table 3).
Interprofessional collaboration
was generally
experienced as being better (means .3.0)
than usual in the regular working
environment at all three sites (see Table 3).
Survey Results: Qualitative
In total, 335 (Utrecht), 193 (New York),
and 292 (Dublin) comments were made
by 211, 83, and 51 participants,
respectively. Thirteen (Utrecht), 6 (New
York), and 15 (Dublin) general themes
emerged for categorization of comments.
The contents aligned in many respects
across locations, and it was not difficult to
consolidate the qualitative results into the
five broad themes discussed below.
Table 4 shows frequencies, summaries,
and examples of comments.
Supervision in Utrecht was generally
reported as being acceptable but
insufficient for the quality of care at a
tertiary level ICU. In addition,
supervision was experienced as being
quite diluted at the height of the surge.
IC-certified nurses regularly commented
that too many inexperienced helpers
required too much direct supervision,
which negatively impacted their available
time for regular patient care. In contrast,
IC-certified personnel in Dublin (where
ICU capacity was stretched less than in
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 9
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Utrecht) reported a high degree of satis-
faction with supervision. Although formal
supervision in New York was experienced
as being reduced, respondents described
“curbside”consultations as more frequent
and accessible, compensating for formal
supervision.
The quality and safety of care were generally
perceived as being lower than at baseline
but as being acceptable at all three sites.
Table 3. Mean score (plus standard deviation) for quality of care and
interprofessional collaboration
Overall quality of
care* and
interprofessional
collaboration
†
UNYD
Care
Quality Collab.
Care
Quality Collab.
Care
Quality Collab.
Medical: not
IC-certified
(number of
respondents:
U = 20, D = N/A)
4.1 (0.4) 4.2 (0.7) ————
Medical:
IC-certified
(number of
respondents:
U = 14, D = 12)
3.9 (0.4) 3.6 (1.1) ———3.8 (0.8)
Nurses: not
IC-certified
(number of
respondents:
U = 121, D = 34)
4.2 (0.5) 3.4 (0.8) ——4.3 (1.3) 3.3 (1.0)
Nurses: IC-certified
(number of
respondents:
U = 79, D = 13)
3.4 (0.7) 3.4 (1.0) ———3.8 (1.0)
Residents: not
IC-certified
(number of
respondents:
NY = 35)
—— 3.7 (1.0) 3.3 (1.1) ——
Fellows: not
IC-certified
(number of
respondents:
NY = 14)
—— 3.3 (0.9) 3.8 (1.2) ——
Attending
physicians: not
IC-certified
(number of
respondents:
NY = 44)
—— 4.2 (0.8) 4.0 (0.9) ——
Definition of abbreviations: collab. = collaboration; D = Dublin; IC = intensive care; N/A = not applicable;
NY = New York; U = Utrecht.
*Overall quality of care (1 = very poor to 5= very good [Utrecht/Dublin scale transformed from 10 to
5-point scale]).
†
Quality of the interprofessional collaboration, compared with baseline (1 = far worse to 5 = far better).
ORIGINAL RESEARCH
10 Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Table 4. Thematically organized qualitative findings
Theme Salient Findings Summarized Quotes from Participants
Supervision (numbers of
comments*: U: 13+, 292;
NY: 23+, 92; D: 3+, 02)
Supervision was considered to be acceptable but
insufficient to deliver the desired quality of care in a
regular ICU. Many nurses with IC experience (U)
commented on the dilution of supervision (number of
experienced doctors and nurses available to supervise
redeployed staff). Care could be accommodated with
skilled alternative employees, but monitoring staff
without satisfactory skill levels diverted resources from
COVID-19 patient care to direct supervision. In a less-
diluted ICU (D), however, IC-trained personnel
deemed supervision to be good to excellent.
U: “Supervision of nonqualified personnel is insufficient at
times due to lack of availability of experienced ICU
nurses and/or lack of time”(#176).
NY: “Everyone is working somewhat outside their comfort
zones and stretched thin. Given the circumstances,
supervision while less than average was adequate”
(#12).
D: “More support from ICU staff (not everyone was
approachable). Try to give less complex patients to ward
staff, only send ward staff to ICU if there is no ICU staff
available”(#110).
Quality and safety of care
(numbers of comments: U:
6+, 342; NY: 2+, 102;D:
9+, 22)
While the quality and safety of the care were described
as being lower than at baseline, respondents
overwhelmingly described the quality as being
acceptable. Again, however, many IC-trained nurses
(U) were critical about the care delivered.
U: “Quality of care is very low, as there is not . . . enough
time for the usual care”(#180).
NY: “I did feel that the safety and quality of care was
reduced solely because of the large volume of patients
and that the providers were not used to taking care of
these patients. Overall, the care was good, but it was
not to the same level as our normal care, simply
because of the nature of the situation”(#37).
D: “excellent consultant cover and support”(#98).
Collaboration,
communication, and
atmosphere (numbers of
comments: U: 184+, 342;
NY: 76+, 112; D: 22+, 72)
The overall picture that emerges is largely one of great
appreciation for the commitment, cooperation, and
willingness of all involved in the COVID-19 ICU work.
Respondents described how everyone in the hospital
supported each other, using terms such as “trust,”
“camaraderie,”“collaboration,”and “spirit of
togetherness.”The collegiality manifested in the
emergence of informal “curbside”consultations as a
commonplace and key way that expertise was shared
efficiently. Collaboration, triage of patients, formal and
informal access to supervision and consultations, and
provision of protocols were cited as significant positive
factors.
U: “Collaboration is great. Many are very grateful for the
support that we are providing. I feel to be part of the
team”(#135).
NY: “There was excellent collaboration among all team
members. Everyone was very helpful and happy to
answer my questions”(#37).
D: “Amazing teamwork and camaraderie observed”(#1).
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 11
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Table 4. (Continued).
Theme Salient Findings Summarized Quotes from Participants
Recruitment, scheduling,
and team composition
(numbers of comments: U:
19+, 1072; NY: 8+, 352;D:
13+, 132)
A key issue was the scheduling of personnel in the five
ICUs. Frequent rotations between wards and within
teams was deemed undesirable, as time spent
becoming familiar with the context of each ICU and
time spent by supervisors estimating the expertise of
new coworkers was lost for patient care. Common
complaints concerned schedules, including knowing
when and where to arrive, knowing who was on your
team, and staggered team start times. Respondents
appreciated teams being composed of less-
experienced personnel paired with more-experienced
personnel and team stability over time.
U: “Assess capabilities of redeployed personnel upfront,
they often require much more input (explaining and
demonstrating how things work) than we get out of it
(number of shifts a person works/their ability to do
things”(#181).
NY: “I think there were issues with scheduling and being
informed what our daily schedules would entail.
Pairing inexperienced surgical attending with medicine
residents was perhaps the ideal way to approach the
unusual nature of deployment”(#106).
D: “Have nurses start at the same time as the ICU nurses
as it was frustrating at 8 or 9 and you had to stop and
hand over again and explain everything you had
already done”(# 10).
Organization and facilities
(numbers of comments: U:
24+, 512; NY: 6+, 132;D:
12+, 112)
There was general appreciation, but lower quality and
safety of care was also attributed to fluctuations in
medication supply, older ventilators and their settings,
patient volume, and limited understanding of the
illness. The most common area for improvement
centered on more intensive orientation to and/or
training for the medicine service as well as the local
systems (including the electronic health record). For
those with less experience, visual algorithms (e.g., if
SpO2is less than x, then do y) were especially valued.
Creative use of facilities, rules, and regulations was
appreciated.
U: “A lot has been achieved in such a short period of
time, a lot of work has been done to be able to open
so many extra ICU beds”(#131). “[There was space] to
act ‘differently’given the circumstances to make the
best of it . . . with daily jokes from the management”
(#90).
NY: “There should be more effective orientation. The
orientation should be in the unit in which we are
working because each unit runs differently. There
should be specific instruction on what our roles are.
We had adequate access to PPE”(#19).
D: “I was put with a scrub nurse on Day 1 and in a room
with poor visibility for other staff with queries”(#158).
Definition of abbreviations: COVID-19 = coronavirus disease; D = Dublin; IC = intensive care; ICU = intensive care unit; NY = New York; PPE = personal protective equipment;
SpO2= oxygen saturation as measured by pulse oximetry; U = Utrecht.
*For the numbers of comments, a plus sign indicates positive comments (tops), and a minus sign indicates negative comments (tips).
ORIGINAL RESEARCH
12 Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
Simple, visual COVID-19 protocols were
particularly valued, especially for those
least familiar (New York). However, one-
third (n= 27) of the IC-trained nurses in
Utrecht perceived the quality of care
delivered to clearly be below standards.
Collaboration, communication, and atmosphere
were overwhelmingly appreciated as being
positive (see Table 4, in which plus signs in
the leftmost column indicate positive
comments), with a vast majority providing
this as a spontaneous comment. The
commitment, cooperation, and willingness
of all involved in the COVID-19 ICU
work were described using terms such as
“trust,”“camaraderie,”and “spirit of
togetherness.”In addition, a willingness to
support each other’s“learning on the job”
responsively was noted at all three sites.
Recruitment, scheduling, and team composition
were often mentioned as critical. Frequent
rotations between wards and teams were
deemed undesirable. Schedules, knowing
when and where to arrive, and team
assignment appeared to be critical.
Having teams that were stable over time
with fewer members but a higher
proportion of experienced personnel was
appreciated. Recruitment of helpers with
little background was criticized (Utrecht).
Assignment of patients to teams with an
appropriate skill mix for patient needs was
identified as being important for both
redeployed and trained personnel.
Organization and facilities were generally
appreciated, but the medication supply,
ventilator quality, use of unfamiliar
electronic health record systems, and
insufficient training and/or orientation
were reported to hamper the work
(Utrecht). The ability to creatively apply
rules and regulations was felt to be
conducive to an effective working
atmosphere. Advance preparation and
training, especially with respect to
personal protective equipment, was
commended, and daily communication of
recent evidence for forward planning was
seen as being critical (Dublin and
Utrecht).
DISCUSSION
Our study yielded a number of salient
findings. A sudden surge of critically ill
patients can be accommodated by a
composite workforce of IC-certified and
redeployed, non–IC-certified personnel.
The majority of our population (268,
69%) was not certified for IC work and
reported often working under direct or indi-
rect supervision (supervisor present or quickly
available if needed, respectively). The qual-
ity of care and amount of supervision,
although lower than usual, were generally
reported as being at least acceptable and
experienced in a superior interprofes-
sional, collaborative atmosphere. Where
IC-certified nurse–to–patient ratios had
decreased most (Utrecht), however, nurses
were the most critical about supervision
and the quality of care.
The overall high degree of appreciation
for the commitment, collaborative
atmosphere, and mutual solidarity across
professions may have affected judgments
regarding the perceived adequacy of
supervision. Discrepant judgments,
notably from IC-certified nurses at UMC
Utrecht about receiving and providing
supervision, may be less surprising, given
their sense of benchmarks for high-quality
ICU care in combination with a lower
imposed ratio of IC nurses to patients.
The dual role of supervisors, serving the
interests of patients and learners, requires
skills to balance and prioritize these inter-
ests differently at different moments.
Hiring coworkers who must be supervised
at level 1 (to be present and only observe)
and level 2 (to act under direct
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 13
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
supervision) for more than a brief
introductory phase is a less efficient
investment in crises and should be
discouraged. Workers requiring only
indirect supervision, on the other hand,
are highly useful; such workers typically
have an adequate background and some
working experience in a similar or related
context in a not too distant past and are
able to access supervision when needed.
Frequent comments on the need for stable
team compositions, mixing more
experienced colleagues with less
experienced colleagues, across
interprofessional boundaries also speaks to
two key features of teams that help
compensate for the inexperience of some.
Working with a team that is stable over
time and includes mixed experience
degrees facilitated both formal and
informal support, even across professional
boundaries. We found that clinical
experience did not always predict who
provided and who received supervision.
Qualified professionals, working
unsupervised, are not necessarily skilled to
act as supervisors in the strict sense, but
informal peer collaboration across
professions and hierarchies may have been
felt to be highly rewarding and conducive
to a collaborative atmosphere. Indeed, this
atmosphere may be explained by
abundant opportunities to receive and
provide peer-level supervision, which is in
concordance with our survey results.
Does this mean that formal supervision
would not be necessary? We do not
believe so. Safe patient care requires
decisions about care that pertain to
coworkers to be made by accountable
supervisors. Employing many alternative
professionals in crises requires a sufficient
volume and quality of supervisors.
Although preparing ICUs for calamities
such as pandemics has been described,
stressing the need for alternate hospital
sites; non-ICU staff being under the super-
vision of those trained in IC; and rapid,
appropriate, “just-in-time”education of
new staff during times of surge has been
highlighted (22–24), but the significance of
having an adequate volume and quality of
supervision has not been described well
and may indicate a possible gap in the
outbreak preparedness of ICUs.
Certified ICU nurses were generally
unfamiliar with and unprepared for
providing supervision while simultaneously
caring for more than one critically ill
patient, which may explain some of the
tensions we found. The Utrecht and
Dublin samples had a similar ratio of
IC-certified to non–IC-certified respond-
ents, but at the peak of the pandemic,
Utrecht faced more patients per experi-
enced ICU nurse (3:1) than Dublin (1–2:1),
although less than 2:1 is recommended
(25). Many ICU nurses, particularly in
Utrecht, were forced to supervise in addi-
tion to managing their own increased
patient load. These ratios are more critical
for nurses than for physicians, who fore-
most make clinical decisions rather than
providing minute-to-minute care. In
Utrecht, the capacity to accommodate
fewer skilled and autonomous “helpers”
and adapt to their deficits seemed to have
been exceeded. This may explain Dublin’s
higher degree of satisfaction with the super-
vision among ICU-certified personnel in
comparison with Utrecht’s. An adequate
ratio of formal supervisors to coworkers,
both for medical and nursing staff, is key
to patient safety and quality of care.
Limitations
Our study has several limitations. It was
conceived and designed using only six
steps of the seven-step process for design-
ing high-quality questionnaires in a very
ORIGINAL RESEARCH
14 Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
brief period of time to capture participant
experiences during a crisis period at three
quite different sites. Furthermore, it should
be recognized that our data reflect the
self-reported (i.e., perceived) quality of
care. Although self-reporting is a common
Table 5. Recommendations for future redeployments
Themes Recommendations
Supervision Redeploying alternative professionals who
require indirect supervision (level 3) is the
predominant target group in situations
such as the COVID-19 pandemic.
Redeploying alternative professionals who
continue to require direct supervision (level
2) is inefficient and should be discouraged,
as providing adequate supervision at this
level distracts from provision of high-
quality care.
Deploying professionals with a supervisory
role (level 5) is critically important in an
adequate ratio to those who need indirect
or direct supervision. Professionals must be
trained or experienced for this role.
Quality and safety of care The ICU ratio of IC-certified nurses to
patients should preferably not exceed 1:2.
Further decreases make preserving team
composition, stability, and scheduling more
critically difficult.
Collaboration, communication, and
atmosphere
As high degrees of interprofessional
collaboration, effort, and camaraderie
compensate for other deficiencies, this
atmosphere must be supported and is
critically important to maintain quality.
Recruitment, scheduling, and team
composition
Selection of alternative professionals
should focus on individuals who are able
to work with indirect supervision (level 3)
after a short, instructive introduction.
ICU teams should include a mix of
designated supervisors (level 5),
experienced professionals (level 4), and
professionals requiring indirect supervision
(level 3).
Team composition should remain as stable
as possible over time.
Organization and facilities Rules and regulations should be applied
judiciously to ease quick decisions if
necessary.
Visible clinical protocols and easy access to
informal (“curbside”) consultations should
be secured.
Definition of abbreviations: COVID-19 = coronavirus disease; IC = intensive care; ICU = intensive care unit.
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 15
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
approach for gathering data in epidemio-
logic and medical research, bias in self-
rating may be of concern (26). Namely, a
recent study among hospitalized patients
with COVID-19 in the United States
showed that mortality in surge settings
during the first wave of the pandemic
appeared to be higher (27). Finally, given
its cross-sectional nature, survey limita-
tions, and differences among settings, insti-
tutions, and/or countries, data could not
be easily merged. Generally, however,
many aspects of the findings converged.
Conclusions
A surge of critically ill patients can be
accommodated by a composite workforce
of IC-certified and redeployed, non–IC-
certified personnel. Although the per-
ceived quality of care, supervisory capac-
ity, training background, and experience
were compromised to meet COVID-19
demands, several key features to maintain
an acceptable degree of perceived quality
of care could be identified. These included
stable teams that matched less experienced
personnel with more experienced person-
nel, allowing for supervision adapted to
needs, constructive interprofessional col-
laboration, a robust culture of informal
consultation, and more flexibility in hospi-
tal rules. Furthermore, deploying profes-
sionals trained or experienced in
supervision is critically important for
future response preparedness. This
requires not only specified training and
certification, and maybe even a funda-
mental definition of competence and qual-
ification for healthcare work (28), but also
an adequate ratio of supervisors to those
needing (indirect or direct) supervision.
On the basis of these findings, several
recommendations for future redeployment
in crisis situations regarding supervision;
quality and safety of care; collaboration,
communication and atmosphere;
recruitment, scheduling, and team
composition; and organization and
facilities were formulated (Table 5). In
addition, further research on the long-
term impact of providing patient care
during the COVID-19 surge on the
healthcare workforce in general seems
warranted.
Acknowledgment:
The authors thank the following
individuals for assistance with data
collection and entry: Ms. Nathalie Mc
Evoy, S.R.N.; Mr. Tije D. Kranenburg;
Ms. Floor E. Kranenburg; Dr. Marc
Lincoln; Dr. Catherine Murphy; Dr.
Tom Ryan; and Dr. Greta Scanlon.
Author disclosures are available with the
text of this article at www.atsjournals.org.
REFERENCES
1. World Health Organization. Coronavirus disease (COVID-19): situation report 111. Geneva,
Switzerland: World Health Organization; 2020 [created 2020 May 10; accessed 2020 May 10].
Available from: https://www.who.int/docs/default-source/coronaviruse/situation-reports/
20200510covid-19-sitrep-111.pdf?sfvrsn=1896976f_6.
2. Fraher EP, Pittman P, Frogner BK, Spetz J, Moore J, Beck AJ, et al. Ensuring and sustaining a
pandemic workforce. N Engl J Med 2020;382:2181–2183.
3. Kim MK, Rabinowitz LG, Nagula S, Chalil J, Xu T, Barna E, et al. A primer for clinician
deployment to the medicine floors from an epicenter of COVID-19. NEJM Catal [online] 4 May
2020; DOI: 10.1056/CAT.20.0180.
ORIGINAL RESEARCH
16 Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
4. Ten Cate O, Hart D, Ankel F, Busari J, Englander R, Glasgow N, et al.; International
Competency-based Medical Education Collaborators. Entrustment decision making in clinical
training. Acad Med 2016;91:191–198.
5. Dennis B, Highet A, Kendrick D, Mazer L, Loiselle S, Bandeh-Ahmadi H, et al. Knowing your
team: rapid assessment of residents and fellows for effective horizontal care delivery in emergency
events. J Grad Med Educ 2020;12:272–279.
6. Sengupta S. With virus surge, dermatologists and orthopedists are drafted. New York, NY: New
York Times: 2020 [accessed 2020 May 12]. Available from: https://www.nytimes.com/2020/04/
03/nyregion/new-york-coronavirus-doctors.html.
7. Barzansky B, Catanese VM. LCME update of medical students, patients, and COVID-19: guiding
principles for early graduation of final-year medical students. Washington, DC: Association of
American Medical Colleges; 2020 [accessed 2020 May 12]. Available from: https://lcme.org/wp-
content/uploads/filebase/March-25–2020-LCME-Guidance-for-Medical-Schools-Considering-
Early-Graduation-Option.pdf.
8. Lingard L, Espin S, Evans C, Hawryluck L. The rules of the game: interprofessional collaboration
on the intensive care unit team. Crit Care 2004;8:R403–R408.
9. ten Cate O. Supervision and entrustment in clinical training: protecting patients, protecting
trainees. Rockville, MD: Agency for Healthcare Research and Quality; 2018 [accessed 2020 May
11]. Available from: https://psnet.ahrq.gov/webmm/case/461.
10. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development
for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach
2015;37:983–1002.
11. O’Dowd E, Lydon S, O’Connor P, Madden C, Byrne D. A systematic review of 7 years of
research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ
2019;53:234–249.
12. Meyer EG, Chen HC, Uijtdehaage S, Durning SJ, Maggio LA. Scoping review of entrustable
professional activities in undergraduate medical education. Acad Med 2019;94:1040–1049.
13. Smit MP, de Hoog M, Brackel HJL, Ten Cate O, Gemke RJBJ. A national process to enhance the
validity of entrustment decisions for Dutch pediatric residents. J Grad Med Educ 2019;11:158–164.
14. Morgan S, Pullon S, McKinlay E. Observation of interprofessional collaborative practice in
primary care teams: an integrative literature review. Int J Nurs Stud 2015;52:1217–1230.
15. Donovan AL, Aldrich JM, Gross AK, Barchas DM, Thornton KC, Schell-Chaple HM, et al.;
University of California, San Francisco Critical Care Innovations Group. Interprofessional care
and teamwork in the ICU. Crit Care Med 2018;46:980–990.
16. Brensilver JM, Smith L, Lyttle CS. Impact of the Libby Zion case on graduate medical education
in internal medicine. Mt Sinai J Med 1998;65:296–300.
17. Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system.
Washington DC: National Academy Press; 2000.
18. Garrouste-Orgeas M, Valentin A. What’s new for patient safety in the ICU? Intensive Care Med
2013;39:1829–1831.
19. Artino AR Jr, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational
research: AMEE guide no. 87. Med Teach 2014;36:463–474.
20. Likert R. Technique for the measurement of attitudes, A. Arch Psychol 1932;22:5–55.
ORIGINAL RESEARCH
| Hennus, Young, Hennessy, et al.: LEAP Study 17
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
21. Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE guide no. 131. Med Teach
2020;42:846–854.
22. Kain T, Fowler R. Preparing intensive care for the next pandemic influenza. Crit Care 2019;23:337.
23. Maves RC, Jamros CM, Smith AG. Intensive care unit preparedness during pandemics and other
biological threats. Crit Care Clin 2019;35:609–618.
24. Dichter JR, Kanter RK, Dries D, Luyckx V, Lim ML, Wilgis J, et al.; Task Force for Mass Critical
Care. System-level planning, coordination, and communication: care of the critically ill and injured
during pandemics and disasters: CHEST consensus statement. Chest 2014;146:e87S–e102S.
25. Marshall JC, Bosco L, Adhikari NK, Connolly B, Diaz JV, Dorman T, et al. What is an intensive
care unit? A report of the task force of the World Federation of Societies of Intensive and Critical
Care Medicine. J Crit Care 2017;37:270–276.
26. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J
Multidiscip Healthc 2016;9:211–217.
27. Block BL, Martin TM, Boscardin WJ, Covinsky KE, Mourad M, Hu LL, et al. Variation in
COVID-19 mortality across 117 US hospitals in high- and low-burden settings. J Hosp Med
2021;16:215–218.
28. ten Cate O, Schultz K, Frank J, Hennus M, Ross S, Schumacher D, et al. Questioning medical
competence: should the COVID-19 crisis affect our views of what it includes? Med Teach (In press)
ORIGINAL RESEARCH
18 Hennus, Young, Hennessy, et al.: LEAP Study |
ATS Scholar. Published August 16, 2021 as 10.34197/ats-scholar.2020-0165OC
Copyright © 2021 by the American Thoracic Society
