Content uploaded by Estelle Michinov
Author content
All content in this area was uploaded by Estelle Michinov on Jan 29, 2025
Content may be subject to copyright.
Review
Crisis management in the operating room: A systematic review of
simulation training to develop non-technical skills
Inas D. Redjem
a,b,*
, Arnaud Huaulm´
e
b
, Pierre Jannin
b
, Estelle Michinov
a
a
Univ Rennes, LP3C (Laboratoire de Psychologie: Cognition, Comportement, Communication), F-35000 Rennes, France
b
Univ Rennes, INSERM, LTSI (Laboratoire du Traitement du Signal et de l'Image) - UMR 1099, F-35000 Rennes, France
ARTICLE INFO
Keywords:
Simulation
Healthcare training
Operating room
Non-technical skills
Crew resource management
Systematic review
ABSTRACT
Aims: Crises in the operating room, often resulting from human factors, endangers patient safety. Simulation-
based training to develop non-technical skills shows promise in managing these crises. This review examines
the simulation techniques, targeted healthcare professionals, non-technical skills, crisis scenarios, and evaluation
metrics used in operating room crisis management training.
Design: Systematic review.
Data sources: MEDLINE, APA PsycInfo and Web of Science databases were searched for peer-reviewed articles
published between January 2004 and March 2024.
Review methods: This systematic review was conducted in accordance with the Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The review protocol has been registered on the
Open Science Framework (OSF) (https://osf.io/7bsc8). The inclusion criteria were as follows: (1) The study
population comprised healthcare and medical professionals or students; (2) the intervention involved a simulated
learning or training experience; (3) the outcomes focused on non-technical skills or crew resource management;
(4) the training setting was the operating room (simulated or real); and (5) the learning scenarios depicted a
crisis or an adverse event.
Results: This systematic review identied 29 eligible articles. The ndings highlight the predominance of high-
delity simulations, primarily targeting medical staff rather than nurses or other healthcare professionals.
Training focused on communication, teamwork, situation awareness, problem solving, and decision making, with
scenarios mostly addressing patient deterioration. Assessments reached up to Kirkpatrick's Level 3, demon-
strating a positive training impact through learners' reactions and learning metrics rather than behavior and
organizational results.
Conclusions: Despite their effectiveness, current training practices exhibit limitations. Incorporating nurses and
other paramedical staff in interprofessional training, as well as emphasizing team-related scenarios and evalu-
ating behavioral changes in practice, could enhance training effectiveness. This has implications for interpro-
fessional healthcare education and skills transfer to real-world settings, ultimately improving patient safety.
1. Introduction
Adverse events during hospitalization can affect patients' health
(Panagioti et al., 2019). Dened as unexpected incidents resulting from
medical management, perioperative adverse events include equipment
or communication failures, as well as procedural mistakes occurring
during surgical procedures (Jung et al., 2019). A large portion of adverse
events are considered preventable because they result from human
factors (Gordo et al., 2021; Smits et al., 2021).
A signicant portion of the errors leading to these “Never Events” can
be attributed to a lack of non-technical skills (Allard et al., 2020; Koleva,
2020), dened as “cognitive, social and personal resource skills that
complement technical skills and contribute to safe and efcient task
performance” (Flin et al., 2008). Non-technical skills are generally
grouped into two categories: social skills (communication, teamwork,
leadership, and coordination) and cognitive skills (situation awareness
and decision-making; Allard et al., 2020). It is well established that a
lack of non-technical skills can delay the response to critical medical
situations, leading to adverse events (Yucel et al., 2020).
Therefore, crew resource management – a set of principles dealing
* Corresponding author at: Universit´
e Rennes 2, D´
epartement de Psychologie, Place du Recteur Henri le Moal, 35043 Rennes Cedex, France.
E-mail addresses: inas.redjem@univ-rennes2.fr (I.D. Redjem), estelle.michinov@univ-rennes2.fr (E. Michinov).
Contents lists available at ScienceDirect
Nurse Education Today
journal homepage: www.elsevier.com/locate/nedt
https://doi.org/10.1016/j.nedt.2025.106583
Received 30 May 2024; Received in revised form 14 January 2025; Accepted 20 January 2025
Nurse Education Today 147 (2025) 106583
Available online 23 January 2025
0260-6917/© 2025 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ ).
with cognitive and interpersonal skills that contribute to optimal team
performance (Buljac-Samardzic et al., 2020) – has played a crucial role
in healthcare education since its introduction to the curricula in the
1990s (Howard et al., 1992). Crew resource management provides
contextualized training in non-technical skills, with the objective of
providing predened responses to critical incidents and guidance on the
coordinated use of all available resources to maximize safe patient
outcomes (Buljac-Samardzic et al., 2020). Recent systematic reviews
have shown that simulation-based training is the most common method
for improving team effectiveness and non-technical skills (Buljac-
Samardzic et al., 2020; Ounounou et al., 2019). Both methodologies
allow healthcare teams to develop these skills in realistic scenarios
without risking patient safety.
Existing systematic reviews examining simulation-based training for
developing non-technical skills in crisis management are limited and
largely outdated (Fung et al., 2015; Meri´
en et al., 2010). The most recent
reviews are narrow in scope, primarily focusing on emergency de-
partments (Alsabri et al., 2022; Brogaard et al., 2022; Thim et al., 2022;
Yucel et al., 2020). There is a noticeable lack of comprehensive coverage
on crisis management in the operating room, with only a few reviews
addressing team training (Brogaard et al., 2022; Cumin et al., 2013) and
non-technical skills in surgery (Dedy et al., 2013; Robertson et al.,
2017), providing limited details on training methodologies and skills
measurements.
This systematic review seeks to ll the gaps in the existing literature
by comprehensively summarizing the research on simulation training on
crisis management in the operating room aimed at healthcare pro-
fessionals and students. The research questions (RQs) are:
RQ1. In what research contexts have simulations for managing crises
in operating rooms been implemented?
RQ2. What simulation methodologies have been used and how are
these simulations organized?
RQ3. What specic scenarios of crisis management and skills are
assessed within simulations?
RQ4. What are the evaluation levels according to Kirkpatrick's model
(Kirkpatrick and Kirkpatrick, 2006)?
2. Method
2.1. Registration and reporting guidelines
This systematic review was conducted in accordance with the
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) guidelines (Page et al., 2021). The review protocol has been
registered on OSF (https://osf.io/7bsc8). The PRISMA checklist is pro-
vided in Supplementary Material 1.
2.2. Data source and search strategy
The Population, Intervention, Comparison and Outcome (PICO)
framework (Richardson et al., 1995) was used to structure a clear review
question. The population was dened as healthcare and medical pro-
fessionals or students; the intervention as a simulated learning or
training experience; the outcomes as non-technical skills or crew
resource management; and the setting as the operating room.
The following comprehensive research equation was developed:
(health* OR nursing professional* OR nurs* OR surgeon* OR nursing
student* OR medical student* OR “clinical”) AND (“simulation” OR
“training” OR “simulation-based training” OR “crew resource manage-
ment” OR “CRM” OR “crisis management training” OR “virtual reality”
OR immersive environment* OR “immersive virtual reality” OR “VR”
OR “HMD” OR immer* OR patient safety training) AND (“operating
room” OR “operating theater” OR “surgery”) AND (human factor* OR
soft skill* OR non-technical skill* OR “decision making” OR “stress
management” OR “leadership” OR “communication” OR team* OR “risk
management” OR “crisis management” OR “crisis” OR critical event* OR
adverse event* OR “CRM” OR medical error*). Full search strategies are
provided in Supplementary Material 2.
Three databases covering the elds of interest (healthcare, simula-
tion, and non-technical skills) were queried: Medline, Web of Science
and APA PsycInfo via the EBSCO platform. Web of Science provides an
interdisciplinary perspective, encompassing research in simulation and
education. APA PsycInfo focuses on behavioral sciences and psychology,
which are key to studying non-technical skills, and Medline is a recog-
nized reference in healthcare. This combination helps reduce publica-
tion bias by offering a diverse range of relevant studies. The availability
of these databases through institutional access ensures the consistent use
of the same Boolean search function across all three. The nal data
extraction was performed on March 6, 2024.
2.3. Eligibility criteria
Reviews, case reports, opinion pieces, viewpoints, conceptual
frameworks, and conference abstracts were excluded. Empirical peer-
reviewed original works written in English and published between
January 2004 and March 2024 were included. This time limit was
selected as it corresponds to the establishment of the Society for Simu-
lation in Healthcare. The inclusion criteria were as follows: (1) The study
population comprised healthcare and medical professionals or students;
(2) the intervention involved a simulated learning or training experi-
ence; (3) the outcomes focused on non-technical skills or crew resource
management; (4) the training setting was the operating room (simulated
or real); and (5) the learning scenarios depicted a crisis or an adverse
event.
2.4. Data extraction and analysis
A data extraction sheet was developed to compile the selected arti-
cles and their key characteristics. The initial screening of articles was
conducted by the rst author (I.R.), who extracted details such as au-
thors, publication year, country, study design, participant occupation,
type of simulation, study objectives, outcomes, and results. For study
objectives, outcomes, and results, only data related to non-technical
skills were included. Relevant information pertaining to the four
research questions was also collected. To address the research questions,
the results of the systematic review were synthesized based on the
research context, simulation methodology and organization, crisis
simulation scenarios and skills measured, as well as the levels of eval-
uation according to Kirkpatrick's model (Kirkpatrick and Kirkpatrick,
2006). The data extraction tables for information pertaining to the four
research questions are provided in Supplementary Material 3.
2.5. Study selection
The study selection was performed according to PRISMA guidelines.
A total of 741 articles were initially retrieved, including 301 duplicates
that were later removed. The titles and abstracts of the articles were rst
screened by I.R in order to exclude those that did not meet all the
eligibility criteria. Out of the 440 articles, 388 were excluded based on
title or abstract as they did not meet the inclusion criteria. Of the
remaining 52 articles, full texts for three were unavailable, leaving 49
articles for the full-text analysis.
Full-text screening was independently performed by two authors (I.R
and E.M), who agreed on the inclusion of 23 articles and the exclusion of
20 articles. Disagreement arose over six articles (Cohen K =0.76)
centered around the denition of a crisis (Bracq et al., 2021; Burkhart
et al., 2013; Gardner and AbdelFattah, 2017; Raison et al., 2018;
Ramjeeawon et al., 2020; Sirihorachai et al., 2018). It was ultimately
agreed that for a simulation scenario to qualify as crisis situation
training, it must include stressors leading to adverse events or depict a
perioperative adverse event.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
2
Following this consensus, the authors re-evaluated the six disputed
articles and determined that they all met this rened criterion. Specif-
ically, two articles (Burkhart et al., 2013; Raison et al., 2018) depicted
clear perioperative adverse events related to patient health. Another two
articles (Bracq et al., 2021; Sirihorachai et al., 2018) focused on training
to manage interruptions caused by stressors, noting that repeated in-
terruptions can increase the risk of adverse events. These stressors
included equipment malfunctions, identity monitoring errors, disrup-
tions during surgery, and interruptions from beepers, radio calls, or
people entering and exiting the room, as well as irrelevant
Fig. 1. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) ow diagram.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
3
communication. The nal two articles (Gardner and AbdelFattah, 2017;
Ramjeeawon et al., 2020) depicted adverse events associated with
stressors such as equipment failure, which were also linked to patient
health issues.
The authors unanimously agreed to include the articles that had
initially been subject to disagreement. As a result, 17 articles were
excluded since they did not involve a crisis situation, and three further
articles were excluded since they did not include any assessment of non-
technical skills. This systematic review ultimately includes 29 articles
(Fig. 1).
2.6. Quality appraisal
The Mixed Methods Appraisal Tool (MMAT) 2018 (Hong et al., 2018)
was used to assess the quality of the studies included after full-text
screening. The MMAT consists of two screening questions and ve
core quality criteria adapted to ve specic study design categories:
qualitative, randomized controlled, non-randomized, quantitative
descriptive, and mixed methods. Studies were considered to be of high
quality if they met 100 % of the criteria, moderate quality if they met
80–99 % of the criteria, average quality if they met 60–79 % of the
criteria, low quality if they met 40–59 % of the criteria, and very low
quality if they met <39 % of the criteria. The overall MMAT scores
ranged from 20 % to 100 %, with a mean quality score of 73 % (SD =
22). The MMAT total scores for studies are presented in Table 1 and the
detailed scoring for each criterion is provided in Supplementary Mate-
rial 4.
3. Results
3.1. RQ1. Research contexts
3.1.1. Study characteristics
The majority of the articles (18 articles) reported on quantitative
non-randomized studies. Four articles were quantitative descriptive
studies, four articles were mixed method studies, and three articles were
quantitative randomized controlled trials. The articles were published
between 2006 and 2023 with a median publication rate of one article
per year (min =1, max =5). Notably, there was a peak in publication
activity between 2015 and 2021. The articles included are predomi-
nantly from the United States (16 articles), followed by the United
Kingdom (5 articles). Table 1 summarizes the characteristics of the 29
studies included in the systematic review.
3.1.2. Surgery specialties
A wide range of surgical specialties were featured in the included
articles, listed in order of frequency: gastrointestinal/abdominal (5 ar-
ticles, Abelson et al., 2015; Gardner and AbdelFattah, 2017; Nguyen
et al., 2015; Pena et al., 2015; Sirihorachai et al., 2018), trauma/
emergency (5 articles, Acero et al., 2012; Lehner et al., 2017; Pena et al.,
2015; Ramjeeawon et al., 2020; Wunder et al., 2020), thoracic (4 arti-
cles, Bierer et al., 2018; Burkhart et al., 2013; Pena et al., 2015; Wunder
et al., 2020) and general surgery (4 articles, Dedy et al., 2016; Pena
et al., 2015; Sirihorachai et al., 2018; Wongsirimeteekul et al., 2018),
urology (3 articles, Gettman et al., 2009; Goldenberg et al., 2018; Raison
et al., 2018), vascular (3 articles, Moorthy et al., 2006; Ramjeeawon
et al., 2020; Taaffe et al., 2021) and head and neck (3 articles, Mai et al.,
2020; Rochlen et al., 2019; Sirihorachai et al., 2018), orthopedics (1
article, Edwards et al., 2023), pediatric surgery (1 article, Lehner et al.,
2017), and ophthalmology (1 article, Wood et al., 2023). Some articles
encompassed multiple specialties (5 articles, Lehner et al., 2017; Pena
et al., 2015; Ramjeeawon et al., 2020; Sirihorachai et al., 2018; Wunder
et al., 2020), highlighting a non-exclusive focus, while several articles
did not specify a surgical specialty reecting a more generalized or non-
disciplinary approach to the subject matter (7 articles, Bracq et al., 2021;
Gao et al., 2021; Leithead et al., 2019; Paige et al., 2014; Shi et al., 2021;
Truong et al., 2022; Weller et al., 2015).
3.1.3. Participants' occupations
The participants in the articles came from a variety of occupations
(see Table 1). In the majority of articles, participants were medical
trainees with various levels of expertise (22 articles, Abelson et al.,
2015; Acero et al., 2012; Burkhart et al., 2013; Dedy et al., 2016;
Edwards et al., 2023; Gettman et al., 2009; Goldenberg et al., 2018;
Leithead et al., 2019; Mai et al., 2020; Moorthy et al., 2006; Nguyen
et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018;
Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Taaffe
et al., 2021; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul
et al., 2018; Wood et al., 2023). The two second most represented
populations were nurses (Acero et al., 2012; Lehner et al., 2017; Mai
et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Sirihorachai et al.,
2018; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al.,
2018) and attending surgeons (Abelson et al., 2015; Bierer et al., 2018;
Gardner and AbdelFattah, 2017; Lehner et al., 2017; Rochlen et al.,
2019; Shi et al., 2021; Truong et al., 2022; Weller et al., 2015; Wood
et al., 2023), targeted as the study population in nine articles each. Other
allied health professionals were also enrolled as participants (7 articles,
Acero et al., 2012; Edwards et al., 2023; Mai et al., 2020; Rochlen et al.,
2019; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al.,
2018), although in fewer studies, alongside anesthesiologist trainees (7
articles, Acero et al., 2012; Gao et al., 2021; Mai et al., 2020; Rochlen
et al., 2019; Shi et al., 2021; Truong et al., 2022; Wongsirimeteekul
et al., 2018), attending anesthetists (4 articles, Lehner et al., 2017;
Rochlen et al., 2019; Truong et al., 2022; Weller et al., 2015), nursing
students (3 articles, Edwards et al., 2023; Leithead et al., 2019; Paige
et al., 2014), student registered nurse anesthetists (3 articles, Leithead
et al., 2019; Paige et al., 2014; Wunder et al., 2020), scrub nurses (2
articles, Acero et al., 2012; Bracq et al., 2021), and certied registered
nurse anesthetists (1 article, Mai et al., 2020).
3.1.4. Simulation groups
The median sample size of participants involved in a simulation
study was 31. In half of the included articles, participants underwent the
simulation individually, rather than in groups (15 articles, Abelson
et al., 2015; Bracq et al., 2021; Dedy et al., 2016; Edwards et al., 2023;
Gardner and AbdelFattah, 2017; Goldenberg et al., 2018; Nguyen et al.,
2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020;
Sirihorachai et al., 2018; Taaffe et al., 2021; Truong et al., 2022; Wood
et al., 2023; Wunder et al., 2020). In simulations involving several
learner participants concurrently working on a task (14 articles), the
groups were either composed of (1) medical staff, anesthesia staff, and
nurses (8/14 articles, Acero et al., 2012; Gao et al., 2021; Leithead et al.,
2019; Mai et al., 2020; Paige et al., 2014; Shi et al., 2021; Weller et al.,
2015; Wongsirimeteekul et al., 2018); (2) medical staff and nurses (2/14
articles, Bierer et al., 2018; Lehner et al., 2017); (3) residents together
(2/14 articles, Burkhart et al., 2013; Gettman et al., 2009); (4) a com-
bination of medical staff, anesthesia staff, nurses, and other allied health
professionals (1/14 articles, Rochlen et al., 2019); or (5) medical staff
and other allied health professionals (1/14 articles, Edwards et al.,
2023). When multiple learners were engaged simultaneously, the
average minimum group size was 4.54 (SD =1.50) participants.
3.2. RQ2. Simulation methodologies and organization
3.2.1. Operating room team simulation
Engaging multiple participants in a scenario can effectively simulate
an operating room team. This was often achieved by using a team
composed of participants and scripted confederates or actors collabo-
rating within the environment (13 articles, Bierer et al., 2018; Dedy
et al., 2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017;
Gettman et al., 2009; Goldenberg et al., 2018; Mai et al., 2020; Moorthy
et al., 2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al.,
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
4
Table 1
Characteristics of studies included in the systematic review.
Article, (year),
country
Design MMAT
(%)
Participants, n Type Objectives Outcome Results
Abelson et al.
(2015), USA
NR 60 Attending Surgeon,
Medical Trainee, n =33
VR Evaluate simulator for
construct and face validity
Decision Making Attendings completed the
simulation faster and had a
higher pass rate than
trainees, but this difference
was not statistically
signicant
Acero et al. (2012),
USA
NR 80 Nurse, Scrub Nurse,
Medical Trainee,
Anesthesiologist Trainee,
OAHP, n =171
HF Improve team performance
during emergency response
activation
Communication, Decision
Making
Understanding of roles and
activation of protocol in OR
emergencies increased from
50 % to 98 %, with
signicant improvements in
task completion time
Bierer et al. (2018),
Canada
NR 60 Attending Surgeon, n =
12
IS Develop simulation for
effective interprofessional
communication and
teamwork
Communication, Teamwork,
Situation Awareness,
Decision Making,
Leadership, Team
Management, Situation
Monitoring, Mutual support,
Team structure
Consultants scored higher
than fellows in NTS,
effectively differentiating
experts from learners
Bracq et al. (2021),
France
Mixed 60 Scrub Nurse, n =26 VR Assess error recognition in a
virtual operating room
Situation Awareness Participants with higher
situation awareness detected
more errors and were faster
Burkhart et al.
(2013), USA
NR 60 Medical Trainee, n =23 HF Test simulation to improving
condence, prociency, and
crisis management
Situation Awareness,
Decision Making, Condence
Condence and crisis
management improved, with
success rates in tasks rising
from 40 % to 69 %
Dedy et al. (2016),
Canada
RCT 100 Medical Trainee, n =22 HF Evaluate the effectiveness of
structured training on NTS
in an OR
Communication, Teamwork,
Situation Awareness,
Decision Making, Leadership
Curriculum-trained residents
improved signicantly in
knowledge, attitude and NTS
performance compared to
conventionally trained
residents
Edwards et al.
(2023), UK
RCT 100 Nurse Student, Medical
Trainee, OAHP, n =40
HF
VR
Compare multiplayer
immersive VR with single-
player for NTS acquisition
Communication, Teamwork,
Situation Awareness,
Decision Making,
Leadership, Task
Management
Team-trained participants
outperformed solo groups on
all NTS
Gao et al. (2021),
China
NR 100 Anesthesiologist Trainee,
n =31
HF Enhance student learning
awareness through
simulation training
Communication, Situation
Awareness, Teamwork,
Decision Making, Team
Management, Condence,
Task Management
Students improved their
responses and completed
tests and scenarios
signicantly faster after
training
Gardner and
AbdelFattah
(2017), USA
NR 60 Attending Surgeon, n =
26
HF Examine role of simulation
in documenting resident NTS
compared to faculty ratings
Communication, Decision
Making, Leadership,
Situation Awareness, Task
Management
Simulation assessments did
not align well with
traditional clinical rotation
evaluations for NTS
Gettman et al.
(2009), USA
NR 80 Medical Trainee, n =19 HF Evaluate teamwork and
communication in a
simulated OR
Communication, Teamwork,
Decision Making,
Leadership, Situation
Awareness, Team
Management
Signicant improvements in
teamwork and adherence to
best practices were observed
between scenarios
Goldenberg et al.
(2018), Canada
NR 60 Medical Trainee, n =15 HF Create simulation tool for
safe NTS assessment
Communication, Teamwork,
Decision Making,
Leadership, Situation
Awareness
Senior residents
outperformed juniors in NTS
Lehner et al. (2017),
Germany
NR 60 Attending
Anesthesiologist,
Attending Surgeon,
Nurse, n =18
HF Establish interdisciplinary
simulation-based training to
improve trauma in pediatric
surgery
Communication, Leadership,
Team Management
Participants reported
improvements in NTS
Leithead et al.
(2019), USA
NR 40 Nurse student, Student
Registered Nurse
Anesthetist, Medical
Trainee, n =152
HF Investigate impact of
simulation on OR team
training across different
professions
Teamwork, Condence Both medical and nursing
students showed signicant
improvements in team-based
attitudes and collaboration
scores
Mai et al. (2020),
USA
QD 20 Nurse, Certied
Registered Nurse
Anesthetist, Medical
Trainee, Anesthesiologist
Trainee, OAHP, n =86
IS Develop simulation to
identify and manage OR re,
reduce adverse outcomes,
and apply crisis
management principles
Teamwork Most participants found the
simulation realistic, relevant,
and effective in promoting
teamwork
Moorthy et al.
(2006), UK
NR 80 Medical Trainee, n =20 HF Develop and evaluate
feasibility, realism, and
validity of surgical crisis
simulation
Communication, Teamwork,
Situation Awareness,
Decision Making,
No signicant differences in
NTS between groups, but
leadership showed a non-
signicant trend and
(continued on next page)
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
5
Table 1 (continued )
Article, (year),
country
Design MMAT
(%)
Participants, n Type Objectives Outcome Results
Leadership, Team
Management
utterances frequency
difference was signicant
Nguyen et al.
(2015), USA
NR 80 Medical Trainee, n =11 HF Examine effect of
simulation-based training on
NTS
Communication,
Professionalism
Residents showed signicant
improvement in completing
perioperative and
intraoperative checklists
after training
Paige et al. (2014),
USA
NR 80 Nurse student, Student
Registered Nurse
Anesthetist, Medical
Trainee, n =66
HF Examine impact of HF OR
team training on team-
related attitudes and
behaviors
Teamwork, Condence Statistically signicant gains
were observed in self-
efcacy and teamwork
performance after simulation
training
Pena et al. (2015),
Australia
NR 100 Medical Trainee, n =40 IS Explore effects of
simulation-based training,
alone or combined on NTS
Communication, Teamwork,
Situation Awareness,
Decision Making, Leadership
Both groups improved
signicantly in NTS, with no
differences between the
simulation-only and
workshop groups
Raison et al. (2018),
UK
RCT 100 Medical Trainee, n =62 HF Investigate the effectiveness
of motor imagery for NTS
training
Communication, Teamwork,
Situation Awareness,
Decision Making, Leadership
No signicant differences
were found between motor
imagery and standard
training in NTS
Ramjeeawon et al.
(2020), UK
NR 80 Medical Trainee, n =16 HF Assess if fully-immersive
simulation training with
structured debrieng
improves the lead surgeon
team-work
Communication, Teamwork,
Situation Awareness,
Leadership, Team
Management, Task
Management, Situation
Monitoring, Mutual support
Structured debrieng
signicantly improved
teamwork performance
across all domains.
Rochlen et al.
(2019), USA
NR 80 Attending
Anesthesiologist,
Attending Surgeon,
Nurse, Medical Trainee,
Anesthesiologist Trainee,
OAHP, n =31
IS Evaluate feasibility and
effectiveness of
interprofessional crisis
management simulation on
improvements in NTS
Communication, Teamwork,
Situation Awareness,
Decision Making,
Leadership, Team
Management
All teams improved in NTS
scores post-intervention, but
the changes were not
statistically signicant.
Shi et al. (2021),
USA
Mixed 60 Attending Surgeon,
Nurse, Medical Trainee,
Anesthesiologist Trainee,
n =14
IS Explore OR professionals'
perspective on teamwork
and communication pre- and
post-interprofessional OR
simulation
Communication, Self
Efcacy
Participants showed high
agreement on their
communication skills and
ability to speak up in both
emergency and non-
emergency situations.
Sirihorachai et al.
(2018), USA
Mixed 100 Nurse, n =30 HF Develop simulation
scenarios to understand
decision-making in response
to interruptions
Interruption Management Experienced nurses were
more likely to allow
interruptions during time-
outs, using cognitive
strategies to manage
distractions.
Taaffe et al. (2021),
USA
NR 80 Medical Trainee, n =40 S Assess feasibility of a
vascular surgery crisis
simulator and evaluate the
performance of residents and
fellows
Situation Awareness,
Decision Making
The vascular surgery crisis
simulation yielded similar
results for residents and
fellows, with a total score of
83 %.
Truong et al. (2022),
USA
QD 100 Attending
Anesthesiologist,
Attending Surgeon,
Nurse, Medical Trainee,
Anesthesiologist Trainee,
OAHP, n =180
VR Evaluate responses to VR-
simulated OR re, examine
impact of experience,
condence, and role on the
number of attempts needed
to pass simulation
Condence Condence in re response
increased signicantly post-
training, with most
participants passing the
simulation within ve
attempts.
Weller et al. (2015),
New Zealand
Mixed 80 Attending
Anesthesiologist,
Attending Surgeon,
Nurse, Medical Trainee,
OAHP, n =120
HF Explore the feasibility and
effectiveness of simulation-
based course for OR team
Communication There was a signicant
improvement in BMRI scores
between the rst and last
scenarios.
Wongsirimeteekul
et al. (2018), USA
QD 20 Nurse, Medical Trainee,
Anesthesiologist Trainee,
OAHP, n =91
IS Develop simulation to apply
core concepts of crisis
resource management.
Teamwork Participants reported that the
simulation improved
teamwork skills and
promoted interprofessional
learning.
Wood et al. (2023),
UK
NR 60 Attending Surgeon,
Medical Trainee, n =17
IS Design a simulation-based
training for managing
posterior capsule rupture
and analyze NTS changes
Communication, Teamwork,
Situation Awareness,
Decision Making,
Leadership, Professionalism
NTS improved signicantly
after simulation-based
training for managing
posterior capsule rupture.
Wunder et al.
(2020), USA
QD 80 Student Registered Nurse
Anesthetist, n =32
HF
AR
Evaluate NTS after mixed
reality simulation of OR re
Teamwork, Situation
Awareness, Decision
Making, Team Management
NTS performance was 91.25
% after the mixed reality
simulation of OR re.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
6
2020; Sirihorachai et al., 2018; Wunder et al., 2020). An alternative
approach involved assigning specic roles to each learner within the
operating room team (12 articles, Acero et al., 2012; Burkhart et al.,
2013; Edwards et al., 2023; Gao et al., 2021; Lehner et al., 2017; Leit-
head et al., 2019; Nguyen et al., 2015; Paige et al., 2014; Rochlen et al.,
2019; Shi et al., 2021; Weller et al., 2015; Wongsirimeteekul et al.,
2018). Two articles (Abelson et al., 2015; Edwards et al., 2023) intro-
duced an innovative approach by using avatars to represent team
members. Finally, four articles focused on the experiences of individual
learners working alone in the operating room (Bracq et al., 2021; Taaffe
et al., 2021; Truong et al., 2022; Wood et al., 2023).
3.2.2. Operating room environment simulation
Simulation environments included in situ settings, virtual reality,
mixed reality, simulation stations, and high-delity simulations. In situ
simulation was sometimes used in the included articles (7 articles, Bierer
et al., 2018; Mai et al., 2020; Pena et al., 2015; Rochlen et al., 2019; Shi
et al., 2021; Wongsirimeteekul et al., 2018; Wood et al., 2023), while
virtual reality simulation (4 articles, Abelson et al., 2015; Bracq et al.,
2021; Edwards et al., 2023; Truong et al., 2022), simulation stations (1
article, Taaffe et al., 2021) and augmented reality simulation (1 article,
Wunder et al., 2020) were less represented. However, the most reported
modality was a form of “simulated operating room” (18 articles), which
does not fall into the previous categories and could be linked to high-
delity simulation. Among these 18 articles, two did not provide suf-
cient details of their denition of “simulated operating room” (Gardner
and AbdelFattah, 2017; Sirihorachai et al., 2018).
Within the remaining 16 articles simulating an operating room in
high-delity, three elements were commonly simulated: the operating
room environment, the operating room equipment, and the patient.
First, a considerable portion of these articles simulated the patient only
(3/16 articles, Acero et al., 2012; Dedy et al., 2016; Lehner et al., 2017).
In each case, this was done using a full-scale high-delity patient
simulator mannequin. Other studies describe the simulated environment
and equipment (2/16 articles, Raison et al., 2018; Ramjeeawon et al.,
2020). In two article, the simulation was performed using a simulated
patient in a simulated OR but without sufcient information about the
instruments (Burkhart et al., 2013; Wunder et al., 2020). Finally, the
majority of articles (9/16 articles) describe a simulated operating room
with both simulated equipment and a simulated patient: the operating
room was consistently an immersive mock operating room, and the
patient simulation was predominantly carried out using a human patient
simulator mannequin. All of these nine articles used real operating room
instruments (Edwards et al., 2023; Gao et al., 2021; Gettman et al.,
2009; Goldenberg et al., 2018; Leithead et al., 2019; Moorthy et al.,
2006; Nguyen et al., 2015; Paige et al., 2014; Weller et al., 2015).
Before 2018, high-delity simulation appears to be the predominant
method for training non-technical skills in crisis management, as evi-
denced by articles published during that period (9/11 articles, Acero
et al., 2012; Dedy et al., 2016; Gardner and AbdelFattah, 2017; Gettman
et al., 2009; Lehner et al., 2017; Moorthy et al., 2006; Nguyen et al.,
2015; Paige et al., 2014; Weller et al., 2015). However, this trend shifted
after 2018, with an increasing adoption of in-situ simulation (6/17 ar-
ticles, Bierer et al., 2018; Mai et al., 2020; Rochlen et al., 2019; Shi et al.,
2021; Wongsirimeteekul et al., 2018; Wood et al., 2023), equaling the
use of high-delity simulation. A notable shift occurred in 2020, char-
acterized by the digitalization of training practices, with a predominant
focus on the use of virtual reality, augmented reality, and simulation
stations in the published articles (5/8 articles, Bracq et al., 2021;
Edwards et al., 2023; Taaffe et al., 2021; Truong et al., 2022; Wunder
et al., 2020).
3.2.3. Brieng phase
In most studies, a brieng was provided to participants before the
simulation (18 articles). Briengs in simulations can encompass various
elements, often combining multiple aspects: orientation regarding the
patient's medical history (13/18 articles, Acero et al., 2012; Bracq et al.,
2021; Dedy et al., 2016; Gardner and AbdelFattah, 2017; Goldenberg
et al., 2018; Mai et al., 2020; Nguyen et al., 2015; Paige et al., 2014;
Ramjeeawon et al., 2020; Weller et al., 2015; Wongsirimeteekul et al.,
2018; Wood et al., 2023; Wunder et al., 2020), simulation objectives (8/
18 articles, Bracq et al., 2021; Gardner and AbdelFattah, 2017; Gold-
enberg et al., 2018; Mai et al., 2020; Moorthy et al., 2006; Sirihorachai
et al., 2018; Wongsirimeteekul et al., 2018; Wunder et al., 2020), and
introduction to the simulation equipment (9/18 articles, Bracq et al.,
2021; Dedy et al., 2016; Gettman et al., 2009; Mai et al., 2020; Moorthy
et al., 2006; Ramjeeawon et al., 2020; Wongsirimeteekul et al., 2018;
Wood et al., 2023; Wunder et al., 2020). Some briengs also included
theoretical clinical aspects required for the scenario (3/18 articles, Mai
et al., 2020; Wongsirimeteekul et al., 2018; Wunder et al., 2020).
3.2.4. Debrieng phase
A debrieng was also provided for participants after the simulation
in most of the included articles (22 articles). Various professionals
conducted the debriengs: moderators (5/22 articles, Acero et al., 2012;
Lehner et al., 2017; Leithead et al., 2019; Paige et al., 2014; Wood et al.,
2023), medical staff (6/22 articles, Bierer et al., 2018; Nguyen et al.,
2015; Pena et al., 2015; Ramjeeawon et al., 2020; Rochlen et al., 2019;
Shi et al., 2021), faculty members or teachers (4/22 articles, Gardner
and AbdelFattah, 2017; Mai et al., 2020; Wongsirimeteekul et al., 2018;
Wunder et al., 2020), and researchers (2/22 articles, Moorthy et al.,
2006; Weller et al., 2015). One debrieng team was composed of both
researchers and teachers (1/22 articles, Bracq et al., 2021). One
debrieng was conducted by a content expert, without further precision
about profession or role (Nguyen et al., 2015). Among this subset of 22
articles, four articles did not specify the composition of the debrieng
team (Dedy et al., 2016; Gao et al., 2021; Gettman et al., 2009; Sir-
ihorachai et al., 2018). Debriengs were often video-based (11/22 ar-
ticles, Acero et al., 2012; Gardner and AbdelFattah, 2017; Gettman
et al., 2009; Lehner et al., 2017; Mai et al., 2020; Moorthy et al., 2006;
Nguyen et al., 2015; Pena et al., 2015; Sirihorachai et al., 2018;
Wongsirimeteekul et al., 2018; Wood et al., 2023) and a portion of the
studies used specic methodologies or frameworks to guide the process
(7/22 articles, Bracq et al., 2021; Gardner and AbdelFattah, 2017;
Leithead et al., 2019; Paige et al., 2014; Pena et al., 2015; Wongsir-
imeteekul et al., 2018; Wunder et al., 2020). When debrieng tech-
niques were not specied (4/22 articles), there was a consistent
emphasis on non-technical skills and crew resource management, per-
formed by reviewing teamwork behaviors (Mai et al., 2020), strategies
for enhancing communication (Rochlen et al., 2019), the interprofes-
sional nature of communication and teamwork (Shi et al., 2021), and the
exploration of mechanisms and barriers to information sharing (Weller
et al., 2015).
3.3. RQ3. Crisis simulation scenarios and skills measured
The crisis scenarios in the included studies can be classied into
three main categories (Fig. 2). The most reported type of crisis is patient
health-related failure (22 articles, Acero et al., 2012; Bierer et al., 2018;
Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al.,
2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg
et al., 2018; Lehner et al., 2017; Leithead et al., 2019; Moorthy et al.,
2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison
et al., 2018; Rochlen et al., 2019; Shi et al., 2021; Taaffe et al., 2021;
Note. The design section in coded such as NR =Non-Random; RCT =Randomized Controlled Trial; QD =Quantitative Descriptive. In the participant section, OAHP =
Other Allied Health Professional. The simulation section is coded such as VR =Virtual Reality; HF =High-Fidelity; IS =In Situ; S =Stations; AR =Augmented Reality.
OR =operating room. NTS =non-technical skills.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
7
Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood et al., 2023),
which refers to crisis scenarios in which a patient experiences a sudden
or critical perioperative deterioration in their health status. These sce-
narios typically require the surgical team to quickly recognize, diagnose,
and manage the crisis to prevent further harm or adverse outcomes.
Another type of scenario is equipment-related failure (11 articles,
Abelson et al., 2015; Bracq et al., 2021; Gardner and AbdelFattah, 2017;
Gettman et al., 2009; Mai et al., 2020; Nguyen et al., 2015; Pena et al.,
2015; Shi et al., 2021; Sirihorachai et al., 2018; Truong et al., 2022;
Wood et al., 2023; Wunder et al., 2020). This refers to crisis scenarios
where the malfunction, misuse, or unavailability of medical equipment
disrupts the surgical procedure or patient care. These scenarios chal-
lenge the surgical team to quickly identify, troubleshoot, and resolve the
issue to minimize delays, prevent harm, and maintain patient safety.
The third category is medical team dynamics failure (7 articles,
Acero et al., 2012; Dedy et al., 2016; Nguyen et al., 2015; Pena et al.,
2015; Raison et al., 2018; Sirihorachai et al., 2018; Wood et al., 2023).
This refers to crisis scenarios where breakdowns in communication,
coordination, leadership, or decision-making within the surgical team
contribute to a critical situation. These failures can lead to delays, errors,
and compromised patient safety.
These crisis scenarios were used to train a wide range of non-
technical skills. Table 1 presents a summary of the non-technical skills
addressed in each included article. The non-technical skills receiving the
most training in crisis simulations were communication (18 articles,
Acero et al., 2012; Bierer et al., 2018; Dedy et al., 2016; Edwards et al.,
2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al.,
2009; Goldenberg et al., 2018; Lehner et al., 2017; Moorthy et al., 2006;
Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon
et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Weller et al., 2015;
Wood et al., 2023), teamwork (17 articles, Bierer et al., 2018; Dedy
et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gettman et al., 2009;
Goldenberg et al., 2018; Leithead et al., 2019; Mai et al., 2020; Moorthy
et al., 2006; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018;
Ramjeeawon et al., 2020; Rochlen et al., 2019; Wongsirimeteekul et al.,
2018; Wood et al., 2023; Wunder et al., 2020), decision making or
problem-solving (17 articles, Abelson et al., 2015; Acero et al., 2012;
Bierer et al., 2018; Burkhart et al., 2013; Dedy et al., 2016; Edwards
et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman
et al., 2009; Goldenberg et al., 2018; Moorthy et al., 2006; Pena et al.,
2015; Raison et al., 2018; Rochlen et al., 2019; Taaffe et al., 2021; Wood
et al., 2023; Wunder et al., 2020), situation awareness (17 articles,
Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy et al.,
2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah,
2017; Gettman et al., 2009; Goldenberg et al., 2018; Moorthy et al.,
2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020;
Rochlen et al., 2019; Taaffe et al., 2021; Wood et al., 2023; Wunder
et al., 2020), and leadership (13 articles, Bierer et al., 2018; Dedy et al.,
2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman
et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Moorthy et al.,
2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020;
Rochlen et al., 2019; Wood et al., 2023).
Additionally, other non-technical skills that do not align directly
with established non-technical skills rating scales were also addressed in
the training. These include team management (8 articles, Bierer et al.,
2018; Gao et al., 2021; Gettman et al., 2009; Lehner et al., 2017;
Moorthy et al., 2006; Ramjeeawon et al., 2020; Rochlen et al., 2019;
Wunder et al., 2020), condence or self-efcacy (6 articles, Burkhart
et al., 2013; Gao et al., 2021; Leithead et al., 2019; Nguyen et al., 2015;
Shi et al., 2021; Truong et al., 2022), task management (4 articles,
Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017;
Raison et al., 2018), situation monitoring (2 articles, Bierer et al., 2018;
Ramjeeawon et al., 2020), mutual support (2 articles, Bierer et al., 2018;
Ramjeeawon et al., 2020), professionalism (2 articles, Nguyen et al.,
2015; Wood et al., 2023), team structure (1 article, Bierer et al., 2018),
and management of interruptions (1 article, Sirihorachai et al., 2018).
Among the 29 articles that focused on non-technical skills, 27
attempted to assess these skills using various methods and tools.
Observation-based assessment was the most commonly used method
(21/27 articles, Acero et al., 2012; Bierer et al., 2018; Burkhart et al.,
2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner
and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018;
Fig. 2. Scenario distribution within each failure type. Panel (a): Patient health-related failures. Panel (b): Medical team-related failures. Panel (c): Equipment-
related failures.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
8
Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al.,
2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019;
Sirihorachai et al., 2018; Taaffe et al., 2021; Weller et al., 2015; Wood
et al., 2023; Wunder et al., 2020). Observation was often paired with a
behavioral rating system, such as the Non-Technical Skills for Surgeons
(NOTSS; Yule et al., 2009) used in six articles (Bierer et al., 2018; Dedy
et al., 2016; Edwards et al., 2023; Goldenberg et al., 2018; Pena et al.,
2015; Raison et al., 2018), Modied Theatre Team Non-Technical Skills
Scoring System (NOTECHS-II; Robertson et al., 2014) used in three ar-
ticles (Edwards et al., 2023; Moorthy et al., 2006; Rochlen et al., 2019),
Operating Room Teamwork Assessment Scales (ORTAS; Paige et al.,
2009a) used in one article (Paige et al., 2014), Objective Structured
Assessment of Nontechnical Skills (OSANTS; Dedy et al., 2015) used in
one article (Dedy et al., 2016), Operating Room Communication
Assessment (ORCA; Rudolph et al., 2006) used in one article (Gardner
and AbdelFattah, 2017), Team Strategies and Tools to Enhance Perfor-
mance and Patient Safety (TeamSTEPPS; Guimond et al., 2009) used in
one article (Bierer et al., 2018), Anaesthetists' Non-Technical Skills
(ANTS; Fletcher et al., 2003) used in one article (Wunder et al., 2020),
HUman Factors in intraoperative Ophthalmic Emergencies Scoring
System (HUFOES; Wood et al., 2021) used in one article (Wood et al.,
2023), Observational Teamwork Assessment for Surgery (OTAS; Undre
et al., 2007) used in one article (Ramjeeawon et al., 2020), Behavioural
Marker of Risk Index (BMRI; Mazzocco et al., 2009) used in one article
(Weller et al., 2015), and Scrub Practitioners' List of Intraoperative Non-
Technical Skills (SPLINTS; Mitchell et al., 2012) used in one article
(Edwards et al., 2023).
While simulation sessions were video-recorded in the majority of
studies (18/27 articles, Acero et al., 2012; Bierer et al., 2018; Dedy et al.,
2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman
et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Mai et al.,
2020; Moorthy et al., 2006; Nguyen et al., 2015; Pena et al., 2015;
Raison et al., 2018; Ramjeeawon et al., 2020; Sirihorachai et al., 2018;
Wongsirimeteekul et al., 2018; Wood et al., 2021; Wunder et al., 2020),
a smaller proportion used these recordings for observation-based
assessment purposes (14/27 articles, Acero et al., 2012; Bierer et al.,
2018; Dedy et al., 2015; Edwards et al., 2023; Gardner and AbdelFattah,
2017; Gettman et al., 2009; Goldenberg et al., 2018; Moorthy et al.,
2006; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ram-
jeeawon et al., 2020; Wood et al., 2023; Wunder et al., 2020). When the
observation-based assessment was not conducted on recorded video, it
was performed live during the simulation itself.
In other studies, non-technical skills were assessed using self-
assessment Likert scales (7/27 articles), some of which were home-
made (5/7 articles, Lehner et al., 2017; Mai et al., 2020; Paige et al.,
2014; Shi et al., 2021; Truong et al., 2022). Other scales used were the
Interprofessional Teamwork Scale (Paige et al., 2009b) used in one
article (Leithead et al., 2019), the Teamwork Assessment Scales (Garbee
et al., 2013) used in one article (Leithead et al., 2019), the TeamSTEPPS
teamwork attitudes questionnaire (Baker et al., 2010) used in one article
(Dedy et al., 2016), the Situation Awareness Rating System (Taylor,
2011) used in one article (Bracq et al., 2021). Finally, non-technical
skills were also evaluated with quizzes to assess knowledge of non-
technical skills and patient safety in the operating room (Dedy et al.,
2016).
3.4. RQ4. Levels of evaluation per Kirkpatrick's model
The Kirkpatrick model's four levels (Kirkpatrick and Kirkpatrick,
2006) were used to investigate training outcomes. The rst level (Re-
action), addressed in the majority of studies, examines participants'
immediate reactions, highlighting their self-reported satisfaction and
engagement (20/29 articles, Abelson et al., 2015; Acero et al., 2012;
Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy et al.,
2016; Gao et al., 2021; Gettman et al., 2009; Lehner et al., 2017; Mai
et al., 2020; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014;
Pena et al., 2015; Ramjeeawon et al., 2020; Rochlen et al., 2019; Truong
et al., 2022; Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood
et al., 2023).
The second level (Learning), also addressed in the majority of
included studies, focuses on skills, attitudes, and knowledge acquisition
as a result of training (25/29 articles, Abelson et al., 2015; Acero et al.,
2012; Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy
et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and
AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Leh-
ner et al., 2017; Leithead et al., 2019; Moorthy et al., 2006; Nguyen
et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018;
Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Taaffe
et al., 2021; Weller et al., 2015; Wood et al., 2023; Wunder et al., 2020).
The measurement tools were mostly third-party observation-based as-
sessments, self-administered questionnaires, and quizzes.
The third level (Behavior) was addressed in a few studies. It in-
vestigates changes in participants' behavior or practice as a result of the
training (6/29 articles, Dedy et al., 2016; Gettman et al., 2009; Paige
et al., 2014; Pena et al., 2015; Ramjeeawon et al., 2020; Weller et al.,
2015). In this review, Level 3 was considered to be achieved when skills
acquired during training were successfully applied in scenarios beyond
the initial training context. The transfer of skills was consistently
assessed through third-party observation of non-technical skills
performances.
The fourth level (Results) was not addressed in any of the included
studies. This level aims to measure the broader impact of training on
organizational outcomes, capturing the ultimate effectiveness of
training in achieving its intended goals.
4. Discussion
4.1. Summary of evidence
This systematic review summarizes research on simulation training
for healthcare professionals and students in crisis management within
the operating room. A comprehensive database search yielded 29 rele-
vant articles. The results show that high-delity simulation is the most
common method used for crisis management training, which is con-
ducted across various surgical specialties and often involving interdis-
ciplinary collaboration. However, there is a predominant focus on
medical professionals and students, to the detriment of nurses and other
healthcare professionals. Communication was identied as the most
critical non-technical skill targeted for training, followed by team-
working, decision-making and situation awareness. Training scenarios
were categorized into three main types: patient health-related, equip-
ment-related, and team-related challenges, with team-related challenges
being notably underutilized for training non-technical skills. The metrics
most employed to evaluate the training were third-party observation-
based assessments of non-technical skills performance. According to
Kirkpatrick's model, assessments reached a maximum of Level 3
(behavioral change), with Level 1 (participant reactions) and Level 2
(learning outcomes) almost always assessed. However, the third level
was measured less frequently, and notably, the fourth level, which
would indicate results in terms of organizational practice or patient
outcomes, was never assessed in the studies included in this review.
4.2. Simulation methods
This systematic review highlights that high-delity simulation is the
primary method used in healthcare education, aligning with existing
literature (Hanshaw and Dickerson, 2020). This prevalence can be
explained by the fact that the method provides immersive and risk-free
environments that enhance both technical and non-technical skills
(Alshehri et al., 2023; Hanshaw and Dickerson, 2020) by supporting
experiential learning, scenario replication, immediate feedback, and the
management of emotional and cognitive load (Newton and Smith, 2024;
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
9
Rogers and Franklin, 2021). However, its implementation involves
substantial costs related to scenario complexity, equipment, and facili-
tator training (Alshehri et al., 2023), as well as logistical challenges such
as team availability and travel to simulation centers, which could limit
its accessibility (Gardner et al., 2020).
Additionally, in situ simulation has gained popularity (Martin et al.,
2020) since 2018, as evidenced by the ndings of this systematic review.
This approach enhances realism by allowing participants to train in their
actual work environment, leading to potential improvements in clinical
outcomes (Calhoun et al., 2024). Despite these advantages, in situ
simulation poses challenges related to time constraints, resource allo-
cation, and its primary suitability for continuing education rather than
initial training (Gardner et al., 2020).
Faced with these limitations, virtual reality emerges as a promising
alternative. The ndings highlight the gradual digitalization of training
since 2020. This shift can be attributed not only to the increasing
accessibility of these technologies for healthcare education organiza-
tions but also to the inherent advantages of virtual reality. Virtual reality
provides cost-effective, repeatable, and standardized training experi-
ences (Pottle, 2019) while facilitating interaction with virtual patients,
colleagues, and crisis scenarios, ultimately enhancing clinical and non-
technical skills (Bracq et al., 2019; Mistry et al., 2023). Virtual reality
also supports tele-simulation, enabling broader access and global
collaboration (Almousa et al., 2021). Its ability to provide high-
frequency, low-stress repetitions makes it particularly suitable for
crisis management scenarios (Heldring et al., 2024).
Nevertheless, the under-adoption of virtual reality can be explained
by several barriers, including initial investment costs, technical exper-
tise requirements, and resistance to change in traditional educational
settings (Kyaw et al., 2019; Lee et al., 2023).
4.3. Target population
With approximately 45 % of the included studies featuring multi-
disciplinarity, this review highlights the potential for improvement in
fostering interprofessional education within simulation training. How-
ever, there remains a noticeable skew towards medical personnel such as
medical trainees, surgeons, and anesthetists at the expense of para-
medical staff like nurses or allied health professionals. This imbalance
may reect traditional institutional biases regarding team roles within
surgical settings.
The prevailing view positions the surgeon as the de facto leader
(Peters et al., 2020), a role that is reinforced by the leadership demands
of the operating room. Surgeons are expected to guide the team and
make critical decisions (Murtagh and Bezemer, 2021), as outlined in the
NOTSS taxonomy (Yule et al., 2008). Despite this traditional hierarchy,
effective performance relies on teamwork (Schmutz et al., 2019) and the
collective expertise of the entire team, including nurses, who represent a
signicant portion of the surgical team, as well as other paramedical
staff (Watkins and Hensley, 2023). This dynamic reinforces the argu-
ment for multidisciplinary simulation-based training programs (Saragih
et al., 2024; Scherer and Winokur, 2019), as evidenced by systematic
reviews on the impact of crisis resource management simulation-based
training for interprofessional and interdisciplinary teams (Fung et al.,
2015).
4.4. Outcomes
This systematic review identied three categories of crisis scenarios
used to train non-technical skills in operating room crisis management:
patient health-related failures, equipment-related failures, and team
dynamics failures. These categories align with the classications of
perioperative adverse events described in the literature, which include
diagnostic errors, tissue injuries, and technical issues related to patient
health; equipment malfunctions, medication errors, and wrong-site/
patient procedures associated with equipment or checklist failures;
and communication failures linked to team dynamics (Jung et al., 2019).
However, the results of this review reveal a pronounced emphasis on
patient health crises over medical team dynamics or equipment mal-
functions despite literature indicating that perioperative errors are often
attributable to non-technical skills related to team dynamics issues
(Allard et al., 2020; Koleva, 2020). This highlights the critical need for
training in team-related crisis management to better prepare healthcare
professionals. Scenarios focused on patient health crises may emphasize
technical skills and immediate clinical management, sidelining non-
technical skills. To address this imbalance, integrating scenarios unre-
lated to patient health, such as equipment failures or team-based deci-
sion-making, could shift the focus away from technical tasks and better
reinforce non-technical skills. Effective crisis management relies on
seamless team collaboration, making it crucial to balance scenarios that
reect real-world communication challenges, such as role ambiguity,
miscommunication under pressure, and leadership conicts. Prioritizing
team dynamics could enhance performance and help reduce adverse
events caused by communication breakdowns.
Another interesting result is that training programs often focus on
non-technical skills like communication, teamwork, and situation
awareness, reecting priorities deemed essential for crisis management.
Indeed, the Interprofessional Education Collaborative (2023) identies
communication and teamwork as core competencies that enhance
healthcare education. Programs focusing on these skills improve crisis
management competence (Buljac-Samardˇ
zi´
c et al., 2021), while
communication signicantly reduces medical errors (Weller and
Webster, 2021), and situation awareness is crucial for recognizing and
reacting to shifts in patient condition during crises (Lei and Palm, 2024).
Teamwork involves broader non-technical skills related to inter-member
interactions, which could explain why it is consistently prioritized in
training.
Finally, over half of the articles in the systematic review report a
positive effect of non-technical skills training. However, these results
rely on diverse methodological frameworks, therefore a nuanced inter-
pretation of this effectiveness is required. Positive impacts of training
are observed at Kirkpatrick's Level 1 (reaction) and Level 2 (learning),
consistent with the literature (Buljac-Samardˇ
zi´
c et al., 2021). The
transfer of trained skills, as expressed by Kirkpatrick's Level 3, was
scarcely evaluated, and no studies achieved Kirkpatrick's Level 4. This
highlights a gap in linking training to actual skills transfer and organi-
zational outcomes. Further, longer-term follow-up is required to deter-
mine if the short-term benets guide future practice and translate into
improvements in the workplace (Hanshaw and Dickerson, 2020).
In summary, this systematic review shows that the predominant use
of high-delity simulation primarily targets medical personnel, while
neglecting other essential team members, thereby limiting the potential
for effective interprofessional crisis management. This narrow focus
reduces the effectiveness of training in real-world team dynamics, as
evidenced by the underutilization of team-based scenarios. The limited
emphasis on interprofessional collaboration is further reected in the
lack of assessments at higher Kirkpatrick levels, indicating a gap be-
tween training exercises and the complex realities encountered in
practice. Consequently, the simulations fall short of fully replicating the
intricate challenges of real-world situations.
Future research should explore strategies for leveraging virtual re-
ality to overcome the limitations of high-delity simulation and examine
how trainer and learner preferences impact its integration into training
programs. As technological advancements progress, virtual reality holds
substantial potential to revolutionize healthcare education by offering
scalable, immersive, and collaborative training solutions. Future studies
could also focus on interprofessional simulation to strengthen non-
technical skills, particularly in communication and teamwork. An
exciting avenue for investigation is in the development of multi-user
virtual reality simulations (Edwards et al., 2023), which enable the
study of team dynamics in high-risk environments. Collaborative sur-
gical team training in immersive virtual reality represents a particularly
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
10
promising area for assessing the effectiveness of this approach in
enhancing team performance
5. Limitations
This systematic review has limitations, including a restricted search
across three databases for articles written exclusively in English, which
may have limited the scope of study inclusion. Incorporating other da-
tabases that index journals specializing in nursing and allied health
sciences, such as CINAHL, might have yielded more comprehensive in-
formation. However, CINAHL employs a different Boolean search
methodology, and was not accessible through the authors' institutional
resources. Moreover, the review did not include gray literature such as
conference proceedings and project reports, potentially omitting addi-
tional insights into the real-world applications of simulation training for
crisis management. These choices were primarily inuenced by time and
resource constraints. Additionally, the reporting of non-technical skills
in the source articles was inconsistent, making their identication
challenging at times. A narrative synthesis of the included studies was
conducted due to the lack of comparability in design, interventions and
heterogeneous outcomes, making it impossible to apply a single sum-
mary measure. A meta-analysis was not feasible due to the diversity of
interventions and outcomes.
6. Theoretical and practical implications
The ndings of this review offer several theoretical and practical
implications. First, they highlight the need for greater interprofessional
education, with increased involvement of nurses and allied healthcare
professionals to foster collaborative practices in crisis management.
Second, training in team-related crises should be conducted in order to
prepare healthcare professionals to effectively handle the situations they
most commonly encounter during adverse events. Finally, demon-
strating real-world skill transfer remains paramount. It is vital to
conduct follow-up evaluations of training programs using level 3 and
level 4 Kirkpatrick metrics to conrm sustained behavioral changes that
ultimately enhance organizational performance and patient outcomes.
7. Conclusion
This systematic review offers valuable insights into the current
research landscape regarding simulation programs for developing non-
technical skills to manage crises in surgical settings. It reveals a strong
preference for high-delity approaches, which are tailored to various
surgical specialties and primarily focus on enhancing skills such as
communication, teamwork, situation awareness and decision-making.
While the scenarios address a broad range of health, equipment, and
team-based failures, they mainly focus on health failures, despite the
fact that adverse events in practice are often due to team-based failures.
Although training shows positive reactions and skills acquisition,
genuine behavioral change assessments are scarce, and the impact on
organizational outcomes remains unclear. By summarizing interprofes-
sional involvement, evaluation practices, crisis training scenarios and
real-world skill transfers, this review highlights promising pathways to
optimize non-technical skills training, thereby enhancing the education
of healthcare professionals and elevating care standards in surgery.
CRediT authorship contribution statement
Inas D. Redjem: Writing – review & editing, Writing – original draft,
Visualization, Methodology, Investigation, Data curation, Conceptuali-
zation. Arnaud Huaulm´
e: Writing – review & editing, Supervision,
Conceptualization. Pierre Jannin: Writing – review & editing, Super-
vision, Methodology, Funding acquisition, Conceptualization. Estelle
Michinov: Writing – review & editing, Supervision, Methodology,
Funding acquisition, Conceptualization.
Funding statement
This work was supported by state aid managed by the French Na-
tional Research Agency under the France 2030 program, bearing the
reference ANR-21-DMES-0001.
Declaration of competing interest
The authors declare that they have no known competing nancial
interests or personal relationships that could have appeared to inuence
the work reported in this paper.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.nedt.2025.106583.
Data availability
Data and supplementary materials are available on our OSF project
page: https://osf.io/jp94c.
References
Abelson, J.S., Silverman, E., Banfelder, J., Naides, A., Costa, R., Dakin, G., 2015. Virtual
operating room for team training in surgery. Am. J. Surg. 210 (3), 585–590. https://
doi.org/10.1016/j.amjsurg.2015.01.024.
Acero, N.M., Motuk, G., Luba, J., Murphy, M., McKelvey, S., Kolb, G., Dumon, K.R.,
Resnick, A.S., 2012. Managing a surgical exsanguination emergency in the operating
room through simulation: an interdisciplinary approach. J. Surg. Educ. 69 (6),
759–765. https://doi.org/10.1016/j.jsurg.2012.06.022.
Allard, M.-A., Blani´
e, A., Brouquet, A., Benhamou, D., 2020. Learning non-technical skills
in surgery. J. Visc. Surg. 157 (3, Supplement 2), S131–S136. https://doi.org/
10.1016/j.jviscsurg.2020.03.001.
Almousa, O., Zhang, R., Dimma, M., Yao, J., Allen, A., Chen, L., Heidari, P., Qayumi, K.,
2021. Virtual reality technology and remote digital application for tele-simulation
and global medical education: an innovative hybrid system for clinical training.
Simul. Gaming 52 (5), 614–634. https://doi.org/10.1177/10468781211008258.
Alsabri, M., Boudi, Z., Lauque, D., Dias, R.D., Whelan, J.S., ¨
Ostlundh, L., Alinier, G.,
Onyeji, C., Michel, P., Liu Jr., S.W., Camargo, C.A., Lindner, T., Slagman, A.,
Bates, D.W., Tazarourte, K., Singer, S.J., Toussi, A., Grossman, S., Bellou, A., 2022.
Impact of teamwork and communication training interventions on safety culture and
patient safety in emergency departments: a systematic review. J. Patient Saf. 18 (1),
e351–e361. https://doi.org/10.1097/PTS.0000000000000782.
Alshehri, F.D., Jones, S., Harrison, D., 2023. The effectiveness of high-delity simulation
on undergraduate nursing students’ clinical reasoning-related skills: a systematic
review. Nurse Educ. Today 121, 105679. https://doi.org/10.1016/j.
nedt.2022.105679.
Baker, D.P., Amodeo, A.M., Krokos, K.J., Slonim, A., Herrera, H., 2010. Assessing
teamwork attitudes in healthcare: development of the TeamSTEPPS teamwork
attitudes questionnaire. Qual. Saf. Health Care 19 (6), e49. https://doi.org/10.1136/
qshc.2009.036129.
Bierer, J., Memu, E., Leeper, W.R., Fortin, D., Fr´
echette, E., Inculet, R., Malthaner, R.,
2018. Development of an in situ thoracic surgery crisis simulation focused on
nontechnical skill training. Ann. Thorac. Surg. 106 (1), 287–292. https://doi.org/
10.1016/j.athoracsur.2018.01.058.
Bracq, M.-S., Michinov, E., Jannin, P., 2019. Virtual reality simulation in nontechnical
skills training for healthcare professionals: a systematic review. Simul. Healthc. 14
(3), 188–194. https://doi.org/10.1097/SIH.0000000000000347.
Bracq, M.-S., Michinov, E., Le Duff, M., Arnaldi, B., Gouranton, V., Jannin, P., 2021.
Training situational awareness for scrub nurses: error recognition in a virtual
operating room. Nurse Educ. Pract. 53, 103056. https://doi.org/10.1016/j.
nepr.2021.103056.
Brogaard, L., Glerup Lauridsen, K., Løfgren, B., Krogh, K., Paltved, C., Boie, S.,
Hvidman, L., 2022. The effects of obstetric emergency team training on patient
outcome: a systematic review and meta-analysis. Acta Obstet. Gynecol. Scand. 101
(1), 25–36. https://doi.org/10.1111/aogs.14263.
Buljac-Samardzic, M., Doekhie, K.D., van Wijngaarden, J.D.H., 2020. Interventions to
improve team effectiveness within health care: a systematic review of the past
decade. Hum. Resour. Health 18, 2. https://doi.org/10.1186/s12960-019-0411-3.
Buljac-Samardˇ
zi´
c, M., Dekker-van Doorn, C.M., Maynard, M.T., 2021. What do we really
know about crew resource management in healthcare?: an umbrella review on crew
resource management and its effectiveness. J. Patient Saf. 17 (8), e929–e958.
https://doi.org/10.1097/PTS.0000000000000816.
Burkhart, H.M., Riley, J.B., Lynch, J.J., Suri, R.M., Greason, K.L., Joyce, L.D., Nuttall, G.
A., Stulak, J., Schaff, H.V., Dearani, J.A., 2013. Simulation-based postcardiotomy
extracorporeal membrane oxygenation crisis training for thoracic surgery residents.
Ann. Thorac. Surg. 95 (3), 901–906. https://doi.org/10.1016/j.
athoracsur.2012.12.015.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
11
Calhoun, A.W., Cook, D.A., Genova, G., Motamedi, S.M.K., Waseem, M., Carey, R.,
Hanson, A., Chan, J.C.K., Camacho, C., Harwayne-Gidansky, I., Walsh, B., White, M.,
Geis, G., Monachino, A.M., Maa, T., Posner, G., Li, D.L., Lin, Y., 2024. Educational
and patient care impacts of in situ simulation in healthcare: a systematic review.
Simul. Healthc. 19 (1S), S23–S31. https://doi.org/10.1097/
SIH.0000000000000773.
Cumin, D., Boyd, M.J., Webster, C.S., Weller, J.M., 2013. A systematic review of
simulation for multidisciplinary team training in operating rooms. Simul. Healthc. 8
(3), 171–179. https://doi.org/10.1097/SIH.0b013e31827e2f4c.
Dedy, N.J., Bonrath, E.M., Zevin, B., Grantcharov, T.P., 2013. Teaching nontechnical
skills in surgical residency: a systematic review of current approaches and outcomes.
Surgery 154 (5), 1000–1008. https://doi.org/10.1016/j.surg.2013.04.034.
Dedy, N.J., Szasz, P., Louridas, M., Bonrath, E.M., Husslein, H., Grantcharov, T.P., 2015.
Objective structured assessment of nontechnical skills: reliability of a global rating
scale for the in-training assessment in the operating room. Surgery 157 (6),
1002–1013. https://doi.org/10.1016/j.surg.2014.12.023.
Dedy, N.J., Bonrath, E.M., Ahmed, N., Grantcharov, T.P., 2016. Structured training to
improve nontechnical performance of junior surgical residents in the operating
room: a randomized controlled trial. Ann. Surg. 263 (1), 43–49. https://doi.org/
10.1097/SLA.0000000000001186.
Edwards, T.C., Soussi, D., Gupta, S., Khan, S., Patel, A., Patil, A., Liddle, A.D., Cobb, J.P.,
Logishetty, K., 2023. Collaborative team training in virtual reality is superior to
individual learning for performing complex open surgery: a randomized controlled
trial. Ann. Surg. 278 (6), 850–857. https://doi.org/10.1097/
SLA.0000000000006079.
Fletcher, G., Flin, R., McGeorge, P., Glavin, R., Maran, N., Patey, R., 2003. Anaesthetists’
non-technical skills (ANTS): evaluation of a behavioural marker system. Br. J.
Anaesth. 90 (5), 580–588. https://doi.org/10.1093/bja/aeg112.
Flin, R., O’Connor, P., Crichton, M., 2008. Safety at the Sharp End: A Guide to Non-
technical Skills. Ashgate Publishing, Ltd.
Fung, L., Boet, S., Bould, M.D., Qosa, H., Perrier, L., Tricco, A., Tavares, W., Reeves, S.,
2015. Impact of crisis resource management simulation-based training for
interprofessional and interdisciplinary teams: a systematic review. J. Interprof. Care
29 (5), 433–444. https://doi.org/10.3109/13561820.2015.1017555.
Gao, P., Wang, C., Liu, S., Tran, K.C., Wen, Q., 2021. Simulation of operating room crisis
management—hypotension training for pre-clinical students. BMC Med. Educ. 21
(1), 60. https://doi.org/10.1186/s12909-020-02477-8.
Garbee, D., Paige, J., Bonanno, L., 2013. Effectiveness of teamwork and communication
education using an interprofessional high-delity human patient simulation critical
care code. J. Nurs. Educ. 3, 1–12.
Gardner, A.K., AbdelFattah, K., 2017. Comparison of simulation-based assessments and
faculty ratings for general surgery resident milestone evaluation: are they telling the
same story? Am. J. Surg. 214 (3), 547–553. https://doi.org/10.1016/j.
amjsurg.2016.07.025.
Gardner, A., DeSandro, S., Pillow, M.T., Ahmed, R., 2020. Optimizing interprofessional
education with in situ simulation. In: Paige, J.T., Sonesh, S.C., Garbee, D.D.,
Bonanno, L.S. (Eds.), Comprehensive Healthcare Simulation: InterProfessional Team
Training and Simulation. Springer International Publishing, pp. 105–120. https://
doi.org/10.1007/978-3-030-28845-7_8.
Gettman, M.T., Pereira, C.W., Lipsky, K., Wilson, T., Arnold, J.J., Leibovich, B.C.,
Karnes, R.J., Dong, Y., 2009. Use of high delity operating room simulation to assess
and teach communication, teamwork and laparoscopic skills: initial experience.
J. Urol. 181 (3), 1289–1296. https://doi.org/10.1016/j.juro.2008.11.018.
Goldenberg, M.G., Fok, K.H., Ordon, M., Pace, K.T., Lee, J.Y., 2018. Simulation-based
laparoscopic surgery crisis resource management training—predicting technical and
nontechnical skills. J. Surg. Educ. 75 (4), 1113–1119. https://doi.org/10.1016/j.
jsurg.2017.11.011.
Gordo, C., Mateo, R., Labiano, J., Nú˜
nez-C´
ordoba, J.M., 2021. Ranking quality and
patient safety challenges: a nationwide survey of healthcare quality experts from
general hospitals in Spain. J. Patient Saf. 17 (8), 541–547. https://doi.org/10.1097/
PTS.0000000000000675.
Guimond, M.E., Sole, M.L., Salas, E., 2009. TeamSTEPPS. Am. J. Nurs. 109 (11), 66.
https://doi.org/10.1097/01.NAJ.0000363359.84377.27.
Hanshaw, S.L., Dickerson, S.S., 2020. High delity simulation evaluation studies in
nursing education: a review of the literature. Nurse Educ. Pract. 46, 102818. https://
doi.org/10.1016/j.nepr.2020.102818.
Heldring, S., Lindstr¨
om, V., Jirwe, M., Wihlborg, J., 2024. Exploring ambulance
clinicians’ clinical reasoning when training mass casualty incidents using virtual
reality: a qualitative study. Scand. J. Trauma Resusc. Emerg. Med. 32 (1), 90.
https://doi.org/10.1186/s13049-024-01255-5.
Hong, Q.N., F`
abregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P.,
Gagnon, M.-P., Grifths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I.,
Pluye, P., 2018. The Mixed Methods Appraisal Tool (MMAT) version 2018 for
information professionals and researchers. Educ. Inf. 34 (4), 285–291. https://doi.
org/10.3233/EFI-180221.
Howard, S.K., Gaba, D.M., Fish, K.J., Yang, G., Sarnquist, F.H., 1992. Anesthesia crisis
resource management training: teaching anesthesiologists to handle critical
incidents. Aviat. Space Environ. Med. 63 (9), 763–770.
Interprofessional Education Collaborative, 2023. IPEC Core Competencies for
Interprofessional Collaborative Practice: Version 3.
Jung, J.J., Elfassy, J., Jüni, P., Grantcharov, T., 2019. Adverse events in the operating
room: denitions, prevalence, and characteristics. A systematic review. World J.
Surg. 43 (10), 2379–2392. https://doi.org/10.1007/s00268-019-05048-1.
Kirkpatrick, D.L., Kirkpatrick, J.D., 2006. Evaluating Training Programs: The Four
Levels. Berrett-Koehler Publishers.
Koleva, S.I., 2020. A literature review exploring common factors contributing to never
events in surgery. J. Perioper. Pract. 30 (9), 256–264. https://doi.org/10.1177/
1750458919886182.
Kyaw, B.M., Saxena, N., Posadzki, P., Vseteckova, J., Nikolaou, C.K., George, P.P.,
Divakar, U., Masiello, I., Kononowicz, A.A., Zary, N., Tudor Car, L., 2019. Virtual
reality for health professions education: systematic review and meta-analysis by the
digital health education collaboration. J. Med. Internet Res. 21 (1), e12959. https://
doi.org/10.2196/12959.
Lee, K., Wang, F., Reeder, B., 2023. Development of an immersive virtual reality platform
for innovation in nursing education. In: Stephanidis, C., Antona, M., Ntoa, S.,
Salvendy, G. (Eds.), HCI International 2023 Posters. Springer Nature Switzerland,
pp. 227–234. https://doi.org/10.1007/978-3-031-36004-6_31.
Lehner, M., Heimberg, E., Hoffmann, F., Heinzel, O., Kirschner, H.-J., Heinrich, M., 2017.
Evaluation of a pilot project to introduce simulation-based team training to pediatric
surgery trauma room care. Int. J. Pediatr. 2017, 1–6. https://doi.org/10.1155/
2017/9732316.
Lei, C., Palm, K., 2024. Crisis resource management training in medical simulation. In:
StatPearls. StatPearls Publishing. http://www.ncbi.nlm.nih.gov/books/N
BK551708/.
Leithead, J., Garbee, D.D., Yu, Q., Rusnak, V.V., Kiselov, V.J., Zhu, L., Paige, J.T., 2019.
Examining interprofessional learning perceptions among students in a simulation-
based operating room team training experience. J. Interprof. Care 33 (1), 26–31.
https://doi.org/10.1080/13561820.2018.1513464.
Mai, C.L., Wongsirimeteekul, P., Petrusa, E., Minehart, R., Hemingway, M., Pian-
Smith, M., Eromo, E., Phitayakorn, R., 2020. Prevention and management of
operating room re: an interprofessional operating room team simulation case.
MedEdPORTAL, 10871. https://doi.org/10.15766/mep_2374-8265.10871.
Martin, A., Cross, S., Attoe, C., 2020. The use of in situ simulation in healthcare
education: current perspectives. Adv. Med. Educ. Pract. 11, 893–903. https://doi.
org/10.2147/AMEP.S188258.
Mazzocco, K., Petitti, D.B., Fong, K.T., Bonacum, D., Brookey, J., Graham, S., Lasky, R.E.,
Sexton, J.B., Thomas, E.J., 2009. Surgical team behaviors and patient outcomes. Am.
J. Surg. 197 (5), 678–685. https://doi.org/10.1016/j.amjsurg.2008.03.002.
Meri´
en, A.E.R., van de Ven, J., Mol, B.W., Houterman, S., Oei, S.G., 2010.
Multidisciplinary team training in a simulation setting for acute obstetric
emergencies: a systematic review. Obstet. Gynecol. 115 (5), 1021–1031. https://doi.
org/10.1097/AOG.0b013e3181d9f4cd.
Mistry, D., Brock, C.A., Lindsey, T., 2023. The present and future of virtual reality in
medical education: a narrative review. Cureus 15 (12), e51124. https://doi.org/
10.7759/cureus.51124.
Mitchell, L., Flin, R., Yule, S., Mitchell, J., Coutts, K., Youngson, G., 2012. Evaluation of
the scrub practitioners’ list of intraoperative non-technical skills system. Int. J. Nurs.
Stud. 49 (2), 201–211. https://doi.org/10.1016/j.ijnurstu.2011.08.012.
Moorthy, K., Munz, Y., Forrest, D., Pandey, V., Undre, S., Vincent, C., Darzi, A., 2006.
Surgical crisis management skills training and assessment: a stimulation-based
approach to enhancing operating room performance. Ann. Surg. 244 (1), 139–147.
https://doi.org/10.1097/01.sla.0000217618.30744.61.
Murtagh, G.M., Bezemer, J., 2021. ‘Hm no-one says anything, did you notice?’:
communication in the operating theatre amidst a hierarchy of expertise. Commun.
Med. 17 (1). https://doi.org/10.1558/cam.15138.
Newton, J., Smith, A.D.A.C., 2024. Developing emotional preparedness and mental
resilience through high-delity simulation: a “bridge too far” for institutions
teaching major trauma management and mass-casualty medicine? BMC Med. Educ.
24 (1), 544. https://doi.org/10.1186/s12909-024-05526-8.
Nguyen, N., Elliott, J.O., Watson, W.D., Dominguez, E., 2015. Simulation improves
nontechnical skills performance of residents during the perioperative and
intraoperative phases of surgery. J. Surg. Educ. 72 (5), 957–963. https://doi.org/
10.1016/j.jsurg.2015.03.005.
Ounounou, E., Aydin, A., Brunckhorst, O., Khan, M.S., Dasgupta, P., Ahmed, K., 2019.
Nontechnical skills in surgery: a systematic review of current training modalities.
J. Surg. Educ. 76 (1), 14–24. https://doi.org/10.1016/j.jsurg.2018.05.017.
Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D.,
Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J.,
Grimshaw, J.M., Hr´
objartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E.,
McDonald, S., Moher, D., 2021. The PRISMA 2020 statement: an updated guideline
for reporting systematic reviews. BMJ 372, n71. https://doi.org/10.1136/bmj.n71.
Paige, J.T., Aaron, D.L., Yang, T., Howell, D.S., Chauvin, S.W., 2009a. Improved
operating room teamwork via SAFETY prep: a rural community hospital’s
experience. World J. Surg. 33 (6), 1281. https://doi.org/10.1007/s00268-009-9952-
2.
Paige, J.T., Kozmenko, V., Yang, T., Paragi Gururaja, R., Hilton, C.W., Cohn, I.,
Chauvin, S.W., 2009b. High-delity, simulation-based, interdisciplinary operating
room team training at the point of care. Surgery 145 (2), 138–146. https://doi.org/
10.1016/j.surg.2008.09.010.
Paige, J.T., Garbee, D.D., Kozmenko, V., Yu, Q., Kozmenko, L., Yang, T., Bonanno, L.,
Swartz, W., 2014. Getting a head start: high-delity, simulation-based operating
room team training of interprofessional students. J. Am. Coll. Surg. 218 (1),
140–149. https://doi.org/10.1016/j.jamcollsurg.2013.09.006.
Panagioti, M., Khan, K., Keers, R.N., Abuzour, A., Phipps, D., Kontopantelis, E.,
Bower, P., Campbell, S., Haneef, R., Avery, A.J., Ashcroft, D.M., 2019. Prevalence,
severity, and nature of preventable patient harm across medical care settings:
systematic review and meta-analysis. BMJ (Clin. Res. Ed.) 366, l4185. https://doi.
org/10.1136/bmj.l4185.
Pena, G., Altree, M., Field, J., Sainsbury, D., Babidge, W., Hewett, P., Maddern, G., 2015.
Nontechnical skills training for the operating room: a prospective study using
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
12
simulation and didactic workshop. Surgery 158 (1), 300–309. https://doi.org/
10.1016/j.surg.2015.02.008.
Peters, W., Picchioni, A., Fleshman, J.W., 2020. Surgical leadership. Clin. Colon Rectal
Surg. 33 (4), 233–237. https://doi.org/10.1055/s-0040-1712976.
Pottle, J., 2019. Virtual reality and the transformation of medical education. Future
Healthc. J. 6 (3), 181–185. https://doi.org/10.7861/fhj.2019-0036.
Raison, N., Ahmed, K., Abe, T., Brunckhorst, O., Novara, G., Buf, N., McIlhenny, C., Van
Der Poel, H., Van Hemelrijck, M., Gavazzi, A., Dasgupta, P., 2018. Cognitive training
for technical and non-technical skills in robotic surgery: a randomised controlled
trial. BJU Int. 122 (6), 1075–1081. https://doi.org/10.1111/bju.14376.
Ramjeeawon, A., Sharrock, A.E., Morbi, A., Martin, G., Riga, C., Bicknell, C., 2020. Using
fully-immersive simulation training with structured debrief to improve nontechnical
skills in emergency endovascular surgery. J. Surg. Educ. 77 (5), 1300–1311. https://
doi.org/10.1016/j.jsurg.2020.03.023.
Richardson, W.S., Wilson, M.C., Nishikawa, J., Hayward, R.S., 1995. The well-built
clinical question: a key to evidence-based decisions. ACP J. Club 123 (3), A12–A13.
Robertson, E.R., Hadi, M., Morgan, L.J., Pickering, S.P., Collins, G., New, S., Grifn, D.,
McCulloch, P., Catchpole, K.C., 2014. Oxford NOTECHS II: a modied theatre team
non-technical skills scoring system. PLoS One 9 (3), e90320. https://doi.org/
10.1371/journal.pone.0090320.
Robertson, J.M., Dias, R.D., Yule, S., Smink, D.S., 2017. Operating room team training
with simulation: a systematic review. J. Laparoendosc. Adv. Surg. Tech. A 27 (5),
475–480. https://doi.org/10.1089/lap.2017.0043.
Rochlen, L.R., Malloy, K.M., Chang, H., Kim, S., Guichard, L., Cassidy, R., Zisblatt, L.,
2019. Pilot one-hour multidisciplinary team training simulation intervention in the
operating room improves team nontechnical skills. J. Educ. Perioper. Med. 21 (2).
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6972973/.
Rogers, B.A., Franklin, A.E., 2021. Cognitive load experienced by nurses in simulation-
based learning experiences: an integrative review. Nurse Educ. Today 99, 104815.
https://doi.org/10.1016/j.nedt.2021.104815.
Rudolph, J.W., Simon, R., Dufresne, R.L., Raemer, D.B., 2006. There’s no such thing as
“nonjudgmental” debrieng: a theory and method for debrieng with good
judgment. Simul. Healthc. 1 (1), 49–55. https://doi.org/10.1097/01266021-
200600110-00006.
Saragih, I.D., Tarihoran, D.E.T.A.U., Lin, W.-T., Lee, B.-O., 2024. Outcomes of scenario-
based simulation courses in nursing education: a systematic review and meta-
analysis. Nurse Educ. Today 136, 106145. https://doi.org/10.1016/j.
nedt.2024.106145.
Scherer, K., Winokur, R.S., 2019. Multidisciplinary team training simulation in
interventional radiology. Tech. Vasc. Interv. Radiol. 22 (1), 32–34. https://doi.org/
10.1053/j.tvir.2018.10.007.
Schmutz, J.B., Meier, L.L., Manser, T., 2019. How effective is teamwork really? The
relationship between teamwork and performance in healthcare teams: a systematic
review and meta-analysis. BMJ Open 9 (9), e028280. https://doi.org/10.1136/
bmjopen-2018-028280.
Shi, R., Marin-Nevarez, P., Hasty, B., Roman-Micek, T., Hirx, S., Anderson, T.,
Schmiederer, I., Fanning, R., Goldhaber-Fiebert, S., Austin, N., Lau, J.N., 2021.
Operating room in situ interprofessional simulation for improving communication
and teamwork. J. Surg. Res. 260, 237–244. https://doi.org/10.1016/j.
jss.2020.11.051.
Sirihorachai, R., Aebersold, M.L., Sarter, N.B., Harris, M., Marsh, V., Redman, R.W.,
2018. Examining interruptions in the operating room using simulation. Clin. Simul.
Nurs. 24, 14–24. https://doi.org/10.1016/j.ecns.2018.08.004.
Smits, M., Langelaan, M., de Groot, J., Wagner, C., 2021. Examining causes and
prevention strategies of adverse events in deceased hospital patients: a retrospective
patient record review study in the Netherlands. J. Patient Saf. 17 (4), 282–289.
https://doi.org/10.1097/PTS.0000000000000586.
Taaffe, J.P., Kabbani, L.S., Goltz, C.J., Bath, J., Mattos, M.A., Caputo, F.J., Singh, P.,
Vogel, T.R., 2021. Feasibility and evaluation of surgical simulation with developed
crisis scenarios: a comparison of performance by vascular surgery training
paradigms. J. Surg. Educ. 78 (6), 2110–2116. https://doi.org/10.1016/j.
jsurg.2021.05.007.
Taylor, R.M., 2011. Situational awareness rating technique (Sart): the development of a
tool for aircrew systems design. In: Situational Awareness. Routledge.
Thim, S., Henriksen, T.B., Laursen, H., Schram, A.L., Paltved, C., Lindhard, M.S., 2022.
Simulation-based emergency team training in pediatrics: a systematic review.
Pediatrics 149 (4), e2021054305. https://doi.org/10.1542/peds.2021-054305.
Truong, H., Qi, D., Ryason, A., Sullivan, A.M., Cudmore, J., Alfred, S., Jones, S.B.,
Parra, J.M., De, S., Jones, D.B., 2022. Does your team know how to respond safely to
an operating room re? Outcomes of a virtual reality, AI-enhanced simulation
training. Surg. Endosc. 36 (5), 3059–3067. https://doi.org/10.1007/s00464-021-
08602-y.
Undre, S., Sevdalis, N., Healey, A.N., Darzi, A., Vincent, C.A., 2007. Observational
teamwork assessment for surgery (OTAS): renement and application in urological
surgery. World J. Surg. 31 (7), 1373–1381. https://doi.org/10.1007/s00268-007-
9053-z.
Watkins, S.C., Hensley, N.B., 2023. Team dynamics in the operating room: how is team
performance optimized? Anesthesiol. Clin. 41 (4), 775–787. https://doi.org/
10.1016/j.anclin.2023.05.004.
Weller, J.M., Webster, C.S., 2021. Normalising good communication in hospital teams.
Br. J. Anaesth. 126 (4), 758–760. https://doi.org/10.1016/j.bja.2020.12.036.
Weller, J., Cumin, D., Torrie, J., Boyd, M., Civil, I., Madell, D., MacCormick, A.,
Gurisinghe, N., Crossan, M., Ng, W.L., Johnson, S., Corter, A., Lee, T., Selander, L.,
Cokorilo, M., Merry, A.F., 2015. Multidisciplinary operating room simulation-based
team training to reduce treatment errors: a feasibility study in New Zealand
hospitals, 128 (1418).
Wongsirimeteekul, P., Mai, C.L., Petrusa, E., Minehart, R., Hemingway, M., Pian-
Smith, M., Eromo, E., Phitayakorn, R., 2018. Identifying and managing
intraoperative arrhythmia: a multidisciplinary operating room team simulation case.
MedEdPORTAL, 10688. https://doi.org/10.15766/mep_2374-8265.10688.
Wood, T.C., Maqsood, S., Zoutewelle, S., Nanavaty, M.A., Rajak, S., 2021. Development
of the HUman Factors in intraoperative Ophthalmic Emergencies Scoring System
(HUFOES) for non-technical skills in cataract surgery. Eye (Lond.) 35 (2), 616–624.
https://doi.org/10.1038/s41433-020-0921-1.
Wood, T.C., Maqsood, S., Saunders, A., Sancha, W., Nanavaty, M.A., Wearne, M.,
Rajak, S., 2023. Non-technical skills simulation-based training model for managing
intraoperative posterior capsule rupture during cataract surgery. Eye 37 (3),
474–479. https://doi.org/10.1038/s41433-022-01962-2.
Wunder, L., Gomez, N.A.G., Gonzalez, J.E., Mitzova-Vladinov, G., Cacchione, M.,
Mato, J., Foronda, C.L., Groom, J.A., 2020. Fire in the operating room: use of mixed
reality simulation with nurse anesthesia students. Informatics 7 (4), 40. https://doi.
org/10.3390/informatics7040040.
Yucel, C., Hawley, G., Terzioglu, F., Bogossian, F., 2020. The effectiveness of simulation-
based team training in obstetrics emergencies for improving technical skills: a
systematic review. Simul. Healthc. 15 (2), 98–105. https://doi.org/10.1097/
SIH.0000000000000416.
Yule, S., Flin, R., Maran, N., Rowley, D., Youngson, G., Paterson-Brown, S., 2008.
Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS
behavior rating system. World J. Surg. 32 (4), 548–556. https://doi.org/10.1007/
s00268-007-9320-z.
Yule, S., Rowley, D., Flin, R., Maran, N., Youngson, G., Duncan, J., Paterson-Brown, S.,
2009. Experience matters: comparing novice and expert ratings of non-technical
skills using the NOTSS system. ANZ J. Surg. 79 (3), 154–160. https://doi.org/
10.1111/j.1445-2197.2008.04833.x.
I.D. Redjem et al.
Nurse Education Today 147 (2025) 106583
13
