Content uploaded by Lotte Abildgren
Author content
All content in this area was uploaded by Lotte Abildgren on May 16, 2022
Content may be subject to copyright.
Available via license: CC BY 4.0
Content may be subject to copyright.
Abildgrenetal. Advances in Simulation (2022) 7:12
https://doi.org/10.1186/s41077-022-00207-2
RESEARCH
The eectiveness ofimproving
healthcare teams’ human factor skills using
simulation-based training: asystematic review
Lotte Abildgren1,2,3* , Malte Lebahn‑Hadidi3,4, Christian Backer Mogensen3, Palle Toft1,5, Anders Bo Nielsen2,5,6,
Tove Faber Frandsen7, Sune Vork Steffensen4,8,9,10 and Lise Hounsgaard2,11,12
Abstract
Background: Simulation‑based training used to train healthcare teams’ skills and improve clinical practice has
evolved in recent decades. While it is evident that technical skills training is beneficial, the potential of human factor
training has not been described to the same extent. Research on human factor training has been limited to marginal
and acute care scenarios and often to validate instruments. This systematic review aimed to investigate the effective‑
ness of simulation‑based training in improving in‑hospital qualified healthcare teams’ human factor skills.
Method: A review protocol outlining the study was registered in PROSPERO. Using the PRISMA guidelines, the
systematic search was conducted on September 28th, 2021, in eight major scientific databases. Three independent
reviewers assessed title and abstract screening; full texts were evaluated by one reviewer. Content analysis was used
to evaluate the evidence from the included studies.
Results: The search yielded 19,767 studies, of which 72 were included. The included studies were published between
2004 and 2021 and covered research from seven different in‑hospital medical specialisms. Studies applied a wide
range of assessment tools, which made it challenging to compare the effectiveness of human factor skills training
across studies. The content analysis identified evidence for the effectiveness. Four recurring themes were identified:
(1) Training human factor skills in qualified healthcare teams; (2) assessment of human factor skills; (3) combined
teaching methods, and (4) retention and transfer of human factor skills. Unfortunately, the human factor skills assess‑
ments are variable in the literature, affecting the power of the result.
Conclusion: Simulation‑based training is a successful learning tool to improve qualified healthcare teams’ human
factor skills. Human factor skills are not innate and appear to be trainable similar to technical skills, based on the find‑
ings of this review. Moreover, research on retention and transfer is insufficient. Further, research on the retention and
transfer of human factor skills from simulation‑based training to clinical practice is essential to gain knowledge of the
effect on patient safety.
Keywords: Systematic review, Simulation‑based training, Medical simulation, Human factor skills, Non‑technical skills
(NTS), Adverse events, Teamwork, Crisis resource management (CRM), Qualified healthcare team, In‑hospital
© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visithttp:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco
mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Open Access
*Correspondence: lotte.abildgren@rsyd.dk
3 Emergency Research Unit, Hospital Sønderjylland, University Hospital
of Southern Denmark, Odense, Denmark
Full list of author information is available at the end of the article
Page 2 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Background
Adverse events1 are common in hospitals all over the
world. ey cause higher mortality and morbidity, along
with more pain and increased healthcare costs [1]. Since
2004, the number of reported adverse events in Den-
mark has increased and has stabilised at a relatively high
level [2]. e Danish Patient Safety Strategy [3] has an
organisational approach that addresses adverse events
by providing knowledge through guidelines, e-learning,
and newsletters [4, 5]. Providing knowledge implies
that adverse events might be avoided through enhanced
guidelines and safety procedures. However, several stud-
ies find that adverse events often occur in non-routine,
complex environments due to interactions between
humans and the systems in which they work. ese
interactions are modifiable due to learning skills (e.g.
leadership-followership, decision-making and coordina-
tion) rather than lack of knowledge [6–9]. e medical
simulation and patient safety literature most often refer
to these aspects as non-technical skills, crisis resource
management or interpersonal relations [9–14]. ese
common concepts are too limited, however, since they
specifically define competence in terms of what is lacking
(non-technical skills), what it is for (crises resource man-
agement) or interaction between people (interpersonal
relations). e comprehensive concept of human factors
includes broader aspects of human interaction, includ-
ing social skills, cognitive skills and decision-making. It
emphasises how the environment, the organisation and
human psychology interact [15, 16]. Based on this reflec-
tion, this article will use human factors skills (HFS) as the
terminology for the skills in focus. Patient safety reports
and root cause analysis indicate that adverse events occur
in interactions between technology, organisation and
human factors, and adverse events are about understand-
ing the interactions among humans and other elements of
a system, including social and cognitive structures [1, 2,
17]. An example is the relocation of healthcare personnel
from their everyday work to COVID-19 units [18]. is
challenged even highly competent personnel and might
have caused an increased number of human errors. Per-
sonnel had to adapt to unfamiliar technical and cognitive
procedures and new surroundings, complications, col-
leagues and workflows. e Danish Patient Safety Data-
base shows a 32% increase in reported adverse events in
2020 [19], with a peak at the beginning of the COVID-19
pandemic.
Research indicates that simulation-based training
(SBT) is a safe and effective tool to develop and increase
competencies in healthcare [20]. However, existing
reviews focus on technical skills (TS), self-confidence,
self-efficacy and the effectiveness of SBT for unquali-
fied healthcare students [21–24] and develop unqualified
healthcare students’ HFS [25, 26]. SBT has been found
to refine qualified healthcare teams’ TS, self-efficacy and
confidence [24, 27]. Existing studies of qualified health-
care teams’ HFS focus on developing curricula, specific
settings or situations or testing new evaluation or rat-
ing instruments [28–32]. Buljac-Samardzic et al. [33]
explored interventions that improved team effectiveness
and concluded that SBT enhances teamwork, though
interventions studies were limited to specific situations,
settings and outcomes. As mentioned, HFS are crucial
to reducing adverse events [34], but evidence concern-
ing the effectiveness of SBT to refine qualified healthcare
teams’ use of HFS is sparse. ere is a need for additional
knowledge about the effectiveness of developing HFS in
qualified healthcare teams with SBT.
Aim
is systematic review aimed to investigate the effective-
ness of in-hospital simulation-based training as a learn-
ing and teaching method to develop qualified healthcare
teams’ human factor skills.
Methods
e AMSTAR 2-criteria (A MeaSurement Tool to Assess
systematic Reviews) were used to prepare the review
[35]. e review report follows the Preferred Report-
ing Items for Systematic Reviews and Meta-Analysis
(PRISMA) statement [36]. Details of the protocol were
registered in the International Prospective Register
of Systematic Reviews (PROSPERO) [37] (record ID:
CRD42021118670).
Search strategy
SPICE (Setting, Perspective/population, Intervention,
Comparison and Evaluation) [38], an alternative to the
qualitative conceptualising model PICO [39], provided
a framework for the formulation of questions, keywords
and the search process. e SPICE elements were out-
lined: Setting = in-hospital healthcare specialisms and
units; Population = all authorised qualified clinical
healthcare personnel, apart from dentists and pharma-
cologists; Intervention = using SBT to teach HFS; Com-
parison = SBT compared to classroom teaching or no
training; and Evaluation = improvements in the person-
nel’s HFS.
1 Adverse events: an event that results in injury or risk of injury during health
professional activity. e incident is unintentional and includes known and
unknown events and errors that are not due to the patient’s illness and that
are either harmful or could have been harmful (near-accident).
Page 3 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Boolean operators were used, combining keywords
and blocks. Furthermore, the databases’ unique thesauri,
truncation, phrase searches and proximity searches were
included. An experienced information specialist (author
TFF) optimised the search. Publications in English, Dan-
ish, Norwegian and Swedish were deemed eligible.
e following databases were searched: CINAHL
(EBSCO), Cochrane Library, EMBASE™ (OVID), ERIC
(EBSCO), MEDLINE® (OVID), PsycINFO (OVID), SCO-
PUS and Teacher Reference Centre (EBSCO), September
28th, 2021. Search histories are available in Supplement A.
Study selection andcritical appraisal
Covidence [40], a screening and data extraction tool,
was used in the study selection process. Except for
reviews, research protocols and conference abstracts,
all study design and publication types were included.
Authors LA, MLH and ABN individually performed
the title and abstract screening using a standardised
pre-piloted guide of inclusion and exclusion criteria
(Table 1). Communication with patients or relatives
and virtual reality were excluded as the focus was on
the performance of qualified healthcare teams. Stud-
ies using role-play were excluded because some team
members role-play it does not resemble the everyday
practice where every team member interacts due to the
situation and competencies. e role-playing personnel
has a role and a script and therefore only acts if given a
significant task.
Conflicts were resolved through dialogue. LA subse-
quently selected eligible studies for inclusion by full-
text reading, and, in cases of doubt, the consensus was
achieved by consulting the authors MLH, ABN, LH and
SVS. Each study was scrutinised for validity, reliability,
generalisability and replicability of the results using the
Critical Appraisal Skills Programme checklists (CASP)
[41], Mixed Methods Appraisal Tool (MMAT) [42] or
Critical Appraisal of a Survey [43]. e studies were
labelled with either a high, medium or low-reliability rat-
ing for use in the analysis of effectiveness.
The analysis process
Content analysis [44, 45] was used to assess the effec-
tiveness. Content analysis is a systematic and objective
research method that enables qualitative and quantita-
tive content analysis. Stemler’s inductive technique was
used to analyse the content. From open coding to cre-
ating themes and abstraction [44]. e following top-
ics framed the content analysis: characteristics, target
population, HFS focus, intervention type and content,
type of assessment, outcome, results and limitations,
summaries of intervention effects for each study. Due
to the variation of the included study types, all assess-
ments and methods were analysed and categorised.
Every theme was verified and, where necessary, revised
or split into two.
Ethical consideration
Ethical approval was not deemed necessary because data
was from previously published studies, but the study
meet(s) the claims of the Helsinki Declaration [46].
Results
e initial search identified 34,846 publications, repre-
senting 19,767 unique studies, after removing duplicates.
After title and abstract screening, 521 studies were iden-
tified for full-text screening, of which 72 were included
for data extraction and synthesis. is process is shown
in the PRISMA flow diagram (Fig.1).
Table 1 Inclusion and exclusion criteria
Page 4 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Result ofquality assessment ofincluded studies
e included studies were of varying quality, as shown in
Table2. e assessment included factors, such as unsuit-
able assessments methods, unclear selection methods, and
uneven weighting of HFS and TS, favouring TS in assess-
ing effectiveness. No studies were excluded following the
quality assessment; however, it was used as an indicator of
validity and reliability of the effectiveness of HFS training.
Fig. 1 PRISMA flow diagram of the screening and selection process
Page 5 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Study characteristics
Included studies were published between 2004 and
2021 and were conducted mostly (n = 70) in Western
countries. e 72 studies used 51 different assessment
methods to measure the outcome of the team training
interventions, including pre-tests, peri-tests and post-
tests, (un)blinded ratings, self-assessments, surveys and
interviews. e methods were validated (n = 30), non-
validated or no information about validation (n = 14)
and modified versions of validated (n = 9) instrument.
e studies reported SBT settings such as simulation
centres (n = 36), in-situ training (n = 24) and the use of
both centre and in-situ training (n = 7). A broad varia-
tion was seen in the size and range of the studies (n =
7 to 675 participants) and represented SBT within seven
different in-hospital medical specialisms: anaesthesiology
(n = 7), emergency medicine (n = 20), intensive care (n
= 9), internal medicine (n = 2), obstetrics (n = 12), pae-
diatrics (n = 6) and surgery (n = 15). A range of teach-
ing methods were used: SBT (n = 30); SBT and didactics
(n = 34); SBT, didactics and workshops (n = 6); and SBT
and workshops (n = 1).
e courses in the included studies were mostly stand-
alone (n = 51), meaning not part of formal educational
(n = 18) progress. e participants were either voluntary
(n = 35), mandatory (n = 16), randomly selected partici-
pants (n = 9) or not stated (n = 12). Participants trained
one or more HFS: communication, coordination, deci-
sion-making, followership, leadership, situational aware-
ness, task management or teamwork.
Team size varied from two to twenty members, typi-
cally training in teams of two to five members. Two-
thirds of the studies were of multidisciplinary teams (n
= 47). Midwives, nurses and physicians were the most
common participants, but 13 different disciplines par-
ticipated. Mono-disciplinary SBT was seen in 20 stud-
ies; physicians (n = 18) were primarily trained separately
from other qualified personnel. An extracted summary of
included studies is shown in Table3, and the whole sum-
mary is available in Supplement B.
Content analysis
e content analysis identified four recurring themes:
(1) Training HFS in qualified teams, (2) assessment of
Table 2 Quality assessment of 72 studies included in a systematic review of The effectiveness of improving healthcare teams’ human
factor skills using simulation‑based training. Green = Yes, Red = No, Grey = Can’t tell, Yellow = Not relevant, Q = Question
Page 6 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Table 3 Extracted summary of studies included in a systematic review of The effectiveness of improving healthcare teams’ human
factor skills using simulation‑based training. The full summary of included studies is available in Supplement B
Page 7 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Table 3 (continued)
Page 8 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Table 3 (continued)
Page 9 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Table 3 (continued)
Page 10 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
HFS, (3) combined teaching methods and (4) retention
and transfer of skills. ese themes will be elaborated on
below.
Training HFS inqualied healthcare teams
e vast majority (n = 65) of the studies concluded that
SBT could develop qualified teams using HFS. In two-
thirds of the studies, HFS as the sole focus of the train-
ing wereseen and associated with enhanced effectiveness
[13, 47–73]. ese studies were mainly conducted in sim-
ulation centres, with smaller teams (n = 2–8 members),
and the SBT-courses were announced. It is a significant
result that HFS usually are trained together with TS, and
when trained on its own, it is taught in centres rather
than insitu and minor teams. Most of the 27 studies (n
= 22) used validated assessment methods and performed
debriefing (n = 24) immediately after every SBT scenario.
Nevertheless, Emani etal. [60] and Jafri etal. [74] show
a correlation between TS scores and HFS scores, which
emphasises that the effect of SBT is evident when HFS is
trained solely in combination with other competencies.
Studies of multi-disciplinary training (n = 47) [13, 48,
53, 56, 58–64, 66–69, 71–102] were generally associated
with greater effectiveness than mono-disciplinary train-
ing, perhaps because multi-disciplinary training better
reflects everyday clinical practice.
ree studies showed potential effect [71, 93, 99], con-
cluding that SBT is a promising tool to train HFS but
that more applicable assessment methods are needed.
Only two studies did not show effect [85, 98]; they men-
tion positive selection bias because high numbers of par-
ticipants withdrew, along with methodological problems
and lack of assessment methods as possible causes of the
non-effect result.
e trainees were mainly personnel from acute or
high-intensity medical departments, and nearly all the
trained situations involved acute life and death situa-
tions. Only four studies [68, 74, 93, 100] trained HFS in
day-to-day work, such as reducing falls, ethical issues,
delirium, the busy ward and caring for older patients and
relatives. A paediatric focus was found in 25 SBT studies,
in anaesthesiology, intensive care and obstetrics [13, 56,
60, 61, 72, 74–77, 80, 81, 83, 86, 88, 90, 91, 98, 102–109].
In total, 3251 of the participants were trained in acute
paediatric scenarios. HFS during resuscitation (n = 20)
was the second most trained situation [10, 13, 49, 52, 53,
59, 61, 62, 65, 72, 76, 78, 87, 89–91, 101, 104–108, 110],
involving 1887 personnel. is illustrates that acute and
high-intensity situations are the main focus of SBT con-
cerning teams’ HFS. Common to these training situations
are available algorithms and checklists of the TS or HFS
(e.g. acute caesarean, cardiopulmonary resuscitation,
Crisis Resource Management), which facilitate a form of
corrective actions. However, compliance with checklists
and training algorithms does not cover the dynamics of
HFS. Checklists and algorithms are task-oriented (check
Table 3 (continued)
Page 11 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
of rhythm, request read-back) that differ from the nature
of HFS, which are social and cognitive processes within
environmental and organisational frames. ese task-ori-
ented approaches increase the risk of changing the focus
from the all-around focus to the tasks themselves. is
could be why the focus on TS overtakes the focus on HFS
in some of the studies, for instance, in Arora etal. and
Siassakos etal. [99, 111].
is demonstrates that SBT increases the HFS among
qualified teams, but due to the lack of high-quality stud-
ies using similar assessment tools, the level of effective-
ness was not established.
Assessment ofHFS
e studies lack an adequate description of how HFS
refinements should be assessed. Existing HFS assess-
ment tools are insufficient, which was emphasised in
28 studies [49, 55, 58, 61, 64, 65, 68, 71, 75, 78, 80, 81,
84, 85, 87, 89, 95, 96, 98, 99, 103, 107, 111–115]. Assess-
ment methods (n = 51) spanned quantitative, qualitative
and mixed methods, validated and non-validated meth-
ods, rating behavioural markers, rating via checklists,
interviews, self-assessments, passing probes of informa-
tion, measuring time and evaluation of reported experi-
ences. Even though the studies used different assessment
methods, they concluded that HFS enhanced among the
participants. In 68 studies, HFS was considered to have
improved and a significant development in HFS as a
result of SBT was shown in 33 studies [10, 47–49, 51–56,
59, 60, 62, 64, 65, 72–77, 79, 80, 83, 87, 90, 100, 101, 104,
107, 108, 114, 116]. In conclusion, SBT can refine HFS.
e primary challenge in assessing HFS was a lack of
definitions for HFS and insufficient coverage of many
different HFS. HFS were, as mentioned, undefined or
broadly described in several studies, or the assessment
was unfit for HFS, such as measuring the time from the
outset of a procedure to a specific action or treatment [13,
51, 61, 83, 89]. For instance, the increased time could also
be due to improvements in the TS and not the HFS. HFS
training associated with specific behaviour markers were
the most successful assessment [10, 49, 54, 59, 65, 72, 73,
79, 101, 102, 114]. Five tools generally inspired the meth-
ods used: crisis resource management [117, 118]; Kirk-
patrick Model: Four Levels of Learning Evaluation [119];
Mayo High-Performance Teamwork Scale [120]; Ottawa
Global Rating Scale [121]; and TeamSTEPPS® [122].
e rating of markers was either blinded or unblinded
by internal or external faculty or assessed by the par-
ticipants themselves. Self-assessments were used in 31
studies. Self-assessment were used in combination with
other methods in 18 studies [47, 53, 57, 60, 65, 67, 68,
72, 78, 81, 85, 88, 93, 95, 97, 98, 108, 116], whereas 13
studies used self-assessment as the only method [82–84,
87, 92, 94, 100, 102, 105, 107, 109, 110, 112]. ere are
inherent challenges in using rating and self-assessments
because assessors must be congruent and unbiased,
and participants tend to overrate their performance
and therefore, the method has been proven unreliable
[123, 124]. Some studies (n = 21) used video recording
and blinded assessors [47, 48, 54, 58, 60, 61, 63, 66, 70,
71, 74, 76, 89, 91, 98, 99, 103, 106, 108, 111, 114], which
increased the validity of the ratings; because the asses-
sors’ could rewind the video and review the situation
multiple times. Other studies rated participants in real-
time, which challenged the assessors’ ability to simul-
taneously watch, listen and rate [10, 49–51, 53, 57, 59,
62–65, 67, 68, 72, 73, 75, 77–79, 81, 85, 93, 96, 101, 107,
115, 125].
e most frequently trained HFS were communication,
leadership and teamwork. e specification of the trained
HFS were described in various ways. Eleven studies [10,
13, 54, 69, 71, 98, 100, 101, 103, 114, 115] described HFS
with behaviour markers, attitudes or as a definition of the
chosen HFS, while others (n = 15) only mentioned the
HFS in broad indefinite terms such as communication or
teamwork [49, 57, 58, 63, 73, 76, 79, 85, 88, 89, 102, 106,
108, 109, 112]. Communication and teamwork were the
two most trained HFS.
Communication and teamwork are both broad terms.
Communication and teamwork are not isolated and une-
quivocal tasks; they depend on and influence each other,
like most HFS. e purpose of outlining and dividing the
tasks into behaviour markers is to simplify a complex
clinical situation, i.e. highlight easily recognisable behav-
iour for the participants, making it easier to acquire and
develop skills [118, 126]. e studies that described HFS
using either behaviour markers or attitudes succeeded to
a greater extent in assessing HFS and developments than
those that described HFS in broad, indefinite terms. It
is difficult to determine and report the effect of training
when the focus is on general terms such as communica-
tion and teamwork without a definition or level of detail.
It is not possible to distinguish between teamwork/com-
munication and cognition. While communication and
teamwork are often immediately recognisable and valid
interpretations for training personnel, they are high-level
concepts difficult to rate to assessors. Maybe because you
know it when you experience it but not always when you
see it. However, the studies that reflected on the use of
high-level concepts and worked to specify these in behav-
iour markers achieved greater internal validity along with
assessed facts, due to the increased transparency [10, 13,
47, 48, 50, 52–55, 65–67, 69–72, 74, 75, 77, 78, 96–98,
100, 101, 103, 107, 114, 116].
Page 12 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
They combined teaching methods
Significant effects on HFS were observed in 32 studies
that combined SBT with didactics and workshops, com-
pared to 12 that just trained SBT. e impact on qualified
teams’ use of HFS was evident, regardless of whether SBT
was combined with didactics and workshops or training
HFS on their own or in combination with TS. HFS train-
ing was combined with TS training in 30 of the studies, of
which 19 showed a significant effect on one or more HFS,
equalling 48 of all the included studies. us, it appears
that the studies in which HFS training was separate from
TS training resulted in the most significant improve-
ments in the teams’ use of HFS.
e studies that combined HFS and TS training tended
to focus more on TS. For instance, Burden etal. and Sias-
sakos etal. covered the results of HFS training with only
a few sentences [99, 125], and Hazwani etal. asserted that
a refined time to first medicine infusion in cardiopulmo-
nary resuscitation training was because of an enhance-
ment in teamwork [13].
Retention andtransfer ofskills
Retention or transfer of HFS was explored in 21 of the
studies. e retention of HFS were measured from par-
ticipants’ knowledge, self-assessment, audits and patient
outcome. Transfer of enhanced HFS are identified in 20
studies, but in two of these [79, 104], the authors iden-
tify transfer due to developed TS. e researchers argue
that improved TS and time decrease in accomplishing
the procedure are due to an increase in HFS skills. Rob-
erts etal. find a transfer of HFS, but with low retention
over time [66]. e transfer of HFS was measured as a
decrease in adverse events and improved patient out-
comes in six studies [49, 59, 79, 95, 97, 104].
Discussion
is systematic review demonstrates that SBT is a
successful learning tool to improve HFS in-hospital
healthcare settings. Unfortunately, we were unable
to show the effect level due to the use of all the dif-
ferent assessment tools. More research is required to
increase knowledge about the transfer of competencies
to daily clinical practice, examining why many studies
use non-validated assessment strategies and the barri-
ers to training HFS. While HFS are widely taught, there
are gaps in the literature regarding efficacy assessment.
ere is a need for more long-term studies and stud-
ies about how we translate assessment of skills to clini-
cal work. However, there is a lack of knowledge about
thetransfer and retention of the HFS developed, from
SBT to actual competencies in clinical practice. e
culture of viewing HFS as innate and complicated to
train could be one of the obstacles.
Although this review revealed support for training
HFS in the clinical setting using SBT, there is a lack of
agreement on which tools are best to assess HFS. ere
are gaps in the literature regarding the assessment of the
HFS. More research and consensus on how we assess
HFS is needed before the level of effectiveness can be
estimated. All assessment methods in SBT should be
supported by valid evidence. Several instruments are
designed to evaluate the effect of HFS skills through
SBT. Still, this review shows that the existing assessment
methods are not solid enough to establish consensus on
the way HFS are assessed. Although tools exist to assess
HFS, methods to study communication and other team-
related processes are far from being standardised, mak-
ing comparison challenging. is raises new questions
abouttraining HFS and future directions for research.
Cognition is an emergent property of the situation and
environment. Knowledge, perceived facts, understanding
and predictions within each team member’s mind inter-
act with displayed information, cues and devices in the
environment to affect decision-making and situational
awareness. Recurrent exposure to these factors can lead
to personal, team and institutional learning. Furthermore,
the environment can be modified and redesigned to sup-
port the team’s improved performance and safety. Cogni-
tion is thus an individual and shared mental process within
the team in all situations [127–129]. erefore, it is essen-
tial to add social, cognitive, environmental and technology
markers to the teaching/learning situations if the goal is
to enhance the teams’ HFS or redesign the environment
to increase patient safety. Nevertheless, 43% of the studies
show significant effectiveness in refining HFS using SBT,
and 92% show some effectiveness. is means that, regard-
less of multiple assessment methods, this review offers a
significant or improved effect of HFS using SBT, and the
outcome was relatively homogeneous—HFS improves
using SBT. A meta-analysis by Salas etal. concludes that
team training is a useful intervention with a moder-
ate, positive effect on team processes [130]. is adds to
the reliability of the present review. erefore, the differ-
ences among the methods in the included studies are not
a weakness of the research but rather a strength for the
results. On the other hand, it makes the results inconsist-
ent because of the lack of comparability. More research
and effort towards a consensus on assessing human factor
skills in the medical simulation society are requested.
e review also demonstrates that studies in which
HFS was trained alone had a more significant effect
than those focused on both HFS and TS. However,
although the increase of HFS was lower in combined TS
and HFS training, HFS was still enhanced in most stud-
ies. In SBT research, HFS are often relegated to an add-
on to develop procedures, algorithms and associated
Page 13 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
TS in specific settings. is may be for several reasons:
everyday clinical situations involve both HFS and TS,
trained together, or it is easier to measure technical
outcomes. HFS often play a minor role in the conclu-
sions drawn. In this way, TS “steal” the focus, and the
focus is on solving the medical problem at hand (e.g.
bleeding or anaphylaxis) rather than improving HFS,
which generally are the cause of most adverse events
[34]. HFS are unfortunately often understood as innate
skills and not skills that can be trained and refined. HFS
are not innate; they are generic and essential in reduc-
ing adverse events within healthcare and need to be
qualified and trained just as seriously as technical skills
and clinical procedures.
e high amount of studies from acute and high-inten-
sity situations and the paediatric speciality shows that
there is awareness of the need for training qualified per-
sonnel, that SBT is not only for the students and novices.
e training mostly around algorithms is unclear and
could be an exciting focus in future research. Neverthe-
less, the results also show that qualified teams mostly
train situations where life is at stake. However, adverse
events not only happens in highly acute situations but
also in slow situations suchas medication administration
[131], receiving and transferring patients [132, 133] and
development of sepsis [134]—all situations where teams
interact. If healthcare teams are trained in everyday care,
it might reflect everyday clinical practice and prevent or
reduce future adverse events.
An interesting result is that the training teams mostly
were 2–5 members, although critical care teams are more
prominent innumerous places in the world. e reasons
for this are unclear, but possible explanations include the
expense of SBT and a high turnover of qualified healthcare
personnel [135]. Moreover, the participants are often vol-
unteers, and the likely absence of volunteers can explain.
It is important to understand learning holistically,
integrating the individual, brain, body and surround-
ings [136]. All levels of education involve both physical
and cognitive stimulations, and if the content is too vast,
the learning decreases. e results suggest that focusing
exclusively on HFS in SBT can lead to a deeper aware-
ness of HFS’s effect on patient safety among teams and,
possibly consequently larger learning potential. How-
ever, further research will have to study to what degree
HFS transfers to competence in clinical practice. e
results show that SBT for HFS alone, combined with
didactics and workshops may lead to the most signifi-
cant improvement in teams’ HFS. is is substantiated
by Maturana’s theory of suitable disturbances [137, 138],
which deals with how disturbances should be moderated.
If a disturbance is too big, the learners might lose atten-
tion, and if the disturbances are too small, the learners
might not even notice. Accordingly, if TS and HFS are
trained together, the educational disturbance to partici-
pants’ behaviour might be too massive for participants to
engage with. However, the link to clinical practice is still
underdeveloped.
Conclusion
is systematic review demonstrates a strong indication
that SBT is an effective learning tool to improve HFS in-
hospital healthcare settings. However, HFS are incon-
sistently described, interpreted, taught and assessed and
the lack of real-world assessment or follow-up makes the
transfer to everyday practice challenging. is systematic
review does not entirely answer if SBT improves HFS in
qualified healthcare teams. Still, it highlights the gaps
in the literature and underpins the necessity of increas-
ing the focus on HFS or routine care in SBT to improve
outcomes. ere is a need for more long-term studies
and studies about how we translate assessment of skills
to clinical work. However, there is a lack of knowledge
about thetransfer and retention of the HFS developed,
from SBT to actual competencies in clinical practice.
e culture of viewing HFS as innate and complicated to
train could be one of the obstacles. Healthcare, in gen-
eral, must support the necessity and significance for HFS.
Otherwise, the HFS will not be effectively transferred to
everyday practice. Also, design issues such as positioning
of the equipment, cognitive aids and process changes are
needed to support ideal human performance such as not
relying on memory or complex decision-making in com-
plex time-pressed situations. More research is required
to increase knowledge about the transfer of competencies
to daily clinical practice, examining why many studies use
non-validated assessment strategies and the barriers to
training HFS.
Limitations
A few limitations of this review need to be highlighted.
Firstly, three authors screened a vast number of studies,
but only the first author did a full-text reading and assess-
ment of the included studies. is increases the possibil-
ity of selection bias and influences the internal validity
and reliability. e bias was sought to be minimised by
bringing any doubts about selected studies to the broader
author group. Nevertheless, the intercoder reliability
is inevitably affected when human coders are used in
content analysis [139]. Secondly, the Hawthorne effect
(behaviour alteration simply because HFS were studied)
represents a possible bias [140]. irdly, 48% of the par-
ticipants in the included studies courses were volunteers,
but the results from volunteer studies do not deviate
from the enhancement among mandatory participants.
Page 14 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
Nevertheless, the number of volunteers could lead to a
positively biased result because they agreed to SBT as a
learning method. Moreover, it is essential to point out
that 20 of the included studies were from an emergency
medicine setting, which can have influenced the results.
A review focusing on HFS, in general, could have eluci-
dated studies from other settings. Finally, the results may
be affected by publication bias because studies with unfa-
vourable results of SBT might not have been published,
which could mean an endorsement of the results in the
direction of a favourable analysis.
Implications forpractice
It is evident that SBT can improve qualified teams’ HFS.
SBT is an effective learning tool for use with novices and
experts, and with unqualified or qualified personnel. A
change of focus is recommended for healthcare provid-
ers to train emergencies or rare situations and everyday
non-emergency situations, such as admission to hospital,
rounds, or the unprepared talk with next-in-kind in the
hallway. is review shows that even qualified teams’ can
develop their HFS significantly through SBT. Using SBT
to train the healthcare personnel for everyday clinical
practice are essential. Firstly, because the everyday rou-
tine takes up most of the performance tasks in the hos-
pitals, the personnel are constantly in different forms of
teamwork. Secondly, as learned from Safety II, it is nec-
essary to enhance the ability to succeed (reduce adverse
events) under varying conditions [141]. irdly, health-
care personnel are constantly interchangeably with new
demands (e.g. professional, environmental and technical)
to the personnel. Finally, yet significantly, the high degree
of personnel turnover in healthcare affects the quality of
care, a quality that the use of continual SBT can increase.
If the personnel’s HFS are capable in everyday practice,
they will in all probability be in acute and high-intensity
situations.
All human interactions in hospitals need to be efficient
and trained just as seriously as TS and clinical procedures
because interactions are just as prone, if not more, to
errors. Cultural, social and people skills, together termed
HFS, are not innate and untrainable. Instead, they are
generic and essential in reducing adverse events within
healthcare and demands an increased focus on system-
atic multidisciplinary training of HFS among healthcare
teams.
Abbreviations
AMSTAR‑2: A measurement tool to assess systematic reviews–2nd edi‑
tion; ANTS: Anaesthetists’ non‑technical skills; CASP: Critical appraisal
skills programme; CINAHL: Cumulative index of nursing and allied health
literature (database); COVID‑19: Coronavirus disease 2019; COVIDENCE: An
online tool that streamlines parts of the systematic review process; CRM:
Crisis resource management; EBSCO: Elton B. Stephens company (online
access); EMBASETM: Excerpta Medica database (database); ERIC: Educational
resources information center (database); HFS: Human factor skills; MMAT:
Mixed methods appraisal tool; MEDLINE: Medical literature analysis and
retrieval system online (database); NTS: Non‑technical skills; Non‑RCT
: Non‑randomized controlled trial; OVID: Part of the Wolters Kluwer group
of companies (online access); PICO: Problem/population, intervention,
comparison, and outcome; PROSPERO: Prospective register of systematic
reviews; PRISMA: Preferred reporting items for systematic reviews and
meta‑analyses; PsycINFO: Psychological information (database); RCT : Rand‑
omized controlled trial; SCOPUS: Elsevier’s abstract and citation database;
SBT: Simulation‑based training; SPICE: Setting, perspective, intervention,
comparison, and evaluation; TS: Technical skills.
Supplementary Information
The online version contains supplementary material available at https:// doi.
org/ 10. 1186/ s41077‑ 022‑ 00207‑2.
Additional le1. Supplement A–Searches.
Additional le2. Supplement B–Results summary.
Acknowledgements
The authors declare no conflict of interest but disclose receipt of the following
financial support for the research and authorship of this article: Anaesthe‑
siology and Critical Care Department, Odense University Hospital; Hospital
Sønderjylland; OPEN, Clinical Research, University of Southern Denmark; and
Centre for Human Interactivity, Department of Language and Communica‑
tion, University of Southern Denmark.
Authors’ contributions
LA drafted the manuscript. LA, ABN and MLH conducted the title and abstract
selection. LA conducted the full‑text reading and the interpretation of data. All
authors critically revised the manuscript and the authors read and approved
the final manuscript.
Funding
Odense University Hospital; Hospital Sønderjylland; University of Southern
Denmark.
Availability of data and materials
The datasets used and analysed during the current study are available from
the corresponding author on reasonable request.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Author details
1 Anesthesiology and Intensive Care Unit, Odense University Hospital, Odense,
Denmark. 2 OPEN, Open Patient data Explorative Network, Odense University
Hospital/Department of Clinical Research, University of Southern Denmark,
Odense, Denmark. 3 Emergency Research Unit, Hospital Sønderjylland, Uni‑
versity Hospital of Southern Denmark, Odense, Denmark. 4 Centre for Human
Interactivity, Department of Language and Communication, University
of Southern Denmark, Odense, Denmark. 5 Department of Clinical Research,
University of Southern Denmark, Odense, Denmark. 6 SimC, Regional Center
for Technical Simulation, Region of Southern Denmark, Odense, Denmark.
7 Department of Design and Communication, University of Southern Denmark,
Kolding, Denmark. 8 Danish Institute for Advanced Study, University of South‑
ern Denmark, Odense, Denmark. 9 Center for Ecolinguistics, South China
Agricultural University, Guangzhou, People’s Republic of China. 10 College
Page 15 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
of International Studies, Southwest University, Chongqing, People’s Republic
of China. 11 Institute of Nursing & Health Science, Ilisimartusarfik, University
of Greenland, Nuuk, Greenland. 12 Center for Mental Health Nursing and Health
Research (CPS), Mental Health Services, Region of Southern Denmark, Univer‑
sity of Southern Denmark, Odense, Denmark.
Received: 21 September 2021 Accepted: 5 April 2022
References
1. de Vries ENE. The incidence and nature of in‑hospital adverse events: a
systematic review. Qual Saf Health Care. 2008;17(3):216–23. https:// doi.
org/ 10. 1136/ qshc. 2007. 023622.
2. Styrelsen for Patientsikkerhed. Årsberetning for Dansk Patientsikker‑
hedsDatabase 2019. Copenhagen: Styrelsen for Patientsikkerhed; 2020.
3. Styrelsen for Patientsikkerhed. Strategiplan 2017‑2021 København.
Denmark: Sundhedsstyrelsen; 2021. [Available from: https:// patie ntsik
kerhed. dk/ conte nt/ uploa ds/ 2017/ 06/ strat egipl an_ 2017_ korre kturl
east. pdf]
4. Pedersen KZ, Mesman J. A transactional approach to patient safety:
understanding safe care as a collaborative accomplishment. J Interprof
Care. 2021;35(4):503–13. https:// doi. org/ 10. 1080/ 13561 820. 2021. 18743
17. Epub 2021 Mar 2.
5. Danish Patient Safety Authority. Learning Strategy. In: Danish Patient
Safety Authority, editor. Online. 1st Denmark: Danish Patient Safety
Authority; 2017.
6. Reason J. Understanding adverse events: human factors. Qual Health
Care. 1995;4(2):80–9.
7. Institute of Medicine Committee on Quality of Health Care in America.
In: Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: build‑
ing a safer health system. Washington (DC): National Academies Press
(US); 2000.
8. Garrouste‑Orgeas M, Philippart F, Bruel C, Max A, Lau N, Misset B.
Overview of medical errors and adverse events. Ann Intensive Care.
2012;2:2.
9. WHO. Topic 2: What is human factors and why is it important to patient
safety? www: WHO; 2021 [Available from: https:// www. who. int/ news‑
room/ fact‑ sheets/ detail/ patie nt‑ safety].
10. Barra FL, Carenzo L, Franc J, Montagnini C, Petrini F, Della Corte F, et al.
Anesthesiology resident induction month: a pilot study showing an
effective and safe way to train novice residents through simulation.
Minerva Anestesiol. 2018;84(12):1377–86.
11. Rodríguez Y, Hignett S. Integration of human factors/ergonomics in
healthcare systems: a giant leap in safety as a key strategy during Covid‑
19. Hum Fact Ergonomics Manufact Serv Ind. 2021. Epub ahead of print.
12. Norris EM, Lockey AS. Human factors in resuscitation teaching. Resusci‑
tation. 2012;83(4):423–7.
13. Hazwani T, Ashraf N, Hasan Z, Antar M, Kazzaz Y, Alali H. 95. Effect of
a pediatric mock code on resuscitation skills and team performance:
an in situ simulation experience over three years. Eur J Emerg Med.
2020;27(Suppl 1):e15–e16. https:// doi. org/ 10. 1097/ 01. mej. 00006 97880.
10650. f1.
14. International Ergonomics Association. Human factors and ergonomics
online: International Ergonomics Association. Switzerland: International
Ergonomics Association; 2021. Available from: https:// iea. cc/.
15. Russ AL, Fairbanks RJ, Karsh B‑T, Militello LG, Saleem JJ, Wears RL. The
science of human factors: separating fact from fiction. BMJ Qual Saf.
2013;22(10):802–8.
16. Wolf L, Parker SH, Gleason JL. Human factors in healthcare. In: Patient
safety and quality Improvement in Healthcare. Switzerland: Springer;
2021. p. 319–33. https:// doi. org/ 10. 1007/ 978‑3‑ 030‑ 55829‑1_ 20.
17. Styrelsen for Patientsikkerhed. Årsberetning for patientombuddet 2015.
In: Patientsikkerhed Sf, www. stps. dk. Kbh. 2016.
18. Vindrola‑Padros C, Andrews L, Dowrick A, Djellouli N, Fillmore H, Gonza‑
lez EB, et al. Perceptions and experiences of healthcare workers during
the COVID‑19 pandemic in the UK. BMJ Open. 2020;10(11):e040503.
19. Styrelsen for Patientsikkerhed. Dansk Patientsikkerhedsdatabase Års‑
beretning 2020. København: Sundhedsministeriet; 2021.
20. Keddington AS, Moore J. Simulation as a method of competency
assessment among health care providers: a systematic review. Nurs
Educ Perspect. 2019;40(2):91–4.
21. Eppich W, Howard V, Vozenilek J, Curran I. Simulation‑based team train‑
ing in healthcare. Simul Healthc. 2011;6(Suppl):S14–9.
22. Griswold‑Theodorson S, Ponnuru S, Dong C, Szyld D, Reed T, McGaghie
WC. Beyond the simulation laboratory: a realist synthesis review of
clinical outcomes of simulation‑based mastery learning. Acad Med.
2015;90(11):1553–60.
23. Boling B, Hardin‑Pierce M. The effect of high‑fidelity simulation on
knowledge and confidence in critical care training: an integrative
review. Nurse Educ Pract. 2016;16(1):287–93.
24. Lucas AE, Marie. Development of crisis resource management skills: a
literature review. Clin Simul Nurs. 2017;13(8):347–58.
25. Krautscheid LC. Improving communication among healthcare provid‑
ers: preparing student nurses for practice. Int J Nurs Educ Scholarsh.
2008;5(1):1–15.
26. Gregory A, Hogg G, Ker J. Innovative teaching in situational awareness.
Clin Teach. 2015;12(5):331–5.
27. Andersen SA, Mik kelsen PT, Konge L, Caye‑Thomasen P, Sorensen MS.
Cognitive load in mastoidectomy skills training: virtual reality simula‑
tion and traditional dissection compared. J Surg Educ. 2016;73(1):45–50.
28. Low XMHD, Brewster DJ. The effects of team‑training in intensive care
medicine: a narrative review. J Crit Care. 2018;48:283–9.
29. Lorello GR, Cook DA, Johnson RL, Brydges R. Simulation‑based train‑
ing in anaesthesiology: A systematic review and meta‑analysis. Br J
Anaesth. 2014;112(2):231–45.
30. Gjeraa K, Møller TP, Ostergaard D. Efficacy of simulation‑based trauma
team training of non‑technical skills. A systematic review. Acta Anaes‑
thesiol Scand. 2014;58(7):775–87.
31. Lapierre A, Bouferguene S, Gauvin‑Lepage J, Lavoie P, Arbour C.
Effectiveness of Interprofessional Manikin‑Based Simulation Training
on Teamwork Among Real Teams During Trauma Resuscitation in
Adult Emergency Departments: A Systematic Review. Simul Healthc.
2020;15(6):409–21. https:// doi. org/ 10. 1097/ SIH. 00000 00000 000443.
32. Weile J, Nebsbjerg MA, Ovesen SH, Paltved C, Ingeman ML. Simulation‑
based team training in time‑critical clinical presentations in emer‑
gency medicine and critical care: a review of the literature. Adv Simul.
2021;6(1):3.
33. Buljac‑Samardzic M, Doekhie KD, van Wijngaarden JDH. Interventions
to improve team effectiveness within health care: a systematic review
of the past decade. Hum Resour Health. 2020;18(1):2.
34. Henriksen K, Dayton E, Keyes MA, Carayon P, Hughes R. Understanding
adverse events: a human factors framework. In: RG H, editor. Patient
safety and quality: an evidence‑based handbook for nurses. Rockville:
Agency for Healthcare Research and Quality (US); 2008.
35. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR
2: a critical appraisal tool for systematic reviews that include ran‑
domised or non‑randomised studies of healthcare interventions, or
both. BMJ. 2017;358:j4008.
36. Page MAO, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD,
et al. The PRISMA 2020 statement: an updated guideline for reporting
systematic reviews. BMJ. 2021;372:n71.https:// doi. org/ 10. 1136/ bmj. n71.
37. Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in
PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32.
38. Booth A. Clear and present questions: formulating questions for evi‑
dence based practice. Library Hi Tech. 2006;24(3):355–68.
39. Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative
evidence synthesis. Qual Health Res. 2012;22(10):1435–43.
40. Veritas Health Innovation Ltd. Covidence systematic review software.
Melbourne: Veritas Health Innovation; 2021.
41. Institute JB. Critical appraisal skills programme. UK: Joanna Briggs Insti‑
tute; 2021. [Available from: https:// casp‑ uk. net/ casp‑ tools‑ check lists/].
42. Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P,
et al. The mixed methods appraisal tool (MMAT) version 2018 for infor‑
mation professionals and researchers. Educ Inf. 2018;34:285–91.
43. Crombie IK. The pocket guide to critical appraisal: A handbook for health
care professionals. London: BMJ Publishing Group; 1997:66.
44. Stemler S. An overview of content analysis. Practical assessment,
research, and evaluation 7.1 (2000):17.
Page 16 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
45. Krippendorff K. Content analysis: an introduction to its methodology.
4th ed. Thousand oaks: SAGE; 2018. 472.
46. World Medical A. World medical association declaration of helsinki:
ethical principles for medical research involving human subjects. JAMA.
2013;310(20):2191–4.
47. Dedy NJ, Bonrath EM, Ahmed N, Grantcharov TP. Structured training
to improve nontechnical performance of junior surgical residents
in the operating room: a randomized controlled trial. Ann Surg.
2016;263(1):43–9.
48. Jonsson K, Brulin C, Härgestam M, Lindkvist M, Hultin M. Do team
and task performance improve after training situation awareness? A
randomized controlled study of interprofessional intensive care teams.
Scand J Trauma Resusc Emerg Med. 2021;29(1):73.
49. Fernandez R, Rosenman ED, Olenick J, Misisco A, Brolliar SM, Chip‑
man AK, et al. Simulation‑based team leadership training improves
team leadership during actual trauma resuscitations: a randomized
controlled trial. Crit Care Med. 2020;48(1):73–82.
50. Yule S, Parker S, Wilkinson J, McKinley A, MacDonald J, Neill A, et al.
Coaching non‑technical skills improves surgical residents’ performance
in a simulated operating room. J Surg Educ. 2015;72(6):1124–30 Avail‑
able from: https:// www. cochr aneli brary. com/ centr al/ doi/ 10. 1002/ centr
al/ CN‑ 01200 595/ full.
51. AbdelFattah KR, Spalding MC, Leshikar D, Gardner AK. Team‑based
simulations for new surgeons: Does early and often make a difference?
Surgery. 2018;163(4):912–5.
52. Rao R, Dumon KR, Neylan CJ, Morris JB, Riddle EW, Sensenig R, et al.
Can simulated team tasks be used to improve nontechnical skills in the
operating room? J Surg Educ. 2016;73(6):e42–e7.
53. Steinemann S, Berg B, Skinner A, Ditulio A, Anzelon K, Terada K, et al.
In situ, multidisciplinary, simulation‑based teamwork training improves
early trauma care. J Surg Educ. 2011;68(6):472–7.
54. Doumouras AG, Engels PT. Early crisis nontechnical skill teaching in
residency leads to long‑term skill retention and improved performance
during crises: a prospective, nonrandomized controlled study. Surgery.
2017;162(1):174–81.
55. Pena G, Altree M, Field J, Sainsbury D, Babidge W, Hewett P, et al.
Nontechnical skills training for the operating room: a prospective study
using simulation and didactic workshop. Surgery. 2015;158(1):300–9.
56. Auerbach M, Roney L, Aysseh A, Gawel M, Koziel J, Barre K, et al. In situ
pediatric trauma simulation: assessing the impact and feasibility of
an interdisciplinary pediatric in situ trauma care quality improvement
simulation program. Pediatr Emerg Care. 2014;30(12):884–91.
57. Bearman M, O’Brien R, Anthony A, Civil I, Flanagan B, Jolly B, et al.
Learning surgical communication, leadership and teamwork through
simulation. J Surg Educ. 2012;69(2):201–7.
58. Burtscher MJ, Manser T, Kolbe M, Grote G, Grande B, Spahn DR, et al.
Adaptation in anaesthesia team coordination in response to a simu‑
lated critical event and its relationship to clinical performance. Br J
Anaesth. 2011;106(6):801–6.
59. Capella J, Smith S, Philp A, Putnam T, Gilbert C, Fry W, et al. Teamwork
training improves the clinical care of trauma patients. J Surg Educ.
2010;67(6):439–43.
60. Emani S, Allan C, Forster T, Fisk A, Lagrasta C, Zheleva B, et al. Simulation
training improves team dynamics and performance in a low‑resource
cardiac intensive care unit. Ann Pediatr Cardiol. 2018;11(2):130–6.
61. Gilfoyle E, Koot DA, Annear JC, Bhanji F, Cheng A, Duff JP, et al. Improved
clinical performance and teamwork of pediatric interprofessional
resuscitation teams with a simulation‑based educational intervention.
Pediatr Crit Care Med. 2017;18(2):e62–e9.
62. Miller D, Crandall C, Washington C 3rd, McLaughlin S. Improving team‑
work and communication in trauma care through in situ simulations.
Acad Emerg Med. 2012;19(5):608–12.
63. Pascual JL, Holena DN, Vella MA, Palmieri J, Sicoutris C, Selvan B,
et al. Short simulation training improves objective skills in estab‑
lished advanced practitioners managing emergencies on the ward
and surgical intensive care unit. J Trauma ‑ Injury, Infect Crit Care.
2011;71(2):330–8.
64. Paull DE, Deleeuw LD, Wolk S, Paige JT, Neily J, Mills PD. The effect of
simulation‑based crew resource management training on measurable
teamwork and communication among interprofessional teams caring
for postoperative patients. J Contin Educ Nurs. 2013;44(11):516–24.
65. Rice Y, DeLetter M, Fryman L, Parrish E, Velotta C, Talley C. Implementa‑
tion and Evaluation of a Team Simulation Training Program. J Trauma
Nurs. 2016;23(5):298–303.
66. Roberts NK, Williams RG, Schwind CJ, Sutyak JA, McDowell C, Griffen D,
et al. The impact of brief team communication, leadership and team
behavior training on ad hoc team performance in trauma care settings.
Am J Surg. 2014;207(2):170–8.
67. Rochlen LR, Malloy KM, Chang H, Kim S, Guichard L, Cassidy R, et al. Pilot
one‑hour multidisciplinary team training simulation intervention in
the operating room improves team nontechnical skills. J Educ Perioper
Med. 2019;21(2):E624.
68. Ross AJ, Anderson JE, Kodate N, Thomas L, Thompson K, Thomas B, et al.
Simulation training for improving the quality of care for older people:
an independent evaluation of an innovative programme for inter‑
professional education. BMJ Qual Saf. 2013;22(6):495–505.
69. Weller J, Cumin D, Civil I, Torrie J, Garden A, MacCormick A, et al.
Improved scores for observed teamwork in the clinical environment
following a multidisciplinary operating room simulation intervention. N
Z Med J. 2016;129(1439):59–67.
70. Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL,
et al. Nontechnical skills in anesthesia crisis management with
repeated exposure to simulation‑based education. Anesthesiology.
2005;103(2):241–8.
71. Shapiro MJ, Morey JC, Small SD, Langford V, Kaylor CJ, Jagminas L, Suner
S, Salisbury ML, Simon R, Jay GD. Simulation based teamwork training
for emergency department staff: does it improve clinical team perfor‑
mance when added to an existing didactic teamwork curriculum? Qual
Saf Health Care. 2004;13(6):417–21. https:// doi. org/ 10. 1136/ qhc. 13.6.
417.
72. Sawyer T, Laubach VA, Hudak J, Yamamura K, Pocrnich A. Improvements
in teamwork during neonatal resuscitation after interprofessional Team‑
STEPPS training. Neonatal Netw. 2013;32(1):26–33.
73. Rosqvist E, Ylönen M, Torkki P, Repo JP, Paloneva J. Costs of hospital
trauma team simulation training: a prospective cohort study. BMJ
Open. 2021;11(6):e046845.
74. Jafri FN, Mirante D, Ellsworth K, Shulman J, Dadario NB, Williams K, Yu S,
Thomas J, Kumar A, Edwards RA, Torres RE, Straff DJ. A Microdebriefing
Crisis Resource Management Program for Simulated Pediatric Resus‑
citation in a Community Hospital: A Feasibility Study. Simul Healthc.
2021;16(3):163–9. https:// doi. org/ 10. 1097/ SIH. 00000 00000 000480.
75. Fransen AF, van de Ven J, Merien AE, de Wit‑Zuurendonk LD, Houter‑
man S, Mol BW, et al. Effect of obstetric team training on team perfor‑
mance and medical technical skills: a randomised controlled trial. Bjog.
2012;119(11):1387–93.
76. Rubio‑Gurung S, Putet G, Touzet S, Gauthier‑Moulinier H, Jordan I, Beis‑
sel A, et al. In situ simulation training for neonatal resuscitation: an RCT.
Pediatrics. 2014;134(3):e790–e7.
77. Skelton T, Nshimyumuremyi I, Mukwesi C, Whynot S, Zolpys L, Living‑
ston P. Low‑cost simulation to teach anesthetists’ non‑technical skills in
Rwanda. Anesth Analg. 2016;123(2):474–80.
78. Mahramus TL, Penoyer DA, Waterval EME, Sole ML, Bowe EM. Two
hours of teamwork training improves teamwork in simulated car‑
diopulmonary arrest events. Clin Nurse Specialist: J Adv Nurs Pract.
2016;30(5):284–91.
79. Marko EK, Fausett MB, Deering S, Staat BC, Stormes S, Freund E, et al.
Reducing perineal lacerations through team‑based simulation. Simul.
2019;14(3):182–7.
80. Colman N, Figueroa J, McCracken C, Hebbar K. Simulation‑based team
training improves team performance among pediatric intensive care
unit staff. J Pediat Intensive Care. 2019;8(2):83–91.
81. Colman N, Figueroa J, McCracken C, Hebbar KB. Can simulation based‑
team training impact bedside teamwork in a pediatric intensive care
unit? J Pediat Intensive Care. 2019;8(4):195–203.
82. Kumar A, Sturrock S, Wallace EM, Nestel D, Lucey D, Stoyles S, Morgan
J, Neil P, Schlipalius M, Dekoninck P. Evaluation of learning from Practi‑
cal Obstetric Multi‑Professional Training and its impact on patient
outcomes in Australia using Kirkpatrick’s framework: a mixed methods
study. BMJ Open. 2018;8(2):e017451. https:// doi. org/ 10. 1136/ bmjop
en‑ 2017‑ 017451.
83. Figueroa MI, Sepanski R, Goldberg SP, Shah S. Improving teamwork,
confidence, and collaboration among members of a pediatric
Page 17 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
cardiovascular intensive care unit multidisciplinary team using
simulation‑based team training. Pediatr Cardiol. 2013;34(3):612–9.
84. Gardner R, Walzer TB, Simon R, Raemer DB. Obstetric simulation as a
risk control strategy: Course design and evaluation. Simul Healthc.
2008;3(2):119–27.
85. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method
for measuring the effectiveness of simulation‑based team training for
improving communication skills. Anesth Anal. 2005;100(5):1375–80.
86. Colacchio K, Johnston L, Zigmont J, Kappus L, Sudikoff SN. An approach
to unit‑based team training with simulation in a neonatal intensive care
unit. J Neonat‑Perinat Med. 2012;5(3):213–9.
87. George KL, Quatrara B. Interprofessional simulations promote
knowledge retention and enhance perceptions of teamwork skills in a
surgical‑trauma‑burn intensive care unit setting. Dimens Crit Care Nurs.
2018;37(3):144–55.
88. Birch L, Jones N, Doyle PM, Green P, McLaughlin A, Champney C, et al.
Obstetric skills drills: evaluation of teaching methods. Nurse Educ
Today. 2007;27(8):915–22.
89. Chamberland C, Hodgetts HM, Kramer C, Breton E, Chiniara G, Tremblay
S. The critical nature of debriefing in high‑fidelity simulation‑based
training for improving team communication in emergency resuscita‑
tion. Appl Cogn Psychol. 2018;32(6):727–38.
90. Cory MJ, Hebbar KB, Colman N, Pierson A, Clarke SA. Multidisciplinary
simulation‑based team training: knowledge acquisition and shifting
perception. Clin Simul Nurs. 2020;41:14–21.
91. De Bernardo G, Sordino D, Cavallin F, Mardegan V, Doglioni N, Tataranno
ML, et al. Performances of low level hospital health caregivers after a
neonatal resuscitation course. Ital J Pediatr. 2016;42(1):1–7.
92. Gum L, Greenhill J, Dix K. Clinical simulation in maternity (CSiM): inter‑
professional learning through simulation team training. Qual Saf Health
Care. 2010;19(5):e19.
93. Kenaszchuk C, MacMillan K, van Soeren M, Reeves S. Interprofessional
simulated learning: short‑term associations between simulation and
interprofessional collaboration. BMC Med. 2011;9:29. https:// doi. org/ 10.
1186/ 1741‑ 7015‑9‑ 29.
94. Meeker K, Brown SK, Lamping M, Moyer MR, Dienger MJ. A high‑fidelity
human patient simulation initiative to enhance communication
and teamwork among a maternity care team. Nurs Womens Health.
2018;22(6):454–62.
95. Mehta N, Boynton C, Boss L, Morris H, Tatla T. Multidisciplinary difficult
airway simulation training: Two year evaluation and validation of a
novel training approach at a District General Hospital based in the UK.
Eur Arch Otorhinolaryngol. 2013;270(1):211–7.
96. Undre S, Koutantji M, Sevdalis N, Gautama S, Selvapatt N, Williams S,
et al. Multidisciplinary crisis simulations: the way forward for training
surgical teams. World J Surg. 2007;31(9):1843–53.
97. Wong AH, Gang M, Szyld D, Mahoney H. Making an “attitude adjust‑
ment”: using a simulation‑enhanced interprofessional education
strategy to improve attitudes toward teamwork and communication.
Simul. 2016;11(2):117–25.
98. Clay‑Williams R, McIntosh CA, Kerridge R, Braithwaite J. Classroom
and simulation team training: a randomized controlled trial. Int J
Qual Health Care. 2013;25(3):314–21. https:// doi. org/ 10. 1093/ intqhc/
mzt027. Epub 2013 Apr 2.
99. Siassakos D, Draycott T, Montague I, Harris M. Content analysis of team
communication in an obstetric emergency scenario. J Obstet Gynaecol.
2009;29(6):499–503.
100. Bursiek AA, Hopkins MR, Breitkopf DM, Grubbs PL, Joswiak ME, Klipfel
JM, et al. Use of high‑fidelity simulation to enhance interdisciplinary
collaboration and reduce patient falls. J Patient Saf. 2020;16(3):245–50.
101. Armstrong P, Peckler B, Pilkinton‑Ching J, McQuade D, Rogan A. Effect
of simulation training on nurse leadership in a shared leadership model
for cardiopulmonary resuscitation in the emergency department.
Emerg Med Australas. 2021;33(2):255–61.
102. Lee MO, Schertzer K, Khanna K, Wang NE, Camargo CAJ, Sebok‑Syer SS.
Using in situ simulations to improve pediatric patient safety in emer‑
gency departments. Acad Med. 2021;96(3):395–8.
103. Sudikoff SN, Overly FL, Shapiro MJ, Sudikoff SN, Overly FL, Shapiro MJ.
High‑fidelity medical simulation as a technique to improve pediatric
residents’ emergency airway management and teamwork: a pilot study.
Pediatr Emerg Care. 2009;25(10):651–6.
104. Andreatta P, Saxton E, Thompson M, Annich G. Simulation‑based mock
codes significantly correlate with improved pediatric patient cardiopul‑
monary arrest survival rates. Pediatr Crit Care Med. 2011;12(1):33–8.
105. Burke RV, Demeter NE, Goodhue CJ, Roesly H, Rake A, Young LC, et al.
Qualitative assessment of simulation‑based training for pediatric
trauma resuscitation. Surgery. 2017;161(5):1357–66.
106. Cordero L, Hart BJ, Hardin R, Mahan JD, Giannone PJ, Nankervis CA.
Pediatrics residents’ preparedness for neonatal resuscitation assessed
using high‑fidelity simulation. J Grad Med Educ. 2013;5(3):399–404.
107. Palmer E, Labant AL, Edwards TF, Boothby J. A collaborative partnership
for improving newborn safety: using simulation for neonatal resuscita‑
tion training. J Contin Educ Nurs. 2019;50(7):319–24.
108. van den Bos‑Boon A, Hekman S, Houmes R‑J, Vloet L, Gischler S, van
der Starre C, et al. Effectiveness of simulation training and assessment
of PICU nurses’ resuscitation skills: a mixed methods study from the
Netherlands. J Pediatr Nurs. 2021;59:e52–60.
109. Lemke DS. Rapid Cycle Deliberate Practice for Pediatric Intern Resusci‑
tation Skills. MedEdPORTAL. 2020;16:11020.
110. Marker S, Mohr M, Østergaard D. Simulation‑based training of junior
doctors in handling critically ill patients facilitates the transition to clini‑
cal practice: an interview study. BMC Med Educ. 2019;19(1):11. https://
doi. org/ 10. 1186/ s12909‑ 018‑ 1447‑0.
111. Arora S, Hull L, Fitzpatrick M, Sevdalis N, Birnbach DJ. Crisis manage‑
ment on surgical wards: a simulation‑based approach to enhanc‑
ing technical, teamwork, and patient interaction skills. Ann Surg.
2015;261(5):888–93.
112. Blum RH, Raemer DB, Carroll JS, Sunder N, Felstein DM, Cooper JB.
Crisis resource management training for an anaesthesia faculty: a new
approach to continuing education. Med Educ. 2004;38(1):45–55.
113. Calcagno HE, Lucke‑Wold B, Noles M, Dillman D, Baskerville M, Spight
D, et al. Integrated otolaryngology and anesthesia simulation model
for crisis management of cavernous carotid artery injury. Arch Neurol
Neuro Disord. 2018;1(1):30–41.
114. Frengley RW, Weller JM, Torrie J, Dzendrowskyj P, Yee B, Paul AM,
et al. The effect of a simulation‑based training intervention on the
performance of established critical care unit teams. Crit Care Med.
2011;39(12):2605–11.
115. Joshi K, Hernandez J, Martinez J, AbdelFattah K, Gardner A. Should
they stay or should they go now? Exploring the impact of team
familiarity on interprofessional team training outcomes. Am J Surg.
2018;215(2):243–9 Available from: https:// www. cochr aneli brary. com/
centr al/ doi/ 10. 1002/ centr al/ CN‑ 01665 586/ full.
116. Caskey RC, Owei L, Rao R, Riddle EW, Brooks AD, Dempsey DT, et al. Inte‑
gration of hands‑on team training into existing curriculum improves
both technical and nontechnical skills in laparoscopic cholecystectomy.
J Surg Educ. 2017;74(6):915–20.
117. Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anesthesia crisis
resource management training: teaching anesthesiologists to handle
critical incidents. Aviat Space Environ Med. 1992;63(9):763–70.
118. Flin R, Maran N. Identifying and training non‑technical skills for teams in
acute medicine. Qual Saf Health Care. 2004;13(Suppl 1):i80–4.
119. Kirkpatrick D, Kirkpatrick J. Transferring learning to behaviour: using the
four levels to improve performance. San Francisco, London: Berrett‑
Koehler McGraw‑Hill distributor; 2005. p. 182. s. p.
120. Malec JF, Torsher LC, Dunn WF, Wiegmann DA, Arnold JJ, Brown DA,
et al. The mayo high performance teamwork scale: reliability and valid‑
ity for evaluating key crew resource management skills. Simul Healthc.
2007;2(1):4–10.
121. Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating
scale and checklist scores in the validation of an evaluation tool to
assess performance in the resuscitation of critically ill patients during
simulated emergencies (abbreviated as "CRM simulator study IB"). Simul
Healthc. 2009 Spring;4(1):6–16. https:// doi. org/ 10. 1097/ SIH. 0b013 e3181
880472.
122. Agency for Healthcare Research and Quality. Publishing and Communi‑
cations Guidelines Government Printing Office: Agency for Healthcare
Research and Quality, Rockville, MD.; 2013 [updated March 2021. Avail‑
able from: https:// www. ahrq. gov/ resea rch/ publi catio ns/ pubco mguide/
index. html]
123. Gordon MJ. A review of the validity and accuracy of self‑assessments in
health professions training. Acad Med. 1991;66(12):762–9.
Page 18 of 18
Abildgrenetal. Advances in Simulation (2022) 7:12
124. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L.
Accuracy of physician self‑assessment compared with observed meas‑
ures of competence: a systematic review. Jama. 2006;296(9):1094–102.
125. Burden AR, Pukenas EW, Deal ER, Coursin DB, Dodson GM, Staman GW,
et al. Using simulation education with deliberate practice to teach lead‑
ership and resource management skills to senior resident code leaders.
J Grad Med Educ. 2014;6(3):463–9.
126. Gaba DM. Crisis resource management and teamwork training in
anaesthesia. Br J Anaesth. 2010;105(1):3–6.
127. Levine JM. Socially‑shared cognition and consensus in small groups.
Curr Opin Psychol. 2018;23:52–6.
128. Resnick LB, Levine JM, Behrend S. Socially shared cognition. Washing‑
ton: American Psychological Association; 1991.
129. Hutchins E. How a cockpit remembers its speeds. Cognit Sci.
1995;19(3):265–88.
130. Salas E, DiazGranados D, Klein C, Burke CS, Stagl KC, Goodwin GF, et al.
Does team training improve team performance? A meta‑analysis. Hum
Factors. 2008;50(6):903–33.
131. Lebahn‑Hadidi M, Abildgren L, Hounsgaard L, Steffensen SV. Integrating
cognitive ethnography and phenomenology: rethinking the study
of patient safety in healthcare organisations. Phenomenol Cognit Sci.
2021. Online published oct. 19th, 2021.
132. Marquet K, Claes N, De Troy E, Kox G, Droogmans M, Schrooten W, et al.
One fourth of unplanned transfers to a higher level of care are associ‑
ated with a highly preventable adverse event: a patient record review
in six Belgian hospitals. Crit Care Med. 2015;43(5):1053–61.
133. Kaboli PJ, Rosenthal GE. Delays in transfer to the ICU: a preventable
adverse advent? J Gen Intern Med. 2003;18(2):155–6.
134. Kronman MP, Hall M, Slonim AD, Shah SS. Charges and lengths of stay
attributable to adverse patient‑care events using pediatric‑specific
quality indicators: a multicenter study of freestanding Children’s Hospi‑
tals. Pediatrics. 2008;121(6):e1653–e9.
135. Daouda OS, Hocine MN, Temime L. Determinants of healthcare worker
turnover in intensive care units: a micro‑macro multilevel analysis. PLoS
One. 2021;16(5):e0251779‑e.
136. Fredens K. Læring med kroppen forrest. 1st ed. Kbh: Hans Reitzel; 2018.
p. 286.
137. Maturana H. Kundskabens træ : den menneskelige erkendelses biolo‑
giske rødder. 1st ed. Århus: Ask; 1987. p. 245.
138. Ravn I, Maturana H. Kærlighedens biologi: interview med Humberto
Maturana. Omverden. 1991;2(7):17–9.
139. O’Connor C, Joffe H. Intercoder Reliability in Qualitative Research:
Debates and Practical Guidelines. International Journal of Qualitative
Methods. 2020. https:// doi. org/ 10. 1177/ 16094 06919 899220.
140. McCambridge J, Witton J, Elbourne DR. Systematic review of the Haw‑
thorne effect: new concepts are needed to study research participation
effects. J Clin Epidemiol. 2014;67(3):267–77.
141. Hollnagel E, Braithwaite J, Wears RL. (Eds.), Resilient health care. Farn‑
ham, UK: Ashgate; 2013.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub‑
lished maps and institutional affiliations.
