No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Measuring non-technical skills of medical emergency teams: An update on the validity and reliability of the Team Emergency Assessment Measure (TEAM)
Abstract
Medical emergency team performance including non-technical skills, is receiving increased attention due to the influences on patient safety. The Team Emergency Assessment Measure (TEAM) was developed to enable standardised performance assessment and structured team debriefing. From several studies, the TEAM has demonstrated a substantial body of normative data confirming its validity and reliability. This includes high uni-dimensional validity, significant subscale relationships between Leadership, Teamwork and Task management (p<0.006), a Cronbach alpha of 0.92 and adequate construct validity. The tool has potential for team training to improve team's non-technical performance. Further testing is required in 'real' clinical settings.
Supplementary resource (1)
... [8][9][10][11] The Teamwork Emergency Assessment Measure (TEAM) is an observational scale designed for measuring team processes and performance, which has been developed and validated in accordance with the psychometric theory. [11][12][13] It comprises 11 items rated on a 4-point Likert response scale and covering 3 dimensions -namely leadership, team work, and task management -and one overall team performance rating item. Originally developed in English, the TEAM scale has never been translated and a French version is currently lacking. ...
... 11 The properties of this score have been evaluated in three different studies showing consistent results. [11][12][13] TEAM is recognized as easy to use providing a quick and global evaluation of NTS during simulation training. 13 The results of this study show that the f-TEAM has similar psychometric properties to the original score (Table 2). ...
... 13 The results of this study show that the f-TEAM has similar psychometric properties to the original score (Table 2). 12,25,26 Thus, translation did not alter the properties of the TEAM score probably thanks to the robust methodology used for translation. Meanwhile, a possible ceiling effect was revealed, which was not previously assessed. ...
Aim:
Evaluation of team performances during medical simulation must rely on validated and reproducible tools. Our aim was to build and validate a French version of the Team Emergency Assessment Measure (TEAM) score, which was developed for the assessment of team performance and non-technical skills during resuscitation.
Methods:
A forward and backward translation of the initial TEAM score was made, with the agreement and the final validation by the original author. Ten medical teams were recruited and performed a standardized cardiac arrest simulation scenario. Teams were videotaped and nine raters evaluate non-technical skills for each team thanks to the French TEAM Score. Psychometric properties of the score were then evaluated.
Results:
French TEAM score showed an excellent reliability with a Cronbach coefficient of 0.95. Mean correlation coefficient between each item and the global score range was 0.78. The inter-rater reliability measured by intraclass correlation coefficient of the global score was 0.93. Finally, expert teams had higher French TEAM score than intermediate and novice teams.
Conclusion:
The French TEAM score shows good psychometric properties to evaluate team performance during cardiac arrest simulation. Its utilization could help in the assessment of non-technical skills during simulation.
... The psychometric properties of some frameworks including NOTSS (J. S. Yule et al., 2008S. Yule et al., , 2009, TEAM (Cooper & Cant, 2014;Cooper et al., 2016;Maignan et al., 2016), Oxford NOTECHS (Etheridge et al., 2021;Mishra et al., 2009;Robertson et al., 2014), and the Trauma Table 5). There is evidence for interrater and testretest reliability when the obstetrics-specific framework, GAOTP, is used with at least eight raters (Morgan et al., 2012). ...
... Raters are usually experienced clinicians from the setting and, although the training of raters is generally recommended (Andersen et al., 2010;Fletcher et al., 2003;S. Yule et al., 2008), some authors of measurement frameworks reported that only minimal training was required for raters (Cooper & Cant, 2014;Malec et al., 2007). ...
... Such research findings would provide valuable information when clinicians or researchers consider which framework to adapt (Higham et al., 2019). However, in terms of psychometric properties and using a multidisciplinary focus during development, the most robust frameworks we identified were TEAM (Cooper & Cant, 2014), Oxford NOTECHS (Mishra et al., 2009), AOTP, and GAOTP (Morgan et al., 2012). ...
Objective
To identify the most suitable nontechnical skills framework to adapt and apply to the air medical transfer of pregnant women.
Data Sources
Embase, PsycINFO, PubMed, MEDLINE, Web of Science, CINAHL, Science Direct, and Google Scholar.
Study Selection
We retrieved potentially relevant articles using a predefined combination of keywords extended with truncation and Boolean operators. Database and manual reference searches yielded 569 peer-reviewed articles. We included articles if they presented empirical data and described nontechnical or cognitive competency skills frameworks for health care professionals. We discussed any ambiguities regarding inclusion, and they were resolved by consensus. We retained 71 full-text articles for final review.
Data Extraction
We coded extracted data under four criteria: nontechnical skill categories, context of use, psychometric properties, and rating system. We generated descriptive summary tables of the characteristics of existing nontechnical skills frameworks based on publication year, method of development, clinical setting, clinical specialty, routine/crisis-based performance, and team/individual performance.
Data Synthesis
We identified 42 nontechnical skills frameworks from a variety of health care settings. We critically examined context of use and how use in various clinical settings may align with air transfers of pregnant women. Our findings illustrate the importance of team-based performance and routine rather than crisis-focused skills. Maintaining situational awareness throughout all stages of the transfer and communicating effectively with team members, the pregnant woman, and her partner are skills that are particularly important to ensure good outcomes.
Conclusion
We selected the Global Assessment of Obstetric Team Performance as the most suitable nontechnical skills framework to adapt to the clinical setting of air medical transfer of pregnant women. We considered the clinical specialty, specific nontechnical skills required in the setting, the framework’s properties, and the requirement to focus on routine team performance.
... The Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR) focuses on individuals as part of a team and has been exclusively tested and used in in-hospital simulation-based training. 10 [9][10][11]14,15 and in in-hospital clinical settings. 9,12,13 While T-NOTECHS was validated for trauma team resuscitations and paediatric emergencies, TEAM specifically focuses on in-hospital cardiac arrests (IHCA) and emergency resuscitations in the clinical setting, primarily in the emergency department (ED). ...
... The Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR) focuses on individuals as part of a team and has been exclusively tested and used in in-hospital simulation-based training. 10 [9][10][11]14,15 and in in-hospital clinical settings. 9,12,13 While T-NOTECHS was validated for trauma team resuscitations and paediatric emergencies, TEAM specifically focuses on in-hospital cardiac arrests (IHCA) and emergency resuscitations in the clinical setting, primarily in the emergency department (ED). ...
... 9,12,13 The TEAM instrument focuses on team performance at the team level. [8][9][10] TEAM with clinical in-hospital resuscitation teams in a variety of resuscitations (neurological, trauma, respiratory, shock-all causes and cardiovascular). 12,13 In this pilot study TEAM is used for the first time in an out-of-hospital clinical resuscitation setting. ...
Aim
To analyse non-technical skills of mobile medical teams during out-of-hospital cardiac arrests (OHCA) using the validated Team Emergency Assessment Measure (TEAM) tool. To research the correlation between non-technical skills and patient outcome.
Methods
Adult patients who experienced an out-of-hospital cardiac arrest between July 2016, and June 2018, and were treated by a mobile medical team from the University Hospital Leuven, were eligible for the study. Resuscitations were video recorded from the team leader’s perspective. Video recordings were reviewed and scored by emergency physicians, using the TEAM evaluation form.
Results
In total 114 OHCAs were analysed. The mean TEAM score was 34.4/44 (SD = 5.5). The mean item score was 3.1/4 (SD = 0.8). On average, ‘effective team communication’ had the lowest score (2.4), while ‘acting with composure and control’ and ‘following of approved standards/guidelines’ scored the highest (3.4). The average non-technical skills theme scores were 2.9 (SD = 0.9) for ‘Leadership’, 3.1 (SD = 0.8) for ‘Teamwork’ and 3.3 (SD = 0.7) for ‘Task management’. ‘Leadership’ was rated significantly lower than ‘Teamwork’ (p = 0.004) and ‘Task management’ (p < 0.001). No significant correlation was found between TEAM and return of spontaneous circulation (p = 0.574) or one month survival (p = 0.225).
Conclusion
The mean overall TEAM score was categorized as good. Task management scored high, while leadership and team communication received lower scores. Future training programs should thus focus on improving leadership and communication. In this pilot study no correlation was found between non-technical skills and survival.
... Various specific team-rating scales have been developed to measure teamwork performances. These assessment tools are based on either self-ratings or observational team performance ratings in real and simulated settings [1,[11][12][13][14]. Commonly used assessment tools are the NRP Megacode Assessment form and also the team strategies and tools to enhance performance and patient safety (TeamSTEPPS) training, which have been used as a template to score neonatal resuscitation performance [1,12]. ...
... Commonly used assessment tools are the NRP Megacode Assessment form and also the team strategies and tools to enhance performance and patient safety (TeamSTEPPS) training, which have been used as a template to score neonatal resuscitation performance [1,12]. The Team Emergency Assessment Measure (TEAM) is reported to be a valid and reliable instrument for rating teamwork during real adult medical emergencies [13,14]. ...
... We used TEAM as an assessment instrument to evaluate the teamwork of newborn emergency teams in simulation training environments [13,14]. Earlier studies of teamwork in newborn resuscitation usually used the NRP Megacode Assessment Form as a template for scoring NRP performance [1]. ...
Background:
Video analyses of real-life newborn resuscitations have shown that Neonatal Resuscitation Program (NRP) guidelines are followed in fewer than 50 % of cases. Multidisciplinary simulation is used as a first-rate tool for the improvement of teamwork among health professionals. In the study we evaluated the impact of the crisis resource management (CRM) and anesthesia non-technical skills instruction on teamwork during simulated newborn emergencies.
Methods:
Ninety-nine participants of two delivery units (17 pediatricians, 16 anesthesiologists, 14 obstetricians, 31 midwives, and 21 neonatal nurses) were divided to an intervention group (I-group, 9 teams) and a control group (C-group, 6 teams). The I-group attended a CRM and ANTS instruction before the first scenario. After each scenario the I-group performed either self- or peer-assessment depending on whether they had acted or observed in the scenario. All the teams participated in two and observed another two scenarios. All the scenarios were video-recorded and scored by three experts with Team Emergency Assessment Measure (TEAM). SPSS software and nlme package were used for the statistical analyses.
Results:
The total TEAM scores of the first scenario between the I- and C-group did not differ from each other. Neither there was an increase in the TEAM scoring between the first and second scenario between the groups. The CRM instruction did not improve the I-group's teamwork performance. Unfortunately the teams were not comparable because the teams had been allowed to self-select their members in the study design. The total TEAM scores varied a lot between the teams. Mixed-model linear regression revealed that the background of the team leader had an impact on differences of the total teamwork scores (D = 6.50, p = 0.039). When an anesthesia consultant was the team leader the mean teamwork improved by 6.41 points in comparison to specialists of other disciplines (p = 0.043).
Conclusion:
The instruction of non-technical skills before simulation training did not enhance the acquisition of teamwork skills of the intervention groups over the corresponding set of skills of the control groups. The teams led by an anesthesiologist scored the best. Experience of team leaders improved teamwork over the CRM instruction.
... Instruments were found which focus on a variety of hospital settings and teams such as the operating room, the resuscitation teams, obstetric teams; trauma teams; nurse anaesthetic teams; healthcare teams in acute settings, and the emergency environment, as shown in Table 1. Most instruments assess three or more non-technical skills, including leadership, task management, teamwork, problem-solving, decisionmaking, resource utilization, situation-awareness, and communication (Baker et al., 2011;Cooper et al., 2010aCooper et al., , 2010bCooper and Cant, 2014;Guise et al., 2008;Healey et al., 2004;Hull et al., 2011;Kim et al., 2006;Lyk-Jensen et al., 2014;Malec et al., 2007;Mishra et al., 2009;Mitchell et al., 2012;Robertson et al., 2014;Sevdalis et al., 2008;Sevdalis et al., 2009;Steinemann et al., 2012;Walker et al., 2011). Other instruments focus only on one or two isolated non-technical skills such as leadership (Cooper and Wakelam, 1999), leadership and teamwork (Cooper and Wakelam, 1999), and communication (Cooper et al., 2007). ...
... The Team Emergency Assessment Measure (TEAM) (Cooper et al., 2010a(Cooper et al., , 2010bCooper and Cant, 2014) was divided in three categories, leadership, teamwork and task management to measure resuscitation teams performance. Encompassed within these categories are nine elements, leadership control, communication, cooperation and coordination, team climate, adaptability, situation awareness (perception), situation awareness (projection), prioritization, and clinical standards. ...
Background:
In nursing, non-technical skills are recognized as playing an important role to increase patient safety and successful clinical outcomes (Pearson and McLafferty, 2011). Non-technical skills are cognitive and social resource skills that complement technical skills and contribute to safe and efficient task performance (Flin et al., 2008). In order to effectively provide non-technical skills training, it is essential to have an instrument to measure these skills.
Methodology:
An online search was conducted. Articles were selected if they referred to and/or described instruments assessing non-technical skills for nurses and/or prelicensure nursing students in educational, clinical and/or simulated settings with validation evidence (inclusion criteria).
Results:
Of the 53 articles located, 26 met the inclusion criteria. Those referred to and/or described 16 instruments with validation evidence developed to assess non-technical skills in multidisciplinary teams including nurses.
Conclusion:
Although articles have shown 16 valid and reliable instruments, to our knowledge, no instrument has been published or developed and validated for the assessment of non-technical skills of only nurses in general, relevant for use in high-fidelity simulation-based training for prelicensure nursing students. Therefore, there is a need for the development of such an instrument.
... Measures. To investigate non-technical performance, one blinded reviewer (an anesthesiologist from a different hospital with 7 years of work experience and 10 years of simulation training experience who did not know the participants of the study) watched the recordings and rated the teams' performances using the Team Emergency Assessment Measure (TEAM), a measurement instrument specifically developed for medical emergency teamwork [10,11]. We aggregated the TEAM questions 1 to 11 and used the percentage of the maximum TEAM score as the dependent variable. ...
... The absence of such a positive association in the study of Everett et al. [14] has been attributed to the measurement tool, which was not validated for their context [43]. We used the same tool (i.e., the TEAM score), albeit on CPR scenarios for which the tool was developed [10,11]. Finally, our results are also in agreement with recent studies on various operating room crises, which reported a significant positive association between individual (i.e., only the team leader was a participant and the other team members were actors) non-technical performance and cognitive aid use [e.g., 37,40,44]. ...
Cognitive aids - artefacts that support a user in the completion of a task at the time - have raised great interest to support healthcare staff during medical emergencies. However, the mechanisms of how cognitive aids support or affect staff remain understudied. We describe the iterative development of a tablet-based cognitive aid application to support in-hospital resuscitation team leaders. We report a summative evaluation of two different versions of the application. Finally, we outline the limitations of current explanations of how cognitive aids work and suggest an approach based on embodied cognition. We discuss how cognitive aids alter the task of the team leader (distributed cognition), the importance of the present team situation (socially situated), and the result of the interaction between mind and environment (sensorimotor coupling). Understanding and considering the implications of introducing cognitive aids may help to increase acceptance and effectiveness of cognitive aids and eventually improve patient safety.
... We evaluated the non-technical components of the team performance using the Team Emergency Assessment Measure (TEAM), a measurement instrument developed for rating non-technical skills of medical emergency teamwork [25]. Arguments have been made for the validity and reliability of the TEAM in contexts similar to our own [26]. TEAM assesses the following: leadership (two items); team work (seven items); task management (two items); and an overall score from 1 to 10. ...
... To lend added insight beyond the other studies of these checklists [10,11], we examined the non-technical aspects of performance using TEAM scores [25,26]. We chose to study non-technical team performance because issues related to team function have been shown to affect negatively surgical performance and outcomes [32,33] and cognitive aids are known to improve non-technical skills in the operating theatre [12,34]. ...
Although the incidence of major adverse events in surgical daycare centres is low, these critical events may not be managed optimally due to the absence of resources that exist in larger hospitals. We aimed to study the impact of operating theatre critical event checklists on medical management and teamwork during whole-team operating theatre crisis simulations staged in a surgical daycare facility. We studied 56 simulation encounters (without and with a checklist available) divided between an initial session and then a retention session several months later. Medical management and teamwork were quantified via percentage adherence to key processes and the Team Emergency Assessment Measure, respectively. In the initial session, medical management was not improved by the presence of a checklist (56% without checklist vs. 62% with checklist; p = 0.50). In the retention session, teams performed significantly worse without the checklists (36% without checklist vs. 60% with checklist; p = 0.04). We did not observe a change in non-technical skills in the presence of a checklist in either the initial or retention sessions (68% without checklist vs. 69% with checklist (p = 0.94) and 69% without checklist vs. 65% with checklist (p = 0.36), respectively). Critical events checklists do not improve medical management or teamwork during simulated operating theatre crises in an ambulatory surgical daycare setting.
... Our results demonstrate that NTS, specifically leadership, communication, and teamwork, can be effectively integrated into clinical prehospital training. Studies examining NTS in emergency care range from leadership [8,[16][17], teamwork [18][19][20][21], safety [22][23][24][25], and communication [1,6,[26][27][28][29][30]. Our developed curriculum focused on three of the five major prehospital NTS domains outlined by Shields and Flin [2]. ...
Background:
Paramedic trainees in developing countries face complex and chaotic clinical environments that demand effective leadership, communication, and teamwork. Providers must rely on non-technical skills (NTS) to manage bystanders and attendees, collaborate with other emergency professionals, and safely and appropriately treat patients. The authors designed a NTS curriculum for paramedic trainees focused on adaptive leadership, teamwork, and communication skills critical to the Indian prehospital environment.
Methods:
Forty paramedic trainees in the first academic year of the 2-year Advanced Post-Graduate Degree in Emergency Care (EMT-paramedic equivalent) program at the GVK-Emergency Management and Research Institute campus in Hyderabad, India, participated in the 6-day leadership course. Trainees completed self-assessments and delivered two brief video-recorded presentations before and after completion of the curriculum.
Results:
Independent blinded observers scored the pre- and post-intervention presentations delivered by 10 randomly selected paramedic trainees. The third-party judges reported significant improvement in both confidence (25 %, p < 0.01) and body language of paramedic trainees (13 %, p < 0.04). Self-reported competency surveys indicated significant increases in leadership (2.6 vs. 4.6, p < 0.001, d = 1.8), public speaking (2.9 vs. 4.6, p < 0.001, d = 1.4), self-reflection (2.7 vs. 4.6, p < 0.001, d = 1.6), and self-confidence (3.0 vs. 4.8, p < 0.001, d = 1.5).
Conclusions:
Participants in a 1-week leadership curriculum for prehospital providers demonstrated significant improvement in self-reported NTS commonly required of paramedics in the field. The authors recommend integrating focused NTS development curriculum into Indian paramedic education and further evaluation of the long term impacts of this adaptive leadership training.
... Our review found several articles referring to the importance of NTS training for healthcare and patient safety (Andersen, Jensen, Lippert, & Ostergaard, 2010;Andrews, 2014;Baker, Capella, Hawkes, Gallo, & Clinic, 2011;Boet et al., 2014;Boet, Reeves, & Bould, 2015;Briggs et al., 2015;Brunckhorst et al., 2015;Burton & Ormrod, 2011;Capella et al., 2010;Clark, 2009;Cooper, Endacott, eISSN: 2357-1330 Cooper & Cant, 2014;Dieckmann, 2010;Dunn et al., 2007;Fletcher et al., 2003;Flin, O'Connor, & Crichton, 2008;Flin & Patey, 2011;Flin & Maran, 2015;Freeth et al., 2009;Gaba et al., 2001;Garbee et al., 2013;Gillman et al., 2015;Gundrosen, Solligård, & Aadahl, 2014;Gururaja, Yang, Paige, & Chauvin, 2008;Hicks, Coke, & Li, 2009;Hull et al., 2012;Irwin & Weidmann, 2015;Kiesewetter & Fischer, 2015;Kodate, Ross, Anderson, & Flin, 2012;Kohn, Corrigan, & Donaldson, 2000;Kutzin, 2010;Légaré el al., 2012;Lindamood et al., 2011;Lyk-Jensen et al., 2014;Martinou et al., 2015;Milligan, 2007;Mitchell & Flin, 2008;Nguyen, Elliott, Watson, & Dominguez, 2015;Paige et al., 2014;Pearson & McLafferty, 2011;Ponton-Carss, Kortbeek, & Ma, 2016;Riley et al., 2011;Roberts et al., 2014;Robertson et al., 2009;Robertson et al., 2014;Sara-aho, 2015, Sevdalis, 2013Shapiro et al., 2004;Thomas et al., 2007;Wisborg & Manser, 2014;Yule et al., 2008;Yule & Paterson-Brown, 2012;Ziesmann et al., 2013) and/or describing studies relating to NTS training in different healthcare settings and/or for different healthcare students and/or professionals (Brunckhorst et al., 2015;Capella et al., 2010;Dunn et al., 2007;Freeth et al., 2009;Garbee et al., 2013;Gillman et al., 2015;Hull et al., 2012;Lindamood et al., 2011;Lyk-Jensen et al., 2014;Martinou et al., 2015;Nguyen, Elliott, Watson, & Dominguez, 2015;Paige et al., 2014;Riley et al., 2011;Roberts et al., 2014;Robertson et al., 2009;Thomas et al., 2007;Ziesmann et al., 2013). ...
Until nowadays, formal training of nurses has focused predominantly on developing knowledge, clinical expertise and technical skills. These skills are necessary but not sufficient to promote and maintain high levels of patient care and safety. Therefore, although recent literature has highlighted the importance of introducing non-technical skills training and assessment within healthcare, nursing education has still to fully include these skills on the training process. International research has shown that many errors and adverse events are due to a lack of nontechnical skills rather than clinical knowledge (Dieckmann, 2010; Irwin & Weidmann, 2015; Lyk-Jensen, Jepsen, Spanager, Dieckmann, & Ostergaard, 2014). Patient safety issues and the incidence of errors are important to all healthcare professionals and public health organizations. Errors do occur in healthcare as in other industries but when these errors involve the risk to human life, the concern is paramount. Thus, one of the most important strategies for error reduction involves prevention (Lindamood, Rachwal, Kappus, Weinstock, & Doherty, 2011). In the light of this, it is essential for undergraduate nursing students to develop not only technical but also nontechnical skills. Moreover, developing and implementing a non-technical skills training course may significantly improve students’ performance and better prepare them for their future clinical practice. Consequently, consideration must be given in integrating NTS training into undergraduate nursing education curriculum. © 2016 Published by Future Academy www.FutureAcademy.org.uk
... Further studies regarding the TEAM scale specifically investigated its psychometric properties, and these studies confirmed the tool as a valid, reliable and feasible instrument for measuring the NTS of medical emergency teams: 12 13 some limitations persisted, specifically the scale did not enable evaluation of single members of the team, but only gave an overall assessment. 14 Moreover, further testing is required in real clinical settings. Similar re-evaluations were not conducted for the other scales. ...
Background Teamwork training has been included in several emergency medicine (EM) curricula; the aim of this study was to compare different scales’ performance in teamwork evaluation during simulation for EM residents.
Methods In the period October 2013–June 2014, we performed bimonthly high-fidelity simulation sessions, with novice (I–III year, group 1 (G1)) and senior (IV–V year, group 2 (G2)) EM residents; scenarios were designed to simulate management of critical patients. Videos were assessed by three independent raters with the following scales: Emergency Team Dynamics (ETD), Clinical Teamwork Scale (CTS) and Team Emergency Assessment Measure (TEAM). In the period March–June, after each scenario, participants completed the CTS and ETD.
Results The analysis based on 18 sessions showed good internal consistency and good to fair inter-rater reliability for the three scales (TEAM, CTS, ETD: Cronbach's α 0.954, 0.954, 0.921; Intraclass Correlation Coefficients (ICC), 0.921, 0.917, 0.608). Single CTS items achieved highly significant ICC results, with 12 of the total 13 comparisons achieving ICC results ≥0.70; a similar result was confirmed for 4 of the total 11 TEAM items and 1 of the 8 total ETD items. Spearman's r was 0.585 between ETD and CTS, 0.694 between ETD and TEAM, and 0.634 between TEAM and CTS (scales converted to percentages, all p<0.0001). Participants gave themselves a better evaluation compared with external raters (CTS: 101±9 vs 90±9; ETD: 25±3 vs 20±5, all p<0.0001).
Conclusions All examined scales demonstrated good internal consistency, with a slightly better inter-rater reliability for CTS compared with the other tools.
... The modified and simplified Team Emergency Assessment Measure questionnaire (TEAM) was used in this study (Fig. 1). The creation and validation of this tool is described elsewhere and its validation was not part of this study [6][7][8][9][10][11]. The TEAM was modified for this study as follows: items one through six, i.e. (1) the team leader let the team know what was expected of them through direction and command, (2) the team leader maintained a global perspective, (3) the team communicated effectively, (4) the team worked together to complete tasks in a timely manner, (5) the team acted with composure and control and (6) team morale was positive, all remained unchanged. ...
Background
Non-technical skills (NTS) are important for the proper functioning of emergency medical ambulance crews but have hardly been researched in the conditions of clinical pre-hospital care.
The primary objective of this study, therefore, is to describe the use of NTS in practice. The secondary objective is to compare if the performance of NTS varies according to the type of case.
Methods
In this multicentric observational study the modified Team Emergency Assessment Measure (TEAM) score was used to assess the performed NTS of two or more crews on site. The evaluation consisted of leadership, teamwork and task management, rated by a field supervisor.
The study observations took place in real clinical pre-hospital emergency medical cases when two or more crews were dispatched between October 2019 and August 2020. The sample size was determined by researchers prior to the study to at least 100 evaluated events per each of the three participating emergency medical services.
The results are presented as median and interquartile range. The internal reliability, consistency and validity of test items and results were evaluated. The Kruskal–Wallis test and the post hoc Mann-Whitney U test with Bonferroni correction were used for multiple comparisons of three groups.
Results
A total of 359 events were evaluated. Surprisingly, the median value for all eight items was as high as 3.0 with a similar interquartile range of 1.0. There were no differences observed by case type (CPR vs. TRAUMA vs. MEDICAL) except from item 1. A post hoc analysis revealed that this difference is in favour of a higher rated performance of non-technical skills in CPR.
Conclusions
The overall result of the performance of non-technical skills can be regarded as very good and can serve for further evaluations. The crews achieved better parameters of NTS in leadership in resuscitation situations in comparison with general medical events.
Trial Registration
The study is registered at Clinical Trials under the ID: NCT04503369.
... 11 Specifically designed for emergency teams the Team Emergency Assessment Measure (TEAM) is a 12-item assessment measure used to rate leadership, teamwork, and task management using a 5 point Likert scale. Previous testing in simulated scenarios has demonstrated that the TEAM is valid and reliable 12,13 with, for example, strong uni-dimensional and high internal consistency (Cronbach alphas of 0.91 and 0.97). Further, the tool is feasible for the emergency workplace-taking less than a minute to complete. ...
Aim:
To test the resuscitation non-technical Team Emergency Assessment Measure (TEAM) for feasibility, validity and reliability, in two Australian Emergency Departments (ED).
Background:
Non-technical (teamwork) skills have been identified as inadequate and as such have a significant impact on patient safety. Valid and reliable teamwork assessment tools are an important element of performance assessment and debriefing processes.
Methods:
A quasi experimental design based on observational ratings of resuscitation non-technical skills in two metropolitan ED. Senior nursing staff rated 106 adult resuscitation team events over a ten month period where three or more resuscitation team members attended. Resuscitation events, team performance and validity and reliability data was collected for the TEAM.
Results:
Most rated events were for full cardiac resuscitation (43%) with 3-15 team members present for an average of 45minutes. The TEAM was found to be feasible and quickly completed with minimal or no training. Discriminant validity was good as was internal consistency with a Cronbach alpha of .94. Uni-dimensional and concurrent validity also reached acceptable standards, .94 and>.63 (p=<.001) respectively, and a single 'teamwork' construct was identified. Non-technical skills overall were good but leadership was rated notably lower than task and teamwork performance indicating a need for leadership training.
Conclusion:
The TEAM is a feasible, valid and reliable non-technical assessment measure in simulated and real clinical settings. Emergency teams need to develop leadership skills through training and reflective debriefing.
... We will investigate the feasibility and reliability of existing teamwork assessment tools, for example, the Behavioral Assessment Tool, 20 the Nontechnical Skills scale, 21 and the TEAM tool. 22 Determined through application of generalizability theory, the instrument with the best discriminatory capacity for team performance will be added to the VIPER registry. With an established metric for measuring important nontechnical team skills, we will have a platform on which to investigate the effects of teamwork dynamics on real patient outcomes. ...
... 8,9 The TEAM tool is easy to use, has a simple scoring system based on the frequency of observations and high inter-rater reliability. [9][10][11][12] The tool is composed of 11 items distributed in 3 domains (Leadership, Teamwork and Task Management), scored from 0 to 4, and a global NTS performance score from 1 to 10. 9 The TEAM tool was translated and is available online in several languages, but the only published validated versions are English (original) and French languages. 9,13 Currently, there is no validated translation of the TEAM tool to Brazilian Portuguese. ...
Objectives
The aim of this study was to conduct the translation and cross-cultural adaptation of the original Team Emergency Assessment Measure (TEAM) tool into the Brazilian Portuguese language and investigate the internal consistency, inter-rater reliability, and concurrent validity of this new version (bp-TEAM).
Methods
Independent medical translators performed forward and backward translations of the TEAM tool between English and Portuguese, creating the bp-TEAM. The authors selected 23 videos from final-year medical students during in-situ emergency simulations. Three independent raters assessed all the videos using the bp-TEAM and provided a score for each of the 12 items of the tool. The authors assessed the internal consistency and the inter-rater reliability of the tool.
Results
Raters assessed all 23 videos. Internal consistency was assessed among the 11 items of the bp-TEAM from one rater, yielding a Cronbach's alpha of 0.89. inter-item correlation analysis yielded a mean correlation coefficient rho of 0.46. Inter-rater reliability analysis among the three raters yielded an intraclass correlation coefficient of 0.86 (95% CI 0.83‒0.89), p < 0.001.
Conclusion
The Brazilian Portuguese version of the TEAM tool presented acceptable psychometric properties, similar to the original English version.
... This kind of skill is particularly important for nurses whose job partly consists in transmitting critical information in the medical team (Mitchell & Flin 2008, Miller et al. 2009). Classically, non-technical skills are defined as corresponding to cognitive, social and personal resources that complement technical skills and contribute to a safe and efficient performance (Cooper & Cant 2014). Among non-technical skills, some frequently identified are the capacity to communicate, to lead, to make decisions or to ensure situational awareness (Reader et al. 2006, Yule et al. 2006. ...
... Incorporating non-technical skills into training and assessment may improve nurses' attitudes towards understanding of human factors and non-technical skills, with the aim of reducing the risk of the potential for adverse events and therefore improve patient care 19 . In recent years, nurses' non-technical skills have been studied in other areas of nursing such as intensive care nursing 45 and emergency department teams 46 . Each clinical environment adapts its own, unique nontechnical skills requirements that are crucial for safe practice. ...
In 1999, the US Institute of Medicine released a report To err is human: Building a safer health system, which estimates that 44,000 and 98,000 patients die as a result of medical errors in the operating room (OR) annually. Despite dramatic improvements in surgical safety knowledge, at least half of the adverse events occur during surgical care. Human failures (for example, miscommunication, teamwork breakdown, leadership and poor decision making) are not uncommon and often lead to errors in surgery. Retained sponges, wrong site surgery, mismatched organ transplants, or blood transfusion can be the result of human errors resulting in many adverse incidents and accidents. Analysis of adverse events in health care suggests that improvement of non-technical skills may reduce surgical errors and enhance patient outcomes. The term 'non-technical skills' refers to "the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance". Subsumed within non-technical skills are the domains of communication, leadership, teamwork, decision making and situation awareness.
... Incorporating non-technical skills into training and assessment may improve nurses' attitudes towards understanding of human factors and non-technical skills, with the aim of reducing the risk of the potential for adverse events and therefore improve patient care 19 . In recent years, nurses' non-technical skills have been studied in other areas of nursing such as intensive care nursing 45 and emergency department teams 46 . Each clinical environment adapts its own, unique nontechnical skills requirements that are crucial for safe practice. ...
To identify and describe the factors that impact on the performance of scrub nurses' non-technical skills performance during the intra-operative phase of surgery.
Non-technical skills have been identified as important precursors to errors in the operating room. However, few studies have investigated factors influencing non-technical skills of scrub nurses.
Prospective observational study.
Structured observations were performed on a sample of 182 surgical procedures across eight specialities by two trained observers from August 2012-April 2013 at two hospital sites. Participants were purposively selected scrub nurses. Bivariate correlations and a multiple linear regression model were used to identify associations among length of surgery, patients' acuity using the American Society of Anesthesiologists classification system, team familiarity, number of occasions scout nurses leave the operating room, change of scout nurse and the outcome, the non-technical skills performance of scrub nurses.
Patient acuity and team familiarity were the strongest predictors of scrub nurses' non-technical skills performance at hospital site A. There were no correlations between the predictors and the performance of scrub nurses at hospital site B.
A dedicated surgical team and patient acuity potentially influence the performance of scrub nurses' non-technical skills. Familiarity with team members foster advanced planning, thus minimizing distractions and interruptions that impact on scrub nurses' performance. Development of interventions aimed at improving non-technical skills has the potential to make a substantial difference and enhance patient care.
© 2015 John Wiley & Sons Ltd.
... This kind of skill is particularly important for nurses whose job partly consists in transmitting critical information in the medical team (Mitchell & Flin 2008, Miller et al. 2009). Classically, non-technical skills are defined as corresponding to cognitive, social and personal resources that complement technical skills and contribute to a safe and efficient performance (Cooper & Cant 2014). Among non-technical skills, some frequently identified are the capacity to communicate, to lead, to make decisions or to ensure situational awareness (Reader et al. 2006, Yule et al. 2006). ...
To explore how nursing performance is impacted by different forms of team communication including a message transmitted through an earpiece which triggers reflective thinking in the simulation of a deteriorating patient situation.
Communication can either support team performance or produce interruptions potentially leading to error. Today, technology offers the opportunity to use devices that can permit communication.
An experimental protocol was used with quantitative and qualitative analyses.
Pairs of nursing students (N = 26) were dispatched to either an experimental group having to wear an earpiece priming reflective thinking, or to a control group. The study was conducted between October 2013-April 2014.
The number of spontaneous information exchanges between pairs of participants was positively correlated with overall performance (actions performed and physician call) and with actions performed at the right moment. The number of questions in the team was positively correlated with overall action performance. No quantitative effect of the earpiece message on the performance indicators was found. But, a qualitative observation showed that this message can allow for error avoidance. Subjective evaluation of the earpiece as an aid was negatively correlated with overall action performance. Its evaluation as a disturbance was also negatively correlated with the measurement of actions performed at the right moment.
The ability to exchange information and to ask questions seems to contribute to performance in care delivery. The use of communication devices to trigger reflective thinking must be studied in more depth to assess their capacity to improve performance.
© 2015 John Wiley & Sons Ltd.
... In an effort to improve CPR outcomes, many hospitals have designated professional CPR teams [4]; however, the most effective resuscitation models for improving outcomes remain controversial [5,6]. Several studies have reported that having either a medical emergency team or a rapid response (RR) team designated for CPR could increase the quality of CPR [7,8]. ...
Background
Previous studies have reported that the quality of cardiopulmonary resuscitation (CPR) is closely associated with patient outcomes. The aim of this study was to compare patient CPR outcomes across resident, emergency medicine, and rapid response teams.
Methods
The records of patients who underwent CPR at the Seoul National University Bundang Hospital from January 1, 2013 to December 31, 2016 were analyzed retrospectively. Return of spontaneous circulation, 10- and 30-day survival, and live discharge after return of spontaneous circulation were compared across patients treated by the three CPR teams.
Results
Of the 1145 CPR cases, 444 (39%) were conducted by the resident team, 431 (38%) by the rapid response team, and 270 (23%) by the emergency medicine team. The adjusted odds ratios for the return of spontaneous circulation and subsequent 10-day survival among patients who received CPR from the resident team compared to the rapid response team were 0.59 (P = 0.001) and 0.71 (P = 0.037), respectively. There were no significant differences in the 30-day survival and rate of live discharge between patients who received CPR from the rapid response and resident teams; likewise, no significant differences were observed between patients who received CPR from the emergency medicine and rapid response teams.
Conclusions
Patients receiving CPR from the rapid response team may have higher 10-day survival and return of spontaneous circulation rates than those who receive CPR from the resident team. However, our results are limited by the differences in approach, time of CPR, and room settings between teams.
... In addition, the tool was designed to be used in emergencies and has been shown to be valid in simulation-based assessments. 27 Our third limitation is that we did not attempt to demonstrate interrater reliability. We chose instead to have 1 video reviewer (T.B.C.) abstract data from all the videos and have a second reviewer (G.L.G.) review a portion of videos to prevent ''drift'' in scoring and ensure consensus. ...
Pediatric emergencies require effective teamwork. These skills are developed and demonstrated in actual emergencies and in simulated environments, including simulation centers (in center) and the real care environment (in situ). Our aims were to compare teamwork performance across these settings and to identify perceived educational strengths and weaknesses between simulated settings. We hypothesized that teamwork performance in actual emergencies and in situ simulations would be higher than for in-center simulations.
A retrospective, video-based assessment of teamwork was performed in an academic, pediatric level 1 trauma center, using the Team Emergency Assessment Measure (TEAM) tool (range, 0-44) among emergency department providers (physicians, nurses, respiratory therapists, paramedics, patient care assistants, and pharmacists). A survey-based, cross-sectional assessment was conducted to determine provider perceptions regarding simulation training.
One hundred thirty-two videos, 44 from each setting, were reviewed. Mean total TEAM scores were similar and high in all settings (31.2 actual, 31.1 in situ, and 32.3 in-center, P = 0.39). Of 236 providers, 154 (65%) responded to the survey. For teamwork training, in situ simulation was considered more realistic (59% vs. 10%) and more effective (45% vs. 15%) than in-center simulation.
In a video-based study in an academic pediatric institution, ratings of teamwork were relatively high among actual resuscitations and 2 simulation settings, substantiating the influence of simulation-based training on instilling a culture of communication and teamwork. On the basis of survey results, providers favored the in situ setting for teamwork training and suggested an expansion of our existing in situ program.
... It also has a global overall rating, on a 10-point scale, for the team's non-technical performance. 13 Advice was sought, from the department of medical statistics at the University of Aberdeen (UoA), in regard to both the required sample size and the subsequent methods of analysis for the assessment tools being used. These would include Shapiro-Wilk test and inspection of Q-Q plots to determine normal distribution of data, along with independent-sample t-testing to identify any statistically significant differences. ...
Aim of the study To assess performance in a simulated resuscitation after participating in either an interprofessional learning (IPL) or uniprofessional learning (UPL) immediate life support (ILS) training course.
Introduction The Team Emergency Assessment Measure (TEAM) is routinely used in Resuscitation Council (UK) Advanced Life Support courses. This study used the psychometrically validated tool to assess if the delivery of an IPL ILS to final year medical and nursing students could improve overall behavioural performance and global TEAM score.
Methods A randomised study of medical (n=48) and nursing (n=48) students, assessing performance in a simulated resuscitation following the IPL or UPL ILS courses. Postcourse completion participants were invited back to undertake a video-recorded simulated-resuscitation scenario. Each of these were reviewed using the TEAM tool, at the time by an experienced advanced life support instructor and subsequently by a clinician, independent to the study and blinded as to which cohort they were reviewing.
Results Inter-rater reliability was tested using a Bland-Altman plot indicating non-proportional bias between raters. Parametric testing and analysis showed statistically significant higher global overall mean TEAM scores for those who had attended the IPL ILS courses.
Conclusion Our results demonstrate that an IPL approach in ILS produced an increased effect on TEAM scores with raters recording a significantly more collaborative team performance. A postscenario questionnaire for students also found a significantly improved experience within the team following the IPL course compared with those completing UPL training. Although this study shows that team behaviour and performance can change and improve in the short-term, we acknowledge further studies are required to assess the long-term effects of IPL interventions. Additionally, through this type of study methodology, other outcomes in regard to resuscitation team performance may be measured, highlighting other potential benefit to patients, at level four of Kirkpatrick’s hierarchy.
... It measures three dimensions of team performance: leadership, teamwork and task management. 16 17 The instrument has been initially developed and validated for simulator-based team training and has been recently validated for the collection of observational ratings of non-technical skills during live resuscitations in emergency departments. 18 Using instruments to score scenarios depend on reliable interpretations of the instrument by the raters, and it might be even more important to ensure the reliability of such interpretations when using an instrument in a non-native language. ...
Objectives
The assessment of situation awareness (SA), team performance and task performance in a simulation training session requires reliable and feasible measurement techniques. The objectives of this study were to test the Airways–Breathing–Circulation–Disability–Exposure (ABCDE) checklist and the Team Emergency Assessment Measure (TEAM) for inter-rater reliability, as well as the application of Situation Awareness Global Assessment Technique (SAGAT) for feasibility and internal consistency.
Design
Methodological approach.
Setting
Data collection during team training using full-scale simulation at a university clinical training centre. The video-recorded scenarios were rated independently by four raters.
Participants
55 medical students aged 22–40 years in their fourth year of medical studies, during the clerkship in anaesthesiology and critical care medicine, formed 23 different teams. All students answered the SAGAT questionnaires, and of these students, 24 answered the follow-up postsimulation questionnaire (PSQ). TEAM and ABCDE were scored by four professionals.
Measures
The ABCDE and TEAM were tested for inter-rater reliability. The feasibility of SAGAT was tested using PSQ. SAGAT was tested for internal consistency both at an individual level (SAGAT) and a team level (Team Situation Awareness Global Assessment Technique (TSAGAT)).
Results
The intraclass correlation was 0.54/0.83 (single/average measurements) for TEAM and 0.55/0.83 for ABCDE. According to the PSQ, the items in SAGAT were rated as relevant to the scenario by 96% of the participants. Cronbach’s alpha for SAGAT/TSAGAT for the two scenarios was 0.80/0.83 vs 0.62/0.76, and normed χ² was 1.72 vs 1.62.
Conclusion
Task performance, team performance and SA could be purposefully measured, and the reliability of the measurements was good.
... First, studies utilizing simulated clinical scenarios as an assessment method were more likely to use validated and reliable assessment tools. For example, the TEAM tool has a substantial body of normative data confirming its validity (Cooper and Cant 2014). However, it only assesses a small component of the non-technical skill spectrum, and thus would only be called upon as an assessment measure of teamwork behaviors. ...
Consensus on how to assess non-technical skills is lacking. This systematic review aimed to evaluate the evidence regarding non-technical skills assessments in undergraduate medical education, to describe the tools used, learning outcomes and the validity, reliability and psychometrics of the instruments. A standardized search of online databases was conducted and consensus reached on included studies. Data extraction, quality assessment, and content analysis were conducted per Best Evidence in Medical Education guidelines. Nine papers met the inclusion criteria. Assessment methods broadly fell into three categories: simulated clinical scenarios, objective structured clinical examinations, and questionnaires or written assessments. Tools to assess non-technical skills were often developed locally, without reference to conceptual frameworks. Consequently, the tools were rarely validated, limiting dissemination and replication. There were clear themes in content and broad categories in methods of assessments employed. The quality of this evidence was poor due to lack of theoretical underpinning, with most assessments not part of normal process, but rather produced as a specific outcome measure for a teaching-based study. While the current literature forms a good starting position for educators developing materials, there is a need for future work to address these weaknesses as such tools are required across health education.
... This study utilised the 12-item Team Emergency Assessment Measure (TEAM TM ) to rate the non-technical skills of 'leadership' (2 items), 'teamwork' (7 items), 'task management' (2 items) (including situation awareness), and a 'global' score, each by a five-point Likert scale rating (0=never/hardly ever; 1=seldom; 2=about as often as not; 3=very often; 4=always/nearly always). [11,14] The TEAM TM instrument has been recognised as a valid, feasible and reliable tool in simulation studies [11][12][13][14] and hospital emergencies. [2,[14][15][16] During the observation additional data including call duration and location, reasons for the call, and other observations were recorded. ...
Background
Medical emergency teams are essential in responding to acute deterioration of patients in hospitals, requiring both clinical and non-technical skills. This study aims to assess the non-technical skills of medical emergency teams during hospital ward emergencies and explore team members perceptions and experiences of the use non-technical skills during medical emergencies.
Methods
A multi-methods study was conducted in two phases. During phase one observation and assessment of non-technical skills used in medical emergencies using the Team Emergency Assessment Measure (TEAM™) was conducted; and in the phase two in-depth interviews were undertaken with medical emergency team members.
Results
Based on 20 observations, mean TEAM™ ratings for non-technical skill domains were: ‘leadership’ 5.0 out of 8 (±2.0); ‘teamwork’ 21.6 out of 28 (±3.6); and ‘task management’ 6.5 out of 8 (±1.4). The mean ‘global’ score was 7.5 out of 10 (±1.5). The qualitative findings identified three areas, ‘individual’, ‘team’ and ‘other’ contributing factors, which impacted upon the non-technical skills of medical emergency teams.
Conclusion
Non-technical skills of hospital medical emergency teams differ, and the impact of the skill mix on resuscitation outcomes was recognised by team members. These findings emphasize the importance non-technical skills in resuscitation training and well-developed processes for medical emergency teams.
... This is further supported by Bosch & Mansell, who suggest that confidence is a prerequisite in building team trust, consequently enhancing team performance [33]. Objective team performance was assessed using the TEAM tool, a previously validated and reliable tool used in emergency scenarios [34,35]. Cant et al. used the tool to assess the NTS of hospital medical emergency teams and concluded that 'it is a valid, reliable and easy to use tool, for both training and clinical settings, with benefits for team performance when used as an assessment and/or debriefing tool'. ...
Objectives
The aim of this study is to evaluate a simulation-based team performance course for medical students and compare its low- and high-fidelity components.
Study design
This is a prospective crossover observational study. Groups participated in one low- and one high-fidelity session twice. Low-fidelity scenarios included management of an emergency case on a simulated-patient, whereas high-fidelity scenarios constituted of multiple-trauma cases where simulated-patients wore a hyper-realistic suit. Team performance was assessed objectively, using the TEAM™ tool, and subjectively using questionnaires. Questionnaires were also used to assess presence levels, stress levels and evaluate the course.
Results
Participants’ team performance was higher in the low-fidelity intervention as assessed by the TEAM™ tool. An overall mean increase in self-assessed confidence towards non-technical skills attitudes was noted after the course, however there was no difference in self-assessed performance between the two interventions. Both reported mean stress and presence levels were higher for the high-fidelity module. Evaluation scores for all individual items of the questionnaire were ≥4.60 in both NTS modules. Students have assessed the high-fidelity module higher (4.88 out of 5, SD = 0.29) compared to low-fidelity module (4.74 out of 5, SD = 0.67).
Conclusions
Both the low- and high-fidelity interventions demonstrated an improvement in team performance of the attending medical students. The high-fidelity intervention was more realistic, yet more stressful. Furthermore, it proved to be superior in harvesting leadership, teamwork and task management skills. Both modules were evaluated highly by the students, however, future research should address retention of the taught skills and adaptability of such interventions.
... The Team Emergency Assessment Measure was designed to assess the performance of healthcare teams. [29][30][31] This instrument consisted of 12 items asking the participants how they perceived the team performed across the duration of the task (eg, leadership, communication and morale). Eleven items were rated on 0-4 scale, with verbal anchors ('never/hardly ever', 'seldom', 'about as often as not', 'often' and 'nearly always/ always'). ...
Introduction
The National Organ Retrieval Service (NORS) 2015 review recommended a Joint Scrub Practitioner for abdominal and cardiac teams during combined organ retrieval. To evaluate the feasibility of this role, and to understand the functional implications, this study explores the use of simulation and provides a novel and comprehensive approach to assess individual and team performance in simulated multiorgan retrievals.
Methods
Two high-fidelity simulations were conducted in an operating theatre with porcine organs, en bloc, placed in a mannequin. For donation after brainstem death (DBD) simulation, an anaesthetic machine provided simulated physiological output. Retrievals following donation after circulatory death (DCD) began with rapid arrival in theatre of the mannequin. Cardiothoracic (lead surgeon) and abdominal (lead and assistant surgeons; joint scrub practitioner, n=9) teams combined for the retrievals. Data collected before, during and after simulations used self-report and expert observers to assess: attitudinal expectations, mental readiness, mental effort, non-technical skills, teamwork, task workload and social validation perceptions.
Results
Attitudinal changes regarding feasibility of a joint scrub practitioner for DBD and DCD are displayed in the main body. There were no significant differences in mental readiness prior to simulations nor in mental effort indicated afterwards; however, variance was noted between simulations for individual team members. Non-technical skills were slightly lower in DCD than in DBD. Global ratings of teamwork were significantly (p<0.05) lower in DCD than in DBD. Measures of attitude indicated less support for the proposed joint scrub practitioner role for DCD than for DBD.
Discussion
The paper posits that the joint scrub practitioner role in DCD multiorgan retrieval may bring serious and unanticipated challenges. Further work to determine the feasibility of the NORS recommendation is required. Measures of team performance and individual psychological response can inform organ retrieval feasibility considerations nationally and internationally.
Objectives:
The introduction of non-technical skills during nursing education is crucial to prepare nurses for the clinical context and increase patient safety. We found no instrument developed for this purpose. to construct, develop and validate a non-technical skills assessment scale in nursing.
Method:
methodological research. Based on the literature review and experience of researchers on non-technical skills in healthcare and the knowledge of the principles of crisis resource management, a list of 63 items with a five-point Likert scale was constructed. The scale was applied to 177 nursing undergraduate students. Descriptive statistics, correlations, internal consistency analysis and exploratory factor analysis were performed to evaluate the psychometric properties of the scale.
Results:
scale items presented similar values for mean and median. The maximum and the minimum values presented a good distribution amongst all response options. Most items presented a significant and positive relationship. Cronbach alpha presented a good value (0.94), and most correlations were significant and positive. Exploratory factor analysis using the Kaiser-Meyer-Olkin test showed a value of 0.849, and the Bartlett's test showed adequate sphericity values (χ2=6483.998; p=0.000). One-factor model explained 26% of the total variance.
Conclusion:
non-technical skills training and its measurement could be included in undergraduate or postgraduate courses in healthcare professions, or even be used to ascertain needs and improvements in healthcare contexts.
Simulation-based methodologies are increasingly used to assess teamwork and communication skills and provide team training. Formative feedback regarding team performance is an essential component. While effective use of simulation for assessment or training requires accurate rating of team performance, examples of rater-training programs in health care are scarce. We describe our rater training program and report interrater reliability during phases of training and independent rating.
We selected an assessment tool shown to yield valid and reliable results and developed a rater training protocol with an accompanying rater training handbook. The rater training program was modeled after previously described high-stakes assessments in the setting of 3 facilitated training sessions. Adjacent agreement was used to measure interrater reliability between raters.
Nine raters with a background in health care and/or patient safety evaluated team performance of 42 in-situ simulations using post-hoc video review. Adjacent agreement increased from the second training session (83.6%) to the third training session (85.6%) when evaluating the same video segments. Adjacent agreement for the rating of overall team performance was 78.3%, which was added for the third training session. Adjacent agreement was 97% 4 weeks posttraining and 90.6% at the end of independent rating of all simulation videos.
Rater training is an important element in team performance assessment, and providing examples of rater training programs is essential. Articulating key rating anchors promotes adequate interrater reliability. In addition, using adjacent agreement as a measure allows differentiation between high- and low-performing teams on video review.
© 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education.
Summary statement:
Simulated environments are frequently used for learner assessment, and a wide array of assessment instruments have been created to assist with this process. It is important, therefore, that clear, compelling evidence for the validity of these assessments be established. Contemporary theory recognizes instrument validity as a unified construct that links a construct to be assessed with a population, an environment of assessment, and a decision to be made using the scores. In this article, we present a primer on 2 current frameworks (Messick and Kane), define the elements of each, present a rubric that can be used by potential authors to structure their work, and offer examples of published studies showing how each framework has been successfully used to make a validity argument. We offer this with the goal of improving the quality of validity-related publications, thereby advancing the quality of assessment in healthcare simulation.
Background: The simulation help improve patient safety, individual and team performance, and the quality of care. We distinguish between technical skills and non-technical skills that include communication proficiency, decision-making, and teamwork. Methods: The aim of this study was to evaluate and assess the effect of simulation in improving non-technical skills in emergency medical crisis management of medical students. The study was conducted at the simulation center. The participants were the medical students of the fifth, sixth and seventh-year. The simulation session was organized with groups of three participants. Each team participated in three different scenarios. The debrief was structured into four phases (reactions, facts, analysis and summary). During the debriefing and in the most relaxed atmosphere possible, the soft skills were assessed by an instructor using the Team Emergency Assessment Measure (TEAM). Results: Eighteen students participated in this study. For each item of non-technikal skills in TEAM there was significant improvement in the mean scores of subjects between their first and second session and their first and third session (p <0.05). Conclusions: Our study shows that simulation sessions improve non-technical skills in all medical students. Consequently, given their crucial importance in ensuring better patient care, they must be integrated early in the training curriculum of medical and nursing students.
Background:
Simulation-based training is an effective method of enhancing the knowledge, skill, and technical abilities of individuals and teams encountering obstetric and gynaecologic emergencies. Simulation may also enhance the non-technical performance of teams resulting in improved patient outcomes. Although simulation-based training is widely recognised as an effective educational approach, issues around feasibility - the lack of simulation experts and malleable outcome measures of team performance - remain critical barriers to their implementation.
Aim:
To evaluate the psychometric properties of the Team Emergency Assessment Measure (TEAM) when used by medical professionals in simulated obstetric and gynaecological emergencies.
Methods:
There were 151 participants (63% female; 60% consultants; 69% no previous simulation-based training) who observed three live high-fidelity obstetric and gynaecological resuscitation simulations and completed the TEAM.
Results:
Confirmatory factor analysis evaluated the construct validity of the TEAM, yielding a second-order structure identified by 'leadership', 'teamwork', and 'team management'. Convergent validity was supported by the average item-to-scale total correlation which was 0.75, P < 0.001 and the average analysis of variance extracted (AVE) 0.88. The individual factors also yielded high factor-to-scale total correlations (mean [M] = 0.87), and AVE (M = 0.89). The internal reliability was high for the whole scale (average alpha = 0.92) and across the sub-factors (average alpha = 0.80). The inter-rater reliability was excellent (inter-class correlation coefficient 1 = 0.98). Participants with differing levels of simulation training experience did not significantly differ.
Conclusion:
The TEAM is a viable instrument for the assessment of non-technical performance during simulated obstetric and gynaecologic emergencies, thus enhancing the feasibility of simulation-based training.
Perhaps one of the most important conversations occurring in the development of a research study concerns the methodology that will be used. In this vignette we will concentrate on quantitative methodology as an example; however, these principles extend to qualitative approaches as well. Without a valid analysis plan, the likelihood a study will be able to meaningfully comment on the research question it was originally intended to address is quite low. This, in turn, will substantially affect its likelihood of acceptance by a peer-reviewed journal.
Scenario building, also referred to as scenario design, is a fundamental component of simulation-based medical education. When done effectively, the scenario can reliably meet the needs of the curriculum and the learners. This chapter explores the rationale for utilizing a formal scenario-building process, outlines many of the theoretical underpinnings important to the endeavor, and describes a practical, six-step approach to designing scenarios. Some of the topics explored in depth include curriculum design, teamwork/interprofessional education, fidelity/realism, the use of distraction, and confederates. The six-step approach involves identifying the target audience, learning objectives and simulator modalities, building a case summary, procuring staging needs (such as moulage, confederates, and adjuncts), writing the script, preprogramming the scenario where appropriate, and finally practicing or pilot-testing the scenario. A sample template is provided and common pitfalls are discussed.
As many errors made in healthcare are due to human factors, training must target these issues in order to improve the quality of care provided. Interprofessional team training has been in existence in several areas of healthcare for many years, including pediatrics. Simulation-based training is the ideal teaching modality to employ for team training, especially for acute care teams (resuscitation teams, operating room teams, etc.). There is much to carefully consider when establishing a simulation-based team training program, including decisions to be made around the purpose of such training, who the participants are, and how to set up the simulated clinical environment. Published research has consistently demonstrated that teams that participate in simulation-based team training improve their knowledge, skills, and behaviors related to teamwork concepts. There are several high-quality assessment tools available that can be used to objectively evaluate teamwork behaviors in the simulated clinical environment. Future research should aim to provide more objective evidence demonstrating that team training in the simulated environment has a positive impact on patient outcomes in the real clinical environment.
Objectives:
High-quality clinical research of resuscitations in a pediatric emergency department is challenging because of the limitations of traditional methods of data collection (chart review, self-report) and the low frequency of cases in a single center. To facilitate valid and reliable research for resuscitations in the pediatric emergency department, investigators from 3 pediatric centers, each with experience completing successful single-center, video-based studies, formed the Videography In Pediatric Emergency Research (VIPER) collaborative.
Methods:
Our initial effort was the development of a multicenter, video-based registry and simulation-based testing of the feasibility and reliability of the VIPER registry. Feasibility of data collection was assessed by the frequency of an indeterminate response for all data elements in the registry. Reliability was assessed by the calculation of Cohen κ for dichotomous data elements and intraclass correlation coefficients for continuous data elements.
Results:
Video-based data collection was completed for 8 simulated pediatric resuscitations, with at least 2 reviewers per case. Data were labeled as indeterminate by at least 1 reviewer for 18 (3%) of 524 relevant data fields. The Cohen κ for all dichotomous data fields together was 0.81 (95% confidence interval, 0.61-1.0). For all continuous (time-based) variables combined, the intraclass correlation coefficient was 0.88 (95% confidence interval, 0.70-0.96).
Conclusions:
Initial simulation-based testing suggests video-based data collection using the VIPER registry is feasible and reliable. Our next step is to assess feasibility and reliability for actual pediatric resuscitations and to complete several prospective, hypothesis-based studies of specific aspects of resuscitative care, including of cardiopulmonary resuscitation, tracheal intubation, and teamwork and communication.
Objectives:
This prospective descriptive study aimed to test the validity and feasibility of the Team Emergency Assessment Measure (TEAM™) for assessing real-world medical emergency teams' non-technical skills. Second, the present study aimed to explore the instrument's contribution to practice regarding teamwork and learning outcomes.
Methods:
Registered nurses (RNs) and medical staff (n = 104) in two hospital EDs in rural Victoria, Australia, participated. Over a 10 month period, the (TEAM™) instrument was completed by multiple clinicians at medical emergency episodes.
Results:
In 80 real-world medical emergency team resuscitation episodes (283 clinician assessments), non-technical skills ratings averaged 89% per episode (39 of a possible 44 points). Twenty-one episodes were rated in the lowest quartile (i.e. ≤37 points out of 44). Ratings differed by discipline, with significantly higher scores given by medical raters (mean: 41.1 ± 4.4) than RNs (38.7 ± 5.4) (P = 0.001). This difference occurred in the Leadership domain. The tool was reliable with Cronbach's alpha 0.78, high uni-dimensional validity and mean inter-item correlation of 0.45. Concurrent validity was confirmed by strong correlation between TEAM™ score and the awarded Global Rating (P < 0.001), with 38.4% of shared variance. RNs praised the instrument as it initiated staff reflection and debriefing discussions around performance improvement.
Conclusion:
Non-technical skills of medical emergency teams are known to often be suboptimal; however, average ratings of 89% were achieved in this real-world study. TEAM™ is a valid, reliable and easy to use tool, for both training and clinical settings, with benefits for team performance when used as an assessment and/or debriefing tool.
The observational research employs certain strategies for the discovery and interpretation of facts through a careful and systematic study designed to answer questions in order to make decisions and obtain new knowledge to solve a problem. Two paradigms guide this research: the positivist (quantitative) and the naturalist or constructivist (qualitative), the first one uses numbers and statistics to explain a phenomenon while the latter interprets the social reality. The educational research in health sciences has increased in recent years; therefore, the interprofessional simulation has also increased. The objective of conducting interprofesional educational research is to identify and evaluate the different factors involved in teamwork and the non-technical skills in the interprofesional clinical education based on simulation. To achieve the expected results, it is important to have high quality tools that are valid and reliable according to its objectives.
Background
Over the past three decades multiple tools have been developed for the assessment of non-technical skills (NTS) in healthcare. This study was designed primarily to analyse how they have been designed and tested but also to consider guidance on how to select them.
Objectives
To analyse the context of use, method of development, evidence of validity (including reliability) and usability of tools for the observer-based assessment of NTS in healthcare.
Design
Systematic review.
Data sources
Search of electronic resources, including PubMed, Embase, CINAHL, ERIC, PsycNet, Scopus, Google Scholar and Web of Science. Additional records identified through searching grey literature (OpenGrey, ProQuest, AHRQ, King’s Fund, Health Foundation).
Study selection
Studies of observer-based tools for NTS assessment in healthcare professionals (or undergraduates) were included if they: were available in English; published between January 1990 and March 2018; assessed two or more NTS; were designed for simulated or real clinical settings and had provided evidence of validity plus or minus usability. 11,101 articles were identified. After limits were applied, 576 were retrieved for evaluation and 118 articles included in this review.
Results
One hundred and eighteen studies describing 76 tools for assessment of NTS in healthcare met the eligibility criteria. There was substantial variation in the method of design of the tools and the extent of validity, and usability testing. There was considerable overlap in the skills assessed, and the contexts of use of the tools.
Conclusion
This study suggests a need for rationalisation and standardisation of the way we assess NTS in healthcare and greater consistency in how tools are developed and deployed.
Aims and objectives:
To examine whether interprofessional simulation training on management of postpartum haemorrhage enhances self-efficacy and collective efficacy and reduces the blood transfusion rate after birth.
Background:
Postpartum haemorrhage is a leading cause of maternal morbidity and mortality worldwide, although it is preventable in most cases. Interprofessional simulation training might help improve the competence of health professionals dealing with postpartum haemorrhage, and more information is needed to determine its potential.
Design:
Multimethod, quasi-experimental, pre-post intervention design.
Methods:
Interprofessional simulation training on postpartum haemorrhage was implemented for midwives, obstetricians and auxiliary nurses in a university hospital. Training included realistic scenarios and debriefing, and a measurement scale for perceived postpartum haemorrhage-specific self-efficacy, and collective efficacy was developed and implemented. Red blood cell transfusion was used as the dependent variable for improved patient outcome pre-post intervention.
Results:
Self-efficacy and collective efficacy levels were significantly increased after training. The overall red blood cell transfusion rate did not change, but there was a significant reduction in the use of ≥5 units of blood products related to severe bleeding after birth.
Conclusion:
The study contributes to new knowledge on how simulation training through mastery and vicarious experiences, verbal persuasion and psychophysiological state might enhance postpartum haemorrhage-specific self-efficacy and collective efficacy levels and thereby predict team performance. The significant reduction in severe postpartum haemorrhage after training, indicated by reduction in ≥5 units of blood transfusions, corresponds well with the improvement in collective efficacy, and might reflect the emphasis on collective efforts to counteract severe cases of postpartum haemorrhage.
Relevance to clinical practice:
Interprofessional simulation training in teams may contribute to enhanced prevention and management of postpartum haemorrhage, shown by a significant increase in perceived efficacy levels combined with an indicated reduction of severe postpartum haemorrhage after training.
Objective:
Non-technical skills (NTS) such as team communication are well recognized determinants of trauma team performance and good patient care. Measuring these competencies during trauma resuscitations is essential, yet few valid and reliable tools are available. We aimed to demonstrate that the TTCA-24 is a valid and reliable instrument that measures communication effectiveness during activations.
Methods:
Two tools with adequate psychometric strength (T-NOTCHES, TEAM) were identified during a systematic review of medical literature and compared with TTCA-24. Three coders used each tool to evaluate 35 stable and 35 unstable patient activations (defined according to ATLS criteria). Inter-rater reliability was calculated between coders using the Intraclass correlation coefficient (ICC). Spearman rank correlation coefficient was used to establish concurrent validity between TTCA-24 and the other two validated tools.
Results:
Coders achieved an ICC of .87 for stable patient activations and .78 for unstable activations scoring excellent on the inter-rater agreement guidelines. The median score for each assessment showed good team communication for all 70 videos (TEAM = 39.8 out of 54, T-NOTECHS = 17.4 out of 25, and TTCA-24 = 87.4 out of 96). A significant correlation between TTTC-24 and T-NOTECHS was revealed (P=0.029), but no significant correlation between TTCA-24 and TEAM (P=0.77). Team communication was rated slightly better across all assessments for stable versus unstable patient activations, but not statistically significant.
Conclusion:
TTCA-24 correlated with T-NOTECHS an instrument measuring NTS for trauma teams, but not TEAM a tool that assess communication in generic emergency settings. TTCA-24 is a reliable and valid assessment that can be a useful adjunct when evaluating interpersonal and team communication during trauma activations.
Level of evidence:
Level II STUDY TYPE: Diagnostic Tests or Criteria.
Background:
Assessing advanced life support (ALS) competence requires validated instruments. Existing instruments include aspects of technical skills (TS), non-technical skills (NTS) or both, but one instrument for detailed assessment that suits all resuscitation situations is lacking. This study aimed to develop an instrument for the evaluation of the overall ALS performance of the whole team.
Methods:
This instrument development study had four phases. First, we reviewed literature and resuscitation guidelines to explore items to include in the instrument. Thereafter, we interviewed resuscitation team professionals (n = 66), using the critical incident technique, to determine possible additional aspects associated with the performance of ALS. Second, we developed an instrument based on the findings. Third, we used an expert panel (n = 20) to assess the validity of the developed instrument. Finally, we revised the instrument based on the experts' comments and tested it with six experts who evaluated 22 video recorded resuscitations.
Results:
The final version of the developed instrument had 69 items divided into adherence to guidelines (28 items), clinical decision-making (5 items), workload management (12 items), team behaviour (8 items), information management (6 items), patient integrity and consideration of laymen (4 items) and work routines (6 items). The Cronbach's α values were good, and strong correlations between the overall performance and the instrument were observed.
Conclusion:
The instrument may be useful for detailed assessment of the team's overall performance, but the numerous items make the use demanding. The instrument is still under development, and more research is needed to determine its psychometric properties.
Statement:
The research literature regarding interprofessional simulation-based medical education has grown substantially and continues to explore new aspects of this educational modality. The aim of this study was to explore the validation evidence of tools used to assess teamwork and nontechnical skills in interprofessional simulation-based clinical education. This systematic review included original studies that assessed participants' teamwork and nontechnical skills, using a measurement tool, in an interprofessional simulated setting. We assessed the validity of each assessment tool using Kane's framework. Medical Education Research Study Quality Instrument scores for the studies ranged from 8.5 to 17.0. Across the 22 different studies, there were 20 different assessment strategies, in which Team Emergency Assessment Measure, Anesthetist's Nontechnical Skills, and Nontechnical Skills for Surgeons were used more than once. Most assessment tools have been validated for scoring and generalization inference. Fewer tools have been validated for extrapolation inference, such as expert-novice analysis or factor analysis.
Situation awareness (SA) is a vital cognitive skill for high-stakes, high-hazard occupations, including military, aviation, and health care. The ability to maintain SA can deteriorate in stressful situations, exposing patients to dangerous errors. The literature regarding how to best teach SA techniques is sparse. This article explores specific techniques to promote and maintain SA in dynamic clinical environments using principles derived from cognitive psychology, neuroscience, and human behavioral and organizational research. The authors propose strategies to help individuals and teams to develop ingrained, subconscious behaviors that can help to maintain effective SA in high-stress environments.
Situation awareness (SA) is critical in high-stakes circumstances, such as the resuscitation of critically ill or injured patients. Exploratory research in psychology, neuroscience, human factors engineering, and to a lesser extent health care has led to a deeper understanding of what SA is and how it can be measured. Unfortunately, little is known about how we can adapt training in order to more consistently create behaviors that heighten SA during dynamic, high-stakes clinical events. In this article, the prevailing theory of SA is reviewed, and the evidence for evaluating it in medicine is presented. In addition, the authors draw from the fields of neuroscience and cognitive psychology to suggest some strategies that can develop effective behaviors that promote SA in resuscitation.
Statement:
The past several decades have seen tremendous growth in our understanding of best practices in simulation-based healthcare education. At present, however, there is limited infrastructure available to assist programs in translation of these best practices into more standardized educational approaches, higher quality of care, and ultimately improved outcomes. In 2014, the International Simulation Data Registry (ISDR) was launched to address this important issue. The existence of such a registry has important implications not just for educational practice but also for research. The ISDR currently archives data related to pulseless arrest, malignant hyperthermia, and difficult airway simulations. Case metrics are designed to mirror the American Heart Association's Get With the Guidelines Registry, allowing for direct comparisons with clinical scenarios. This article describes the rationale for the ISDR, and outlines its development. Current data are presented to highlight the educational and research value of this approach. Projected future developments are also discussed.
Cambridge Core - Communications - The Cambridge Handbook of Group Interaction Analysis - edited by Elisabeth Brauner
Introduction:
Non-technical skills (NTS) teamwork training can enhance clinicians' understanding of roles and improve communication. We evaluated a quality improvement project rating teams' NTS performance to determine the value of formal rating and debriefing processes.
Methods:
In two Australian emergency departments the NTS of resuscitation teams were rated by senior nurses and medical staff. Key measures were leadership, teamwork, and task management using a valid instrument: Team Emergency Assessment Measure (TEAM™). Emergency nurses were asked to attend a focus group from which key themes around the quality improvement process were identified.
Results:
Main themes were: 'Team composition' (allocation of resuscitation team roles), 'Resuscitation leadership' (including both nursing and medical leadership roles) and 'TEAM™ ratings promote reflective practice' (providing staff a platform to discuss team effectiveness). Objective ratings were seen as enabling staff to provide feedback to other team members. Reflection on practice and debriefing were thought to improve communication, help define roles and responsibilities, and clarify leadership roles.
Conclusion:
Use of a non-technical skills rating scheme such as TEAM™ after team-based clinical resuscitation events was seen by emergency department nurses as feasible and a useful process for examining and improving multi-disciplinary practice, while improving team performance.
Background:
Communication failures in healthcare teams are associated with medical errors and negative health outcomes. These findings have increased emphasis on training future health professionals to work effectively within teams. The Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) communication training model, widely employed to train healthcare teams, has been less commonly used to train student interprofessional teams. The present study reports the effectiveness of a simulation-based interprofessional TeamSTEPPS training in impacting student attitudes, knowledge and skills around interprofessional communication.
Methods:
Three hundred and six fourth-year medical, third-year nursing, second-year pharmacy and second-year physician assistant students took part in a 4 h training that included a 1 h TeamSTEPPS didactic session and three 1 h team simulation and feedback sessions. Students worked in groups balanced by a professional programme in a self-selected focal area (adult acute, paediatric, obstetrics). Preassessments and postassessments were used for examining attitudes, beliefs and reported opportunities to observe or participate in team communication behaviours.
Results:
One hundred and forty-nine students (48.7%) completed the preassessments and postassessments. Significant differences were found for attitudes toward team communication (p<0.001), motivation (p<0.001), utility of training (p<0.001) and self-efficacy (p=0.005). Significant attitudinal shifts for TeamSTEPPS skills included, team structure (p=0.002), situation monitoring (p<0.001), mutual support (p=0.003) and communication (p=0.002). Significant shifts were reported for knowledge of TeamSTEPPS (p<0.001), advocating for patients (p<0.001) and communicating in interprofessional teams (p<0.001).
Conclusions:
Effective team communication is important in patient safety. We demonstrate positive attitudinal and knowledge effects in a large-scale interprofessional TeamSTEPPS-based training involving four student professions.
To describe a funded proposal for the development of an on-line evidence based educational program for the management of deteriorating patients.
There are international concerns regarding the management of deteriorating patients with issues around the ‘failure to rescue’. The primary response to these issues has been the development of medical emergency teams with little focus on the education of primary first responders.
A mixed methods triangulated convergent design.
In this four phase proposal we plan to 1. examine nursing student team ability to manage deteriorating patients and based upon these findings 2. develop web based educational material, including interactive scenarios. This educational material will be tested and refined in the third Phase 3, prior to evaluation and dissemination in the final phase.
This project aims to enhance knowledge development for the management of deteriorating patients through rigorous assessment of team performance and to produce a contemporary evidence-based online training program.
Background and aim
Following high profile errors resulting in patient harm and attracting negative publicity, the healthcare sector has begun to focus on training non-technical teamworking skills as one way of reducing the rate of adverse events. Within the area of resuscitation, two tools have been developed recently aiming to assess these skills – TEAM and OSCAR. The aims of the study reported here were:1.To determine the inter-rater reliability of the tools in assessing performance within the context of resuscitation.2.To correlate scores of the same resuscitation teams episodes using both tools, thereby determining their concurrent validity within the context of resuscitation.3.To carry out a critique of both tools and establish how best each one may be utilised.
Methods
The study consisted of two phases – reliability assessment; and content comparison, and correlation. Assessments were made by two resuscitation experts, who watched 24 pre-recorded resuscitation simulations, and independently rated team behaviours using both tools. The tools were critically appraised, and correlation between overall score surrogates was assessed.
Results
Both OSCAR and TEAM achieved high levels of inter-rater reliability (in the form of adequate intra-class coefficients) and minor significant differences between Wilcoxon tests. Comparison of the scores from both tools demonstrated a high degree of correlation (and hence concurrent validity). Finally, critique of each tool highlighted differences in length and complexity.
Conclusion
Both OSCAR and TEAM can be used to assess resuscitation teams in a simulated environment, with the tools correlating well with one another. We envisage a role for both tools – with TEAM giving a quick, global assessment of the team, but OSCAR enabling more detailed breakdown of the assessment, facilitating feedback, and identifying areas of weakness for future training.
To evaluate the effectiveness of training and institutionalizing teamwork behaviors, drawn from aviation crew resource management (CRM) programs, on emergency department (ED) staff organized into caregiver teams.
Nine teaching and community hospital EDs.
A prospective multicenter evaluation using a quasi-experimental, untreated control group design with one pretest and two posttests of the Emergency Team Coordination Course (ETCC). The experimental group, comprised of 684 physicians, nurses, and technicians, received the ETCC and implemented formal teamwork structures and processes. Assessments occurred prior to training, and at intervals of four and eight months after training. Three outcome constructs were evaluated: team behavior, ED performance, and attitudes and opinions. Trained observers rated ED staff team behaviors and made observations of clinical errors, a measure of ED performance. Staff and patients in the EDs completed surveys measuring attitudes and opinions.
Hospital EDs were the units of analysis for the seven outcome measures. Prior to aggregating data at the hospital level, scale properties of surveys and event-related observations were evaluated at the respondent or case level.
A statistically significant improvement in quality of team behaviors was shown between the experimental and control groups following training (p = .012). Subjective workload was not affected by the intervention (p = .668). The clinical error rate significantly decreased from 30.9 percent to 4.4 percent in the experimental group (p = .039). In the experimental group, the ED staffs' attitudes toward teamwork increased (p = .047) and staff assessments of institutional support showed a significant increase (p = .040).
Our findings point to the effectiveness of formal teamwork training for improving team behaviors, reducing errors, and improving staff attitudes among the ETCC-trained hospitals.
Objectives
This paper reports the quantitative findings of the first phase of a larger program of ongoing research: Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends (FIRST2ACTTM). It specifically aims to identify the characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients.DesignMixed-method multi-centre study.SettingHigh fidelity simulated acute clinical environment in three Australian universities.ParticipantsA convenience sample of 97 final year nursing students enrolled in an undergraduate Bachelor of Nursing or combined Bachelor of Nursing degree were included in the study.Method
In groups of three, participants proceeded through three phases: (i) pre-briefing and completion of a multi-choice question test, (ii) three video-recorded simulated clinical scenarios where actors substituted real patients with deteriorating conditions, and (iii) post-scenario debriefing. Clinical performance, teamwork and situation awareness were evaluated, using a validated standard checklist (OSCE), Team Emergency Assessment Measure (TEAM) score sheet and Situation Awareness Global Assessment Technique (SAGAT). A Modified Angoff technique was used to establish cut points for clinical performance.ResultsStudent teams engaged in 97 simulation experiences across the three scenarios and achieved a level of clinical performance consistent with the experts' identified pass level point in only 9 (1%) of the simulation experiences. Knowledge was significantly associated with overall teamwork (p = .034), overall situation awareness (p = .05) and clinical performance in two of the three scenarios (p = .032 cardiac and p = .006 shock). Situation awareness scores of scenario team leaders were low overall, with an average total score of 41%.Conclusions
Final year undergraduate nursing students may have difficulty recognising and responding appropriately to patient deterioration. Improving pre-requisite knowledge, rehearsal of first response and team management strategies need to be a key component of undergraduate nursing students' education and ought to specifically address clinical performance, teamwork and situation awareness.
The study purpose was to evaluate the effectiveness of a structured education curriculum with simulation training in educating undergraduate Baccalaureate of Science in Nursing (BSN) students to recognize and respond to patients experiencing acute deterioration as first responders. Researchers have demonstrated a lack of adequate clinical reasoning skills in new graduate nurses is a factor in critical patient incidents. A mixed methods design using a quasi-experimental, repeated measures and a descriptive, qualitative approach was used. A convenience sample of 48 BSN students was recruited. Statistically significant increases were shown in knowledge, self-confidence, and perceptions of teamwork. Six categories emerged from the qualitative data analysis: sources of knowledge, knowledge as a person, knowledge as a group, reasoning under pressure, feelings, real person versus simulation, and values. Nursing educators need to use innovative teaching strategies to ameliorate or even eliminate the theory-practice gap in nursing.
The purpose of this article is to review critically the Angoff (1971) and modified Angoff methods for setting cut-scores. The criteria used in this review were originally proposed by Berk (1986). The assumptions of the Angoff method and other current issues surrounding this method are also discussed. Recommendations are made for using the Angoff method. In addition, several issues that are relevant to cut-score setting that are not addressed by Berk's criteria arose while reviewing the Angoff method. These issues are addressed separately.
An abstract is unavailable. This article is available as HTML full text and PDF.
Objective To assess the ability of rural Australian nurse teams to manage deteriorating patients.
Methods This quasi-experimental design used pre- and post-intervention assessments and observation to evaluate nurses' simulated clinical performance. Registered nurses (n=44) from two hospital wards completed a formative knowledge assessment and three team-based video recorded scenarios (Objective Structured Clinical Examinations (OSCE)). Trained patient actors simulated deteriorating patients. Skill performance and situation awareness were measured and team performance was rated using the Team Emergency Assessment Measure.
Results Knowledge in relation to patient deterioration management varied (mean 63%, range 27–100%) with a median score of 64%. Younger nurses with a greater number of working hours scored the highest (p=0.001). OSCE performance was generally low with a mean performance of 54%, but performance was maintained despite the increasing complexity of the scenarios. Situation awareness was generally low (median 50%, mean 47%, range 17–83%, SD 14.03) with significantly higher levels in younger participants (r=−0.346, p=0.021). Teamwork ratings averaged 57% with significant associations between the subscales (Leadership, Teamwork and Task Management) (p<0.006), the global rating scale (p<0.001) and two of the OSCE measures (p<0.049). Feedback from participants following the programme indicated significant improvements in knowledge, confidence and competence (p<0.001).
Conclusion Despite a satisfactory knowledge base, the application of knowledge was low with notable performance deficits in these demanding and stressful situations. The identification and management of patient deterioration needs to be taught in professional development programmes incorporating high fidelity simulation techniques. The Team Emergency assessment tool proved to be a valid measure of team performance in patient deterioration scenarios.
Nontechnical skills are "the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance." Our research team developed and evaluated the task of developing and validating a behavioral marker system for the observational assessment of emergency physicians' nontechnical skills.
The development of the tool was divided into 3 phases and used triangulation of data from a number of sources. During phase 1, a provisional assessment tool was developed according to published literature and curricula. Phase 2 used analysis of staff interviews and field observations to determine whether the skill list contained any significant omissions. These studies were also used to identify behavioral markers linked to nontechnical skills in the context of the emergency department (ED) and establish whether skills included in the tool were observable. Phase 3 involved evaluating the content validity index of exemplar behaviors, using a survey of experts.
A behavioral marker system was developed that comprised 12 emergency medicine-specific nontechnical skills, grouped into 4 categories. Content validity was assessed with a survey of 148 emergency medicine staff, and 75% of items achieved the recommended content validity index greater than 0.75. Data from the survey enabled further refinement of the behavioral markers to produce a final version of the tool.
Although further evaluative studies are needed, this behavioral marker system provides a structured approach to the assessment and training of nontechnical skills in the ED.
There is an increasing interest in human factors within the healthcare environment reflecting the understanding of their impact on safety. The aim of this paper is to explore how human factors might be taught on resuscitation courses, and improve course outcomes in terms of improved mortality and morbidity for patients. The delivery of human factors training is important and this review explores the work that has been delivered already and areas for future research and teaching.
Medline was searched using MESH terms Resuscitation as a Major concept and Patient or Leadership as core terms. The abstracts were read and 25 full length articles reviewed.
Critical incident reporting has shown four recurring problems: lack of organisation at an arrest, lack of equipment, non functioning equipment, and obstructions preventing good care. Of these, the first relates directly to the concept of human factors. Team dynamics for both team membership and leadership, management of stress, conflict and the role of debriefing are highlighted. Possible strategies for teaching them are discussed.
Four strategies for improving human factors training are discussed: team dynamics (including team membership and leadership behaviour), the influence of stress, debriefing, and conflict within teams. This review illustrates how human factor training might be integrated further into life support training without jeopardising the core content and lengthening the courses.
To develop a valid, reliable and feasible teamwork assessment measure for emergency resuscitation team performance.
Generic and profession specific team performance assessment measures are available (e.g. anaesthetics) but there are no specific measures for the assessment of emergency resuscitation team performance.
(1) An extensive review of the literature for teamwork instruments, and (2) development of a draft instrument with an expert clinical team. (3) Review by an international team of seven independent experts for face and content validity. (4) Instrument testing on 56 video-recorded hospital and simulated resuscitation events for construct, consistency, concurrent validity and reliability and (5) a final set of ratings for feasibility on fifteen simulated 'real time' events.
Following expert review, selected items were found to have a high total content validity index of 0.96. A single 'teamwork' construct was identified with an internal consistency of 0.89. Correlation between the total item score and global rating (rho 0.95; p<0.01) indicated concurrent validity. Inter-rater (k 0.55) and retest reliability (k 0.53) were 'fair', with positive feasibility ratings following 'real time' testing. The final 12 item (11 specific and 1 global rating) are rated using a five-point scale and cover three categories leadership, teamwork and task management.
In this primary study TEAM was found to be a valid and reliable instrument and should be a useful addition to clinicians' tool set for the measurement of teamwork during medical emergencies. Further evaluation of the instrument is warranted to fully determine its psychometric properties.
We describe an example of simulation-based interprofessional continuing education, the multidisciplinary obstetric simulated emergency scenarios (MOSES) course, which was designed to enhance nontechnical skills among obstetric teams and, hence, improve patient safety. Participants' perceptions of MOSES courses, their learning, and the transfer of learning to clinical practice were examined.
Participants included senior midwives, obstetricians, and obstetric anesthetists, including course faculty from 4 purposively selected delivery suites in England. Telephone or e-mail interviews with MOSES course participants and facilitators were conducted, and video-recorded debriefings that formed integral parts of this 1-day course were analyzed.
The team training was well received. Participants were able to check out assumptions and expectations of others and develop respect for different roles within the delivery suite (DS) team. Skillful facilitation of debriefing after each scenario was central to learning. Participants reported acquiring new knowledge or insights, particularly concerning the role of communication and leadership in crisis situations, and they rehearsed unfamiliar skills. Observing peers working in the simulations increased participants' learning by highlighting alternative strategies. The learning achieved by individuals and groups was noticeably dependent on their starting points. Some participants identified limited changes in their behavior in the workplace following the MOSES course. Mechanisms to manage the transfer of learning to the wider team were weakly developed, although 2 DS teams made changes to their regular update training.
Interprofessional, team-based simulations promote new learning.
Full-scale simulation training is an accepted learning method for gaining behavioural skills in team-centred domains such as aviation, the nuclear power industry and, recently, medicine. In this study we evaluated the effects of a simulator team training method based on targets and known principles in cognitive psychology.
This method was developed and adapted for a medical emergency team. In particular, we created a trauma team course for novices, and allowed 15 students to practise team skills in 5 full-scale scenarios. Students' team behaviour was video-recorded and students' attitude towards safe teamwork was assessed using a questionnaire before and after team practice.
Nine of 10 observed team skills improved significantly in response to practice, in parallel with a global rating of team skills. In contrast, no change in attitude toward safe teamwork was registered.
The use of team skills in 5 scenarios in a full-scale patient simulator environment implementing a training method based on targets and known principles in cognitive psychology improved individual team skills but had no immediate effect on attitude toward safe patient care.
Simulation training is an essential educational strategy for health care systems to improve patient safety. The strength of simulation training is its suitability for multidisciplinary team training. There is good evidence that simulation training improves provider and team self-efficacy and competence on manikins. There is also good evidence that procedural simulation improves actual operational performance in clinical settings. However, no evidence yet shows that crew resource management training through simulation, despite its promise, improves team operational performance at the bedside. Also, no evidence to date proves that simulation training actually improves patient outcome. Even so, confidence is growing in the validity of medical simulation as the training tool of the future. The use of medical simulation will continue to grow in the context of multidisciplinary team training for patient safety.
High-reliability organizations have stressed the importance of non-technical skills for safety and of regularly providing such training to their teams. Recently safety skills training has been applied in the practice of medicine. In this study, we developed and piloted a module using multidisciplinary crisis scenarios in a simulated operating theatre to train entire surgical teams.
Twenty teams participated (n = 80); each consisted of a trainee surgeon, anesthetist, operating department practitioner (ODP), and scrub nurse. Crisis scenarios such as difficult intubation, hemorrhage, or cardiac arrest were simulated. Technical and non-technical skills (leadership, communication, team skills, decision making, and vigilance), were assessed by clinical experts and by two psychologists using relevant technical and human factors rating scales. Participants received technical and non-technical feedback, and the whole team received feedback on teamwork.
Trainees assessed the training favorably. For technical skills there were no differences between surgical trainees' assessment scores and the assessment scores of the trainers. However, nurses overrated their technical skill. Regarding non-technical skills, leadership and decision making were scored lower than the other three non-technical skills (communication, team skills, and vigilance). Surgeons scored lower than nurses on communication and teamwork skills. Surgeons and anesthetists scored lower than nurses on leadership.
Multidisciplinary simulation-based team training is feasible and well received by surgical teams. Non-technical skills can be assessed alongside technical skills, and differences in performance indicate where there is a need for further training. Future work should focus on developing team performance measures for training and on the development and evaluation of systematic training for technical and non-technical skills to enhance team performance and safety in surgery.
The crew resource management training program was developed by the aviation industry in response to critical and fatal errors by the flight team. This article examines the evolution and application of crew resource management to the healthcare industry. The goal of this evolution was to increase patient safety through better communication and teamwork. To accomplish this goal, teamwork training programs, such as MedTeams, are being introduced to healthcare professionals. Clinical studies have yet to show conclusive results of these training programs. Further studies are ongoing and necessary.
To compare the management of and neonatal injury associated with shoulder dystocia before and after introduction of mandatory shoulder dystocia simulation training.
This was a retrospective, observational study comparing the management and neonatal outcome of births complicated by shoulder dystocia before (January 1996 to December 1999) and after (January 2001 to December 2004) the introduction of shoulder dystocia training at Southmead Hospital, Bristol, United Kingdom. The management of shoulder dystocia and associated neonatal injuries were compared pretraining and posttraining through a review of intrapartum and postpartum records of term, cephalic, singleton births in which difficulty with the shoulders was recorded during the two study periods.
There were 15,908 and 13,117 eligible births pretraining and posttraining, respectively. The shoulder dystocia rates were similar: pretraining 324 (2.04%) and posttraining 262 (2.00%) (P=.813). After training was introduced, clinical management improved: McRoberts' position, pretraining 95/324 (29.3%) to 229/262 (87.4%) posttraining (P<.001); suprapubic pressure 90/324 (27.8%) to 119/262 (45.4%) (P<.001); internal rotational maneuver 22/324 (6.8%) to 29/262 (11.1%) (P=.020); delivery of posterior arm 24/324 (7.4%) to 52/262 (19.8%) (P<.001); no recognized maneuvers performed 174/324 (50.9%) to 21/262 (8.0%) (P<.001); documented excessive traction 54/324 (16.7%) to 24/262 (9.2%) (P=.010). There was a significant reduction in neonatal injury at birth after shoulder dystocia: 30/324 (9.3%) to 6/262 (2.3%) (relative risk 0.25 [confidence interval 0.11-0.57]).
The introduction of shoulder dystocia training for all maternity staff was associated with improved management and neonatal outcomes of births complicated by shoulder dystocia.
II.
Strategies and tools to enhance performance and patient safety. U.S. Department of Health and Human Services
- Agency For Healthcare Research
- Quality
Agency for Healthcare Research Quality. Strategies and tools to enhance performance and patient safety. U.S. Department of Health and Human Services; 2009
http://teamstepps.ahrq.gov/index.htm (accessed April 2011).
Sydney: Allen and Unwin
- J Pallant
Pallant J. SPSS Survival Manual. 4th ed. Sydney: Allen and Unwin; 2011.