No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Tactical and operational response to major incidents: Feasibility and reliability of skills assessment using novel virtual environments
Abstract
Objectives:
To determine feasibility and reliability of skills assessment in a multi-agency, triple-site major incident response exercise carried out in a virtual world environment.
Methods:
Skills assessment was carried out across three scenarios. The pre-hospital scenario required paramedics to triage and treat casualties at the site of an explosion. Technical skills assessment forms were developed using training syllabus competencies and national guidelines identified by pre-hospital response experts. Non-technical skills were assessed using a seven-point scale previously developed for use by pre-hospital paramedics. The two in-hospital scenarios, focusing on a trauma team leader and a silver/clinical major incident co-ordinator, utilised the validated Trauma-NOTECHS scale to assess five domains of performance. Technical competencies were assessed using an ATLS-style competency scale for the trauma scenario. For the silver scenario, the assessment document was developed using competencies described from a similar role description in a real-life hospital major incident plan. The technical and non-technical performance of all participants was assessed live by two experts in each of the three scenarios and inter-assessor reliability was computed. Participants also self-assessed their performance using identical proformas immediately after the scenarios were completed. Self and expert assessments were correlated (assessment cross-validation).
Results:
Twenty-three participants underwent all scenarios and assessments. Performance assessments were feasible for both experts as well as the participants. Non-technical performance was generally scored higher than technical performance. Very good inter-rater reliability was obtained between expert raters across all scenarios and both technical and non-technical aspects of performance (reliability range 0.59-0.90, Ps<0.01). Significant positive correlations were found between self and expert assessment in technical skills across all three scenarios (correlation range 0.52-0.84, Ps<0.05), although no such correlations were observed in non-technical skills.
Conclusions:
This study establishes feasibility and reliability of virtual environment technical and non-technical skills assessment in major incident scenarios for the first time. The development for further scenarios and validated assessment scales will enable major incident planners to utilise virtual technologies for improved major incident preparation and training.
... Nonetheless, a number of studies have abundantly highlighted that the adoption of a multimodal approach combining frontal lectures and interactive simulations has the potential to foster students' engagement and knowledge retention [35,[55][56][57]. All this considered, there is a compelling need to aim for standardization in the design and assessment of the simulation exercises incorporated in the courses, with the option of integrating a series of predefined performance indicators to guide their development [58,59]. ...
... • Should be based on strong scientific evidence • Should be reproducible • Should comprise DM's transdisciplinarity [9] • Should follow the existing educational frameworks [9] • A multimodal approach of theoretical lectures and interactive simulations should be used, for increasing students' engagement and knowledge retention [35,[61][62][63] • If organizing a full scale exercise would be out of reach, virtual reality or computer aided simulation should be used [48,49,58,61,64,65] • Predefined performance indicators should be used to evaluate students' performance [58] Training approach Duration ...
... • Should be based on strong scientific evidence • Should be reproducible • Should comprise DM's transdisciplinarity [9] • Should follow the existing educational frameworks [9] • A multimodal approach of theoretical lectures and interactive simulations should be used, for increasing students' engagement and knowledge retention [35,[61][62][63] • If organizing a full scale exercise would be out of reach, virtual reality or computer aided simulation should be used [48,49,58,61,64,65] • Predefined performance indicators should be used to evaluate students' performance [58] Training approach Duration ...
Introduction: Disaster Medicine (DM) is currently underrepresented in medical schools’ curricula worldwide, and existing DM courses for medical students are extremely heterogeneous due to the lack of pragmatic and standardized guidelines. Moreover, there is a gap in knowledge regarding the curriculum development methodology used for DM courses. This study aims to identify DM courses for medical students worldwide and to map their curriculum development methodologies by reviewing available literature.
Method: The search was conducted on three databases using the terms “Disaster medicine” AND “Education”. Following the PRISMA approach, twenty-five articles that described the content and implementation of DM curricula were included in the analysis.
Results: Nine studies thoroughly described the curriculum development process. Expert opinion and literature review were the methodologies mostly used to develop DM curricula. Only four studies followed a multi-method process made up of four different methodologies, including expert opinion, literature review, survey, and Delphi methodology. Most of the courses adopted a face-to-face approach combining different training modalities, including the use of virtual reality simulations and drills.
Conclusion: This systematic review provides a compendious analysis of the curricula and curriculum development processes in DM training for medical students. The scarce usage of reproductible, comprehensive curriculum development methodologies and consequently a great heterogenicity of the covered topics and course design were brought forward. Therefore, there is a need for standardization in DM education. Overall, this systematic review highlights the need for evidence-based educational curricula in DM and provides recommendations for developing DM courses following a scientific approach.
... Nonetheless, a number of studies have abundantly highlighted that the adoption of a multimodal approach combining frontal lectures and interactive simulations has the potential to foster students' engagement and knowledge retention [35,[55][56][57]. All this considered, there is a compelling need to aim for standardization in the design and assessment of the simulation exercises incorporated in the courses, with the option of integrating a series of predefined performance indicators to guide their development [58,59]. ...
... • Should be based on strong scientific evidence • Should be reproducible • Should comprise DM's transdisciplinarity [9] • Should follow the existing educational frameworks [9] • A multimodal approach of theoretical lectures and interactive simulations should be used, for increasing students' engagement and knowledge retention [35,[61][62][63] • If organizing a full scale exercise would be out of reach, virtual reality or computer aided simulation should be used [48,49,58,61,64,65] • Predefined performance indicators should be used to evaluate students' performance [58] Training approach Duration ...
... • Should be based on strong scientific evidence • Should be reproducible • Should comprise DM's transdisciplinarity [9] • Should follow the existing educational frameworks [9] • A multimodal approach of theoretical lectures and interactive simulations should be used, for increasing students' engagement and knowledge retention [35,[61][62][63] • If organizing a full scale exercise would be out of reach, virtual reality or computer aided simulation should be used [48,49,58,61,64,65] • Predefined performance indicators should be used to evaluate students' performance [58] Training approach Duration ...
Disaster Medicine (DM) is currently underrepresented in medical schools’ curricula worldwide, and existing DM courses for medical students are extremely heterogeneous due to the lack of pragmatic and standardized guidelines. Moreover, there is a gap in knowledge regarding the curriculum development methodology used for DM courses. This study aims to identify DM courses for medical students worldwide and to map their curriculum development methodologies by reviewing available literature. The search was conducted on three databases using terms “Disaster medicine” AND “Education”. Following the PRISMA approach, twenty-five articles that described the content and implementation of DM curricula were included in the analysis. Nine studies thoroughly described the curriculum development process. Expert opinion and literature review were the methodologies mostly used to develop DM curricula. Only four studies followed a multi-method process made up of four different methodologies, including expert opinion, literature review, survey, and Delphi methodology. Most of the courses adopted a face-to-face approach combining different training modalities, including the use of virtual reality simulations and drills. Overall, this systematic review highlights the need for evidence-based educational curricula in DM and provides recommendations for developing DM courses following a scientific approach.
... 17 Second, virtual worlds are three-dimensional virtual environments based on multiplayer online gaming, allowing users to free themselves from geographical proximity or time constraints (individual connection and full time access). [18][19][20] For health professionals, medical furniture, instruments, devices, tools, and characters are added to create dedicated medical virtual worlds. 21 Lastly, immersive VR environments combine three-dimensional imaging, interactions with the environment, possible haptic feedback, and head-mounted displays (HMDs) or cave automatic virtual environments (CAVE, room-sized cube VR environments) to immerse the user and occlude the real world to provide a feeling of presence. ...
... Regarding sample size, the average number of participants was 39.69 (min = 10, max = 148, SD = 37.33). Looking more closely at their study design, 17 studies used an experimental design (pretest/posttest, group comparisons, or control/test group comparisons), [29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44] nine were observational studies, [18][19][20][45][46][47][48][49][50] and one was based on qualitative interviews. 51 ...
... Emergency medicine 18,19,32,44,47 and health education 20,37,41,50,51 stand out from the rest with five articles each, but the latter includes issues that potentially concern all specialties or health professionals in different sectors. Next are urology 29,30,40,48 and gastroenterology 33,34,45,46 with four articles each. ...
Statement:
This systematic review, conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, is aimed to review current research in virtual reality (VR) for healthcare training, specifically pertaining to nontechnical skills. PsycInfo and Medline databases were queried for relevant articles published through December 2017. Of the 1377 publications identified, 80 were assessed for eligibility and 26 were finally included in the qualitative synthesis. Overall, the use of virtual training for nontechnical skills is recent in healthcare education and has increased since 2010. Screen-based VR simulators or virtual worlds are the most frequently used systems. The nontechnical skills addressed in VR simulation include mainly teamwork, communication, and situation awareness. Most studies evaluate the usability and acceptability of VR simulation, and few studies have measured the effects of VR simulation on nontechnical skills development.
... 6 This need is especially evident within the context of simulated multiple patient casualty training-defined as medical response training efforts composed of a large number of causalities occurring within a short period. 7 Cohen et al 8 identify lack of effective teamwork as a challenge to first responders and further link poor teamwork to reduced decision-making and performance outcomes. 8 Team training is highlighted as a needed solution, given that effective first responder care can decrease patient mortality. ...
... 7 Cohen et al 8 identify lack of effective teamwork as a challenge to first responders and further link poor teamwork to reduced decision-making and performance outcomes. 8 Team training is highlighted as a needed solution, given that effective first responder care can decrease patient mortality. 9 Although team training has been found to be effective within several different industries and contexts, the extent to which it improves major incident response is presently unclear. ...
... 9 Although team training has been found to be effective within several different industries and contexts, the extent to which it improves major incident response is presently unclear. 8,10 This deficiency in the multicasualty literature highlights the need for identifying specific, observable behaviors that are indicative of multicasualty team performance. Existing frameworks of teamwork processes in multicasualty scenarios or similar contexts do not provide this level of detail as most focus on high-level constructs such as effective communication about critical events. ...
Introduction:
Teamwork is a critical aspect of patient care and is especially salient in response to multiple patient casualties. Effective training and measurement improve team performance. However, the literature currently lacks a scientifically developed measure of team performance within multiple causality scenarios, making training and feedback efforts difficult. The present effort addresses this gap by integrating the input of subject matter experts and the science of multicasualty teams and training to (1) identify overarching teamwork processes and corresponding behaviors necessary for team performance and (2) develop a behavioral observation tool to optimize teamwork in multicasualty training efforts.
Method:
A search of articles including team performance frameworks associated with team training was conducted, leading to the identification of a total of 14 articles. Trained coders extracted teamwork processes and the corresponding team behaviors indicative of effective performance from these articles. Five subject matter experts were interviewed using the critical incident technique to identify additional behaviors.
Results:
Team situation awareness, team leadership, coordination, and information exchange emerged as the four core team processes required for team performance in scenarios with multiple patient casualties. Relevant behaviors and subbehaviors within these overarching processes were identified to inform a pilot behavioral framework of team performance.
Conclusions:
The processes and associated behaviors identified within this effort serve as scientifically grounded behaviors of team performance in the case of multiple patient casualties simulated training scenarios. Future work can use and further refine these results to ensure that measures of team performance are grounded in specific, observable, and scientifically delineated behaviors.
... However, they show the technical feasibility of providing medical education via virtual worlds [10]. A total of 3 of the Second Life-embedded virtual worlds were designed for team-based training in medical education-Second Health London is provided by the Imperial College London and was used in a pilot feasibility study including a first validation [13]. ...
... The literature search uncovered 5 articles that included validation studies, however, no study assessed all levels of validity. One of the available VW simulators validated the coherence between simulated content and real content (ie, content validity) in training of emergency procedures [13]. The correlation between the outcome of simulator training and alternative training methods was proved in 2 virtual worlds [16,20] (ie, concordance validity)-in all 3 validation studies, simulator training was comparable to alternative training in terms of outcome. ...
... However, all current IPSs are used in the blended learning context, as postprocedural review of students' performance immediately after training is known to be essential for an adequate knowledge gain [32]. Moreover, there is evidence that preexisting knowledge has a positive impact on simulator performance (ie, construct validity), although the group size of this study was low [13]. None of the articles assessed all forms of validity. ...
BACKGROUND:
Immersive patient simulators (IPSs) combine the simulation of virtual patients with a three-dimensional (3D) environment and, thus, allow an illusionary immersion into a synthetic world, similar to computer games. Playful learning in a 3D environment is motivating and allows repetitive training and internalization of medical workflows (ie, procedural knowledge) without compromising real patients. The impact of this innovative educational concept on learning success requires review of feasibility and validity.
OBJECTIVE:
It was the aim of this paper to conduct a survey of all immersive patient simulators currently available. In addition, we address the question of whether the use of these simulators has an impact on knowledge gain by summarizing the existing validation studies.
METHODS:
A systematic literature search via PubMed was performed using predefined inclusion criteria (ie, virtual worlds, focus on education of medical students, validation testing) to identify all available simulators. Validation testing was defined as the primary end point.
RESULTS:
There are currently 13 immersive patient simulators available. Of these, 9 are Web-based simulators and represent feasibility studies. None of these simulators are used routinely for student education. The workstation-based simulators are commercially driven and show a higher quality in terms of graphical quality and/or data content. Out of the studies, 1 showed a positive correlation between simulated content and real content (ie, content validity). There was a positive correlation between the outcome of simulator training and alternative training methods (ie, concordance validity), and a positive coherence between measured outcome and future professional attitude and performance (ie, predictive validity).
CONCLUSIONS:
IPSs can promote learning and consolidation of procedural knowledge. The use of immersive patient simulators is still marginal, and technical and educational approaches are heterogeneous. Academic-driven IPSs could possibly enhance the content quality, improve the validity level, and make this educational concept accessible to all medical students.
... In two different studies, we have recently shown the efficacy of live simulation in assessment of triage skills after a disaster medicine training session [19,32]. However, live simulations, 'whilst accepted to be the 'gold-standard' are challenging to organize, expensive and disruptive' [33]. In the light of the above, in recent years, virtual reality (VR) simulation has been shown to be a valid, clinically appropriate, and cost-effective training method able to achieve a good degree of realism [24][25][26][27][28][29][30]. ...
... In the light of the above, in recent years, virtual reality (VR) simulation has been shown to be a valid, clinically appropriate, and cost-effective training method able to achieve a good degree of realism [24][25][26][27][28][29][30]. However, little is known about the possibility of using this simulation methodology to test the MCT skills acquired after brief teaching sessions [33]. ...
... These findings undoubtedly showed that both simulations equally detect the same level of knowledge and expertise in medical students and they are able to similarly assess the improvement in MCT skills from an identical baseline. Moreover, they accurately reflect the conclusions of a recent study [33], and provide further evidence on 'the feasibility of utilizing these immersive, low-cost virtual training environments for skills assessment as an adjunct to existing training and assessment tools in major incident preparation' [33]. ...
Objectives:
This study tested the hypothesis that virtual reality simulation is equivalent to live simulation for testing naive medical students' abilities to perform mass casualty triage using the Simple Triage and Rapid Treatment (START) algorithm in a simulated disaster scenario and to detect the improvement in these skills after a teaching session.
Methods:
Fifty-six students in their last year of medical school were randomized into two groups (A and B). The same scenario, a car accident, was developed identically on the two simulation methodologies: virtual reality and live simulation. On day 1, group A was exposed to the live scenario and group B was exposed to the virtual reality scenario, aiming to triage 10 victims. On day 2, all students attended a 2-h lecture on mass casualty triage, specifically the START triage method. On day 3, groups A and B were crossed over. The groups' abilities to perform mass casualty triage in terms of triage accuracy, intervention correctness, and speed in the scenarios were assessed.
Results:
Triage and lifesaving treatment scores were assessed equally by virtual reality and live simulation on day 1 and on day 3. Both simulation methodologies detected an improvement in triage accuracy and treatment correctness from day 1 to day 3 (P<0.001). The time to complete each scenario and its decrease from day 1 to day 3 were detected equally in the two groups (P<0.05).
Conclusion:
Virtual reality simulation proved to be a valuable tool, equivalent to live simulation, to test medical students' abilities to perform mass casualty triage and to detect improvement in such skills.
... In addition, the interrater reliability of our Hebrew version of T-NOTECHS was excellent (ICC, 0.786), consistent with prior reports for T-NOTECHS, in simulated applications, and a Finnish translation. 14,22,23 Since the simulation performance assessment reliability in our study was also excellent, we believe that this finding validates the use of T-NOTECHS to assess performance in multicasualty events. ...
... This is supported by the report of Steinemann et al. 15 that simulatorbased teamwork training improved specific communication skill performance and overall outcomes in real-life resuscitations. Another viable option involves the utilization of virtual reality; Cohen et al. 22 demonstrated the possibility of T-NOTECHS team training in such an environment. Finally, our study shows the feasibility of using T-NOTECHS to evaluate medical teams during multicasualty event training. ...
BACKGROUND
Multicasualty events present complex medical challenges. This is the first study to investigate the role of nontechnical skills in prehospital multicasualty trauma care. We assessed the feasibility of using the Trauma Nontechnical Skills Scale (T-NOTECHS) instrument, which has not yet been investigated to evaluate these scenarios.
METHODS
We conducted an observational study involving military medical teams with Israel Defense Forces Military Trauma Life Support training to assess the T-NOTECHS' utility in predicting prehospital medical team performance during multicasualty event simulations. These teams were selected from a pool of qualified military Advanced Life Support providers. Simulations were conducted in a dedicated facility resembling a field setting, with video recordings to ensure data accuracy. Teams faced a single multicasualty scenario, assessed by two instructors, and were evaluated using a 37-item checklist. The T-NOTECHS scores were analyzed using regression models to predict simulation performance.
RESULTS
We included 27 teams for analysis, led by 28% physicians and 72% paramedics. Interrater reliability for simulation performance and T-NOTECHS scores showed good agreement. Overall T-NOTECHS scores were positively correlated with simulation performance scores ( R = 0.546, p < 0.001). Each T-NOTECHS domain correlated with simulation performance. The Communication and Interaction domain explained a unique part of the variance ( β = 0.406, p = 0.047). Assessment and Decision Making had the highest correlation ( R = 0.535, p < 0.001). These domains significantly predicted specific items on the simulation performance checklist. Cooperation and Resource Management showed the least correlation with checklist items.
CONCLUSION
This study confirms the T-NOTECHS' reliability in predicting prehospital trauma team performance during multicasualty scenarios. Key nontechnical skills, especially Communication and Interaction, and Assessment and Decision Making, play vital roles. These findings underscore the importance of training in these skills to enhance trauma care in such scenarios, offering valuable insights for medical team preparation.
LEVEL OF EVIDENCE
Diagnostic Tests or Criteria; Level II.
... Once a defined set of parameters is used, an existing standard or common practice is used as a reference (Whitmore 2005;Walker et al. 2014;Salmon et al. 2009;Rüter and Vikstrom 2009;Rådestad et al. 2012;Pleban et al. 2002;Nilsson and Rüter 2008;Nilsson et al. 2013;Gebbie et al. 2006;Qari et al. 2019;Djalali et al. 2014;Cohen et al. 2013). The parameters include elements of the standard such as the conduct of meetings, the content of defense plans, information to higher and lower levels. ...
... The observations focus on the object of investigation, e.g., team behavior or task completion. Tools such as questionnaires, checklists, or protocols (Cohen et al. 2013;Garvin and Miller 1981) augment the procedure for this purpose (Peck et al. 2017). These observations can be supported by behavioral anchors: Behavioral anchors are a common method supported by the use of checklists or rating scales (Bearman et al. 2018) such as the Air Warfare Team Performance Index (APTI) (Johnston et al. 2013;Reeves et al. 1998). ...
Command and control are critical components of emergency management when disaster strikes. Command posts act as a support system within the command chain for leaders, and the performance of these teams is essential for the successful operation. This paper presents the findings of a scoping review on the performance of command posts, identifying six performance factors: scenario, resilience, situational awareness, decision making, team structures and teamwork, and operational execution. Furthermore, this paper presents objective indicators for performance measurement in three dimensions: process, effectiveness, and efficiency. Additionally, it provides insights and methodologies for evaluating command post exercises.
... Full-scale exercises are the most complex form of operation-based exercises, demonstrating psychological, physical, and environmental fidelity [24,25]. They offer the greatest level of realism compared to other exercises, allowing the whole response structure to be tested under stressful conditions, including intra-and inter-agency coordination, and implementation of emergency plans, personnel, and equipment [26][27][28][29]. However, designing and delivering full-scale exercises requires the largest resource investment of all emergency planning exercises, making them less practical to deliver regularly [24,25]. ...
... Full-scale exercises are a tool for enhancing emergency preparedness. Their goal is to allow the whole response structure to be tested under stressful conditions, including intra-and inter-agency coordination, and implementation of emergency plans, personnel, and equipment [26][27][28][29]. This rapid evidence assessment also sought to highlight whether participation in full-scale exercises was beneficial for developing disaster response knowledge and skills in emergency responders, and what aspects of exercise planning and de-livery were important for facilitating this learning. ...
... Virtual worlds could be useful in healthcare contexts when staff or patients meet remotely [14], and when the virtual environment can be used to facilitate discussions or collaborative activities [15,16]. Documents can be worked on collaboratively within the virtual environment [15], the environment can be modified to represent any form of physical setting [17] and tools for collaboration can be built that would be difficult or costly to represent in equivalent face-to-face meetings [6,15]. Virtual worlds can give users the impression that they are all sharing the same physical space, resulting in a sense of 'togetherness' that cannot be created through video link or any other remote communication method [18,19]. ...
... com). Second Life was selected because it is publically accessible, free to use, and has been used for healthcare training [6,36] and education. ...
Background:
Healthcare teams often consist of geographically dispersed members. Virtual worlds can support immersive, high-quality, multimedia interaction between remote individuals; this study investigated use of virtual worlds to support remote healthcare quality improvement team meetings.
Methods:
Twenty individuals (12 female, aged 25-67 [M = 42.3, SD = 11.8]) from 6 healthcare quality improvement teams conducted collaborative tasks in virtual world or face-to-face settings. Quality of collaborative task performances were measured and questionnaires and interviews were used to record participants' experiences of conducting the tasks and using the virtual world software.
Results:
Quality of collaborative task outcomes was high in both face-to-face and virtual world settings. Participant interviews elicited advantages for using virtual worlds in healthcare settings, including the ability of the virtual environment to support tools that cannot be represented in equivalent face-to-face meetings, and the potential for virtual world settings to cause improvements in group-dynamics. Reported disadvantages for future virtual world use in healthcare included the difficulty that people with weaker computer skills may experience with using the software. Participants tended to feel absorbed in the collaborative task they conducted within the virtual world, but did not experience the virtual environment as being 'real'.
Conclusions:
Virtual worlds can provide an effective platform for collaborative meetings in healthcare quality improvement, but provision of support to those with weaker computer skills should be ensured, as should the technical reliability of the virtual world being used. Future research could investigate use of virtual worlds in other healthcare settings.
... The infrequent nature of MCIs limits the amount of exposure of emergency personnel to these events, rendering many ill-equipped to deal with the sheer number of casualties and the allocation of resources. It is therefore necessary to ensure that paramedics are provided with the appropriate systematic training and preparation to equip them with the necessary skills to ensure an effective and safe response (1). ...
... The emergency preparedness of many healthcare organizations and individuals in responding to MCIs is far from adequate (1)(2)(3)(4)(5). One of the most essential components of MCI response is patient triage. ...
Mass-casualty incidents (MCIs) can arise from natural or man-made disasters. The nature of such incidents and the multiple casualties involved can rapidly overwhelm response personnel. Mass-casualty triage training is traditionally taught via didactic lectures or table top exercises. This training fails to provide an opportunity for practical application or experiential learning in immersive conditions. Further, large-scale simulations are heavily resource-intensive, logistically challenging, require the coordination and time of multiple personnel, and are costly to replicate. This study compared the simulation efficacy of a bespoke virtual-reality (VR) MCI simulation with an equivalent live simulation scenario designed for undergraduate paramedicine students.
Both simulations involved ten injured patients resulting from a police car chase and shooting. Twenty-nine second-year paramedicine students completed the live and VR simulation in a random order. Measures of student immersion, task-difficulty, clinical decision-making (i.e. triage card allocation accuracy and timeliness), learning satisfaction and cost were taken.
While perceived physical demand was higher in the live simulation compared to VR (p < 0.001), no differences were observed across mental demand, temporal demand, performance, effort or frustration domains. No differences were found for participant satisfaction. No differences were observed in the number of triage cards correctly allocated to patients in each platform. However, participants were able to allocate cards far quicker in VR (p<.001). Variable costs of running the VR came to AUD 9,413.71 (staff time, moulage, actors, director, prop vehicle), approximately 13 times more expensive. When including VR content development costs, cost neutrality occurs after 145 participants.
The VR simulation provided comparable simulation efficacy for paramedicine students compared to the live simulation. VR MCI training resources represent an exciting new direction for authentic and cost-effective (particularly for institutions with large student numbers) education and training for medical professionals.
... The concern about human behavior and performance have become a major issue in almost every economic activity these days. The intuitive notion of human factors as one of the main causes of accidents, is being confirmed by studies in areas such as aviation [1], chemical industry [2], nuclear power generation, [3] and water supply [4], among others, turning what was intuitive into a scientific "de facto". ...
... The approach has already been discussed in the literature in cases of virtual environments generating data to HRA studies or to validate concepts [1]- [3], but about the subject of how the Virtual Environment (VE) was created, very few, or no info at all is given. At the best of authors' knowledge, Ref [4] is the only that is a bit clearer on the subject of environment creation though it is not tailored for HRA. ...
As man-made activities are still undoubtedly necessary, especially in complex and cognitive tasks, human behavior and performance have become a major concern in almost every economic activity. Human factors are quite often pointed, direct or indirectly, as one of the main accident causes in areas such as aviation, chemical industry, nuclear power generation and water supply among many others. The present work proposes to discuss the use of game engines in the construction of virtual environments for Human Reliability Assessment (HRA) studies. Two well known facts motivate this work: the lack of good datasets for HRA estimations, and the shortage of funding for scientific research, generating small teams with poor interdisciplinarity. This paper discusses how to port real-world scenarios to virtual The rapidly evolving technology turned industrial process automation into tangible reality. environments aiming to extract useful data for HRA and the use of in game Analytics, from Serious Games (SG) gameplay sessions, under the small teams’ perspective. The work discusses how to adapt scenarios, create analytics generation setups and shows which data types can be obtained. Also, a real scenario accident of an evacuation and toxic cloud release is modeled and analyzed to study the proposed methodology feasibility.
... Lack of appropriate training in mass-casualty response can lead to mass-casualty incidents becoming chaotic catastrophes, underscoring the importance of effective training programs (Dal Ponte, Dornelles, Arquilla, Bloem, & Roblin, 2015). The optimal method of training to ensure effectiveness at a major incident remains unclear (Cohen et al., 2013), however a combination of didactic and practical training appears to be key in improving triage efficacy (Ersoy & Akpinar, 2010;Rehn et al., 2010). Short didactic training sessions are often used to educate responders on the application of triage algorithms, however disagreement exists as to whether such training, in isolation, is actually effective (Deluhery et al., 2011;Lampi, Vikstrom, & Jonson, 2013). ...
... While questions remain about the efficacy of short didactic sessions in isolation, short (15 minute) training modules delivered via the internet have been demonstrated to be effective in dramatically improving triage accuracy in responders who have not been exposed to triage training previously, with improvements sustained after one month (Báez, Sztajnkrycer, Smester, Giraldez, & Vargas, 2005). Likewise, studies employing virtual reality to train in triage methodology appear promising, with two studies demonstrating skill improvements that rival the results gleaned from 'real life' mass-casualty scenarios (Cohen et al., 2013;Cone et al., 2011). While further work remains to be done in this area, early indications suggest that didactic and practical training sessions may, at least in part, be able to be replaced by online training modules and virtual reality exercises with no reduction in responder efficacy. ...
Background
Mass-casualty triage algorithms are a ubiquitous tool aimed at helping responders to major incidents identify and categorise patients most in need of care. This review seeks to present the current literature regarding triage algorithms, including training methods, validation attempts, and their usability during exercises and incidents.
Methods
A search was undertaken of MEDLINE, EMBASE and CINAHL, producing 382 unique results. Following review using pre-determined inclusion and exclusion criteria, 31 publications were selected for inclusion. These covered four broad themes: accuracy, training, indications and validation.
Discussion
Triage is an important component of incident management, but may not be necessary in all mass-casualty incidents. Responders tend to over-triage more but under-triage less during incidents when compared to exercises and retrospective studies. Algorithms must strike a balance between sensitivity and specificity; the SALT algorithm appears to be the best from those studied. Optimal training is critical to accuracy, and best results come from a combination of didactic and practical training, although virtual reality training shows promise. Validation of algorithms remains notoriously difficult and no standardised criteria exist to assess algorithms against.
Conclusions
Incident managers must consider the scale of the incident and the resources available to determine if patients will benefit from full implementation of a mass-casualty triage system. Responder training needs to be effective to ensure accuracy and minimise under- and over-triage - including didactic and practical sessions. The lack of validation for algorithms remains a significant issue, and much work remains to ensure that mass-casualty patients receive the best of evidence-based care.
... Canada could collaborate with African experts to seamlessly provide such programs by leveraging on validated virtual technologies in trauma education (41), (42). Evidence from Canada shows that virtual simulations provide an ideal starting point to identify team errors that can be recti ed when a critical mass of trainers and trainees are accumulated (43). ...
Background
Research shows that trauma team formation could potentially improve effectiveness of injury care in rural settings. The aim of this study was to determine the feasibility of the use of rural medical trainees and road traffic law enforcement professionals in the formation of rural trauma teams in Uganda.
Methods
Multi-center interrupted time series of interventional rural health professions education, using the American College of Surgeons’ 4th edition of rural trauma team development course model. Trauma related multiple choice questions (MCQs) were administered pre-and post-training between September 2019- August 2023. Acceptability of the training for promulgation to other rural regions and its relevance to participants’ work needs were evaluated on 5- and 3-point Likert scales respectively. The median MCQ scores (IQR) were compared before and after training at 95% CI, regarding p < 0.05 as statistically significant. Triangulation with open-ended questions was obtained. Time series regression models were applied to test for autocorrelation in performance using Stata 15.0. Ethical approval was obtained from Uganda National Council for Science and Technology (Ref: SS 5082).
Results
A total of 500 participants including: 66 (13.2%) traffic police officers, 30 (6.0%) intern doctors, 140 (28.0%) fifth year and 264 (52.8%) third-year medical students were trained. The overall median pre- and post-test scores were 60%, IQR (50–65) and 80%, IQR (70–85) respectively. Overall, the mean difference between pre- and post-test scores was statistically significant (z = 16.7%, P|z|=<0.0001). Most participants strongly agreed to promulgation 389 (77.8%), relevance to their educational 405 (81.0%), and work needs 399 (79.8%). All the course elements scored above 76.0% as being very relevant.
Conclusion
This study demonstrates that rural trauma team development training had a positive effect on the test scores of course participants. The training is feasible, highly acceptable and regarded as relevant amongst medical trainees and traffic law enforcement professionals who provide first-aid to trauma patients in resource-limited settings. The findings could inform the design of future trauma teams in rural communities.
Trial registration
Retrospective registration (UIN: researchregistry9450)
... Virtual worlds are 3D virtual environments based on multiplayer web-based gaming, freeing users from the constraints of location and time. Virtual worlds representing a clinical setting have been used to train emergency personnel on the management of situations involving mass casualties or major incidents [10][11][12]. Avatars representing patients can be generated to provide a more realistic simulation for the user [13]. Mobile VR refers to VR modalities designed for use on a touch screen mobile phone or tablet; examples include the Touch Surgery app [14]. ...
Background:
Virtual reality (VR) produces a virtual manifestation of the real world and has been shown to be useful as a digital education modality. As VR encompasses different modalities, tools, and applications, there is a need to explore how VR has been used in medical education.
Objective:
The objective of this scoping review is to map existing research on the use of VR in undergraduate medical education and to identify areas of future research.
Methods:
We performed a search of 4 bibliographic databases in December 2020. Data were extracted using a standardized data extraction form. The study was conducted according to the Joanna Briggs Institute methodology for scoping reviews and reported in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines.
Results:
Of the 114 included studies, 69 (60.5%) reported the use of commercially available surgical VR simulators. Other VR modalities included 3D models (15/114, 13.2%) and virtual worlds (20/114, 17.5%), which were mainly used for anatomy education. Most of the VR modalities included were semi-immersive (68/114, 59.6%) and were of high interactivity (79/114, 69.3%). There is limited evidence on the use of more novel VR modalities, such as mobile VR and virtual dissection tables (8/114, 7%), as well as the use of VR for nonsurgical and nonpsychomotor skills training (20/114, 17.5%) or in a group setting (16/114, 14%). Only 2.6% (3/114) of the studies reported the use of conceptual frameworks or theories in the design of VR.
Conclusions:
Despite the extensive research available on VR in medical education, there continue to be important gaps in the evidence. Future studies should explore the use of VR for the development of nonpsychomotor skills and in areas other than surgery and anatomy.
International registered report identifier (irrid):
RR2-10.1136/bmjopen-2020-046986.
... VR-based education has also been shown to be a feasible and reliable means for technical and nontechnical skills assessment (20). To date, the efficacy of VR in procedural training has been demonstrated and more broadly adopted in surgical specialties than in medicine and nursing; however, there is increasing evidence of VR's utility in teaching a broad range of nonsurgical procedures (15,21). ...
Background:
Advancements in technology continue to transform the landscape of medical education. The need for technology-enhanced distance learning has been further accelerated by the coronavirus disease (COVID-19) pandemic. The relatively recent emergence of virtual reality (VR), augmented reality (AR), and alternate reality has expanded the possible applications of simulation-based education (SBE) outside of the traditional simulation laboratory, making SBE accessible asynchronously and in geographically diverse locations.
Objective:
In this review, we will explore the evidence base for use of emerging technologies in SBE as well as the strengths and limitations of each modality in a variety of settings.
Methods:
PubMed was searched for peer-reviewed articles published between 1995 and 2021 that focused on VR in medical education. The search terms included medical education, VR, simulation, AR, and alternate reality. We also searched reference lists from selected articles to identify additional relevant studies.
Results:
VR simulations have been used successfully in resuscitation, communication, and bronchoscopy training. In contrast, AR has demonstrated utility in teaching anatomical correlates with the use of diagnostic imaging, such as point-of-care ultrasound. Alternate reality has been used as a tool for developing clinical reasoning skills, longitudinal patient panel management, and crisis resource management via multiplayer platforms.
Conclusion:
Although each of these modalities has a variety of educational applications in health profession education, there are benefits and limitations to each that are important to recognize prior to the design and implementation of educational content, including differences in equipment requirements, cost, and scalability.
... The use of virtual worlds representing a clinical setting has been used in training emergency personnel on the management of mass-casualty or major incident situations. [13][14][15] Avatars can be generated representing patients, which provides a more realistic simulation for the user. 16 The use of VR in medical education can be applied in to two major areas. ...
Background
Virtual reality (VR) is a technology that produces a virtual manifestation of the real world. In recent years, VR has been increasingly used as a tool in medical education. The use of VR in medical education has large potential, as it allows for distance learning and training which may be challenging to deliver in real life. VR encompasses different tools and applications. There is a need to explore how VR has been employed in medical education to date.
Objective
The objective of this scoping review is to conceptualise the VR tools available and the applications of VR in undergraduate medical education as reported in the literature. This scoping review will identify any gaps in this field and provide suggestions for future research.
Methods and analysis
The relevant studies will be examined using the Joanna Briggs Institute methodological framework for scoping studies. A comprehensive search from a total of six electronic databases and grey literature sources will be performed. The reference list of included studies will be screened for additional studies. The screening and data extraction will be done in parallel and independently by two review authors. Any discrepancies will be resolved through consensus or discussion with a third review author. A data extraction form has been developed using key themes from the research questions. The extracted data will be qualitatively analysed and presented in a diagrammatic or tabular form, alongside a narrative summary, in line with Preferred Reporting Items for Systematic Reviews and Meta-Analysis: extension for Scoping Reviews reporting guidelines.
Ethics and dissemination
All data will be collected from published and grey literature. Ethics approval is therefore not a requirement. We will present our findings at relevant conferences and submit them for publications in peer-reviewed journals.
... A limitation of ISWP's Basic Wheelchair Service Provider (WSP) certification, which employs the Wheelchair Service Provision-Basic Test, is that it does not yet include a practical or skills assessment. Other certifications are available, such as the Seating and Mobility Specialist, offered by the Rehabilitation Engineering Society of North America (RESNA), which offers credentialing for clinicians who demonstrate competence in seating and mobility [32]. However, it requires knowledge of complex rehabilitation equipment which may not be suitable for those in some low or middle-income countries (LMIC) or volunteers who are assisting with service provision and distribution abroad. ...
Aim
The purpose of this study was to develop, validate and conduct a feasibility study of three remote basic skills assessment modalities for wheelchair service providers (WSP) including an online case study quiz (m1), an in-person skills assessment (m2) and a video conference skills assessment (m3).
Methods
We conducted a literature review; developed and validated 3 basic skills assessments; and conducted a feasibility study of each modality.
Results
The literature review revealed that a validated remote basic skills assessment for WSP that reflects all World Health Organization (WHO) 8 wheelchair service provision steps did not exist. We recruited a total of 12 participants for the feasibility study. Two participants dropped out of the study prior to completing a second testing modality. Related to test performance, the results show that our first hypothesis was rejected because only m1 mean score was comparable to the International Society of Wheelchair Professionals (ISWP) Basic Knowledge Test (SD = 0.44). This is in contrast with the Wilcoxon signed-rank test results that show a statistically significant difference between these two modalities. We are therefore not confident that the knowledge test was an appropriate comparison to m1 skills assessment. Hypothesis two was not rejected. The feasibility results reveal 86% success.
Conclusion
M1, 2 and 3 have the potential to serve as remote basic skills assessments. However, according to both test performance and feasibility criteria, we believe that m2 has the highest potential to be included in certification processes for basic wheelchair service providers, like the one offered by ISWP.
•
IMPLICATIONS FOR REHABILITATION
• A universal remote basic skills assessment that can be accessed across the globe, especially in remote locations where a skilled and experienced provider is not available, is needed.
• Such test can be an asset to training or professional organisations like ISWP as a way to test WSP competency or to warrant certification.
• WSP clinical knowledge and skills are essential for the prescription of an appropriate wheelchair to avoid physical harm, abandonment of the device and unnecessary expenses.
• Properly prescribed wheelchairs allow people with impaired mobility to gain increased ability to perform ADLs, participate in communities, and reduce secondary medical complications such as upper limb repetitive strain injuries, pain, and/or pressure sores.
... Other interactive models such as table-top exercises and computerized simulation models are frequently used in MCI triage training [67][68][69]. ...
Triage, derived from the French word for sorting, aims to assess and prioritize injured patients, regardless of whether the injuries are sustained from everyday road traffic accident with few injured or a mass casualty incident. Triage seeks to provide the greatest benefit to the largest number of casualties in order to minimize morbidity and mortality. Triage in a pre-hospital setting entails management and sorting of patients according to an assessment of medical need, prioritization, and evacuation. In-hospital triage aims to rapidly identify the most injured and ensure timely and appropriate treatment according to the patient’s clinical urgency. A number of different systems for performing triage have been established and implemented globally. The methodology is recognized and utilized but there is still a need for an evidence-based strategy to optimize training and the efficacy of the different systems.
The main aim of this thesis was to determine triage performance among prehospital personnel and investigate the potential advantage of a triage system for trauma patients. The papers included in this thesis evaluated the triage skills of physicians, pre-hospital personnel, and rescue services personnel by testing their performance before and after an educational intervention. The last paper evaluated potential benefits of using a triage system for trauma patients admitted to the emergency department at MOI Teaching and Referral Hospital in Eldoret, Kenya.
The results presented in this thesis illustrate that triage skills are lacking among physicians. Experienced pre-hospital personnel are more skilled in performing triage than physicians. The triage skills of the rescue services personnel improved significantly after the educational intervention. Moreover, the potential benefit to trauma patients of implementing an in-hospital triage system in a resource-poor environment was shown. In conclusion, health care personnel, especially physicians without experience but highly involved in trauma patient management, seem to be in need of triage training. How to train, how to implement, and how to evaluate triage skills must be considered in order to develop effective training.
... With regard to procedural knowledge, a significant effect was noted for one out of six categories, and this effect was only present in students who had actually been exposed to the respective case while playing the game. Only students that worked with the case "perforated sigmoid diverticulitis" also showed an increase in performance in the OSCE questions about the "non-perorated sigmoid diverticulitis", which is a sign for positive concordance validity [24]. ...
Background:
Serious games enable the simulation of daily working practices and constitute a potential tool for teaching both declarative and procedural knowledge. The availability of educational serious games offering a high-fidelity, three-dimensional environment in combination with profound medical background is limited, and most published studies have assessed student satisfaction rather than learning outcome as a function of game use.
Objective:
This study aimed to test the effect of a serious game simulating an emergency department ("EMERGE") on students' declarative and procedural knowledge, as well as their satisfaction with the serious game.
Methods:
This nonrandomized trial was performed at the Department of General, Visceral and Cancer Surgery at University Hospital Cologne, Germany. A total of 140 medical students in the clinical part of their training (5th to 12th semester) self-selected to participate in this experimental study. Declarative knowledge (measured with 20 multiple choice questions) and procedural knowledge (measured with written questions derived from an Objective Structured Clinical Examination station) were assessed before and after working with EMERGE. Students' impression of the effectiveness and applicability of EMERGE were measured on a 6-point Likert scale.
Results:
A pretest-posttest comparison yielded a significant increase in declarative knowledge. The percentage of correct answers to multiple choice questions increased from before (mean 60.4, SD 16.6) to after (mean 76.0, SD 11.6) playing EMERGE (P<.001). The effect on declarative knowledge was larger in students in lower semesters than in students in higher semesters (P<.001). Additionally, students' overall impression of EMERGE was positive.
Conclusions:
Students self-selecting to use a serious game in addition to formal teaching gain declarative and procedural knowledge.
... Some of these are outlined by Mossel et al. [9]. Other example uses of virtual worlds include Second Life and Open Simulator [4,3]. ...
Autonomous robotics and artificial intelligence techniques can be used to support human personnel in the event of critical incidents. These incidents can pose great danger to human life. Some examples of such assistance include: multi-robot surveying of the scene; collection of sensor data and scene imagery, real-time risk assessment and analysis; object identification and anomaly detection; and retrieval of relevant supporting documentation such as standard operating procedures (SOPs). These incidents, although often rare, can involve chemical, biological, radiological/nuclear or explosive (CBRNE) substances and can be of high consequence. Real-world training and deployment of these systems can be costly and sometimes not feasible. For this reason, we have developed a realistic 3D model of a CBRNE scenario to act as a testbed for an initial set of assisting AI tools that we have developed (This research has received funding from the European Union’s Horizon 2020 Programme under grant agreement No. 700264.).
... 63 Based on this model, one could then train different types of decision-makers through virtual simulation. Some virtual simulations already exist for MCIs, currently focusing primarily on clinical skills, 64 but the approach could be used at managerial level to refine understanding of decision-making in these exceptional conditions and to train participants. Finally, based on the results of constructive simulations and virtual simulations, a scenario for a full-scale live drill could be elaborated. ...
Simulation is a technique that evokes or replicates substantial aspects of the real world, in order to experiment with a simplified imitation of an operations system, for the purpose of better understanding and/or improving that system. Simulation provides a safe environment for investigating individual and organisational behaviour and a risk-free testbed for new policies and procedures. Therefore, it can complement or replace direct field observations and trial-and-error approaches, which can be time consuming, costly and difficult to carry out. However, simulation has low adoption as a research and improvement tool in healthcare management and policy-making. The literature on simulation in these fields is dispersed across different disciplinary traditions and typically focuses on a single simulation method. In this article, we examine how simulation can be used to investigate, understand and improve management and policy-making in healthcare organisations. We develop the rationale for using simulation and provide an integrative overview of existing approaches, using examples of in vivo behavioural simulations involving live participants, pure in silico computer simulations and intermediate approaches (virtual simulation) where human participants interact with computer simulations of health organisations. We also discuss the combination of these approaches to organisational simulation and the evaluation of simulation-based interventions.
... Current evidence supports the use of VRS as a training method, but only a few studies have used experimental designs. [25][26][27][28][29][30] The reviewed literature related to VRS used in overall evacuation training is predominately focused on model crowd response. This research supports the findings of Ribeiro, Almeida, and Rossetti et al. 11 and Garrett and MacMahon, 12 who effectively used VRS to train participants in evacuation. ...
Objective
This study examined differences in learning outcomes among newborn intensive care unit (NICU) workers who underwent virtual reality simulation (VRS) emergency evacuation training versus those who received web-based clinical updates (CU). Learning outcomes included a) knowledge gained, b) confidence with evacuation, and c) performance in a live evacuation exercise.
Methods
A longitudinal, mixed-method, quasi-experimental design was implemented utilizing a sample of NICU workers randomly assigned to VRS training or CUs. Four VRS scenarios were created that augmented neonate evacuation training materials. Learning was measured using cognitive assessments, self-efficacy questionnaire (baseline, 0, 4, 8, 12 months), and performance in a live drill (baseline, 12 months). Data were collected following training and analyzed using mixed model analysis. Focus groups captured VRS participant experiences.
Results
The VRS and CU groups did not statistically differ based upon the scores on the Cognitive Assessment or perceived self-efficacy. The virtual reality group performance in the live exercise was statistically ( P <.0001) and clinically (effect size of 1.71) better than that of the CU group.
Conclusions
Training using VRS is effective in promoting positive performance outcomes and should be included as a method for disaster training. VRS can allow an organization to train, test, and identify gaps in current emergency operation plans. In the unique case of disasters, which are low-volume and high-risk events, the participant can have access to an environment without endangering themselves or clients. ( Disaster Med Public Health Preparedness. 2018; page 1 of 8)
... Some of these are outlined by Mossel et al. [9]. Other example uses of virtual worlds include Second Life and Open Simulator [4,3]. ...
Autonomous robotics and artificial intelligence techniques can be used to support human personnel in the event of critical incidents. These incidents can pose great danger to human life. Some examples of such assistance include: multi-robot surveying of the scene; collection of sensor data and scene imagery, real-time risk assessment and analysis; object identification and anomaly detection; and retrieval of relevant supporting documentation such as standard operating procedures (SOPs). These incidents, although often rare, can involve chemical, biological, radiological/nuclear or explosive (CBRNE) substances and can be of high consequence. Real-world training and deployment of these systems can be costly and sometimes not feasible. For this reason, we have developed a realistic 3D model of a CBRNE scenario to act as a testbed for an initial set of assisting AI tools that we have developed.
... An advantage of the T-nOTECHS is that it is an internationally used validated scale that has been modified specifically for evaluating trauma teams' nTS. Its interrater reliability is from fair to moderate (12,17,18). In the study by Pucher et al. (6) using T-nOTECHS, the interrater reliability scores were not reported. ...
Background and aims:
As conducting the regular trauma team simulation training is expensive and time-consuming, its effects must be explored. The objective was to evaluate the efficacy of a structured 2-h in situ multiprofessional trauma team simulation training course on non-technical skills.
Materials and methods:
This prospective study comprised 90 trauma teams with 430 participants. The structured, 2-h course consisted of an introductory lecture and two different simulations with debriefings. Data were collected using a pre-post self-assessment questionnaire. In addition, the expert raters used the T-NOTECHS scale.
Results:
The following non-technical skills improved significantly among both medical doctors and nurses: knowledge of the trauma resuscitation guidelines, problem identification, decision making, situation awareness/coping with stress, communication and interaction, time management, being under authority, and confidence in one's role in a team. The teams improved significantly in leadership, cooperation and resource management, communication and interaction, assessment and decision making, and situation awareness/coping with stress.
Conclusion:
A short, structured 2-h in situ trauma team simulation training course is effective in improving non-technical skills.
... 10,11 Thus, temporal placement of training close to mass casualty events is not feasible. 12,13 Screen-based simulations are defined by the Society for Simulation in Healthcare as computer-generated video game simulators that create scenarios that require real-time decision making in a virtual environment. 14 The ability to access these simulations or games on any device with access to the Internet enables low-cost, time-efficient, and generalizable standardized training. ...
Objectives
Paramedics and emergency medical technicians (EMTs) perform triage at disaster sites. There is a need for disaster triage training. Live simulation training is costly and difficult to deliver. Screen‐based simulations may overcome these training barriers. We hypothesized that a screen‐based simulation, 60 Seconds to Survival (60S), would be associated with in‐game improvements in triage accuracy.
Methods
This was a prospective cohort study of a screen‐based simulation intervention, 60S. Participants included emergency medical services (EMS) personnel from 21 EMS agencies across 12 states. Participants performed assessments (e.g., check for pulse) and actions (e.g., reposition the airway) for 12 patients in each scenario and assigned color‐coded triage levels (red, yellow, green, or black) to each patient. Participants received on‐screen feedback about triage performance immediately after each scenario. A scoring system was designed to encourage accurate and timely triage decisions. Participants who played 60S included practicing EMTs, paramedics, and nurses as well as students studying to assume these roles. Participants played the game at least three times over 13 weeks.
Results
In total, 2,234 participants began game play and 739 completed the study and were included in the analysis. Overall, the median number of plays of the game was just above the threshold inclusion criteria (three or more plays) with a median of four plays during the study period (interquartile range [IQR] = 3–7). There was a significant difference in triage accuracy from the first play of the game to the last play of the game. Median baseline triage accuracy in the game was 89.7% (IQR = 82.1%–94.9%), which then increased to a median of 100% at the last game play (IQR = 87.5%–100.0%; p < 0.001). There was some variability in median triage accuracy on fourth through 11th game plays, ranging from 95% to 100%, and on the 12th to 16th plays, the median accuracy was sustained at 100%. There was a significant decrease in the rate of undertriage: from 10.3% (IQR = 5.1%–18.0%) to 0 (IQR = 0%–12.5%; p < 0.001).
Conclusion
60 Seconds to Survival is associated with improved in‐game triage accuracy. Further study of the correlation between in‐game triage accuracy and improvements in live simulation or real‐world triage decisions is warranted.
... Virtual worlds can be used to generate realistic representations of real-world locations, which can result in a considerably immersive user experience (Hall, Conboy-Hill & Taylor, 2011;Warburton, 2009). Two key advantages of using virtual worlds for health-related education are that (1) they can be used to create simulations that would be difficult or expensive to construct in real life (Cohen et al, 2013;Kulendran, Taylor, Taylor & Darzi, 2013) and (2) they can be used to connect geographically dispersed individuals (Taylor et al, 2013a). The main advantage of using virtual worlds to communicate electronically, in comparison with alternatives such as Skype TM is that users sharing the 3D environment can get a sense of "togetherness" from being in the "same" (albeit virtual) location (Ma & Agarwal, 2007): communicating in a virtual world can give users a subjective impression that they are sharing physical space with another person, who in fact is communicating from a different place entirely. ...
Recent advances in communication technologies enable potential provision of remote education for patients using computer-generated environments known as virtual worlds. Previous research has revealed highly variable levels of patient receptiveness to using information technologies for healthcare-related purposes. This preliminary study involved implementing a questionnaire investigating attitudes and access to computer technologies of respiratory outpatients in order to assess potential for use of virtual worlds to facilitate health-related education for this sample. Ninety-four patients with a chronic respiratory condition completed surveys, which were distributed at a Chest Clinic. In accordance with our prediction, younger participants were more likely to be able to use and have access to a computer, and some patients were keen to explore use virtual worlds for healthcare-related purposes: Of those with access to computer facilities, 14.50% expressed a willingness to attend a virtual world focus group. Results indicate that future virtual world health education facilities should be designed to cater to younger patients because this group is most likely to accept and use such facilities. Within the study sample, this is likely to comprise of people diagnosed with asthma. Future work could investigate the potential of creating a virtual world asthma education facility.
... However, to accurately measure the success of SBT, there is a need for performance measures to continuously assess the outcome of the training scenarios and give feed-back all along the play by providing a detailed learning-focused feedback and evaluation. Finally, scenarios must be realistic with no exaggeration and what happens before, during and after the training intervention must be considered equally critical in making a successful SBT [1,5,8,12,16,19,39,45,[49][50][51][52]. ...
Disasters and major incidents are inevitable, but can be mitigated by good
planning and education. The educational approach to emergencies must be
multidisciplinary and engage all organizations involved. It should consist of
ancillary material, but also practical sequences, in which participants may
exercise their knowledge, skills and competency individually and in group,
based on diverse scenarios.
In this short review, major types of simulation training are discussed with
regard to their effectiveness in teaching and knowledge retention.
Keywords: Education; Emergency medicine; Disaster management;
Simulation
... The difference in training level of the raters (1 attending physician and 1 PGY 3 resident for each assessment tool) is not likely to have contributed to the low interrater reliability for T-NOTECHS, as the NOTSS assessment was also performed by 1 resident and 1 attending physician and demonstrated strong interrater reliability. Our study is consistent with prior studies specifically using T-NOTECHS for team NTS assessment, which have demonstrated an interrater reliability of 0.59 for a simulated in-hospital trauma resuscitation scenario in a study 12 and 0.44 for a real-time assessment of simulated resuscitation and 0.71 for video assessment of simulated resuscitation in another study 9 ; a study using T-NOTECHS did not report interrater reliability scores. 10 Another potential limitation of the study is that the residents undergoing TTT were junior residents without extensive trauma resuscitation experience, and therefore, their response to the scenario may be different from that of more senior residents. ...
Trauma team training provides instruction on crisis management through debriefing and discussion of teamwork and leadership skills during simulated trauma scenarios. The effects of team leader's nontechnical skills (NTSs) on technical performance have not been thoroughly studied. We hypothesized that team's and team leader's NTSs correlate with technical performance of clinical tasks.
Retrospective cohort study.
Brigham and Women's Hospital, STRATUS Center for Surgical Simulation PARTICIPANTS: A total of 20 teams composed of surgical residents, emergency medicine residents, emergency department nurses, and emergency services assistants underwent 2 separate, high-fidelity, simulated trauma scenarios. Each trauma scenario was recorded on video for analysis and divided into 4 consecutive sections. For each section, 2 raters used the Non-Technical Skills for Surgeons framework to assess NTSs of the team. To evaluate the entire team's NTS, 2 additional raters used the Modified Non-Technical Skills Scale for Trauma system. Clinical performance measures including adherence to guidelines and time to perform critical tasks were measured independently.
NTSs performance by both teams and team leaders in all NTS categories decreased from the beginning to the end of the scenario (all p < 0.05). There was significant correlation between team's and team leader's cognitive skills and critical task performance, with correlation coefficients between 0.351 and 0.478 (p < 0.05). The NTS performance of the team leader highly correlated with that of the entire team, with correlation coefficients between 0.602 and 0.785 (p < 0.001).
The NTSs of trauma teams and team leaders deteriorate as clinical scenarios progress, and the performance of team leaders and teams is highly correlated. Cognitive NTS scores correlate with critical task performance. Increased attention to NTSs during trauma team training may lead to sustained performance throughout trauma scenarios. Decision making and situation awareness skills are critical for both team leaders and teams and should be specifically addressed to improve performance.
Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.
Simulation is a crucial tool for the development of surgical skills. Many important skills can be taught outside of the operating room in a low-risk environment, allowing for detailed feedback and assessment. With new technologies, such as virtual reality or augmented reality, simulation based training resembles more realistic scenarios with imbedded feedback and haptics. Robot assisted surgery also requires the use of simulation to develop competency. Technology and simulation are both critical for a surgeon in training.
Background:
Paramedics who perform patient triaging in case of a mass casualty incident generally receive triage training during their undergraduate educational program. Triage training can be facilitated using various simulation modalities together with theoretical training.
Objectives:
The aim of this study is to determine the effectiveness of online scenario-based Visually Enhanced Mental Simulation (VEMS) on developing the casualty triage and management skills of paramedic students.
Design:
The study was conducted using a single-group pre-test/post-test quasi-experimental design.
Settings and participants:
The study was carried out in October 2020 with 20 volunteer students studying in the First and Emergency Aid program of a university in Turkey.
Methods:
After the online theoretical crime scene management and triage course, students completed a demographic questionnaire and a pre-VEMS assessment. They then took part in the online VEMS training and eventually completed the post-VEMS assessment. At the end of the session, they filed an online survey concerning VEMS.
Results:
There is a statistically significant increase in the scores obtained by the students between the pre- and post-educational intervention assessment (p < 0.05). The majority of the students gave positive feedback concerning VEMS as an educational approach.
Conclusion:
The results show that online VEMS is effective in helping paramedic students acquire casualty triage and management skills and that students thought it was an effective educational approach.
Summary statement:
Disaster medicine (DM) training aims to recreate stressful, mass casualty scenarios faced by medical professionals in the field with high fidelity. Virtual (VR) and augmented reality (AR) are well suited to disaster training as it can provide a safe, socially distant simulation with a high degree of realism. The purpose of this literature review was to summarize the current use of VR or AR for simulation training of healthcare providers in DM education. A systematic review of peer-reviewed articles was performed from January 1, 2000, to November 21, 2020, on PubMed, Embase, and OVID. Exclusion criteria included non-English articles, computer-generated models without human participants, or articles not relating to DM, VR or AR. Thirty-two articles were included. Triage accuracy was evaluated in 17 studies. Participants reported improved confidence and positive satisfaction after the simulations. The studies suggest VR or AR can be considered for disaster training in addition to other, more traditional simulation methods. More research is needed to create a standardized educational model to incorporate VR and AR into DM training and to understand the relationship between disaster simulation and improved patient care.
Purpose
Conduct a scoping review to critically appraise the development and summarize the evidence on the measurement properties of T-NOTECHS including sensibility, reliability, and validity.
Methods
A literature search was performed using Pubmed and Ovid databases. Studies that described the development process of T-NOTECHS and primary studies that presented evidence of reliability and validity were identified and included. Measurement properties of T-NOTECHS was assessed and summarized under the following: scale development, sensibility, reliability, and validity.
Results
The literature search yielded 245 articles with 24 studies meeting inclusion criteria. The T-NOTECHS was developed with an acceptable robust methodology. It has good sensibility with adequate content, face validity, and feasibility. It is a reliable measure of non-technical skills in the setting of trauma video review, which improves with expert raters or extensive training. The T-NOTECHS is a valid discriminative and evaluative instrument that measures non-technical skills of multidisciplinary trauma teams.
Conclusions
T-NOTECHS provides reliable and valid measurements of non-technical skills of trauma teams, particularly when assessing trauma video review and non-technical skills training interventions by expert raters.
Objectives
To (1) develop a simulation software environment to conduct prehospital research during the COVID-19 pandemic on paramedics’ teamwork and use of mobile computing devices, and (2) establish its feasibility for use as a research and training tool.
Background
Simulation-based research and training for prehospital environments has typically used live simulation, with highly realistic equipment and technology-enhanced manikins. However, such simulations are expensive, difficult to replicate, and require facilitators and participants to be at the same location. Although virtual simulation tools exist for prehospital care, it is unclear how best to use them for research and training.
Methods
We present SPECTRa—Simulated Prehospital Emergency Care for Team Research—an online simulated prehospital environment that lets participants care concurrently for single or multiple patients remotely. Patient scenarios are designed using Laerdal’s SimDesigner. SPECTRa records data about scenario states and participants’ virtual interaction with the simulated patients. SPECTRa’s supporting environment records participants’ verbal communication and their visual and physical interactions with their interface and devices using Zoom conferencing and audiovisual recording. We discuss a pilot research implementation to assess SPECTRa’s feasibility.
Results
SPECTRa allows researchers to systematically test small-team interaction in single- or multipatient care scenarios and assess the impact of mobile devices on participants’ assessment and care of patients. SPECTRa also supports pedagogical features that could allow prehospital educators to provide individual trainees or teams with online simulation training and evaluation.
Conclusions
SPECTRa, an online tool for simulating prehospital patient care, shows potential for remote healthcare research and training.
Background:
The objective of this scoping review was to identify assessment tools of trauma team performance (outside of technical skills) and assess the validity and reliability of each tool in assessing trauma team performance.
Methods:
We searched Embase, Cochrane Library, Web of Science, Ovid Medline, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) from inception to June 1st, 2021. English studies that evaluated trauma team performance using nontechnical skill assessment tools in a simulation or real-world setting were included. Studies were assessed by two independent reviewers for meeting inclusion/exclusion criteria. Data regarding team assessment tools were extracted and synthesized into behaviour domains. Each tool was then assessed for validity and reliability.
Results:
The literature search returned 4215 articles with 29 meeting inclusion criteria. Our search identified 12 trauma team performance assessment tools. Most studies were conducted in the United States (n = 20, 69%). 20 studies (69%) assessed trauma team performance in a simulation setting; Team Assessment Measure (TEAM) and Trauma-Nontechnical Skills Scale (T-NOTECHS) were the only tools to be applied in a simulation and real-world setting. Most studies assessed trauma team performance using video review technology (n = 17, 59%). Five overarching themes were designed to encompass behavioural domains captured across the 12 tools: 1) Leadership; 2) Communication; 3) Teamwork; 4) Assessment; 5) Situation Awareness. The reliability and validity of T-NOTECHS was investigated by the greatest number of studies (n = 13); however, TEAM had the most robust evidence of reliability and validity.
Conclusions:
We identified 12 trauma team performance tools which assessed nontechnical skills to varying degrees. T-NOTECHS and TEAM tools had the most evidence to support their reliability and validity. Considering the limited research in the impact of trauma team performance on patient outcomes, future studies could utilize video review technology in authentic trauma cases to further study this important relationship.
Level of evidence:
Systematic Reviews & Meta-analyses, Level IV.
During emergency responses, public health leaders frequently serve in incident management roles that differ from their routine job functions. Leaders’ familiarity with incident management principles and functions can influence response outcomes. Therefore, training and exercises in incident management are often required for public health leaders. To describe existing methods of incident management training and exercises in the literature, we queried 6 English language databases and found 786 relevant articles. Five themes emerged: (1) experiential learning as an established approach to foster engaging and interactive learning environments and optimize training design; (2) technology-aided decision support tools are increasingly common for crisis decision-making; (3) integration of leadership training in the education continuum is needed for developing public health response leaders; (4) equal emphasis on competency and character is needed for developing capable and adaptable leaders; and (5) consistent evaluation methodologies and metrics are needed to assess the effectiveness of educational interventions.
These findings offer important strategic and practical considerations for improving the design and delivery of educational interventions to develop public health emergency response leaders. This review and ongoing real-world events could facilitate further exploration of current practices, emerging trends, and challenges for continuous improvements in developing public health emergency response leaders.
Introduction: In pandemic times, in which the “lockdown strategy” has been adopted, the use of innovations using technological resources such as the creation of instruments that can replace traditional teaching-learning methods in the training of health professionals is essential. Objective: the aim of this study was to develop and evaluate the usability of a realistic interactive simulation computer system using three-dimensional imaging technology and virtual reality with free-access computational tools available on the web. Methods: the development of a prototype (OSCE 3D) was based on the steps used for the construction of a “Serious Game” simulation software. The free-access version of the Unity Editor 3D platform (Unity Technologies, version 2018), used for developing educational games, the software GNU Image Manipulation Program (GIMP, version 2.10.12), Blender (version 2.79) and MakeHuman (version 1.1.1) were utilized for creating textures and building models of the 3D environments. An experimental phase was carried out to assess usability, through a questionnaire based on the System Usability Scale. The study was approved by the Research Ethics Committee of the institution and all participants signed the Informed Consent Form. Results: a total of 39 undergraduate medical students attending the 6th semester of a private university center of northeastern Brazil voluntarily participated in the evaluation of the OSCE 3D. The usability evaluation resulted in a mean score of 75.4 with a margin of error of 3.2, which is considered a good usability score according to the literature. Conclusions: this study allowed the development of a low-cost prototype, using a three-dimension realistic simulation system for clinical skills assessment. This product, even in the prototype phase, showed good usability.
Introduction: In pandemic times, in which the “lockdown strategy” has been adopted, the use of innovations using technological resources such as the creation of instruments that can replace traditional teaching-learning methods in the training of health professionals is essential. Objective: the aim of this study was to develop and evaluate the usability of a realistic interactive simulation computer system using three-dimensional imaging technology and virtual reality with free-access computational tools available on the web. Methods: the development of a prototype (OSCE 3D) was based on the steps used for the construction of a “Serious Game” simulation software. The free-access version of the Unity Editor 3D platform (Unity Technologies, version 2018), used for developing educational games, the software GNU Image Manipulation Program (GIMP, version 2.10.12), Blender (version 2.79) and MakeHuman (version 1.1.1) were utilized for creating textures and building models of the 3D environments. An experimental phase was carried out to assess usability, through a questionnaire based on the System Usability Scale. The study was approved by the Research Ethics Committee of the institution and all participants signed the Informed Consent Form. Results: a total of 39 undergraduate medical students attending the 6th semester of a private university center of northeastern Brazil voluntarily participated in the evaluation of the OSCE 3D. The usability evaluation resulted in a mean score of 75.4 with a margin of error of 3.2, which is considered a good usability score according to the literature. Conclusions: this study allowed the development of a low-cost prototype, using a three-dimension realistic simulation system for clinical skills assessment. This product, even in the prototype phase, showed good usability.
Sensor-based, mobile behavioral analytics have much potential for adaptive human-machine interactivity in team- and multiteam-based, live simulation training. This paper will explore a human-technology adaptive system where real-time data is generated from multiple sensor systems to inform multiteam-based training. Examples from first responder law enforcement training contexts will be discussed as well as the future potential of these sensor-based technologies to iteratively and adaptively inform both the smart technology system and the human system in a reciprocal learning cycle.
Background : In pandemic times where the “lockdown strategy” has been adopted, the use of innovations using technological resources such as the creation of instruments that can replace traditional teaching-learning methods in the training of health professionals is essential. The aim of this study was to develop and evaluate the usability of a realistic interactive simulation computer system using three-dimensional imaging technology and virtual reality with free-access computational tools available on the web.
Methods : the development of a prototype (OSCE 3D) was based on the steps used for the construction of simulation software of a "Serious Game". An experimental phase was carried out to assess usability, through a questionnaire based on the System Usability Scale. The study was approved by the Research Ethics Committee of the institution and all patients signed the Informed Consent Form.
Results : a total of 39 undergraduate medical students from the 6th semester of a private university center of northeast do Brazil voluntarily participated in the evaluation of the OSCE 3D. The usability evaluation presented a mean score of 75.4 with a margin of error of 3.2, considered a good usability according to the literature.
Conclusions : this work allowed the development of a low-cost prototype, using a three-dimension realistic simulation system for OSCE assessment stations. This product, even in the prototype phase, showed good usability.
Objectives
To summarize characteristics and commonalities of non-technical competency frameworks for health professionals in emergency and disaster.
Methods
An electronic literature search was conducted in PubMed, MEDLINE, ERIC, Scopus, Cochrane database, and Google Scholar to identify original English-language articles related to development, evaluation or application of the nontechnical competency frameworks. Reviewers assessed identified articles for exclusion/inclusion criteria and abstracted data on study design, framework characteristics, and reliability/validity evidence.
Results
Of the 9627 abstracts screened, 65 frameworks were identified from 94 studies that were eligible for result extraction. Sixty (63.8%) studies concentrated on clinical settings. Common scenarios of the studies were acute critical events in hospitals (44;46.8%) and nonspecified disasters (39;41.5%). Most of the participants (76; 80.9%) were clinical practitioners, and participants in 36 (38.3%) studies were multispecialty. Thirty-three (50.8%) and 42 (64.6%) frameworks had not reported evidence on reliability and validity, respectively. Fourteen of the most commonly involved domains were identified from the frameworks.
Conclusions
Nontechnical competency frameworks applied to multidisciplinary emergency health professionals are heterogeneous in construct and application. A fundamental framework with standardized terminology for the articulation of competency should be developed and validated so as to be accepted and adapted universally by health professionals in all-hazard emergency environment.
Global health security (GHS) is dependent upon having an adequate and prepared health security workforce. There are currently numerous challenges in establishing and maintaining a health security workforce. The frequency and magnitude of disasters have increased significantly over the past 30 years. Current and future GHS threats, both manmade and natural, require a prepared and flexible healthcare provider workforce ready to respond to current or emerging GHS threats. Developing and maintaining GHS -specific skills in the healthcare workforce is a tremendous logistical challenge. Innovative education technologies, including simulation and digital learning, can be leveraged to achieve preparedness for GHS threats.
Multiteam systems (MTSs) are comprised of two or more teams working toward shared superordinate goals but with unique subgoals. In large MTSs operating in extreme environments, coordination difficulties have repeatedly been found, which compromise response effectiveness. Research is needed that examines MTSs in situ within extreme environments to develop temporal theories of inter-team processes and understanding of how coordination may be improved within these challenging contexts. Live disaster exercises replicate the complexities of extreme environments, providing a valuable avenue for observing inter-team processes in situ. This article seeks to contribute to MTS research by highlighting (i) a mixed-method framework for collecting data during live disaster exercises that uses both inductive and deductive approaches to promote methodological and measurement fit; (ii) ways in which data can be collected and combined to meet the appropriate standards of their methodological class; and (iii) a case example of a National exercise.
Human error is one of the primary reasons for accidents in complex industries like aviation, nuclear power plant management, and health care. Physical and cognitive workload, flawed information processing, and poor decision making are some of the reasons that make humans vulnerable to error and lead to failures and accidents. In many accidents and failures, oftentimes, vulnerabilities that are embedded in the system, in the form of design deficiencies and poor human factors, lead to latent or catastrophic failures, but the last link is a human operator who gets blamed or worse, injured. This paper introduces an early design human performance assessment framework to identify what type of digital prototyping methodologies are appropriate to detect the deviation of the operator's performance due to an emergency condition. Fire in a civilian aircraft cockpit was introduced as a performance shaping factor (PSF). Ergonomics performance was evaluated using two prototyping strategies: (1) a computational prototyping framework includes digital human modeling (DHM) and computer-aided design; and (2) a novel mixed prototyping framework includes motion capture, DHM, and virtual reality. Results showed that the mixed prototyping framework can simulate emergency scenarios with increased realism and also has the potential to incorporate subjective aspects of ergonomics outcomes, overcoming the underlying lack of design knowledge in conventional early design methodologies.
This book is the proceedings of the International Workshop on A.I. in Security. The workshop was at ECML which was in Dublin, Ireland. The book contains 6 research articles and 2 system demos which address the use of A.I in security.
Immersive, 3D conferences are becoming viable using OpenSimulator, open source software. The act of planning for an immersive conference using the software dependent on the conference success strengthens the community of users that participate in the platform. This paper describes three conference events held from 2013–2015 involving an emerging consortium of leading developers and researchers of virtual worlds. The implications of technological success of immersive conferences hold promise for government and military agencies facing training requirements under fiscal restrictions. A workshop was conducted during the writing of this paper establishing the inaugural, immersive workshop for the Federal Consortium of Virtual Worlds sponsored by the US Army and Avacon Incorporated, a non-profit organization producing conference events.
Simulation is a powerful tool for education and quality improvement in emergency medicine, and simulation-based curricula demonstrate improved health care outcomes in select domains. Simulation-the use of trained actors, anatomic models, computer-based task trainers and mannequins, and virtual reality environments to create realistic care scenarios in a purely educational setting-can provide crucial training in the procedural, communications, and teamwork skills required to provide high-quality medical care. This chapter identifies the performance gaps in clinical training, the learning theories, and evaluation methods that support the efficacy of simulation-based education, and discusses how simulation training is currently being used to address the identified performance gaps in emergency medicine.
Unremitting natural disasters, deliberate threats, pandemics, and humanitarian suffering resulting from conflict situations necessitate swift and effective response paradigms. The European Union's (EU) increasing visibility as a disaster response enterprise suggests the need not only for financial contribution but also for instituting a coherent disaster response approach and management structure. The DITAC (Disaster Training Curriculum) project identified deficiencies in current responder training approaches and analyzed the characteristics and content required for a new, standardized European course in disaster management and emergencies.
Over 35 experts from within and outside the EU representing various organizations and specialties involved in disaster management composed the DITAC Consortium. These experts were also organized into 5 specifically tasked working groups. Extensive literature reviews were conducted to identify requirements and deficiencies and to craft a new training concept based on research trends and lessons learned. A pilot course and program dissemination plan was also developed.
The lack of standardization was repeatedly highlighted as a serious deficiency in current disaster training methods, along with gaps in the command, control, and communication levels. A blended and competency-based teaching approach using exercises combined with lectures was recommended to improve intercultural and interdisciplinary integration.
The goal of a European disaster management course should be to standardize and enhance intercultural and inter-agency performance across the disaster management cycle. A set of minimal standards and evaluation metrics can be achieved through consensus, education, and training in different units. The core of the training initiative will be a unit that presents a realistic situation "scenario-based training." (Disaster Med Public Health Preparedness. 2015;0:1-11).
Background:
Mass casualty incidents are unfortunately becoming more common. The coordination of mass casualty incident response is highly complex. Currently available options for training, however, are limited by either lack of realism or prohibitive expense and by a lack of assessment tools. Virtual worlds represent a potentially cost-effective, immersive, and easily accessible platform for training and assessment. The aim of this study was to assess feasibility of a novel virtual-worlds-based system for assessment and training in major incident response.
Methods:
Clinical areas were modeled within a virtual, online hospital. A major incident, incorporating virtual casualties, allowed multiple clinicians to simultaneously respond with appropriate in-world management and transfer plans within limits of the hospital's available resources. Errors, delays, and completed actions were recorded, as well as Trauma-NOnTECHnical Skills (T-NOTECHS) score. Performance was compared between novice and expert clinician groups.
Results:
Twenty-one subjects participated in three simulations: pilot (n = 7), novice (n = 8), and expert groups (n = 6). The novices committed more critical events than the experts, 11 versus 3, p = 0.006; took longer to treat patients, 560 (299) seconds versus 339 (321) seconds, p = 0.026; and achieved poorer T-NOTECHS scores, 14 (2) versus 21.5 (3.7), p = 0.003, and technical skill, 2.29 (0.34) versus 3.96 (0.69), p = 0.001. One hundred percent of the subjects thought that the simulation was realistic and superior to existing training options.
Conclusion:
A virtual-worlds-based model for the training and assessment of major incident response has been designed and validated. The advantages of customizability, reproducibility, and recordability combined with the low cost of implementation suggest that this potentially represents a powerful adjunct to existing training methods and may be applicable to further areas of surgery as well.
Background and aim
Following high profile errors resulting in patient harm and attracting negative publicity, the healthcare sector has begun to focus on training non-technical teamworking skills as one way of reducing the rate of adverse events. Within the area of resuscitation, two tools have been developed recently aiming to assess these skills – TEAM and OSCAR. The aims of the study reported here were:1.To determine the inter-rater reliability of the tools in assessing performance within the context of resuscitation.2.To correlate scores of the same resuscitation teams episodes using both tools, thereby determining their concurrent validity within the context of resuscitation.3.To carry out a critique of both tools and establish how best each one may be utilised.
Methods
The study consisted of two phases – reliability assessment; and content comparison, and correlation. Assessments were made by two resuscitation experts, who watched 24 pre-recorded resuscitation simulations, and independently rated team behaviours using both tools. The tools were critically appraised, and correlation between overall score surrogates was assessed.
Results
Both OSCAR and TEAM achieved high levels of inter-rater reliability (in the form of adequate intra-class coefficients) and minor significant differences between Wilcoxon tests. Comparison of the scores from both tools demonstrated a high degree of correlation (and hence concurrent validity). Finally, critique of each tool highlighted differences in length and complexity.
Conclusion
Both OSCAR and TEAM can be used to assess resuscitation teams in a simulated environment, with the tools correlating well with one another. We envisage a role for both tools – with TEAM giving a quick, global assessment of the team, but OSCAR enabling more detailed breakdown of the assessment, facilitating feedback, and identifying areas of weakness for future training.
The purpose of the trauma team is to provide advanced simultaneous care from relevant specialists to the seriously injured trauma patient. When functioning well, the outcome of the trauma team performance should be greater than the sum of its parts. Trauma teams have been shown to reduce the time taken for resuscitation, as well as time to CT scan, to emergency department discharge and to the operating room. These benefits are demonstrated by improved survival rates, particularly for the most severely injured patients, both within and outside of dedicated trauma centres. In order to ensure the best possible performance of the team, the leadership skills of the trauma team leader are essential and their non-technical skills have been shown to be particularly important. Team performance can be enhanced through a process of audit and assessment of the workings of the team and the evidence currently available suggests that this is best facilitated through the process of video review of the trauma resuscitation. The use of human patient simulators to train and assess trauma teams is becoming more commonplace and this technique offers a safe environment for the future education of trauma team staff.
Trauma teams are a key component of most programmes which set out to improve trauma care. This article reviews the background of trauma teams, the evidence for benefit and potential techniques of performance assessment. The review was written after a PubMed, Ovid, Athens, Cochrane and guideline literature review of English language articles on trauma teams and their performance and hand searching of references from the relevant searched articles.
In order to prepare the healthcare system and healthcare personnel to meet the health needs of populations affected by disasters, educational programs have been developed by numerous academic institutions, hospitals, professional organizations, governments, and non-government organizations. Lacking standards for best practices as a foundation, many organizations and institutions have developed "core competencies" that they consider essential knowledge and skills for disaster healthcare personnel.
The Nursing Section of the World Association for Disaster and Emergency Medicine (WADEM) considered the possibility of endorsing an existing set of competencies that could be used to prepare nurses universally to participate in disaster health activities. This study was undertaken for the purpose of reviewing published disaster health competencies to determine commonalities and universal applicability for disaster preparedness.
In 2007, a review of the electronic literature databases was conducted using the major keywords: disaster response competencies; disaster preparedness competencies; emergency response competencies; disaster planning competencies; emergency planning competencies; public health emergency preparedness competencies; disaster nursing competencies; and disaster nursing education competencies. A manual search of references and selected literature from public and private sources also was conducted. Inclusion criteria included: English language; competencies listed or specifically referred to; competencies relevant to disaster, mass-casualty incident (MCI), or public health emergency; and competencies relevant to healthcare.
Eighty-six articles were identified; 20 articles failed to meet the initial inclusion criteria; 27 articles did not meet the additional criteria, leaving 39 articles for analysis. Twenty-eight articles described competencies targeted to a specific profession/discipline, while 10 articles described competencies targeted to a defined role or function during a disaster. Four of the articles described specific competencies according to skill level, rather than to a specific role or function. One article defined competencies according to specific roles as well as proficiency levels. Two articles categorized disaster nursing competencies according to the phases of the disaster management continuum. Fourteen articles described specified competencies as "core" competencies for various target groups, while one article described "cross-cutting" competencies applicable to all healthcare workers.
Hundreds of competencies for disaster healthcare personnel have been developed and endorsed by governmental and professional organizations and societies. Imprecise and inconsistent terminology and structure are evident throughout the reviewed competency sets. Universal acceptance and application of these competencies are lacking and none have been validated. Further efforts must be directed to developing a framework and standardized terminology for the articulation of competency sets for disaster health professionals that can by accepted and adapted universally.
Medical teams depend on technical skills (TS) as well as non-technical skills (NTS) for successful management of critical events. Simulated scenarios are an opportunity for presentation of similar crisis situations. The aim of this study was to test whether TS and NTS are assessable with satisfactory interrater reliability (IRR) during a regular paramedic training.
Thirty paramedics were rated by two independent observers using video-recording and previously validated checklists while managing two simulated emergency scenarios as a team of two. The observed items of the team's TS included type, order, and time of adequate medical care. The NTS were restricted to six team-oriented dimensions. The IRR was quantified by calculating the intraclass correlation coefficient (ICC). The z-transformed values of the TS and NTS were correlated by Pearson's correlation. Internal consistency was controlled using Cronbach's alpha.
The average measures ICC for the IRR was between 0.97 [95% confidence interval (CI) 0.91-0.99] and 0.98 (95% CI 0.94-0.99) for the TS sum-score, and was 0.94 (95% CI 0.87-0.97) for the NTS sum-score; the Cronbach's alpha of this NTS sum-score was 0.86. There is a positive correlation between the normalised TS and NTS sum-scores (r=0.53; P<0.05).
Assessment of TS and NTS is feasible and reliable during paramedic training in emergency scenarios. TS can be reliably assessed by one trained observer; for NTS, two trained raters provide a suitable condition for excellent observations. There is a significant positive correlation between TS and NTS.
Hospitals the world over have been involved in disasters, both internal and external. These two types of disasters are independent, but not mutually exclusive. Internal disasters are isolated to the hospital and occur more frequently than do external disasters. External disasters affect the community as well as the hospital. This paper first focuses on common problems encountered during acute-onset disasters, with regards to hospital operations and caring for victims. Specific injury patterns commonly seen during natural disasters are reviewed. Second, lessons learned from these common problems and their application to hospital disaster plans are reviewed.
An extensive review of the available literature was conducted using the computerized databases Medline and Healthstar from 1977 through March 1999. Articles were selected if they contained information pertaining to a hospital response to a disaster situation or data on specific disaster injury patterns. Selected articles were read, abstracted, analyzed, and compiled.
Hospitals continually have difficulties and failures in several major areas of operation during a disaster. Common problem areas identified include communication and power failures, water shortage and contamination, physical damage, hazardous material exposure, unorganized evacuations, and resource allocation shortages.
Lessons learned from past disaster-related operational failures are compiled and reviewed. The importance and types of disaster planning are reviewed.
Recently, mass-casualty incident (MCI) preparedness and training has received increasing attention at the hospital level.
To review the existing evidence on the effectiveness of disaster drills, technology-based interventions and tabletop exercises in training hospital staff to respond to an MCI.
A systematic, evidence-based process was conducted incorporating expert panel input and a literature review with the key terms: "mass casualty", "disaster", "disaster planning", and "drill". Paired investigators reviewed citation abstracts to identify articles that included evaluation of disaster training for hospital staff. Data were abstracted from the studies (e.g., MCI type, training intervention, staff targeted, objectives, evaluation methods, and results). Study quality was reviewed using standardized criteria.
Of 243 potentially relevant citations, 21 met the defined criteria. Studies varied in terms of targeted staff, learning objectives, outcomes, and evaluation methods. Most were characterized by significant limitations in design and evaluation methods. Seventeen addressed the effectiveness of disaster drills in training hospital staff in responding to an MCI, four addressed technology-based interventions, and none addressed tabletop exercises. The existing evidence suggests that hospital disaster drills are effective in allowing hospital employees to become familiar with disaster procedures, identify problems in different components of response (e.g., incident command, communications, triage, patient flow, materials and resources, and security) and provide the opportunity to apply lessons learned to disaster response. The strength of evidence on other training methods is insufficient to draw valid recommendations.
Current evidence on the effectiveness of MCI training for hospital staff is limited. A number of studies suggest that disaster drills can be effective in training hospital staff. However, more attention should be directed to evaluating the effectiveness of disaster training activities in a scientifically rigorous manner.
Although Britain is no stranger to terrorist attacks, the London attacks of July 7 represent a shift to a new scale and a new modus operandi. Drs. Jim Ryan and Hugh Montgomery write that the attacks were unprecedented in scale and severity for London, but all the emergency services had prepared extensively for such attacks.
1969 to 2003, 34 years.
Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus.
Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.
The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the 'grey literature' were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality.
Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention.
Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol.
Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis. HEADLINE RESULTS: Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following: providing feedback--51 (47%) journal articles reported that educational feedback is the most important feature of simulation-based medical education; repetitive practice--43 (39%) journal articles identified repetitive practice as a key feature involving the use of high-fidelity simulations in medical education; curriculum integration--27 (25%) journal articles cited integration of simulation-based exercises into the standard medical school or postgraduate educational curriculum as an essential feature of their effective use; range of difficulty level--15 (14%) journal articles address the importance of the range of task difficulty level as an important variable in simulation-based medical education; multiple learning strategies--11 (10%) journal articles identified the adaptability of high-fidelity simulations to multiple learning strategies as an important factor in their educational effectiveness; capture clinical variation--11 (10%) journal articles cited simulators that capture a wide variety of clinical conditions as more useful than those with a narrow range; controlled environment--10 (9%) journal articles emphasized the importance of using high-fidelity simulations in a controlled environment where learners can make, detect and correct errors without adverse consequences; individualized learning--10 (9%) journal articles highlighted the importance of having reproducible, standardized educational experiences where learners are active participants, not passive bystanders; defined outcomes--seven (6%) journal articles cited the importance of having clearly stated goals with tangible outcome measures that will more likely lead to learners mastering skills; simulator validity--four (3%) journal articles provided evidence for the direct correlation of simulation validity with effective learning.
While research in this field needs improvement in terms of rigor and quality, high-fidelity medical simulations are educationally effective and simulation-based education complements medical education in patient care settings.
In high-risk industries such as aviation, the skills not related directly to technical expertise, but crucial for maintaining safety (e.g. teamwork), have been categorized as non-technical skills. Recently, research in anaesthesia has identified and developed a taxonomy of the non-technical skills requisite for safety in the operating theatre. Although many of the principles related to performance and safety within anaesthesia are relevant to the intensive care unit (ICU), relatively little research has been done to identify the non-technical skills required for safe practice within the ICU. This review focused upon critical incident studies in the ICU, in order to examine whether the contributory factors identified as underlying the critical incidents, were associated with the skill categories (e.g. task management, teamwork, situation awareness and decision making) outlined in the Anaesthetists' Non-technical Skills (ANTS) taxonomy. We found that a large proportion of the contributory factors underlying critical incidents could be attributed to a non-technical skill category outlined in the ANTS taxonomy. This is informative both for future critical incident reporting, and also as an indication that the ANTS taxonomy may provide a good starting point for the development of a non-technical skills taxonomy for intensive care. However, the ICU presents a range of unique challenges to practitioners working within it. It is therefore necessary to conduct further non-technical skills research, using human factors techniques such as root-cause analyses, observation of behaviour, attitudinal surveys, studies of cognition, and structured interviews to develop a better understanding of the non-technical skills important for safety within the ICU. Examples of such research highlight the utility of these techniques.
There is a dire need to have complementary form of disaster training which is cost effective, relatively easy to conduct, comprehensive, effective and acceptable. This will complement field drills training. A classroom-based training and simulation module was built by combining multiple tools: Powerpoint lectures, simulations utilising the Kuala Lumpur International Airport (KLIA) schematic module into 'floortop' model and video show of previous disaster drill. 76 participants made up of medical responders, categorised as Level 1 (specialists and doctors), Level 2 (paramedics), Level 3 (assistant paramedics) and Level 4 (health attendants and drivers) were trained using this module. A pre-test with validated questions on current airport disaster plans was carried out before the training. At the end of training, participants answered similar questions as post-test. Participants also answered questionnaire for assessment of training's acceptance. There was a mean rise from 47.3 (18.8%) to 84.0 (18.7%) in post-test (p<0.05). For Levels 1, 2, 3 and 4 the scores were 94.8 (6.3)%, 90.1 (11)%, 80.3 (20.1)% and 65 (23.4)% respectively. Nevertheless Level 4 group gained most increase in knowledge rise from baseline pre-test score (51.4%). Feedback from the questionnaire showed that the training module was highly acceptable. A classroom-based training can be enhanced with favourable results. The use of classroom training and simulation effectively improves the knowledge of disaster plan significantly on the back of its low cost, relatively-easy to conduct, fun and holistic nature. All Levels of participants (from specialists to drivers) can be grouped together for training. Classroom training and simulation can overcome the problem of "dead-document" phenomenon or "paper-plan syndrome".
Since 2001, state and local health departments in the United States (US) have accelerated efforts to prepare for high-impact public health emergencies. One component of these activities has been the development and conduct of exercise programs to assess capabilities, train staff and build relationships. This paper summarizes lessons learned from tabletop exercises about public health emergency preparedness and about the process of developing, conducting, and evaluating them.
We developed, conducted, and evaluated 31 tabletop exercises in partnership with state and local health departments throughout the US from 2003 to 2006. Participant self evaluations, after action reports, and tabletop exercise evaluation forms were used to identify aspects of the exercises themselves, as well as public health emergency responses that participants found more or less challenging, and to highlight lessons learned about tabletop exercise design.
Designing the exercises involved substantial collaboration with representatives from participating health departments to assure that the scenarios were credible, focused attention on local preparedness needs and priorities, and were logistically feasible to implement. During execution of the exercises, nearly all health departments struggled with a common set of challenges relating to disease surveillance, epidemiologic investigations, communications, command and control, and health care surge capacity. In contrast, performance strengths were more varied across participating sites, reflecting specific attributes of individual health departments or communities, experience with actual public health emergencies, or the emphasis of prior preparedness efforts.
The design, conduct, and evaluation of the tabletop exercises described in this report benefited from collaborative planning that involved stakeholders from participating health departments and exercise developers and facilitators from outside the participating agencies. While these exercises identified both strengths and vulnerabilities in emergency preparedness, additional work is needed to develop reliable metrics to gauge exercise performance, inform follow-up action steps, and to develop re-evaluation exercise designs that assess the impact of post-exercise interventions.
To compare the effectiveness of serious games technology and standard small group techniques in the teaching of major incident triage.
Pragmatic controlled trial.
Learners on a standard short major incident (MIMMS) course were allocated to receiving triage training by playing a serious game or by attending a standard small group practical workshop. The triage trainer (game group) learned by …
The increase in stress related medical leave did not occur in large numbers until months after the September 11 attacks. Repeated exposures at the site and the increasing number of funerals and memorial services that firefighters attended during the next 11 months might have contributed to stress related problems. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Introductory Sentence. We hypothesize that resident education is inadequate with respect to management of multiple/mass casualty incidents that may involve chemical, biological, and nuclear exposures. Methods. After IRB approval, chief level residents in Surgery (10), Emergency Medicine (10), and Anesthesia (8) at our university based level-one trauma center were asked to anonymously complete a survey questionnaire. Responses were tabulated and statistically analyzed with Mann-Whitney rank sum, Fisher exact test, and Kruskal-Wallis one-way ANOVA. Results. Every chief resident responded. All the residents were similar with respect to age, sex, and intended setting of clinical practice. Only a single resident reported military experience. Two residents (7.1%), including the one with military experience, had administered medical care while wearing a protective suit. Compared to emergency medicine residents, surgical residents reported significantly less formal teaching in mass casualty incidents (P = 0.0249), trauma triage (P = 0.0139), and nuclear, biological, chemical agents (P = 0.0027). When surgical residents were compared to anesthesia residents, there was significantly less training for surgical residents in nuclear, chemical, and biological agents (P = 0.0295). Multiple/mass casualty incident experience did not differ between residents. However, the most common incidents involved only 3–5 patients with blunt trauma. Emergency medicine residents were significantly more comfortable in treating patients with exposure to anthrax (P = 0.0110), sarin (P = 0.048), and nuclear exposure (P = 0.0108). Conclusions. Surgical residents have significantly less formal training in mass casualties, triage, and chemical, biological, and nuclear exposures than residents in other specialties. Therefore, surgical residents are less comfortable treating these types of patients. Since surgeons are often expected to take leadership roles in mass casualty incidents, surgical education should be modified to match or exceed that of other residents.
Objectives To create, develop, test and evaluate an interactive system for education and training in disaster medicine, adjustable to any organization and giving an objective assessment of training and testing. Methods A system that is simple to use and does not require sophisticated equipment was developed, based on magnetic symbols representing patients, staff and resources, movable markers indicating priority and treatment, and a large patient-bank with a protocol giving results of treatment based on trauma score. The methodology was tested in two models: classroom mode and advanced interactive mode, at all levels, regionally, nationally and internationally. Results During 1987-2001, a total of 8777 students were trained with the system with the classroom mode in 94 regional, 37 national and 8 international courses. The advanced interactive mode was used for 2518 students in 14 regional, 22 national and 18 international courses. The students' evaluation of the quality of the methodology based on a scoring system of 1-5 was 4.32 ± 0.19 for regional courses, 4.70 ± 0.12 for national courses and 4.58 ± 0.16 for international courses. The students' subjective appreciation of accuracy of the methodology was 4.58 ± 0.16 for national courses and 4.65 ± 0.13 for international courses. Corresponding figures for the advanced interactive mode were 4.78 ± 0.12 for national and 4.81 ± 0.11 for international courses for the methodology and 4.70 ± 0.13 for national and 4.65 ± 0.17 for international courses for evaluation of accuracy. Conclusion The methodology was well received and the subjective evaluation of the participants on all levels showed very high scores. On the international level many participants had extensive experience. An objective evaluation of accuracy (validity) of the methodology should be carried out as a next step parallel to further utilization of the considerable potential of this methodology.
Background:
There is growing evidence that health systems in developed countries are poorly prepared to deal with major incidents.
Study objectives:
This study aimed to determine the skills required for successful major incident response, the factors that contribute to a successful major incident exercise, and whether there is a role for using novel simulation training (virtual worlds) in preparing for major incidents.
Methods:
This was a qualitative semi-structured interview study. Fourteen health care staff with experience of major incident planning and training in the United Kingdom were recruited. Interviews were content-analyzed to identify emergent themes.
Results:
The aims and benefits of current exercises were categorized into three major themes: Organizational, Interpersonal, and Cognitive. Participants felt that the main objective of current exercises is to see how a major incident plan is implemented, rather than training individual staff. Communications was the most frequently commented-on area requiring improvement. Participants felt that lack of constructive feedback reduced the effectiveness of the exercises. All participants commented that virtual worlds technology could be successfully utilized for training. The creation of an immersive environment, increased training opportunity, and improved participant feedback were thought to be amongst the greatest benefits.
Conclusion:
There are clear deficiencies with current major incident preparation. Utilizing virtual worlds technology as an adjunct to existing exercises could improve training and response in the future.
Objectives:
To determine the feasibility of evidence-based design and use of low-cost virtual world environments for preparation and training in multi-agency, multi-site, major incident response.
Methods:
A prospective cohort feasibility study was carried out. One pre-hospital, and two in-hospital major incident scenarios, were created in an accessible virtual world environment. 23 pre-hospital and hospital-based clinicians each took part in one of three linked major incident scenarios: a pre-hospital bomb blast site, focusing on the roles of the team leader and triage person; a blast casualty in a resuscitation room, focusing on the role of the trauma team leader; a hospital command and control scenario focusing on the role of the clinical major incident co-ordinator/silver commander. Participants supplied both quantitative and qualitative feedback.
Results:
Using a systematic, evidence-based approach, three scenarios were successfully developed and tested using low-cost virtual worlds (Second Life and OpenSimulator). All scenarios were run to completion. 95% of participants expressed a desire to use virtual environments for future training and preparation. Pre-hospital responders felt that the immersive virtual environment enabled training in surroundings that would be inaccessible in real-life.
Conclusions:
The feasibility and face/content validity of using low-cost virtual worlds for multi-agency major incident simulation has been established. Major incident planners and trainers should explore utilising this technology as an adjunct to existing methodologies. Future work will involve development of robust assessment metrics.
Failures in nontechnical and teamwork skills frequently lie at the heart of harm and near-misses in the operating room (OR). The purpose of this systematic review was to assess the impact of nontechnical skills on technical performance in surgery.
MEDLINE, EMBASE, PsycINFO databases were searched, and 2,041 articles were identified. After limits were applied, 341 articles were retrieved for evaluation. Of these, 28 articles were accepted for this review. Data were extracted from the articles regarding sample population, study design and setting, measures of nontechnical skills and technical performance, study findings, and limitations.
Of the 28 articles that met inclusion criteria, 21 articles assessed the impact of surgeons' nontechnical skills on their technical performance. The evidence suggests that receiving feedback and effectively coping with stressful events in the OR has a beneficial impact on certain aspects of technical performance. Conversely, increased levels of fatigue are associated with detriments to surgical skill. One article assessed the impact of anesthesiologists' nontechnical skills on anesthetic technical performance, finding a strong positive correlation between the 2 skill sets. Finally, 6 articles assessed the impact of multiple nontechnical skills of the entire OR team on surgical performance. A strong relationship between teamwork failure and technical error was empirically demonstrated in these studies.
Evidence suggests that certain nontechnical aspects of performance can enhance or, if lacking, contribute to deterioration of surgeons' technical performance. The precise extent of this effect remains to be elucidated.
A modified nontechnical skills (NOTECHS) scale for trauma (T-NOTECHS) was developed to teach and assess teamwork skills of multidisciplinary trauma resuscitation teams. In this study, T-NOTECHS was evaluated for reliability and correlation with clinical performance.
Interrater reliability (intraclass correlation coefficient) and correlation with the speed and completeness of resuscitation tasks were assessed during simulation-based teamwork training and during actual trauma resuscitations.
For T-NOTECHS ratings done in real time, intraclass correlation coefficients were .44 for simulated and .48 for actual resuscitations. Reliability was higher (intraclass correlation coefficient = .71) for video review of resuscitations. Better T-NOTECHS scores were correlated with better performance during simulations, evidenced by a greater number of completed resuscitation tasks (r = .50, P < .01) and faster time to completion (r = -.38, P < .05) In actual resuscitations, T-NOTECHS ratings improved after teamwork training (P < .001). Higher T-NOTECHS scores were correlated with better clinical performance, evidenced by faster resuscitation (r = -.13, P < .05) and fewer unreported resuscitation tasks (r = -.16, P < .05).
Improvement in T-NOTECHS scores after teamwork training, and correlation with clinical parameters in simulated and actual trauma resuscitations, suggest its clinical relevance. Further evaluation, aiming to improve reliability, may be warranted.
The training of medical personnel to provide care for disaster victims is a priority for the physician community, the federal government, and society as a whole. Course development for such training guided by well-accepted standardized core competencies is lacking, however. This project identified a set of core competencies and performance objectives based on the knowledge, skills, and attitudes required by the specific target audience (emergency department nurses, emergency physicians, and out-of-hospital emergency medical services personnel) to ensure they can treat the injuries and illnesses experienced by victims of disasters regardless of cause. The core competencies provide a blueprint for the development or refinement of disaster training courses. This expert consensus project, supported by a grant from the Robert Wood Johnson Foundation, incorporated an all-hazard, comprehensive emergency management approach addressing every type of disaster to minimize the effect on the public's health. An instructional systems design process was used to guide the development of audience-appropriate competencies and performance objectives. Participants, representing multiple academic and provider organizations, used a modified Delphi approach to achieve consensus on recommendations. A framework of 19 content categories (domains), 19 core competencies, and more than 90 performance objectives was developed for acute medical care personnel to address the requirements of effective all-hazards disaster response. Creating disaster curricula and training based on the core competencies and performance objectives identified in this article will ensure that acute medical care personnel are prepared to treat patients and address associated ramifications/consequences during any catastrophic event.
Accurate assessment is imperative for learning, feedback and progression. The aim of this study was to examine whether surgeons can accurately self-assess their technical and nontechnical skills compared with expert faculty members' assessments.
Twenty-five surgeons performed a laparoscopic cholecystectomy (LC) in a simulated operating room. Technical and nontechnical performance was assessed by participants and faculty members using the validated Objective Structured Assessment of Technical Skills (OSATS) and the Non-Technical Skills for Surgeons scale (NOTSS).
Assessment of technical performance correlated between self and faculty members' ratings for experienced (median score, 30.0 vs 31.0; ρ = .831; P = .001) and inexperienced (median score, 22.0 vs 28.0; ρ = .761; P = .003) surgeons. Assessment of nontechnical skills between self and faculty members did not correlate for experienced surgeons (median score, 8.0 vs 10.5; ρ = -.375; P = .229) or their more inexperienced counterparts (median score, 9.0 vs 7.0; ρ = -.018; P = .953).
Surgeons can accurately self-assess their technical skills in virtual reality LC. Conversely, formal assessment with faculty members' input is required for nontechnical skills, for which surgeons lack insight into their behaviours.
This study assessed the implementation of a theater induction curriculum through a didactic lecture, an online Second Life operating room, and a simulated operating suite.
Sixty operating room novices were randomized into 4 groups: control (n = 15), didactic lecture (n = 15), Second Life (n = 15), and simulated operating suite (n = 15). The study followed a pretest and posttest design with a training intervention between operating room attendances. Outcome measures were knowledge, skills, and attitudes, measured using observed behavior and a self-report scale, with knowledge further assessed using multiple-choice questionnaires.
The lecture, Second Life, and simulated operating suite groups demonstrated significant improvements in all outcome measures. After the intervention, these 3 groups had significantly higher behavior (P < .001), self-report (P < .05), and knowledge (P < .05) scores than the control group.
This study demonstrates the value of delivering a theater induction curriculum for operating room preparation.
Training emergency personnel on the clinical management of a mass-casualty incident (MCI) with prior chemical, biological, radioactive, nuclear, or explosives (CBRNE) -exposed patients is a component of hospital preparedness procedures.
The objective of this research was to determine whether a Virtual Emergency Department (VED), designed after the Stanford University Medical Center's Emergency Department (ED) and populated with 10 virtual patient victims who suffered from a dirty bomb blast (radiological) and 10 who suffered from exposure to a nerve toxin (chemical), is an effective clinical environment for training ED physicians and nurses for such MCIs.
Ten physicians with an average of four years of post-training experience, and 12 nurses with an average of 9.5 years of post-graduate experience at Stanford University Medical Center and San Mateo County Medical Center participated in this IRB-approved study. All individuals were provided electronic information about the clinical features of patients exposed to a nerve toxin or radioactive blast before the study date and an orientation to the "game" interface, including an opportunity to practice using it immediately prior to the study. An exit questionnaire was conducted using a Likert Scale test instrument.
Among these 22 trainees, two-thirds of whom had prior Code Triage (multiple casualty incident) training, and one-half had prior CBRNE training, about two-thirds felt immersed in the virtual world much or all of the time. Prior to the training, only four trainees (18%) were confident about managing CBRNE MCIs. After the training, 19 (86%) felt either "confident" or "very confident", with 13 (59%) attributing this change to practicing in the virtual ED. Twenty-one (95%) of the trainees reported that the scenarios were useful for improving healthcare team skills training, the primary objective for creating them. Eighteen trainees (82%) believed that the cases also were instructive in learning about clinical skills management of such incidents.
These data suggest that training healthcare teams in online, virtual environments with dynamic virtual patients is an effective method of training for management of MCIs, particularly for uncommonly occurring incidents.
By exploiting video games technology, serious games strive to deliver affordable, accessible and usable interactive virtual worlds, supporting applications in training, education, marketing and design. The aim of the present study was to evaluate the effectiveness of such a serious game in the teaching of major incident triage by comparing it with traditional training methods.
Pragmatic controlled trial.
During Major Incident Medical Management and Support Courses, 91 learners were randomly distributed into one of two training groups: 44 participants practiced triage sieve protocol using a card-sort exercise, whilst the remaining 47 participants used a serious game. Following the training sessions, each participant undertook an evaluation exercise, whereby they were required to triage eight casualties in a simulated live exercise. Performance was assessed in terms of tagging accuracy (assigning the correct triage tag to the casualty), step accuracy (following correct procedure) and time taken to triage all casualties. Additionally, the usability of both the card-sort exercise and video game were measured using a questionnaire.
Tagging accuracy by participants who underwent the serious game training was significantly higher than those who undertook the card-sort exercise [Chi2=13.126, p=0.02]. Step accuracy was also higher in the serious game group but only for the numbers of participants that followed correct procedure when triaging all eight casualties [Chi2=5.45, p=0.0196]. There was no significant difference in time to triage all casualties (card-sort=435+/-74 s vs video game=456+/-62 s, p=0.155).
Serious game technologies offer the potential to enhance learning and improve subsequent performance when compared to traditional educational methods.
As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise.
The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.
Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self reported scores and external evaluators' scores ranged from moderate to good.
The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.