Measuring team performance in simulation-based training: adopting best practices for healthcare.

Department of Psychology, University of Central Florida, Florida, USA.
Simulation in healthcare: journal of the Society for Simulation in Healthcare (Impact Factor: 1.64). 02/2008; 3(1):33-41. DOI: 10.1097/SIH.0b013e3181626276
Source: PubMed

ABSTRACT Team performance measurement is a critical and frequently overlooked component of an effective simulation-based training system designed to build teamwork competencies. Quality team performance measurement is essential for systematically diagnosing team performance and subsequently making decisions concerning feedback and remediation. However, the complexities of team performance pose a challenge to effectively measuring team performance. This article synthesizes the scientific literature on this topic and provides a set of best practices for designing and implementing team performance measurement systems in simulation-based training.

1 Bookmark
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: As a conceptual review, this paper will debate relevant learning theories to inform the development, design and delivery of an effective educational programme for simulated team training relevant to health professionals. Kolb's experiential learning theory is used as the main conceptual framework to define the sequence of activities. Dewey's theory of reflective thought and action, Jarvis modification of Kolb's learning cycle and Schon's reflection-on-action serve as a model to design scenarios for optimal concrete experience and debriefing for challenging participants' beliefs and habits. Bandura's theory of self-efficacy and newer socio-cultural learning models outline that for efficient team training, it is mandatory to introduce the social-cultural context of a team. The ideal simulated team training programme needs a scenario for concrete experience, followed by a debriefing with a critical reflexive observation and abstract conceptualisation phase, and ending with a second scenario for active experimentation. Let them re-experiment to optimise the effect of a simulated training session. Challenge them to the edge: The scenario needs to challenge participants to generate failures and feelings of inadequacy to drive and motivate team members to critical reflect and learn. Not experience itself but the inadequacy and contradictions of habitual experience serve as basis for reflection. Facilitate critical reflection: Facilitators and group members must guide and motivate individual participants through the debriefing session, inciting and empowering learners to challenge their own beliefs and habits. To do this, learners need to feel psychological safe. Let the group talk and critical explore. Motivate with reality and context: Training with multidisciplinary team members, with different levels of expertise, acting in their usual environment (in-situ simulation) on physiological variables is mandatory to introduce cultural context and social conditions to the learning experience. Embedding in situ team training sessions into a teaching programme to enable repeated training and to assess regularly team performance is mandatory for a cultural change of sustained improvement of team performance and patient safety.
    BMC Medical Education 04/2014; 14(1):69. · 1.41 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: To determine the impact of a low-resource-demand, easily disseminated computer-based teamwork process training intervention on teamwork behaviors and patient care performance in code teams. A randomized comparison trial of computer-based teamwork training versus placebo training was conducted from August 2010 through March 2011. This study was conducted at the simulation suite within the Kado Family Clinical Skills Center, Wayne State University School of Medicine. Participants (n = 231) were fourth-year medical students and first-, second-, and third-year emergency medicine residents at Wayne State University. Each participant was assigned to a team of four to six members (nteams = 45). Teams were randomly assigned to receive either a 25-minute computer-based training module targeting appropriate resuscitation teamwork behaviors or a placebo training module. Teamwork behaviors and patient care behaviors were video recorded during high-fidelity simulated patient resuscitations and coded by trained raters blinded to condition assignment and study hypotheses. Teamwork behavior items (e.g., "chest radiograph findings communicated to team" and "team member assists with intubation preparation") were standardized before combining to create overall teamwork scores. Similarly, patient care items ("chest radiograph correctly interpreted"; "time to start of compressions") were standardized before combining to create overall patient care scores. Subject matter expert reviews and pilot testing of scenario content, teamwork items, and patient care items provided evidence of content validity. When controlling for team members' medically relevant experience, teams in the training condition demonstrated better teamwork (F [1, 42] = 4.81, p < 0.05; ηp = 10%) and patient care (F [1, 42] = 4.66, p < 0.05; ηp = 10%) than did teams in the placebo condition. Computer-based team training positively impacts teamwork and patient care during simulated patient resuscitations. This low-resource team training intervention may help to address the dissemination and sustainability issues associated with larger, more costly team training programs.
    Critical care medicine 08/2013; · 6.37 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Teamwork is a critical component during critical events. Assessment is mandatory for remediation and to target training programmes for observed performance gaps. The primary purpose was to test the feasibility of team-based self-monitoring of crisis resource management with a validated teamwork assessment tool. A secondary purpose was to assess item-specific reliability and content validity in order to develop a modified context-optimised assessment tool.We conducted a prospective, single-centre study to assess team-based self-monitoring of teamwork after in-situ inter-professional simulated critical events by comparison with an assessment by observers. The Mayo High Performance Teamwork Scale (MHPTS) was used as the assessment tool with evaluation of internal consistency, item-specific consensus estimates for agreement between participating teams and observers, and content validity. 105 participants and 58 observers completed the MHPTS after a total of 16 simulated critical events over 8 months. Summative internal consistency of the MHPTS calculated as Cronbach's alpha was acceptable with 0.712 for observers and 0.710 for participants. Overall consensus estimates for dichotomous data (agreement/non-agreement) was 0.62 (Cohen's kappa; IQ-range 0.31-0.87). 6/16 items had excellent (kappa > 0.8) and 3/16 good reliability (kappa > 0.6). Short questions concerning easy to observe behaviours were more likely to be reliable. The MHPTS was modified using a threshold for good reliability of kappa > 0.6. The result is a 9 item self-assessment tool (TeamMonitor) with a calculated median kappa of 0.86 (IQ-range: 0.67-1.0) and good content validity. Team-based self-monitoring with the MHPTS to assess team performance during simulated critical events is feasible. A context-based modification of the tool is achievable with good internal consistency and content validity. Further studies are needed to investigate if team-based self-monitoring may be used as part of a programme of assessment to target training programmes for observed performance gaps.
    BMC Emergency Medicine 12/2013; 13(1):22.

Full-text (2 Sources)

Available from
Jun 2, 2014