Faculty evaluation of simulation-based modules for assessment of intraoperative decision making
ABSTRACT Previous studies using simulation-based curricula have focused largely on technical skills. We developed a set of simulation-based modules that focus on intraoperative decision making. The objective of this study was to conduct a faculty evaluation of: (1) the usefulness of 4 newly developed, simulation-based modules; (2) the curricular need to train and assess intraoperative decision making skills of the residents; and (3) potential for resident benefit.
Simulation-based modules were developed using a cognitive task analysis (CTA) framework. The CTA framework involved faculty interviews focusing on 4 operative tasks that span a range of complexity: (1) creation of small and large bowel stoma, (2) laparoscopic ventral hernia repair, (3) pancreaticojejunostomy, and (4) lymph node biopsy during a mediastinoscopy. An experienced psychologist conducted task-specific, one-on-one interviews with fellowship-trained specialists who perform these operations in their practice. Two faculty were interviewed for each procedure. The interviews lasted a minimum of 1 hour and focused on critical decisions, error prevention, error recognition, and error rescue strategies. The coded interview summaries were used as development guides for the simulation-based learning modules. Each module included locally developed physical models for the simulated operative tasks combined with oral and paper-based questions. The physical models were fabricated in such a way that simulated operative tasks could be performed using standard surgical instruments. To assess the newly developed simulation-based modules, 8 volunteer faculty (50% overlap with the interview pool) participated in a simulation-based exercise during a one-on-one session and then completed an 8-item survey cast on a 5-point Likert agreement scale (1 = strongly disagree, 5 = strongly agree). One of the items was worded negatively to ensure internal consistency. An independent observer recorded faculty session times and assessed faculty engagement in the task (1 = not engaged, 5 = extremely engaged).
On average, faculty spent 60 minutes completing each simulation-based exercise. Over 80% of this time was spent performing the operative tasks as they would during a real-life procedure. Mean engagement rating was 4.9 (maximum 5.0, SD = 0.3). Survey results show strong agreement on the importance of training and assessing intraoperative decision making, and that residents would likely benefit from the simulation-based modules.
We developed 4 high-fidelity simulation-based modules to assess intraoperative decision making. Faculty agree strongly on the importance and need for additional modules.
SourceAvailable from: Jan F Kukleta
Article: DOI 10.1007/s00464-014-3917-8
[Show abstract] [Hide abstract]
ABSTRACT: Background Surgical errors result from faulty decision-making, misperceptions and the application of suboptimal problem-solving strategies, just as often as they result from technical failure. To date, surgical training curricula have focused mainly on the acquisition of technical skills. The aim of this review was to assess the validity of methods for improving situational awareness in the surgical theatre.MethodsA search was conducted in PubMed, Embase, the Cochrane Library and PsycINFO® using predefined inclusion criteria, up to June 2014. All study types were considered eligible. The primary endpoint was validity for improving situational awareness in the surgical theatre at individual or team level.ResultsNine articles were considered eligible. These evaluated surgical team crisis training in simulated environments for minimally invasive surgery (4) and open surgery (3), and training courses focused at training non-technical skills (2). Two studies showed that simulation-based surgical team crisis training has construct validity for assessing situational awareness in surgical trainees in minimally invasive surgery. None of the studies showed effectiveness of surgical crisis training on situational awareness in open surgery, whereas one showed face validity of a 2-day non-technical skills training course.Conclusion To improve safety in the operating theatre, more attention to situational awareness is needed in surgical training. Few structured curricula have been developed and validation research remains limited. Strategies to improve situational awareness can be adopted from other industries.British Journal of Surgery 01/2015; 102(1). DOI:10.1002/bjs.9643 · 4.84 Impact Factor
[Show abstract] [Hide abstract]
ABSTRACT: Recent literature has called into question resident readiness for operative independence at the end of general surgery training. We used a simulation-based exit examination to assess resident readiness. Six chief residents performed 3 simulated procedures: bowel anastomosis, laparoscopic ventral hernia (LVH) repair, and pancreaticojejunostomy. Faculty assessed resident performance using task-specific checklists, Objective Structured Assessment of Technical Skills (OSATS), and final product analysis. Residents' individual task-specific checklist scores ranged from 25% to 100% across all 3 procedures. Mean OSATS scores ranged from 4.06 to 4.23/5.0. Residents scored significantly higher on "instrument knowledge" (mean = 4.78, standard deviation [SD] = 23) than "time and motion" (mean = 3.94, SD = .48, P = .025) and "ability to adapt to individual pathologic circumstances" (mean = 4.06, SD =.12, P = .002). Final product analysis revealed a range of errors, including incorrect technique and poor intraoperative planning. Despite relatively high OSATS ratings, residents had a wide range of errors and procedure outcomes. Exit assessments using multiple evaluation metrics may improve awareness of residents' learning needs. Copyright © 2014 Elsevier Inc. All rights reserved.10/2014; 209(1). DOI:10.1016/j.amjsurg.2014.10.002