A Comparison of Orthopaedic Resident Performance on Surgical Fixation of an Ulnar Fracture Using Virtual Reality and Synthetic Models

Department of Surgery, The University of Calgary, Calgary, Alberta, Canada
The Journal of Bone and Joint Surgery (Impact Factor: 5.28). 05/2013; 95(9):e601-6. DOI: 10.2106/JBJS.K.01284
Source: PubMed


Surgical trainees develop surgical skills using various techniques, with simulators providing a safe learning environment. Fracture fixation is the most common procedure in orthopaedic surgery, and residents may benefit from simulated fracture fixation. The performance of residents on a virtual simulator that allows them to practice the surgical fixation of fractures by providing a sense of touch (haptics) has not yet been compared with their performance using other methods of practicing fracture fixation, such as a Sawbones simulator model. The purpose of this study was to assess whether residents performed similarly on a newly developed virtual simulator compared with a Sawbones simulator fracture fixation model.
A stratified, randomized controlled study involving twenty-two orthopaedic surgery residents was performed. The residents were randomized to first perform surgical fixation of the ulna on either the virtual or the Sawbones simulator, after which they performed the same procedure on the other simulator. Their performance was evaluated by examiners experienced in fracture fixation who completed a task-specific checklist, global rating scale (GRS) form, and time-to-completion record for each participant on each simulator.
Both simulators distinguished between differing experience levels, demonstrating construct validity; for the Sawbones simulator, the Cohen d value (effect size) was >0.90, and for the virtual simulator, d was >1.10 (p < 0.05 for both). The participants achieved significantly better scores on the virtual simulator compared with the Sawbones simulator (p < 0.05) for all measures except time to completion. The GRS scores showed a high level of internal consistency (Cronbach α, >0.80). However, Pearson product-moment correlation analysis showed no significant correlations between the results on the two simulators; therefore, concurrent validity was not achieved.
The newly developed virtual ulnar surgical fixation simulator, which incorporates haptics, shows promise for helping surgical trainees learn and practice basic skills, but it did not attain the same standards as the current standard Sawbones simulator. The procedural measures used to assess resident performance demonstrated good reliability and validity, and both the Sawbones and the virtual simulator showed evidence of construct validity.

Download full-text


Available from: Justin Eugene Matthew LeBlanc
  • Source
    • "This model is not without risk, though. Inexperienced trainees contribute to a higher rate of re-admissions and reoperations (Palm et al. 2007, Leblanc et al. 2013). A review of 609 studies found that virtual-reality simulation training improved operative skills (Cook et al. 2011), and simulation-based training in orthopedic surgery is starting to emerge. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background and purpose Internal fixation of hip fractures is a common and important procedure that orthopedic surgeons must master early in their career. Virtual-reality training could improve initial skills, and a simulation-based test would make it possible to ensure basic competency of junior surgeons before they proceed to supervised practice on patients. The aim of this study was to develop a reliable and valid test with credible pass/fail standards. Methods 20 physicians (10 untrained novices and 10 experienced orthopedic surgeons) each performed 3 internal fixation procedures of an undisplaced femoral neck fracture: 2 hook-pins, 2 screws, and a sliding hip screw. All procedures were preformed on a trauma simulator. Performance scores for each procedure were obtained from the predefined metrics of the simulator. The inter-case reliability of the simulator metrics was explored by calculation of intra-class correlation coefficient. Validity was explored by comparison between novices’ and experts’ scores using independent-samples t-test. A pass/fail standard was set by the contrasting-groups method and the consequences were explored. Results The percentage of maximum combined score (PM score) showed an inter-case reliability of 0.83 (95% CI: 0.65–0.93) between the 3 procedures. The mean PM score was 30% (CI: 7–53) for the novices and 76% (CI: 68–83) for the experienced surgeons. The pass/fail standard was set at 58%, resulting in none of the novices passing the test and a single experienced surgeon failing the test. Interpretation The simulation-based test was reliable and valid in our setting, and the pass/fail standard could discriminate between novices and experienced surgeons. Potentially, training and testing of future junior surgeons on a virtual-reality simulator could ensure basic competency before proceeding to supervised practice on patients.
    Full-text · Article · Apr 2014 · Acta Orthopaedica
  • [Show abstract] [Hide abstract]
    ABSTRACT: Surgical training has followed the master-apprentice model for centuries but is currently undergoing a paradigm shift. The traditional model is inefficient with no guarantee of case mix, quality, or quantity. There is a growing focus on competency-based medical education in response to restrictions on doctors' working hours and the traditional mantra of "see one, do one, teach one" is being increasingly questioned. The medical profession is subject to more scrutiny than ever before and is facing mounting financial, clinical, and political pressures. Simulation may be a means of addressing these challenges. It provides a way for trainees to practice technical tasks in a protected environment without putting patients at risk and helps to shorten the learning curve. The evidence for simulation-based training in orthopedic surgery using synthetic models, cadavers, and virtual reality simulators is constantly developing, though further work is needed to ensure the transfer of skills to the operating theatre.
    No preview · Article · Apr 2014 · Current Reviews in Musculoskeletal Medicine
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective To develop and conduct a pilot study of a curriculum of 4 surrogate bone training modules to assess and track progress in basic orthopedic manual skills outside the operating room. Design Four training modules were developed with faculty and resident input. The modules include (1) cortical drilling, (2) drill trajectory, (3) oscillating saw, and (4) pedicle probing. Orthopedic resident's performance was evaluated. Validity and reliability results were calculated using standard analysis of variance and multivariate regression analysis accounting for postgraduate year (PGY) level, number of attempts, and specific outcome target results specific to the simulation module. Setting St. Mary's Medical Center in San Francisco, CA. Participants These modules were tested on 15 orthopedic surgery residents ranging from PGY 1 to PGY 5 experience. Results The cortical drilling module had a mean success rate of 56% ± 5%. There was a statistically significant difference in performance according to the diameter of the drill used from 33% ± 7% with large diameter to 70% ± 6% with small diameter. The drill trajectory module had a success rate of 85% ± 3% with a trend toward improvement across PGY level. The oscillating saw module had a mean success rate of 25% ± 5% (trajectory) and 84% ± 6% (depth). We observed a significant improvement in trajectory performance during the second attempt. The pedicle probing module had a success rate of 46% ± 10%. Conclusion The results of this pilot study on a small number of residents are promising. The modules were inexpensive and easy to administer. Conclusions of statistical significance include (1) residents who could easily detect changes in surrogate bone thickness with a smaller diameter drill than with a larger diameter drill and (2) residents who significantly improved saw trajectory with an additional attempt at the module.
    No preview · Article · Aug 2014 · Journal of Surgical Education
Show more