Article

Validation of a Global Assessment of Arthroscopic Skills in a Cadaveric Knee Model

Pan Am Clinic, Winnipeg, Manitoba, Canada
Arthroscopy The Journal of Arthroscopic and Related Surgery (Impact Factor: 3.19). 11/2012; 29(1). DOI: 10.1016/j.arthro.2012.07.010
Source: PubMed

ABSTRACT PURPOSE: The purpose of this study was to determine whether a global assessment of arthroscopic skills was valid for blinded assessment of cadaveric diagnostic knee arthroscopy. METHODS: A global skills assessment for arthroscopy was created using a published theory of the development of expertise. Faculty surgeons, fellows, and residents were consented and enrolled in this institutional review board-approved validation study. All participants were oriented to the equipment and procedures for diagnostic arthroscopy of the knee. After reviewing the anatomic structures to be visualized, participants were allowed 10 minutes to complete a diagnostic arthroscopy of the knee. The hands and arthroscopic view were recorded during this attempt. Resident participants completed a second filmed diagnostic arthroscopy 1 week after the initial attempt. Five blinded reviewers watched the synchronized videos and assessed arthroscopic skills with a procedure-specific checklist and the newly developed global skills assessment. The agreement between reviewers was determined by intraclass correlation coefficient. Internal consistency was determined with Cronbach's α. Test-retest reliability was measured by correlating repeated arthroscopies by residents. The ability of the global assessment to discriminate skill levels was determined with between-group Mann-Whitney U tests. RESULTS: The agreement between global assessment scores was strong (I.C.C. = 0.80, 95% C.I. 0.68-0.92). The internal consistency of evaluations was excellent (Cronbach's α = 0.97), and the test-retest reliability was strong (r = 0.52). The global assessment score was shown to be able to discriminate between skill levels by an analysis of variance indicating the difference in means among the various levels of training (P < .0001). CONCLUSIONS: The Objective Assessment of Arthroscopic Skills is a useful adjunct to arthroscopic educators and learners and could be used for in-training evaluations. CLINICAL RELEVANCE: The Objective Assessment of Arthroscopic Skills is an instrument that can be employed to measure the impact of skills curricula, including but not limited to simulation.

1 Follower
 · 
102 Views
  • Source
    • "It was able to distinguish between the novice, experienced and expert arthroscopists in the cadaver knee model. The Objective Assessment of Arthroscopic Skills (OAAS) instrument consists of multiple skill domains each rated on an expertise-based scale with 5 skill-level options [57]. When combined with an anatomical checklist, the OAAS discriminated between skills levels of various levels of training with excellent internal consistency and test-retest reliability. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background The current paradigm of arthroscopic training lacks objective evaluation of technical ability and its adequacy is concerning given the accelerating complexity of the field. To combat insufficiencies, emphasis is shifting towards skill acquisition outside the operating room and sophisticated assessment tools. We reviewed (1) the validity of cadaver and surgical simulation in arthroscopic training, (2) the role of psychomotor analysis and arthroscopic technical ability, (3) what validated assessment tools are available to evaluate technical competency, and (4) the quantification of arthroscopic proficiency. Methods The Medline and Embase databases were searched for published articles in the English literature pertaining to arthroscopic competence, arthroscopic assessment and evaluation and objective measures of arthroscopic technical skill. Abstracts were independently evaluated and exclusion criteria included articles outside the scope of knee and shoulder arthroscopy as well as original articles about specific therapies, outcomes and diagnoses leaving 52 articles citied in this review. Results Simulated arthroscopic environments exhibit high levels of internal validity and consistency for simple arthroscopic tasks, however the ability to transfer complex skills to the operating room has not yet been established. Instrument and force trajectory data can discriminate between technical ability for basic arthroscopic parameters and may serve as useful adjuncts to more comprehensive techniques. There is a need for arthroscopic assessment tools for standardized evaluation and objective feedback of technical skills, yet few comprehensive instruments exist, especially for the shoulder. Opinion on the required arthroscopic experience to obtain proficiency remains guarded and few governing bodies specify absolute quantities. Conclusions Further validation is required to demonstrate the transfer of complex arthroscopic skills from simulated environments to the operating room and provide objective parameters to base evaluation. There is a deficiency of validated assessment tools for technical competencies and little consensus of what constitutes a sufficient case volume within the arthroscopy community.
    BMC Medical Education 05/2013; 13(1):61. DOI:10.1186/1472-6920-13-61 · 1.41 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Introduction Traditional orthopaedic training has followed an apprenticeship model whereby trainees enhance their skills by operating under guidance. However the introduction of limitations on training hours and shorter training programmes mean that alternative training strategies are required. Aims To perform a literature review on simulation training in arthroscopy and devise a framework that structures different simulation techniques that could be used in arthroscopic training. Methods A systematic search of Medline, Embase, Google Scholar and the Cochrane Databases were performed. Search terms included “virtual reality OR simulator OR simulation” and “arthroscopy OR arthroscopic”. Results 14 studies evaluating simulators in knee, shoulder and hip arthroplasty were included. The majority of the studies demonstrated construct and transference validity but only one showed concurrent validity. More studies are required to assess its potential as a training and assessment tool, skills transference between simulators and to determine the extent of skills decay from prolonged delays in training. We also devised a “ladder of arthroscopic simulation” that provides a competency-based framework to implement different simulation strategies. Conclusion The incorporation of simulation into an orthopaedic curriculum will depend on a coordinated approach between many bodies. But the successful integration of simulators in other areas of surgery supports a possible role for simulation in advancing orthopaedic education.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Traditional orthopaedic training has followed an apprenticeship model whereby trainees enhance their skills by operating under guidance. However the introduction of limitations on training hours and shorter training programmes mean that alternative training strategies are required. To perform a literature review on simulation training in arthroscopy and devise a framework that structures different simulation techniques that could be used in arthroscopic training. A systematic search of Medline, Embase, Google Scholar and the Cochrane Databases were performed. Search terms included "virtual reality OR simulator OR simulation" and "arthroscopy OR arthroscopic". 14 studies evaluating simulators in knee, shoulder and hip arthroplasty were included. The majority of the studies demonstrated construct and transference validity but only one showed concurrent validity. More studies are required to assess its potential as a training and assessment tool, skills transference between simulators and to determine the extent of skills decay from prolonged delays in training. We also devised a "ladder of arthroscopic simulation" that provides a competency-based framework to implement different simulation strategies. The incorporation of simulation into an orthopaedic curriculum will depend on a coordinated approach between many bodies. But the successful integration of simulators in other areas of surgery supports a possible role for simulation in advancing orthopaedic education.
    International Journal of Surgery (London, England) 04/2014; 12(6). DOI:10.1016/j.ijsu.2014.04.005 · 1.65 Impact Factor
Show more