Article

Validation of a Global Assessment of Arthroscopic Skills in a Cadaveric Knee Model

Pan Am Clinic, Winnipeg, Manitoba, Canada
Arthroscopy The Journal of Arthroscopic and Related Surgery (Impact Factor: 3.1). 11/2012; 29(1). DOI: 10.1016/j.arthro.2012.07.010
Source: PubMed

ABSTRACT PURPOSE: The purpose of this study was to determine whether a global assessment of arthroscopic skills was valid for blinded assessment of cadaveric diagnostic knee arthroscopy. METHODS: A global skills assessment for arthroscopy was created using a published theory of the development of expertise. Faculty surgeons, fellows, and residents were consented and enrolled in this institutional review board-approved validation study. All participants were oriented to the equipment and procedures for diagnostic arthroscopy of the knee. After reviewing the anatomic structures to be visualized, participants were allowed 10 minutes to complete a diagnostic arthroscopy of the knee. The hands and arthroscopic view were recorded during this attempt. Resident participants completed a second filmed diagnostic arthroscopy 1 week after the initial attempt. Five blinded reviewers watched the synchronized videos and assessed arthroscopic skills with a procedure-specific checklist and the newly developed global skills assessment. The agreement between reviewers was determined by intraclass correlation coefficient. Internal consistency was determined with Cronbach's α. Test-retest reliability was measured by correlating repeated arthroscopies by residents. The ability of the global assessment to discriminate skill levels was determined with between-group Mann-Whitney U tests. RESULTS: The agreement between global assessment scores was strong (I.C.C. = 0.80, 95% C.I. 0.68-0.92). The internal consistency of evaluations was excellent (Cronbach's α = 0.97), and the test-retest reliability was strong (r = 0.52). The global assessment score was shown to be able to discriminate between skill levels by an analysis of variance indicating the difference in means among the various levels of training (P < .0001). CONCLUSIONS: The Objective Assessment of Arthroscopic Skills is a useful adjunct to arthroscopic educators and learners and could be used for in-training evaluations. CLINICAL RELEVANCE: The Objective Assessment of Arthroscopic Skills is an instrument that can be employed to measure the impact of skills curricula, including but not limited to simulation.

0 Bookmarks
 · 
88 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Introduction Traditional orthopaedic training has followed an apprenticeship model whereby trainees enhance their skills by operating under guidance. However the introduction of limitations on training hours and shorter training programmes mean that alternative training strategies are required. Aims To perform a literature review on simulation training in arthroscopy and devise a framework that structures different simulation techniques that could be used in arthroscopic training. Methods A systematic search of Medline, Embase, Google Scholar and the Cochrane Databases were performed. Search terms included “virtual reality OR simulator OR simulation” and “arthroscopy OR arthroscopic”. Results 14 studies evaluating simulators in knee, shoulder and hip arthroplasty were included. The majority of the studies demonstrated construct and transference validity but only one showed concurrent validity. More studies are required to assess its potential as a training and assessment tool, skills transference between simulators and to determine the extent of skills decay from prolonged delays in training. We also devised a “ladder of arthroscopic simulation” that provides a competency-based framework to implement different simulation strategies. Conclusion The incorporation of simulation into an orthopaedic curriculum will depend on a coordinated approach between many bodies. But the successful integration of simulators in other areas of surgery supports a possible role for simulation in advancing orthopaedic education.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Traditional orthopaedic training has followed an apprenticeship model whereby trainees enhance their skills by operating under guidance. However the introduction of limitations on training hours and shorter training programmes mean that alternative training strategies are required. To perform a literature review on simulation training in arthroscopy and devise a framework that structures different simulation techniques that could be used in arthroscopic training. A systematic search of Medline, Embase, Google Scholar and the Cochrane Databases were performed. Search terms included "virtual reality OR simulator OR simulation" and "arthroscopy OR arthroscopic". 14 studies evaluating simulators in knee, shoulder and hip arthroplasty were included. The majority of the studies demonstrated construct and transference validity but only one showed concurrent validity. More studies are required to assess its potential as a training and assessment tool, skills transference between simulators and to determine the extent of skills decay from prolonged delays in training. We also devised a "ladder of arthroscopic simulation" that provides a competency-based framework to implement different simulation strategies. The incorporation of simulation into an orthopaedic curriculum will depend on a coordinated approach between many bodies. But the successful integration of simulators in other areas of surgery supports a possible role for simulation in advancing orthopaedic education.
    International Journal of Surgery (London, England) 04/2014; DOI:10.1016/j.ijsu.2014.04.005 · 1.44 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: To assess orthopaedic trainees performing diagnostic knee arthroscopies and evaluate procedural competence using a novel statistical method, the Cumulative Summation Test for Learning Curve (LC-CUSUM). Twenty orthopaedic trainees in postgraduate year (PGY) 1 through 5 performed diagnostic knee arthroscopy and were evaluated intraoperatively with a validated 10-point knee task-specific checklist (TSCL) and 50-point global rating scale (GRS). A score of 40 points or greater (of 50 points) for the GRS and 8 points or greater (of 10 points) for the knee TSCL was considered to indicate a successful procedure. For the LC-CUSUM analysis, adequate performance was defined as a 10% failure rate, inadequate performance was defined as a 30% failure rate, and an acceptable deviation from adequate performance was defined as 10%. A limit h equal to 1.6 was selected to give a true-discovery rate of 90% and a false-discovery rate of 10% over 50 procedures. A total of 340 consecutive procedures were performed by 20 trainees during the 12-month study period. The cumulative number of arthroscopic procedures performed by trainees before study start increased with increasing PGY. The median number of arthroscopic procedures performed per trainee was 16.5 (interquartile range, 14 to 21.75). Competency in knee arthroscopy for the TSCL was achieved by 8 trainees (40%), after a median of 16 procedures (interquartile range, 13 to 20), and for the GRS by only a single trainee (5%), after 14 procedures. Threshold-adjusted curves stratified by PGY level enabled multiple trainees to achieve competency for both the TSCL and GRS. The LC-CUSUM can be successfully applied to knee arthroscopy to provide an individualized assessment of performance and quantitatively demonstrate competency for basic arthroscopic tasks. The LC-CUSUM is an effective method to evaluate procedure competence in arthroscopic training and can provide objective feedback and benchmarks in the learning phase.
    Arthroscopy The Journal of Arthroscopic and Related Surgery 05/2014; 30(5):613-21. DOI:10.1016/j.arthro.2014.02.021 · 3.10 Impact Factor