Reliable and valid assessment of performance in thoracoscopy.

Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100 Copenhagen, Denmark.
Surgical Endoscopy (Impact Factor: 3.43). 12/2011; 26(6):1624-8. DOI: 10.1007/s00464-011-2081-7
Source: PubMed

ABSTRACT As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery. The purpose of this study is to create such an assessment tool, and to explore its reliability and validity.
An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement, respect for tissue, precision of operative technique, creation and placement of ports, localization of pathologic tissue, use of staplers, retrieval of tissue in bag and placement of chest tube. Fifty consecutive thoracoscopic wedge resections were recorded and assessed blindly and independently by two experts using the tool.
Four residents, four fellows and five consultants performed 1-10 (median 4) operations each. The fellows performed significantly better than the residents (P = 0.03; effect size, ES = 0.72). The consultants scored 11% higher than the fellows, but this difference was not significant (P = 0.10, ES = 0.64). The inter-rater reliability was acceptable (Cronbach's alpha 0.71).
This tool for assessing performance in thoracoscopy is reliable and valid. It can provide unbiased feedback to trainees, and can be used to evaluate new teaching curricula, i.e. simulation-based training. Furthermore, it has potential to aid in certification of new thoracic surgeons.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Video-assisted thoracic surgery is gradually replacing conventional open thoracotomy as the method of choice for the treatment of early-stage non-small cell lung cancers, and thoracic surgical trainees must learn and master this technique. Simulation-based training could help trainees overcome the first part of the learning curve, but no virtual-reality simulators for thoracoscopy are commercially available. This study aimed to investigate whether training on a laparoscopic simulator enables trainees to perform a thoracoscopic lobectomy. Twenty-eight surgical residents were randomized to either virtual-reality training on a nephrectomy module or traditional black-box simulator training. After a retention period they performed a thoracoscopic lobectomy on a porcine model and their performance was scored using a previously validated assessment tool. The groups did not differ in age or gender. All participants were able to complete the lobectomy. The performance of the black-box group was significantly faster during the test scenario than the virtual-reality group: 26.6 min (SD 6.7 min) versus 32.7 min (SD 7.5 min). No difference existed between the two groups when comparing bleeding and anatomical and non-anatomical errors. Simulation-based training and targeted instructions enabled the trainees to perform a simulated thoracoscopic lobectomy. Traditional black-box training was more effective than virtual-reality laparoscopy training. Thus, a dedicated simulator for thoracoscopy should be available before establishing systematic virtual-reality training programs for trainees in thoracic surgery.
    Surgical Endoscopy 01/2014; · 3.43 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: INTRODUCTION: An increasing proportion of thoracic procedures are performed using video-assisted thoracic surgery. This minimally invasive technique places special demands on the surgeons. Using simulation-based training on artificial models or animals has been proposed to overcome the initial part of the learning curve. This study aimed to investigate the effect of simulation-based training and to compare self-guided and educator-guided training. METHODS: The study included a surgeon group (n = 10) and 30 randomized novices in 3 groups. A control group (n = 10) and the group of surgeons (n = 10) were tested with no previous simulator training. A self-guided training group (n = 10) and an educator-guided training group (n = 10) trained for 3 hours on 3 scenarios of increasing fidelity and difficulty before taking a standardized test consisting of performing a wedge resection on a porcine lung, which was recorded and assessed blindly and independently by 2 thoracoscopic experts using a modified version of a validated assessment tool. RESULTS: Interrater reliability was acceptable (Spearman ρ = 0.73, P < 0.001). The control group and the self-guided training group performed significantly worse than the experienced surgeons (P = 0.012 and P = 0.010, respectively). There was no significant difference between the educator-guided training group and the experienced surgeons (P = 0.271). CONCLUSIONS: This randomized study concerning simulation-based training for thoracoscopy showed that 3 hours of intensive simulator training with a dedicated educator enables novices to perform an acceptable wedge resection in a simple, simulated model. Although not significant, it seemed that having an educator present during training had a beneficiary effect. Transfer studies are required for further conclusions.
    Simulation in healthcare: journal of the Society for Simulation in Healthcare 04/2013; · 1.64 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Simulation may reduce the risks associated with the complex operations of cardiothoracic surgery and help create a more efficient, thorough, and uniform curriculum for cardiothoracic surgery fellowship. Here, we review the current status of simulation in cardiothoracic surgical training and provide an overview of all simulation models applicable to cardiothoracic surgery that have been published to date. We completed a comprehensive search of all publications pertaining to simulation of cardiothoracic surgical procedures by using PubMed. Numerous cardiothoracic surgical simulators at various stages of development, assessment, and commercial manufacturing have been published to date. There is currently a predominance of models simulating coronary artery bypass grafting and bronchoscopy and a relative paucity of simulators of open pulmonary and esophageal procedures. Despite the wide range of simulators available, few models have been formally assessed for validity and educational value. Surgical simulation is becoming an increasingly important educational tool in training cardiothoracic surgeons. Our next steps forward will be to develop an objective, standardized way to assess surgical simulation training compared with the current apprenticeship model.
    The Journal of thoracic and cardiovascular surgery 01/2014; 147(1):18-24.e2. · 3.41 Impact Factor