Reliable and valid assessment of performance in thoracoscopy.

Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100 Copenhagen, Denmark.
Surgical Endoscopy (Impact Factor: 3.43). 12/2011; 26(6):1624-8. DOI: 10.1007/s00464-011-2081-7
Source: PubMed

ABSTRACT As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery. The purpose of this study is to create such an assessment tool, and to explore its reliability and validity.
An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement, respect for tissue, precision of operative technique, creation and placement of ports, localization of pathologic tissue, use of staplers, retrieval of tissue in bag and placement of chest tube. Fifty consecutive thoracoscopic wedge resections were recorded and assessed blindly and independently by two experts using the tool.
Four residents, four fellows and five consultants performed 1-10 (median 4) operations each. The fellows performed significantly better than the residents (P = 0.03; effect size, ES = 0.72). The consultants scored 11% higher than the fellows, but this difference was not significant (P = 0.10, ES = 0.64). The inter-rater reliability was acceptable (Cronbach's alpha 0.71).
This tool for assessing performance in thoracoscopy is reliable and valid. It can provide unbiased feedback to trainees, and can be used to evaluate new teaching curricula, i.e. simulation-based training. Furthermore, it has potential to aid in certification of new thoracic surgeons.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions. Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4). Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach's alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01). This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.
    Surgical endoscopy. 11/2014;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Simulation may reduce the risks associated with the complex operations of cardiothoracic surgery and help create a more efficient, thorough, and uniform curriculum for cardiothoracic surgery fellowship. Here, we review the current status of simulation in cardiothoracic surgical training and provide an overview of all simulation models applicable to cardiothoracic surgery that have been published to date. We completed a comprehensive search of all publications pertaining to simulation of cardiothoracic surgical procedures by using PubMed. Numerous cardiothoracic surgical simulators at various stages of development, assessment, and commercial manufacturing have been published to date. There is currently a predominance of models simulating coronary artery bypass grafting and bronchoscopy and a relative paucity of simulators of open pulmonary and esophageal procedures. Despite the wide range of simulators available, few models have been formally assessed for validity and educational value. Surgical simulation is becoming an increasingly important educational tool in training cardiothoracic surgeons. Our next steps forward will be to develop an objective, standardized way to assess surgical simulation training compared with the current apprenticeship model.
    The Journal of thoracic and cardiovascular surgery 01/2014; 147(1):18-24.e2. · 3.41 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Video-assisted thoracic surgery is gradually replacing conventional open thoracotomy as the method of choice for the treatment of early-stage non-small cell lung cancers, and thoracic surgical trainees must learn and master this technique. Simulation-based training could help trainees overcome the first part of the learning curve, but no virtual-reality simulators for thoracoscopy are commercially available. This study aimed to investigate whether training on a laparoscopic simulator enables trainees to perform a thoracoscopic lobectomy. Twenty-eight surgical residents were randomized to either virtual-reality training on a nephrectomy module or traditional black-box simulator training. After a retention period they performed a thoracoscopic lobectomy on a porcine model and their performance was scored using a previously validated assessment tool. The groups did not differ in age or gender. All participants were able to complete the lobectomy. The performance of the black-box group was significantly faster during the test scenario than the virtual-reality group: 26.6 min (SD 6.7 min) versus 32.7 min (SD 7.5 min). No difference existed between the two groups when comparing bleeding and anatomical and non-anatomical errors. Simulation-based training and targeted instructions enabled the trainees to perform a simulated thoracoscopic lobectomy. Traditional black-box training was more effective than virtual-reality laparoscopy training. Thus, a dedicated simulator for thoracoscopy should be available before establishing systematic virtual-reality training programs for trainees in thoracic surgery.
    Surgical Endoscopy 01/2014; · 3.43 Impact Factor