Reliable and valid assessment of performance in thoracoscopy.
ABSTRACT As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery. The purpose of this study is to create such an assessment tool, and to explore its reliability and validity.
An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement, respect for tissue, precision of operative technique, creation and placement of ports, localization of pathologic tissue, use of staplers, retrieval of tissue in bag and placement of chest tube. Fifty consecutive thoracoscopic wedge resections were recorded and assessed blindly and independently by two experts using the tool.
Four residents, four fellows and five consultants performed 1-10 (median 4) operations each. The fellows performed significantly better than the residents (P = 0.03; effect size, ES = 0.72). The consultants scored 11% higher than the fellows, but this difference was not significant (P = 0.10, ES = 0.64). The inter-rater reliability was acceptable (Cronbach's alpha 0.71).
This tool for assessing performance in thoracoscopy is reliable and valid. It can provide unbiased feedback to trainees, and can be used to evaluate new teaching curricula, i.e. simulation-based training. Furthermore, it has potential to aid in certification of new thoracic surgeons.
- SourceAvailable from: ncbi.nlm.nih.govBMJ (Clinical research ed.). 12/2003; 327(7422):1032-7.
- [show abstract] [hide abstract]
ABSTRACT: If, as neuropsychologists, we think of the relationship between brain and behavior as the same as that between truth and reality, we must be equipped with statistical procedures that are coherent in terms of what we measure and what it represents. I believe that this necessary statistical procedure is effect size analysis, and without it, I believe that we fail to tell the truth, the whole truth, and nothing but the truth when describing our neuropsychological research. Accordingly, I review here the standard calculations of commonly employed effect sizes in two group designs and show how to adjust some familiar (and perhaps not so familiar) formulae using illustrative numerical examples. I also put forth an argument to adopt Cohen's measure as an expression of effect size based on its apropos to neuropsychological research. It is also argued that the interpretation of the magnitude of an effect size should depend on context, and not on pre-established heuristic benchmarks. It is noted, however, that effect sizes greater than 3.0 (OL%<5) might seem particularly appropriate when evaluating the sensitivity of neuropsychological tasks and in establishing test markers in neuropsychological disorders.Archives of Clinical Neuropsychology 10/2001; 16(7):653-67. · 2.00 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: Only a few simulators are available which offer training in video-assisted thoracic surgery (VATS). We have developed a VATS training model for surgeons. The simulator consists of a training module housing three disposable components: the lung (made of polyurethane), the bronchus, and the artificial circulatory pulmonary vessels (made of polyvinyl chloride), connected to a pump. VATS procedures were videotaped and evaluated using a checklist assessment method. This unique module has been tested at several VATS seminars in Japan. In the questionnaire, training participants strongly agreed that the trainer was helpful and prepared them well for VATS lobectomy prior to performing actual surgery. Evaluation of the dexterity score for thoracoscopic surgery with our simulator correlated with the surgeons' experience with actual surgery. Technical factors were well taught using this model. Our unique trainer may enhance the skill of VATS surgeons at a national level.The Thoracic and Cardiovascular Surgeon 03/2008; 56(1):32-6. · 0.93 Impact Factor