-
[show abstract]
[hide abstract]
ABSTRACT: OBJECTIVE:: To investigate the nature of process failures in postoperative care, to assess their frequency and preventability, and to explore their relationship to adverse events. BACKGROUND:: Adverse events are common and are frequently caused by failures in the process of care. These processes are often evaluated independently using clinical audit. There is little understanding of process failures in terms of their overall frequency, relative risk, and cumulative effect on the surgical patient. METHODS:: Patients were observed daily from the first postoperative day until discharge by an independent surgeon. Field notes on the circumstances surrounding any nonroutine or atypical event were recorded. Field notes were assessed by 2 surgeons to identify failures in the process of care. Preventability, the degree of harm caused to the patient, and the underlying etiology of process failures were evaluated by 2 independent surgeons. RESULTS:: Fifty patients undergoing major elective general surgery were observed for a total of 659 days of postoperative care. A total of 256 process failures were identified, of which 85% were preventable and 51% directly led to patient harm. Process failures occurred in all aspects of care, the most frequent being medication prescribing and administration, management of lines, tubes, and drains, and pain control interventions. Process failures accounted for 57% of all preventable adverse events. Communication failures and delays were the main etiologies, leading to 54% of process failures. CONCLUSIONS:: Process failures are common in postoperative care, are highly preventable, and frequently cause harm to patients. Interventions to prevent process failures will improve the reliability of surgical postoperative care and have the potential to reduce hospital stay.
Annals of surgery 10/2012; · 7.90 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Effective communication is imperative to safe surgical practice. Previous studies have typically focused upon the operating theatre. This study aimed to explore the communication and information transfer failures across the entire surgical care pathway.
Using a qualitative approach, semi-structured interviews were conducted with 18 members of the multidisciplinary team (seven surgeons, five anaesthetists and six nurses) in an acute National Health Service trust. Participants' views regarding information transfer and communication failures at each phase of care, their causes, effects and potential interventions were explored. Interviews were recorded, transcribed verbatim, and submitted to emergent theme analysis. Sampling ceased when categorical and theoretical saturation was achieved.
Preoperatively, lack of communication between anaesthetists and surgeons was the most common problem (13/18 participants). Incomplete handover from the ward to theatre (12/18) and theatre to recovery (15/18) were other key problems. Work environment, lack of protocols and primitive forms of information transfer were reported as the most common cause of failures. Participants reported that these failures led to increased morbidity and mortality. Healthcare staff were strongly supportive of the view that standardisation and systematisation of communication processes was essential to improve patient safety.
This study suggests communication failures occur across the entire continuum of care and the participants opined that it could have a potentially serious impact on patient safety. This data can be used to plan interventions targeted at the entire surgical pathway so as to improve the quality of care at all stages of the patient's journey.
BMJ quality & safety 07/2012; 21(10):843-9.
-
[show abstract]
[hide abstract]
ABSTRACT: Laparoscopic bariatric surgery is a safe and effective way of treating morbid obesity. However, the operations are technically challenging and training opportunities for junior surgeons are limited. This study aims to assess whether virtual reality (VR) simulation is an effective adjunct for training and assessment of laparoscopic bariatric technical skills.
Twenty bariatric surgeons of varying experience (Five experienced, five intermediate, and ten novice) were recruited to perform a jejuno-jejunostomy on both cadaveric tissue and on the bariatric module of the Lapmentor VR simulator (Simbionix Corporation, Cleveland, OH). Surgical performance was assessed using validated global rating scales (GRS) and procedure specific video rating scales (PSRS). Subjects were also questioned about the appropriateness of VR as a training tool for surgeons.
Construct validity of the VR bariatric module was demonstrated with a significant difference in performance between novice and experienced surgeons on the VR jejuno-jejunostomy module GRS (median 11-15.5; P = .017) and PSRS (median 11-13; P = .003). Content validity was demonstrated with surgeons describing the VR bariatric module as useful and appropriate for training (mean Likert score 4.45/7) and they would highly recommend VR simulation to others for bariatric training (mean Likert score 5/7). Face and concurrent validity were not established.
This study shows that the bariatric module on a VR simulator demonstrates construct and content validity. VR simulation appears to be an effective method for training of advanced bariatric technical skills for surgeons at the start of their bariatric training. However, assessment of technical skills should still take place on cadaveric tissue.
Surgery 06/2012; 151(6):779-84. · 3.10 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Clinical handover (handoff, sign out) is frequently implicated as a cause of adverse events in hospitalised patients. Complex social interactions such as handover are subject to the teamwork skills of the participants and there is increasing evidence that the quality of teamwork in handover affects outcome. Teamwork skills have been assessed in one-to-one handovers but the applicability of these measurement tools to healthcare team shift handovers remains unproven. This study aimed to assess the feasibility of measurement of teamwork skills in shift handover and the applicability of adapted teamwork skills rating scales to a shift handover environment.
Morning surgical shift handovers were assessed for completeness of information transfer, duration, interruptions and handover attendance. Handover teamwork skills were evaluated using two validated rating scales, adapted from one-to-one handovers and intra-operative teamwork skill measurement.
50 handovers, including 306 patients were observed. Communication checklist completion was 97% but the quality of teamwork skills varied widely between handovers. There was very good concurrent validity between the two teamwork skill rating scales (Spearman's rho = 0.67, p < 0.001). There was no significant correlation between content completion, duration, interruptions or attendance and teamwork skill ratings.
Teamwork skills vary widely between handovers and can be consistently scored using both rating scales. It is feasible to use adapted teamwork skill rating scales in shift handover and they appear to measure different constructs to traditional handover measures such as interruptions and communication checklist completion. The assessment of teamwork skills is a necessary complement to the assessment of completeness of information transfer when evaluating the overall quality of handover.
International journal of surgery (London, England) 05/2012; 10(7):355-9.
-
[show abstract]
[hide abstract]
ABSTRACT: Education and training of health care professionals is necessary to achieve sustainable improvements in patient safety. Despite its inherently risky nature, little training specifically in safety has been conducted in the surgical disciplines. In this study we explored the effects of a safety skills training program on surgical residents' knowledge, attitudes, and awareness of patient safety.
A half-day training program incorporating safety awareness, analysis, and improvement skills was delivered to surgical residents from 19 hospitals in London, United Kingdom. Participants were assessed in terms of safety knowledge (MCQs) and attitudes to safety (validated questionnaire; scale 1 to 5) before and after training. To determine long-term effects, 6 months after training participants identified and reported on observed safety events in their own workplace by using an observational form for data collection.
A total of 27 surgeons participated in the training program. Knowledge of safety significantly improved after the course (mean pre = 45.26% vs mean post = 70.59%, P < .01) as did attitudes to error analysis and improving safety (mean pre 3.50 vs mean post 3.97, P < .001) and ability to influence safety (mean pre 3.22 vs mean post 3.49, P < .01). After the course, participants reported richer, detailed sets of observations demonstrating enhanced understanding, recognition, and analysis of patient safety issues in their workplace.
Safety skills training with positive educational outcomes can be delivered in a half day. Such a course may allow patient safety to be integrated into any curriculum, thereby training the next generation of the healthcare workforce to maintain the safety momentum.
Surgery 04/2012; 152(1):26-31. · 3.10 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: It is well known that many healthcare systems have poor reliability; however, the size and pervasiveness of this problem and its impact has not been systematically established in the UK. The authors studied four clinical systems: clinical information in surgical outpatient clinics, prescribing for hospital inpatients, equipment in theatres, and insertion of peripheral intravenous lines. The aim was to describe the nature, extent and variation in reliability of these four systems in a sample of UK hospitals, and to explore the reasons for poor reliability.
Seven UK hospital organisations were involved; each system was studied in three of these. The authors took delivery of the systems' intended outputs to be a proxy for the reliability of the system as a whole. For example, for clinical information, 100% reliability was defined as all patients having an agreed list of clinical information available when needed during their appointment. Systems factors were explored using semi-structured interviews with key informants. Common themes across the systems were identified.
Overall reliability was found to be between 81% and 87% for the systems studied, with significant variation between organisations for some systems: clinical information in outpatient clinics ranged from 73% to 96%; prescribing for hospital inpatients 82-88%; equipment availability in theatres 63-88%; and availability of equipment for insertion of peripheral intravenous lines 80-88%. One in five reliability failures were associated with perceived threats to patient safety. Common factors causing poor reliability included lack of feedback, lack of standardisation, and issues such as access to information out of working hours.
Reported reliability was low for the four systems studied, with some common factors behind each. However, this hides significant variation between organisations for some processes, suggesting that some organisations have managed to create more reliable systems. Standardisation of processes would be expected to have significant benefit.
BMJ quality & safety 04/2012; 21(6):466-72.
-
[show abstract]
[hide abstract]
ABSTRACT: Accurate assessment is imperative for learning, feedback and progression. The aim of this study was to examine whether surgeons can accurately self-assess their technical and nontechnical skills compared with expert faculty members' assessments.
Twenty-five surgeons performed a laparoscopic cholecystectomy (LC) in a simulated operating room. Technical and nontechnical performance was assessed by participants and faculty members using the validated Objective Structured Assessment of Technical Skills (OSATS) and the Non-Technical Skills for Surgeons scale (NOTSS).
Assessment of technical performance correlated between self and faculty members' ratings for experienced (median score, 30.0 vs 31.0; ρ = .831; P = .001) and inexperienced (median score, 22.0 vs 28.0; ρ = .761; P = .003) surgeons. Assessment of nontechnical skills between self and faculty members did not correlate for experienced surgeons (median score, 8.0 vs 10.5; ρ = -.375; P = .229) or their more inexperienced counterparts (median score, 9.0 vs 7.0; ρ = -.018; P = .953).
Surgeons can accurately self-assess their technical skills in virtual reality LC. Conversely, formal assessment with faculty members' input is required for nontechnical skills, for which surgeons lack insight into their behaviours.
American journal of surgery 10/2011; 202(4):500-6. · 2.36 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To assess the feasibility, validity, and reliability of a postoperative Handover Assessment Tool (PoHAT) and to evaluate the current practices of the postoperative handover at 2 large European hospitals.
Postoperative handover is one of the most critical phases in the care of a patient undergoing surgery. However, handovers are largely informal and variable. A thorough understanding of the problem is necessary before safety solutions can be considered.
Postoperative Handover Assessment Tool (PoHAT) was developed through task analysis, semistructured interviews, literature review, and learned society guidelines. Subsequent validation was done by the Delphi technique. Feasibility and reliability were then evaluated by direct observation of handovers at 2 large European hospitals. Outcomes measures included information omissions, task errors, teamwork evaluation, duration of handover, and number of distractions.
The tool was feasible to use and inter-rater reliability was excellent (r = 0.96, P < 0.001). Evaluation of handover at the 2 study sites revealed a median of 8 information omissions per handover at both the centers (IQR 7-10). There were a median of 3 task errors per handover (IQR 2-4). Thirty-five percent of handovers had distractions, which included competing demands for nurse attention, bleeps, and case-irrelevant communication.
This study has established the feasibility, validity, and reliability of a tool for evaluating postoperative handover. In addition to serving as an objective measure of postoperative handover, the tool can also be used to evaluate the efficacy of any intervention developed to improve this process. The study has also shown that postoperative handover is characterized by incomplete transfer of information and failures in the performance of key tasks.
Annals of surgery 04/2011; 253(4):831-7. · 7.90 Impact Factor
-
Cognition, Technology & Work. 01/2011; 13:1-10.
-
[show abstract]
[hide abstract]
ABSTRACT: In Britain over 39,000 reports were received by the National Patient Safety Agency relating to failures in documentation in 2007 and the UK Health Services Journal estimated in 2008 that over a million hospital outpatient visits each year might take place without the full record available. Despite these high numbers, the impact of missing clinical information has not been investigated for hospital outpatients in the UK.Studies in primary care in the USA have found 13.6% of patient consultations have missing clinical information, with this adversely affecting care in about half of cases, and in Australia 1.8% of medical errors were found to be due to the unavailability of clinical information.Our objectives were to assess the frequency, nature and potential impact on patient care of missing clinical information in NHS hospital outpatients and to assess the principal causes. This is the first study to present such figures for the UK and the first to look at how clinicians respond, including the associated impact on patient care.
Prospective descriptive study of missing information reported by surgeons, supplemented by interviews on the causes.Data were collected by surgeons in general, gastrointestinal, colorectal and vascular surgical clinics in three teaching hospitals across the UK for over a thousand outpatient appointments. Fifteen interviews were conducted with those involved in collating clinical information for these clinics.The study had ethics approval (Hammersmith and Queen Charlotte's & Chelsea Research Ethics Committee), reference number (09/H0707/27). Participants involved in the interviews signed a consent form and were offered the opportunity to review and agree the transcript of their interview before analysis. No patients were involved in this research.
In 15% of outpatient consultations key items of clinical information were missing. Of these patients, 32% experienced a delay or disruption to their care and 20% had a risk of harm. In over half of cases the doctor relied on the patient for the information, making a clinical decision despite the information being missing in 20% of cases. Hospital mergers, temporary staff and non-integrated IT systems were contributing factors.
If these findings are replicated across the NHS then almost 10 million outpatients are seen each year without key clinical information, creating over a million unnecessary appointments, and putting nearly 2 million patients at risk of harm. There is a need for a systematic, regular audit of the prevalence of missing clinical information. Only then will we know the impact on clinical decision making and patient care of new technology, service reorganisations and, crucially given the present financial climate, temporary or reduced staffing levels. Further research is needed to assess the relationship between missing clinical information and diagnostic errors; to examine the issue in primary care; and to consider the patients perspective.
BMC Health Services Research 01/2011; 11:114. · 1.66 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To assess the trends in uptake of minimal invasive esophagectomy in England over the last 12 years (1996/1997-2007/2008) and to compare their clinical outcomes with those after open esophagectomy.
Around 7400 people are affected each year in the United Kingdom. Prognosis following esophageal resection is, however, poor. Even after "curative" surgery, 5-year survival rates do not exceed 25%. The minimally invasive approach to esophagectomy has attracted attention as a potentially less invasive alternative to conventional surgery.
Data on patients undergoing esophagectomy for esophageal cancer were extracted from a national administrative database. The outcomes of interest were in-hospital mortality, 30-day in-hospital mortality, 30-day total (ie, in and out of hospital) mortality, 365-day total mortality, 28-day emergency readmission rates, and length of hospital stay. Hierarchical logistic regression was used to identify the effect of minimal invasive esophagectomy (MIE) on the outcomes after adjustment for age, gender, socioeconomic deprivation, and comorbidity.
A total of 18,673 esophagectomies were performed over the 12-year study period. The use of minimal access surgery increased exponentially over time (from 0.6% in 1996/1997 to 16.0% in 2007/2008). There was a suggestion that patients undergoing MIE had better 1-year survival rates than patients receiving open esophagectomy (OR = 0.68, 95% CI = 0.46-1.01, P = 0.058).
The uptake of MIE in England is increasing exponentially. With the possible exception of 1-year survival, patients selected for MIE demonstrated similar mortality and length of stay outcomes when compared with those undergoing conventional surgery. These results need to be confirmed in large-scale randomized controlled trials.
Annals of surgery 08/2010; 252(2):292-8. · 7.90 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We conducted a systematic review of published literature to gain a better understanding of interprofessional information transfer and communication (ITC) in hospital setting in the field of surgical and anesthetic care.
Communication breakdowns are a common cause of surgical errors and adverse events.
Medline, Embase, PsycINFO, Cochrane Database of Systematic Reviews, and hand search of articles bibliography.
Of the 4027 citations identified through the initial electronic search and screened for possible inclusion, 110 articles were retained following title and abstract reviews. Of these, 38 were accepted for this review.
Data were extracted from the studies about objectives, clinical domain, methodology including study design, sample population, tools for assessing communication, results, and limitations.
Information transfer failures are common in surgical care and are distributed across the continuum of care. They not only lead to errors in care provision but also lead to patient harm. Most of the articles have focused on ITC process in different phases especially in operating room. None of the studies have looked at whole of the surgical care process. No standard tool has been developed to capture the ITC process in different teams and to evaluate the effect of various communication interventions. Uses of standardized communication through checklist, proformas, and technology innovations have improved the ITC process, with an effect on clinical and patient outcomes.
ITC deficits adversely affect patient care. There is a need for standard measures to evaluate this process. Effective and standardized communication among healthcare professionals during the perioperative process facilitates surgical safety.
Annals of surgery 08/2010; 252(2):225-39. · 7.90 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To evaluate information transfer and communication (ITC) across the surgical care pathway with the use of Information Transfer and Communication Assessment Tool for Surgery (ITCAS).
Communication failures are the leading cause of surgical errors and adverse events. It is vital to assess the ITC across the entire surgical continuum of care to understand the process, to study teams, and to prioritize the phases for intervention.
Twenty patients undergoing major gastrointestinal procedures were followed through their entire surgical care, and ITC process was assessed using ITCAS. ITCAS consisted of 4 checklists for 4 phases of the surgical care.
ITC failures are distributed across the entire surgical continuum of care. Preprocedural teamwork and postoperative handover phases have the maximum number of ITC failures (61.7% and 52.4%, respectively). Moreover, it was found that information degrades as it crosses from one phase to another. Of patients, 75% had clinical incidents or adverse events because of ITC failures.
The study demonstrated that ITC failures are ubiquitous across surgical care pathway and there is an imminent need to modify current ITC practices. Standardization of ITC through use of checklists, protocols, or information technology is essential to reduce these communication failures.
Annals of surgery 08/2010; 252(2):402-7. · 7.90 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To identify the information transfer and communication problems in postoperative handover and to develop and validate a novel protocol for standardizing this communication.
Effective clinical handover ensures continuity of patient care. Patient handovers within surgical units are largely informal. A thorough understanding of the problem is vital to develop standardized protocols.
A qualitative semistructured interview study was conducted with 18 healthcare professionals to uncover the problems with postoperative handover and to identify solutions, including components of a postoperative handover protocol. Interviews were recorded, transcribed verbatim, and submitted to emergent theme analysis. Multiple blind coders were used to ensure triangulation and reliability of the coding process. A Delphi method was used to elicit consensus from a group of 50 surgical professionals so as to validate the handover protocol.
Many of the information transfer and communication failures at the postoperative phase are deemed to be due to an incomplete handover. All the interviewed healthcare professionals agreed that postoperative handover should be structured in the form of a standardized protocol so as to prevent omissions of any critical information. Based on this, 28 items were submitted to the Delphi process. Of these, 21 items had a mean importance score greater than 4.0 and were included in the final postoperative handover proforma under the following headings: patient-specific information, surgical information, and anesthetic information.
The present study identified that the postoperative handover is informal, unstructured and inconsistent with often incomplete information transfer. Based on end-user input, a handover protocol was successfully developed and validated. Use of this may facilitate standardization of this critical activity and thereby improve the quality of patient care.
Annals of surgery 07/2010; 252(1):171-6. · 7.90 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Open esophagectomy for cancer is a major oncological procedure, associated with significant morbidity and mortality. Recently, thoracoscopic procedures have offered a potentially advantageous alternative because of less operative trauma compared with thoracotomy. The aim of this study was to utilize meta-analysis to compare outcomes of open esophagectomy with those of minimally invasive esophagectomy (MIE) and hybrid minimally invasive esophagectomy (HMIE).
Literature search was performed using Medline, Embase, Cochrane Library, and Google Scholar databases for comparative studies assessing different techniques of esophagectomy. A random-effects model was used for meta-analysis, and heterogeneity was assessed. Primary outcomes of interest were 30-day mortality and anastomotic leak. Secondary outcomes included operative outcomes, other postoperative outcomes, and oncological outcomes in terms of lymph nodes retrieved.
A total of 12 studies were included in the analysis. Studies included a total of 672 patients for MIE and HMIE, and 612 for open esophagectomy. There was no significant difference in 30-day mortality; however, MIE had lower blood loss, shorter hospital stay, and reduced total morbidity and respiratory complications. For all other outcomes, there was no significant difference between the two groups.
Minimally invasive esophagectomy is a safe alternative to the open technique. Patients undergoing MIE may benefit from shorter hospital stay, and lower respiratory complications and total morbidity compared with open esophagectomy. Multicenter, prospective large randomized controlled trials are required to confirm these findings in order to base practice on sound clinical evidence.
Surgical Endoscopy 07/2010; 24(7):1621-9. · 4.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Health care failure mode and effect analysis identifies critical processes prone to information transfer and communication failures and suggests interventions to improve these failures.
Failure mode and effect analysis.
Academic research.
A multidisciplinary team consisting of surgeons, anesthetists, nurses, and a psychologist involved in various phases of surgical care was assembled.
A flowchart of the whole process was developed. Potential failure modes were identified and evaluated using a hazard matrix score. Recommendations were determined for certain critical failure modes using a decision tree.
The process of surgical care was divided into the following 4 main phases: preoperative assessment and optimization, preprocedural teamwork, postoperative handover, and daily ward care. Most failure modes were identified in the preoperative assessment and optimization phase. Forty-one of 132 failures were classified as critical, 26 of which were sufficiently covered by current protocols. Recommendations were made for the remaining 15 failure modes.
Modified health care failure mode and effect analysis proved to be a practical approach and has been well received by clinicians. Systematic analysis by a multidisciplinary team is a useful method for detecting failure modes.
Archives of surgery (Chicago, Ill.: 1960) 06/2010; 145(6):582-8. · 4.32 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To describe national trends in bariatric surgery and examine the factors influencing outcome in bariatric surgery in England.
Observational population cohort study.
Hospital Episode Statistics database.
All patients who had primary gastric bypass, gastric banding, or sleeve gastrectomy procedures between April 2000 and March 2008.
30 day mortality, mortality at one year after surgery, unplanned readmission to hospitalwithin 28 days, and duration of stay in hospital.
6953 primary bariatric procedures were carried out during the study period, of which 3649 were gastric band procedures, 3191 were gastric bypass procedures, and 113 were sleeve gastrectomy procedures. A marked increase occurred in the numbers of bariatric procedures done, from 238 in 2000 to 2543 in 2007, with an increase in the percentage of laparoscopic procedures over the study period (28% (66/238) laparoscopic procedures in 2000 compared with 74.5% (1894/2543) in 2007). Overall, 0.3% (19/6953) patients died within 30 days of surgery. The median length of stay in hospital was 3 (interquartile range 2-6) days. An unplanned readmission to hospital within 28 days of surgery occurred in 8% (556/6953) of procedures. No significant increase in mortality or unplanned readmission was seen over the study period, despite the exponential increase in minimal access surgery and consequently bariatric surgery.
Bariatric surgery has increased exponentially in England. Although postoperative weight loss and reoperation rates were not evaluated in this observational population cohort study, patients selected for gastric banding had lower postoperative mortality and readmission rates and a shorter length of stay than did those selected for gastric bypass.
BMJ (Clinical research ed.). 01/2010; 341:c4296.
-
[show abstract]
[hide abstract]
ABSTRACT: The advancement of surgical technology has made surgery an increasingly suitable management option for an increasing number of medical conditions. Yet there is also a growing concern about the number of patients coming to harm as a result of surgery. Studies show that this harm can be prevented by better teamwork and communication in operating theatres. This article discusses the extent of adverse events in surgery and how effective teamwork and communication can improve patient safety. It also highlights the role checklists and briefing in improving teamwork and reducing human error in surgery.
Journal of perioperative practice 10/2009; 19(10):330-4.
-
Surgery 07/2009; 145(6):689-90. · 3.10 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To determine the feasibility, validity, inter-rater, and intertest reliability of 4 previously published video-based rating scales, for technical skills assessment on a benchmark laparoscopic procedure.
Assessment of technical skills is crucial to the demonstration and maintenance of competent healthcare practitioners. Traditional assessment methods are prone to subjectivity through a lack of proven validity and reliability.
Nineteen surgeons (6 novice and 13 experienced) performed a median of 2 laparoscopic cholecystectomies each (range 1-5) on 53 patients within 2 Academic Surgical Departments. All patients had a diagnosis of biliary colic. Surgical technical skills were rated posthoc in a blinded manner by 2 experienced observers on 4 video-based rating scales. The different scales used had been developed to assess generic or procedure-specific technical skills in a global manner, or on a procedure-specific checklist.
Six of 53 procedures were excluded on the basis of intraoperative difficulty. Of the remaining 47 procedures, 14 were performed by 6 novice surgeons and 33 by the 13 experienced surgeons. There were statistically significant differences between performance of the 2 groups on the generic global rating scale (median 24 vs. 27, P = 0.031), though not on procedural or checklist-based scales. All scales demonstrated inter-rater reliability (alpha = 0.58-0.76), though only the global rating scales exhibited intertest reliability (alpha = 0.72).
Video-based technical skills evaluation in the operating room is feasible, valid and reliable. Global rating scales hold promise for summative assessment, though further work is necessary to elucidate the value of procedural rating scales.
Annals of Surgery 03/2008; 247(2):372-9. · 7.49 Impact Factor
