[Show abstract][Hide abstract] ABSTRACT: Competency-based education and simulation are being used more frequently in surgical skills curricula. We explored a novel student-led learning (SLL) paradigm, which allows trainees to become more active participants in the learning process while maintaining expert guidance and supervision.
[Show abstract][Hide abstract] ABSTRACT: Ensuring competence remains a seminal objective of endoscopy training programs, professional organizations, and accreditation bodies; however, no widely accepted measure of endoscopic competence currently exists.
By using Delphi methodology, we aimed to develop and establish the content validity of the Gastrointestinal Endoscopy Competency Assessment Tool for colonoscopy.
An international panel of endoscopy experts rated potential checklist and global rating items for their importance as indicators of the competence of trainees learning to perform colonoscopy. After each round, responses were analyzed and sent back to the experts for further ratings until consensus was reached.
Consensus was defined a priori as ≥80% of experts, in a given round, scoring ≥4 of 5 on all remaining items.
Fifty-five experts agreed to be part of the Delphi panel: 43 gastroenterologists, 10 surgeons, and 2 endoscopy managers. Seventy-three checklist and 34 global rating items were generated through a systematic literature review and survey of committee members. An additional 2 checklist and 4 global rating items were added by Delphi panelists. Five rounds of surveys were completed before consensus was achieved, with response rates ranging from 67% to 100%. Seven global ratings and 19 checklist items reached consensus as good indicators of the competence of clinicians performing colonoscopy.
Further validation required.
Delphi methodology allowed for the rigorous development and content validation of a new measure of endoscopic competence, reflective of practice across institutions. Although further evaluation is required, it is a promising step toward the objective assessment of competency for use in colonoscopy training, practice, and research.
[Show abstract][Hide abstract] ABSTRACT: Valid and reliable techniques for assessing performance are essential to surgical education, especially with the emergence of competency-based frameworks. Despite this, there is a paucity of adequate tools for the evaluation of skills required during joint replacement surgery. In this scoping review, we examine current methods for assessing surgeons' competency in joint replacement procedures in both simulated and clinical environments. The ability of many of the tools currently in use to make valid, reliable and comprehensive assessments of performance is unclear. Furthermore, many simulation-based assessments have been criticised for a lack of transferability to the clinical setting. It is imperative that more effective methods of assessment are developed and implemented in order to improve our ability to evaluate the performance of skills relating to total joint replacement. This will enable educators to provide formative feedback to learners throughout the training process to ensure that they have attained core competencies upon completion of their training. This should help ensure positive patient outcomes as the surgical trainees enter independent practice. Cite this article: Bone Joint J 2013;95-B:1445-9.
The bone & joint journal. 11/2013; 95-B(11):1445-9.
[Show abstract][Hide abstract] ABSTRACT: Previous studies have presented compelling data that a 1-month "boot-camp"-style course can be a highly effective mechanism for teaching and developing targeted technical skills. In the current study, we examine whether performance of these targeted skills is improved when residents are trained using directed, student-led (SL) learning methods compared with traditional instructor-led (IL) learning methods.
Twelve first-year orthopedic residents began their training with a 1-month, intensive skills course. Six residents were taught basic surgical skills using a format that focused on deliberate, SL exploration and practice of the skills under instructor supervision (SL group). The remaining residents were taught the same surgical skills using more traditional IL methods that included complete demonstration of the surgical task by an orthopedic surgeon, followed by an extended period of instruction (IL group). Performance on 4 targeted technical skills (sawing, bone drilling, suturing, and plaster splint application) was tested using an objective, structured assessment of technical skills examination for the 2 groups at the beginning and the end of the skills course.
Before the start of the skills course, there were no differences in performance scores between the 2 groups. On completion of the skills course, mean global rating scores for the 4 surgical skills tasks were greater for the SL group compared with the IL group: SL, 3.95 ± 0.1; IL, 3.42 ± 0.1; F(1,10) = 7.66 P < .02. A similar pattern of results was revealed by the checklists scores, with the SL group outperforming the IL group: SL, 94.9 ± 2.1; IL, 86.4 ± 2.1; F(1,10) = 8.512; P < .02.
Previous work has demonstrated the effectiveness of teaching basic surgical skills through an intensive course at the onset of residency. The present study shows that allowing surgical trainees to take a directed, student-regulated approach to learning basic surgical skills can further improve performance of these skills.
Surgery 07/2013; 154(1):29-33. · 3.37 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: To develop and evaluate an objective method of technical skills assessment for graduating subspecialists in colorectal (CR) surgery-the Colorectal Objective Structured Assessment of Technical Skill (COSATS).
It may be reasonable for the public to assume that surgeons certified as competent have had their technical skills assessed. However, technical skill, despite being the hallmark of a surgeon, is not directly assessed at the time of certification by surgical boards.
A procedure-based, multistation technical skills examination was developed to reflect a sample of the range of skills necessary for CR surgical practice. These consisted of bench, virtual reality, and cadaveric models. Reliability and construct validity were evaluated by comparing 10 graduating CR residents with 10 graduating general surgery (GS) residents from across North America. Expert CR surgeons, blinded to level of training, evaluated performance using a task-specific checklist and a global rating scale. The mean global rating score was used as the overall examination score and a passing score was set at "borderline competent for CR practice."
The global rating scale demonstrated acceptable interstation reliability (0.69) for a homogeneous group of examinees. Both the overall checklist and global rating scores effectively discriminated between CR and GS residents (P < 0.01), with 27% of the variance attributed to level of training. Nine CR residents but only 3 GS residents were deemed competent.
The Colorectal Objective Structured Assessment of Technical Skill effectively discriminated between CR and GS residents. With further validation, the Colorectal Objective Structured Assessment of Technical Skill could be incorporated into the colorectal board examination where it would be the first attempt of a surgical specialty to formally assess technical skill at the time of certification.
[Show abstract][Hide abstract] ABSTRACT: The current methods used to train residents to become orthopaedic surgeons are based on tradition, not evidence-based models. Educators have only a limited ability to assess trainees for competency using validated tests in various domains. The reduction in resident work hours limits the time available for clinical training, which has resulted in some calls for lengthening the training process. Another approach to address limited training hours is to focus training in a program that allows residents to graduate from a rotation based on demonstrated competency rather than on time on a service. A pilot orthopaedic residency curriculum, which uses a competency-based framework of resident training and maximizes the use of available training hours, has been designed and is being implemented.
[Show abstract][Hide abstract] ABSTRACT: OBJECTIVE:: To develop and validate an ex vivo comprehensive curriculum for a basic laparoscopic procedure. BACKGROUND:: Although simulators have been well validated as tools to teach technical skills, their integration into comprehensive curricula is lacking. Moreover, neither the effect of ex vivo training on learning curves in the operating room (OR), nor the effect on nontechnical proficiency has been investigated. METHODS:: This randomized single-blinded prospective trial allocated 20 surgical trainees to a structured training and assessment curriculum (STAC) group or conventional residency training. The STAC consisted of case-based learning, proficiency-based virtual reality training, laparoscopic box training, and OR participation. After completion of the intervention, all participants performed 5 sequential laparoscopic cholecystectomies in the OR. The primary outcome measure was the difference in technical performance between the 2 groups during the first laparoscopic cholecystectomy. Secondary outcome measures included differences with respect to learning curves in the OR, technical proficiency of each sequential laparoscopic cholecystectomy, and nontechnical skills. RESULTS:: Residents in the STAC group outperformed residents in the conventional group in the first (P = 0.004), second (P = 0.036), third (P = 0.021), and fourth (P = 0.023) laparoscopic cholecystectomies. The conventional group demonstrated a significant learning curve in the OR (P = 0.015) in contrast to the STAC group (P = 0.032). Residents in the STAC group also had significantly higher nontechnical skills (P = 0.027). CONCLUSIONS:: Participating in the STAC shifted the learning curve for a basic laparoscopic procedure from the operating room into the simulation laboratory. STAC-trained residents had superior technical proficiency in the OR and nontechnical skills compared with conventionally trained residents. (The study registration ID is NCT01560494.).
[Show abstract][Hide abstract] ABSTRACT: We examined retention rates for basic surgical skills taught through a 1-month intensive laboratory boot camp-style course at the onset of residency.
We present data from 3 groups, each composed of 6 residents. The first group consisted of residents from a new competency-based curriculum (CBC). They started residency training with the Toronto Orthopaedic Boot Camp course. The other 2 groups were junior (JR) and senior (SR) residents from a traditional program whose residency training included no such course. Performance on targeted technical skills was tested using an objective structured assessment of technical skills examination 7 months after the onset of training for the CBC and JR groups and at least 43 months after the onset of training for the SR group.
The mean global rating scale score for the CBC group immediately after the skills course was 4.3, which was maintained 6 months later. There were no significant performance differences between the CBC and SR groups. Both the CBC and SR groups performed significantly better than the JR group (mean global rating scale 3.7; F[2, 15] = 12.269, P < .001).
We conclude that a surgical skills course at the onset of residency is an effective mechanism for teaching targeted technical skills and that skills taught in this manner can have excellent retention rates. Furthermore, an early focus on technical skills allows junior residents to perform at the same level as senior residents for certain tasks and may privilege later learning.
Surgery 06/2012; 151(6):803-7. · 3.37 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: To compare the effectiveness and cost of 2 ex vivo training curricula for laparoscopic suturing.
Although simulators have been developed to teach laparoscopic suturing, a barrier to their wide implementation in training programs is a lack of knowledge regarding their relative training benefit and their associated cost.
This prospective single-blinded randomized trial allocated 24 surgical residents to train to proficiency using either a virtual reality (VR) simulator or box trainer. All residents then placed intracorporeal laparoscopic stitches during a Nissen fundoplication on a patient. The operating room (OR) cases were video-recorded and technical proficiency was assessed using 2 validated tools. OR performance of both groups was compared to that of conventionally trained residents and to fellowship-trained surgeons. A cost analysis of box training, VR training, and conventional residency training across Canadian surgical programs was performed.
After ex vivo training, no significant differences in laparoscopic suturing in the OR were found between the 2 groups with respect to time (P = 0.74)-global rating score (P = 0.65) or checklist score (P = 0.97). It took conventionally trained residents 6 practice attempts in the OR to achieve the technical proficiency of the ex vivo trained groups (P = 0.83). VR training was more efficient than box training (transfer effectiveness ratio of 2.31 vs 1.13). The annual cost of training 5 residents on the FLS trainer box was $11,975.00, on the VR simulator was $77,500.00, and conventional residency training was $17,380.00. Over 5 years, box training was the most cost-effective option for all programs, and VR training was more cost-effective for programs with more 10 residents.
Training on either a VR simulator or on a box trainer significantly decreased the learning curve necessary to learn laparoscopic suturing. VR training, however, is the more efficient training modality, whereas box training the more cost-effective option.
Annals of surgery 05/2012; 255(5):833-9. · 7.90 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Changes in health care across the globe have had a profound impact on the number of hands-on surgical training opportunities that are available to residents. In the current study, we examine whether an intensive laboratory-based skills course at the start of orthopedic surgical training is an effective mechanism for teaching core technical skills.
First-year residents were divided into 3 groups (on-service, n = 8; off-service, n = 8; and a new, competency-based program that has as a major element of the curriculum a focused, intensive skills laboratory-based experience, n = 6). Baseline surgical skills were assessed prior to commencing training. The intensive skills laboratory group was then given an intensive surgical skills course, whereas the other 2 groups embarked on traditional residency. After the surgical skills course, all the residents were assessed for core surgical skills using an objective structured assessment of technical skills (OSATS) procedure.
Pretraining scores revealed no differences between the groups of residents using both checklist (F[2,19] = 0.852, P = .442) and global rating scores (F[2,19] = 0.704, P = .507). Post-training scores revealed a significant difference, with residents from the intensive skills laboratory group performing better on both the checklists (on-service = 78.9, off-service = 78.6, intensive skills laboratory = 92.3; F[2,19] = 6.914, P < .01) and global rating scores (on-service = 3.4, off-service = 3.4, intensive skills laboratory = 4.3; F[2,19] = 5.722, P < .01), than the other groups who showed no differences between them.
The intensive skills course used in this study was highly effective at teaching and developing targeted surgical skills in first-year orthopedic residents. We predict that allowing residents to acquire key technical skills at the start of their training will enhance learning opportunities at later stages of training.
Surgery 06/2011; 149(6):745-9. · 3.37 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: The use of networks for sharing and distributing information, for institutional collaboration, and action programs is commonplace. In 1989, the Medical Council of Canada began the implementation of a national clinical licensing examination to assess physicians for practice skills and decision making using standardized or simulated patients in an Objective Structured Clinical Examination format. Once fully implemented, the examination was administered through a network of medical schools at 16 locations across Canada in two languages twice yearly. That network has functioned successfully for 17 years. This article reviews the literature and examines the reasons and incentives for the long-term sustainability of the network. Based on that assessment, a framework is presented for analyzing, designing, and sustaining a national simulation network. It emphasizes the need for an iterative approach and identifies the success factors that can facilitate the adoption of a national simulation network for use in professional credentialing and licensure.
Simulation in healthcare: journal of the Society for Simulation in Healthcare 04/2011; 6(2):94-100. · 1.64 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND Suboptimal communication within healthcare teams can lead to adverse patient outcomes. Team briefings were previously associated with improved communication patterns, and we assessed the impact of briefings on clinical practice. To quantify the impact of the preoperative team briefing on direct patient care, we studied the timing of preoperative antibiotic administration as compared to accepted treatment guidelines. STUDY DESIGN A retrospective pre-intervention/post-intervention study design assessed the impact of a checklist-guided preoperative team briefing on prophylactic antibiotic administration timing in surgical cases (N=340 pre-intervention and N=340 post-intervention) across three institutions. χ(2) Analyses were performed to determine whether there was a significant difference in timely antibiotic administration between the study phases. RESULTS The process of collecting and analysing these data proved to be more complicated than expected due to great variability in documentation practices, both between study sites and between individual practitioners. In cases where the timing of antibiotics administration was documented unambiguously in the chart (n=259 pre-intervention and n=283 post-intervention), antibiotic prophylaxis was on time for 77.6% of cases in the pre-intervention phase of the study, and for 87.6% of cases in the post-intervention phase (p<0.01). CONCLUSIONS Use of a preoperative team checklist briefing was associated with improved physician compliance with antibiotic administration guidelines. Based on the results, recommendations to enhance timely antibiotic therapy are provided.
[Show abstract][Hide abstract] ABSTRACT: As the medical education community celebrates the 100th anniversary of the seminal Flexner Report, medical education is once again experiencing significant pressure to transform. Multiple reports from many of medicine's specialties and external stakeholders highlight the inadequacies of current training models to prepare a physician workforce to meet the needs of an increasingly diverse and aging population. This transformation, driven by competency-based medical education (CBME) principles that emphasize the outcomes, will require more effective evaluation and feedback by faculty.Substantial evidence suggests, however, that current faculty are insufficiently prepared for this task across both the traditional competencies of medical knowledge, clinical skills, and professionalism and the newer competencies of evidence-based practice, quality improvement, interdisciplinary teamwork, and systems. The implication of these observations is that the medical education enterprise urgently needs an international initiative of faculty development around CBME and assessment. In this article, the authors outline the current challenges and provide suggestions on where faculty development efforts should be focused and how such an initiative might be accomplished. The public, patients, and trainees need the medical education enterprise to improve training and outcomes now.
Academic medicine: journal of the Association of American Medical Colleges 02/2011; 86(4):460-7. · 2.34 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: To assess the relationship between changes in clinician attitude and changes in postoperative outcomes following a checklist-based surgical safety intervention.
Pre- and post intervention survey.
Eight hospitals participating in a trial of a WHO surgical safety checklist.
Clinicians actively working in the designated study operating rooms at the eight hospitals. SURVEY INSTRUMENT: Modified operating-room version Safety Attitudes Questionnaire (SAQ).
Change in mean safety attitude score and correlation between change in safety attitude score and change in postoperative outcomes, plus clinician opinion of checklist efficacy and usability.
Clinicians in the preintervention phase (n=281) had a mean SAQ score of 3.91 (on a scale of 1 to 5, with 5 representing better safety attitude), while the postintervention group (n=257) had a mean of 4.01 (p=0.0127). The degree of improvement in mean SAQ score at each site correlated with a reduction in postoperative complication rate (r=0.7143, p=0.0381). The checklist was considered easy to use by 80.2% of respondents, while 19.8% felt that it took a long time to complete, and 78.6% felt that the programme prevented errors. Overall, 93.4% would want the checklist used if they were undergoing operation.
Improvements in postoperative outcomes were associated with improved perception of teamwork and safety climate among respondents, suggesting that changes in these may be partially responsible for the effect of the checklist. Clinicians held the checklist in high regard and the overwhelming majority would want it used if they were undergoing surgery themselves.
[Show abstract][Hide abstract] ABSTRACT: Although the objective in European Union and North American surgical residency programmes is similar-to train competent surgeons-residents' working hours are different. It was hypothesized that practice-ready surgeons with more working hours would perform significantly better than those being educated within shorter working week curricula.
At each test site, 21 practice-ready candidate surgeons were recruited. Twenty qualified Canadian and 19 qualified Dutch surgeons served as examiners. At both sites, three validated outcome instruments assessing multiple aspects of surgical competency were used.
No significant differences were found in performance on the integrative and cognitive examination (Comprehensive Integrative Puzzle) or the technical skills test (Objective Structured Assessment of Technical Skill; OSATS). A significant difference in outcome was observed only on the Patient Assessment and Management Examination, which focuses on skills needed to manage patients with complex problems (P < 0.001). A significant interaction was observed between examiner and candidate origins for both task-specific OSATS checklist (P = 0.001) and OSATS global rating scale (P < 0.001) scores.
Canadian residents, serving many more working hours, perform equivalently to Dutch residents when assessed on technical skills and cognitive knowledge, but outperformed Dutch residents in skills for patient management. Secondary analyses suggested that cultural differences influence the assessment process significantly.
British Journal of Surgery 03/2010; 97(3):443-9. · 4.84 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: To explore perceptions of how professionalism is learned in the current academic environment. Professionalism is a core competency in surgery (as in all of medical practice), and its presence or absence affects all aspects of clinical education and practice, but the ways in which professional values and attitudes are best transmitted to developing generations of surgeons have not been well defined.
The authors conducted 34 semistructured interviews of individual surgery residents and faculty members at two academic institutions from 2004 to 2006. Interviews consisted of open-ended questions on how the participants learned professionalism and what they perceived as challenges to learning professionalism. Two researchers analyzed the interview transcripts for emergent themes by using a grounded-theory approach.
Faculty members' and residents' perceptions of how they learned professionalism reflected four major themes: (1) personal values and upbringing, including premedical education experiences, (2) learning by example from professional role models, (3) the structure of the surgery residency, and (4) formal instruction on professionalism. Of these, role modeling was the dominant theme: Participants identified observation, reflection, and reinforcement as playing key roles in their learning from role models and in distinguishing the sometimes blurred boundary between positive and negative role models.
The theoretical framework generated out of this study proposes a focus on specific activities to improve professional education, including an active approach to role modeling through the intentional and explicit demonstration of professional behavior during the course of everyday work; structured, reflective self-examination; and timely and meaningful evaluation and feedback for reinforcement.
Academic medicine: journal of the Association of American Medical Colleges 01/2010; 85(1):134-9. · 2.34 Impact Factor