Systematic review on mentoring and simulation in laparoscopic colorectal surgery.

Abstract

To identify and evaluate the influence of mentoring and simulated training in laparoscopic colorectal surgery (LCS) and define the key components for learning advanced technical skills.
Laparoscopic colorectal surgery is a complex procedure, often being self-taught by senior surgeons. Educational issues such as inadequate training facilities or a shortfall of training fellowships may result in a slow uptake of LCS. The effectiveness of mentored and simulated training, however, remains unclear.
We conducted a systematic search, using Ovid databases. Four study categories were identified: mentored versus nonmentored cases, training case selection, simulation, and assessment. We performed a meta-analysis and a mixed model regression on the difference of the main outcome measures (conversion rates, morbidity, and mortality) for mentored trainees and expert surgeons. We also compared conversion rates of mentored and nonmentored. Meta-analysis of risk factors for conversion was performed using published and unpublished data sets requested from various investigators. For studies on simulation, we compared scores of surveys on the perception of different training courses.
Thirty-seven studies were included. Pooled weighted outcomes of mentored cases (n = 751) showed a lower conversion rate (13.3% vs 20.5%, P = 0.0332) compared with nonmentored cases (n = 695). Compared to expert case series (n = 5313), there was no difference in conversion (P = 0.2835), anastomotic leak (P = 0.8342), or mortality (P = 0.5680). A meta-analysis of training case selection data (n = 4444) revealed male sex (P < 0.0001), previous abdominal surgery (P = 0.0200), a BMI greater than 30 (P = 0.0050), an ASA of less than 2 (P < 0.0001), colorectal cancer (P < 0.0001) and intra-abdominal fistula (P < 0.0001), but not older than 64 years (P = 0.4800), to significantly increase conversion risk. Participants on cadaveric courses were highly satisfied with the teaching value yet trainees on an animal course gave less positive feedback. Structured assessment for LCS has been partially implemented.
This review and meta-analysis supports evidence that trainees can obtain similar clinical results like expert surgeons in laparoscopic colorectal surgery if supervised by an experienced trainer. Cadaveric models currently provide the best value for training in a simulated environment. There remains a need for further research into technical skills assessment and the educational value of simulated training.

Full text

Page 1

REVIEW PAPER
Systematic Review on Mentoring and Simulation in
Laparoscopic Colorectal Surgery
Danilo Miskovic, MD, FRCS, Susannah M. Wyles, MSc, MRCS, Melody Ni, MSc,
Ara W. Darzi, FMedSci HonFREng KBE, and George B. Hanna, PhD, FRCS
Objective: To identify and evaluate the influence of mentoring and simu-
lated training in laparoscopic colorectal surgery (LCS) and define the key
components for learning advanced technical skills.
Background: Laparoscopic colorectal surgery is a complex procedure, often
being self-taught by senior surgeons. Educational issues such as inadequate
training facilities or a shortfall of training fellowships may result in a slow
uptakeofLCS.Theeffectivenessofmentoredandsimulatedtraining,however,
remains unclear.
Methods:Weconductedasystematicsearch,usingOviddatabases.Fourstudy
categories were identified: mentored versus nonmentored cases, training case
selection, simulation, and assessment. We performed a meta-analysis and a
mixed model regression on the difference of the main outcome measures
(conversion rates, morbidity, and mortality) for mentored trainees and expert
surgeons. We also compared conversion rates of mentored and nonmentored.
Meta-analysis of risk factors for conversion was performed using published
and unpublished data sets requested from various investigators. For studies
on simulation, we compared scores of surveys on the perception of different
training courses.
Results: Thirty-seven studies were included. Pooled weighted outcomes of
mentored cases (n = 751) showed a lower conversion rate (13.3% vs 20.5%,
P = 0.0332) compared with nonmentored cases (n = 695). Compared to
expert case series (n = 5313), there was no difference in conversion (P =
0.2835), anastomotic leak (P = 0.8342), or mortality (P = 0.5680). A meta-
analysis of training case selection data (n = 4444) revealed male sex (P
< 0.0001), previous abdominal surgery (P = 0.0200), a BMI greater than
30 (P = 0.0050), an ASA of less than 2 (P < 0.0001), colorectal cancer (P
< 0.0001) and intra-abdominal fistula (P < 0.0001), but not older than 64
years (P = 0.4800), to significantly increase conversion risk. Participants on
cadaveric courses were highly satisfied with the teaching value yet trainees on
an animal course gave less positive feedback. Structured assessment for LCS
has been partially implemented.
Conclusion: This review and meta-analysis supports evidence that trainees
canobtainsimilarclinicalresultslikeexpertsurgeonsinlaparoscopiccolorec-
talsurgeryifsupervisedbyanexperiencedtrainer.Cadavericmodelscurrently
provide the best value for training in a simulated environment. There remains
aneedforfurtherresearchintotechnicalskillsassessment andtheeducational
value of simulated training.
(Ann Surg 2010;252:943–951)
E
vidence exists for lengthy proficiency gain curves for technically
complex interventional procedures.1–4In laparoscopic colorec-
From the Department of Surgery and Cancer, Imperial College, St Mary’s Hospital,
Praed St, London, United Kingdom.
Supplementaldigitalcontentisavailableforthisarticle.DirectURLcitationappears
in the printed text and is provided in the HTML and PDF versions of this article
on the journal’s Web site (www.annalsofsurgery.com).
This work is funded by the National Cancer Action Team as part of the National
Training Programme in Laparoscopic Colorectal Surgery.
Reprints:GeorgeB.Hanna,PhD,FRCS,DepartmentofSurgeryandCancer,Impe-
rial College, St Mary’s Hospital, Praed St, London W2 1NY, United Kingdom.
E-mail: g.hanna@imperial.ac.uk.
CopyrightC ?2010 by Lippincott Williams & Wilkins
ISSN: 0003-4932/10/25206-0943
DOI: 10.1097/SLA.0b013e3181f662e5
Annals of SurgeryrVolume 252, Number 6, December 2010
tal surgery (LCS), this is coupled with a limited dissemination of
the technique despite the supportive evidence from randomized con-
trolled clinical trials.5–9In 2006, 32.6% of all colorectal resections
in the United States, and 10.4% in the United Kingdom, were per-
formed laparoscopically.10,11Among other factors, the limited uptake
of LCS may also be attributed to educational issues such as inade-
quatetrainingfacilitiesorashortfalloffellowshipsandothertraining
posts.12–14Furthermore, at present only limited structured guidelines
exist for the training of LCS.15The increasing emphasis on patient
safety and governance issues necessitate a prudent uptake of techni-
cally complex procedures and the framework for training in this con-
text remains unclear. In most Western countries, structured curricula
have been set up for training basic laparoscopic procedures, yet there
are few reports on dedicated programs for advanced techniques.15–18
LCS is a good example of a common complex procedure that is
technology-dependent, often being learnt by senior trainees, and
has been considered as an index procedure for advanced skills
training.19
The aim of this systematic review was to identify and eval-
uate the influence of mentoring and simulated training in LCS and
delineate the key components for learning advanced technical skills
without compromising patient safety.
METHODS
Question and Search Strategy
We searched Ovid MEDLINE (1950–2009, August Week 3),
EMBASE (1980–2009 Week 34), and PsycINFO (1967–2009, Au-
gust Week 3). We used 3 different domains of MeSH-terms and key
words combined by “AND,” and in each domain the terms were
combined by “OR.” The first domain contained terms on training,
the second on surgical anatomy (colon and rectum), and the third
on laparoscopy (Supplemental Digital Content 1). The selection was
not restricted by language. Two investigators reviewed titles and ab-
stracts resulting from the search, and those clearly unrelated to the
topic and any duplicates were excluded. For those possibly fulfilling
inclusion criteria, full text was retrieved and assessed for eligibility.
At this stage, one article was identified through cross-referencing and
included for final evaluation.20
Eligibility and Study Categorization
On reviewing the selected articles, 4 main categories were
identified. The first included reports on patient outcome data from
mentored and nonmentored training surgeons. Mentored, in this con-
text, referred to whether an expert laparoscopic colorectal surgeon
provided technical support, advice, and guidance to the trainee dur-
ing the operation, either by actually being present in the operating
room or remotely via video links (telementoring). We defined the
first 50 cases as representative of the early learning curve to be con-
sistent with studies demonstrating a plateau in the learning curve at
this level of experience.2,21
The second category was training case selection. These stud-
ies assessed certain patient attributes to determine factors that might
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
www.annalsofsurgery.com | 943

Page 2

Miskovic et al
Annals of SurgeryrVolume 252, Number 6, December 2010
FIGURE 1. Flow chart of the selection process for studies included in the systematic review.
predict a more difficult operation and thus increase the likelihood of
conversion from a laparoscopic to an open procedure. The third cat-
egory was simulation, which included studies on different simulated
training techniques. The fourth, skills assessment, incorporated arti-
cles on assessment methods used to describe the quality of technical
performance of LCS trainees.
We excluded case reports, letters or comments, editorials,
bulletins, reviews, descriptions of techniques, and studies on hand-
assisted,open,orroboticsurgery,andreportswithnoextractabledata.
If there were incomplete published data, we asked the corresponding
author for this by e-mail. If investigators failed to respond to a repeat
request or were unable to provide the desirable data, their study was
excluded. The search was performed with a good interrater reliability
(Cohen’s κ = 0.77).
Data Extraction, Outcome Measures, and Analysis
Two reviewers applied inclusion criteria independently and
extracted data into a standardized electronic format (Excel 2004 for
Mac,MicrosoftCorporation,Redmond,WA),guidedbytheCochrane
Reviewers’ Handbook.22Included articles were first allocated to the
4 groups specified within the study objectives, and then relevant data
were extracted according to the category.
The outcome measures were determined by conventional
clinical outcomes and desirable criteria for simulation and
assessment.18,23–25The end points for each category were as follows:
1. Mentored versus nonmentored: clinical outcome (morbidity, mor-
tality, conversion from laparoscopic to open procedure);
2. Training case selection: clinical outcome (morbidity, operation
time, mortality, conversion from laparoscopic to open procedure);
3. Simulation:
fidelity, effect
satisfaction.18(Fidelity in this context is defined as the degree
to which a simulation matches the real system and/or the environ-
ment in terms of physical and functional characteristics.)26
4. Skills assessment: reliability, validity and feasibility, acceptability,
educational impact, cost-effectiveness.23–25
on clinicaltraining,trainee
Two authors extracted conversion rates and morbidity data
from reports on 3 subgroups; mentored, and nonmentored cases, and
expert cases. The data from the expert series were taken as a control.
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
944 | www.annalsofsurgery.com
C ?2010 Lippincott Williams & Wilkins

Page 3

Annals of SurgeryrVolume 252, Number 6, December 2010 Mentoring and Simulation in Laparoscopy
For the analysis, we used the software Comprehensive Meta-Analysis
(Version 2.0, Biostat, Englewood, NJ).We performed ameta-analysis
for each subgroup separately, using a fixed and random effects model
for event rates. We did a subgroup analysis, using the mixed model
(method of moments) for levels of significance. A P value of less
than 0.05 was considered to be significant. Sensitivity analysis was
performed in each subgroup by running the analysis several times
and each time removing one study. No outliers could be identified in
this analysis.
For data pertaining to training case selection, two authors ex-
tracted conversion rates for different risk factors (age, BMI, sex,
neoplasia, ASA score, presence of intra-abdominal fistula, and pre-
vious open abdominal surgery) from multiple expert series and com-
puted odds ratios using Review Manager [Version 4.2.10 (November
13, 2006)].27Assuming heterogeneity among these studies, we ap-
pliedtheDerSimonian-Lairdrandom-effectsmethodfordichotomous
data.28Heterogeneity was tested by integrating Cochrane’s Q into the
I2-formula.29Nonpublished data for certain studies were obtained
directly from the investigators.10,21,30–32
Quality Assessment
It was not possible to apply a classical bias risk assessment
method for the included articles as no randomized controlled trials
were identified. Two reviewers independently assessed these studies
by applying the validated Newcastle-Ottawa scale for cohort studies
for bias risk assessment.33For a more comprehensive quality assess-
ment, we used the “signal-to-noise” concept. The noise represents
the methodological weakness of a study (1-method score). The sig-
nal represents the study in the context of the review question and
contains 3 key elements: relevance, applicability, and effect size.34,35
For each of these 3 components, there were 3 further items to rate
(Supplemental Digital Content 2). By multiplication of the relatives
ofsignaland method (1-noise)score,wecomputed a“signal-to-noise
ratio” with a score range of 0 to 1. Thus, only articles with maximum
signal and minimum noise would score 1, whereas studies with ei-
ther minimal signal or maximal noise would score 0, independent of
the size of the other factor. Interrater reliability was calculated using
Bland-Altman analysis.36Because this scoring system is yet to be
validated, it was used merely for descriptive purposes, and, therefore,
studieswereincludedorexcludedindependentoftheirsignal-to-noise
ratio. However, none of the included studies scored 0.
RESULTS
After removing duplicates, we retrieved 3420 citations from
the literature search (MEDLINE 3060 studies, EMBASE 360 stud-
ies, PsycINFO no studies). One hundred fifteen reports potentially
fulfilled our inclusion criteria. A total of 37 studies (22 cohort studies
andcaseseries,13interventionstudies,and2surveys)wereincluded.
Of these studies, 22 were used for mentored and nonmentored train-
ing, 8 for case selection, 6 for simulation, and 3 for assessment.
Several studies fell into more than 1 category (Fig. 3).
TABLE 1. Comparison of Pooled Weighted Outcomes of
Mentored and Expert Series
ParameterMentored∗
Experts∗
P-value†
Conversions
Complications
Anastomotic leak
Mortality
0.13 (0.10–0.17)
0.20 (0.16–0.24)
0.03 (0.01–0.06)
0.02 (0.01–0.04)
0.10 (0.77–0.13)
0.25 (0.19–0.31)
0.04 (0.02–0.08)
0.02 (0.01–0.02)
0.2853
0.4933
0.3682
0.5680
∗Pooled mean rates, random effects model (95% confidence intervals).
†Mixed effect model (method of moments).
Weexcludedatotalof79reports.Forty-ninestudieshadnonus-
able or nonrelevant data. In 28 studies, there was insufficient data on
initialexperience(“earlylearningcurve”).Onestudyonlaparoscopic
rectum resections was excluded as the surgeons performing 36 of the
first 50 resections had already previously performed more than 50
colon resections.37One further study could not be included as the
investigators were unable to provide more detailed data on individual
patients.38A table of excluded studies may be requested from the
authors; a table with the full description of all included studies is
available on the Internet (Supplemental Digital Content 3).
The average signal-to-noise ratio was 0.43 (0.22–0.70), the
methodscore0.65(0.44–0.83),andthesignalscore0.66(0.46–0.88).
The interrater reliability showed a variation of only +0.01 (−0.22 to
0.25) in the Bland-Altmann analysis (Fig. 2).
Mentored Versus Nonmentored Training
Five studies addressed the impact of a mentor’s physical pres-
ence in the operating department on the trainees’ performance, tak-
ing mortality, and morbidity data as end points from a total of 713
cases.31,39–43A further 2 studies, with 38 cases, assessed the effect of
a mentor present in the operating department only at the beginning
of the training phase, and then with subsequent training via a remote
video-link device for the further cases (telementoring).44,45No publi-
cationbiaswasdetectedforthesestudiesasnosignificantasymmetry
wasshownonthefunnelplot.46Acomparisonofthepooledweighted
outcomes of all mentored cases (n = 751) compared with those of
large expert case series (n = 5313) showed no significant difference
in conversion, anastomotic leak, or mortality rate (Table 1).2,21,30,47–52
Studies on mentored programs were compared with the early
case series (n = 695) of nonmentored (self-taught) surgeons in a
similar way.2,30,31,47,48,50,53–62Apart from conversion rate, data were
insufficient for the nonmentored group, which limited the possible
analysis. Nevertheless, a comparison of conversion rates revealed a
significant difference supporting the mentored group (13% for men-
tored, 20% for nonmentored, mixed model P = 0.0332, fixed model
P = 0.0005). Furthermore, the conversion rates were significantly
different for nonmentored and expert surgeons (20% vs 11%, both
models P < 0.0001), but no clear difference was found between men-
tored trainees and expert surgeons (13% vs 11%, mixed model P =
0.2835, fixed model P = 0.0127) (Table 2 and Fig. 3).
Selection of Training Case
We identified 8 studies that focused on training case selec-
tion. One was a survey study among expert surgeons that aimed to
rank different types of resection in terms of difficulty, and overall
rated right-sided colonic resections as easier than left-sided.63An-
other investigated the effect an increased visceral to body fat index,
rather than simply a high BMI, had on the rate of conversion.64Six
studies on case selection with a total of 4444 cases were used in
a meta-analysis to delineate predictive risk factors for conversion
and thereby determine case difficulty.2,21,30,32,51,52A meta-analysis re-
vealed that male gender (P < 0.0001), previous abdominal surgery
(P = 0.0200), a BMI greater than 30 (P = 0.0050), an ASA of 3 and
4 (P < 0.0001), the presence of colorectal cancer (P < 0.0001), or
of an intra-abdominal fistula (P < 0.0001) were factors that led to
an increased risk of conversion. Age of 65 years and greater did not
increase the conversion risk (P = 0.4800) (Table 3). Only 3 studies
provided data on postoperative and intraoperative complications or
readmission rates, but these data were defined or reported inconsis-
tently and, therefore, not suitable for analysis.2,21,32
Simulation
There were several studies on reconstructed nonlive animal
models, simulation with synthetic tissue, computer-based learning,
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
C ?2010 Lippincott Williams & Wilkins
www.annalsofsurgery.com | 945

Page 4

Miskovic et al
Annals of SurgeryrVolume 252, Number 6, December 2010
FIGURE 2. Equation for the signal-to-noise ratio and interrater reliability (Bland-Altman plot).
TABLE 2. Comparison of Pooled Weighted Conversion Rates
for Mentored, Nonmentored and Expert (Control) Series
[mixed effect model (Method of Moments)]
Statistical
Model
Mentored
(n = 751)
0.14 (0.12–0.17)
0.13 (0.10–0.17)
P = 0.2835
Nonmentored
(n = 665)
0.22 (0.19–0.25)
0.21 (0.15–0.26)
P = 0.0002021
Expert (n = 5313)
0.12 (0.11–0.13)
0.12 (0.10–0.13)
P = 0.0332
Fixed effects
Random effects
and cadaveric and animal LCS training courses.39,65–73Of these, only
4 had extractable data, which pertained to candidate satisfaction, and
these were included for analysis.39,71–73There were no reliable data
obtained for fidelity (tissue property, anatomical landmarks, envi-
ronmental context). And again, unlike for basic laparoscopic skills
in other specialties, objective data on the impact of the course on
clinical skills were not demonstrated.18Having adjusted the satisfac-
tion scores to comparable ranges of Likert scale, the synthesis of data
showedthatparticipantsofcourseswithcadavericmodelswerehighly
satisfied with the teaching value and the reliability of the materials
used[mean4.5(range,4–5)].72,73Traineesontheanimal(pig)course,
by comparison, reported considerably less favorable feedback [2.15
(range,1.5–3)].39ThenumberofdelegatesperformingLCSincreased
from 53% to 81% after attending a cadaveric training course, with
26% of graduates performing, on average, one colectomy a week.71
One study demonstrated construct validity of a synthetic tissue-based
simulator with expert surgeons performing significantly better than
novices, and another showed a self-perceived increase in resident’s
knowledge after completing a multimedia course.69,70
Assessment
There were 3 studies on the implementation of assessment
tools in LCS training. A global assessment score, which has been
partially validated previously, was used to evaluate the trainee’s abil-
ity in 2 studies, and the completion of the operation in another.43,74,75
The studies using a global assessment score show good interitem and
interrater reliability [Cronbach α = 0.88 and 0.82, respectively, inter-
rater reliability 0.76 (Pearson correlation)].74,75None of the studies
tested specifically for predictive validity, acceptability, educational
impact, or cost-effectiveness.
DISCUSSION
Thisreviewshowsthatmentoredtraineesachievegoodclinical
results in LCS. Surgeons perceive that teaching on cadaveric models
provides a realistic method for training in a simulated environment.
There remains a lack of data on the assessment of technical skills.
Our study is the first report with a comparative meta-analysis
of clinical outcome data in training and was possible only by retriev-
ing nonpublished data directly from the investigators. Analyzed data
of 6064 patients reflect that trainees with an appropriate level of su-
pervision generate the same complication, conversion, and mortality
rates as expert surgeons. Similar results have been shown for carotid
endarterectomy in a large, single-center study.76
It is difficult to measure accurately manual performance dur-
ing training. In this review, only 3 studies fell into the category of
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
946 | www.annalsofsurgery.com
C ?2010 Lippincott Williams & Wilkins

Page 5

Annals of SurgeryrVolume 252, Number 6, December 2010Mentoring and Simulation in Laparoscopy
FIGURE 3. (a and b) Summary of
conversion rates and 95% confidence
intervals for nonmentored and mentored
cases (fixed and random effects model). By
comparison, the expert group that
included a total of 5313 cases had a
conversion rate of 0.12 (95%CI: 0.10-0.13)
and was not different to the mentored case
group (see also Table 2).
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
C ?2010 Lippincott Williams & Wilkins
www.annalsofsurgery.com | 947

Page 6

Miskovic et al
Annals of SurgeryrVolume 252, Number 6, December 2010
TABLE 3. Meta-analysis of Odds Ratios for Risk Factors for
Conversion (Random Effects Model)
Risk FactorN∗
n∗
OR (95% CI)
P
Age > 64 yr
Male sex
PAS
BMI > 30
ASA 3 and 4
Cancer
Fistula
4444
4444
2219
3684
2684
3430
2482
2340
2108
659
513
614
1047
1.15 (0.68–1.94)
1.72 (1.40–2.11)
1.65 (1.19–2.28)
1.66 (1.26–2.20)
1.737 (1.331–2.266)
1.798 (1.443–2.241)
4.229 (2.575–6.944)
0.61
<0.0001
0.003
0.0003
<0.0001
<0.0001
<0.0001
87
∗ N, total patients; n, group exposed to risk factor.
ASA indicates American Society of Anesthesiologist (score); BMI, body mass in-
dex (BMI) (kg/m2); 95% CI, 95% confidential interval; OR, odds ratio; PAS, previous
abdominal surgery.
assessment of technical skills in LCS.43,74,75Two demonstrated the
use of dedicated global assessment tools during the training period of
laparoscopic colorectal surgeons. Such scales are practical and easy-
to-use and can help to estimate the performance level of a trainee.77
However, despite being partially validated and used widely, these
scales have limitations as they demonstrate a ceiling effect and lack
precision.Suchtoolsoftenfocusonbasicsurgicalskills,whichshould
alreadyhavebeenmasteredbyanadvancedtrainee.Theseissuesmay
beaddressedbyhumanreliabilitymethodsthatanalyzeerrorsenacted
during procedures and identify the underlying performance shaping
factors.78Nevertheless, these methods remain in their infancy.79–81
Anotherapproachismorbidityandmortalitydata,whichgiveaglobal
indicator of performance, but here again lie several limitations. First,
audit results can be assessed only when both large numbers and long-
term follow-up data are available. This takes time and does not avoid
unfavorable outcomes accumulating while the audit is in progress.
Second, surgical competence is multidimensional and includes oper-
ative skills, cognitive factors, personality traits, and decision making.
Third, morbidity and mortality data cannot provide a prescriptive
method for specifying how the performance of a procedure can be
improved.Despitetheseshortfalls,anauditprocessremainsessential,
as it promotes monitoring to ensure the maintenance of safe clinical
practice throughout training. In many studies, conversion rate was
used as a performance parameter, although some authors state that
this should not be considered to be a complication in itself, because
the decision to convert may also reflect an experienced surgeon’s
sound judgment.2,10,44,82,83Nevertheless, it is expected for surgeons
tohavesimilarcomplicationandconversionratesatthesamestageof
training. Interestingly, this review demonstrated a significant differ-
ence in conversion rates between mentored and nonmentored trainees
andhencethisislikelytorepresentadifferenceinthetechnicalability
to complete a case laparoscopically. Although trainees at this level
are often already proficient in basic technical skills, an experienced
trainer may further aid intraoperative decision making and the com-
prehensionofanatomy,andguidethetraineetoutilizeerrorreduction
mechanisms and thus potentially minimize the rate of unnecessary
conversion. Patient factors leading to conversion were analyzed and
taken as indicators of case difficulty, thus allowing for appropriate
training case selection in the early learning curve. Furthermore, for
the first time, the meta-analyzed data provide enough supportive ev-
idence to show that previous abdominal surgery is also a significant
risk factor for conversion.
Our results reflect the effects of cognitive apprenticeship and
the scaffolding instruction strategy on performance. Cognitive ap-
prenticeshipisthecombinedprocessofsimultaneouslyteachingcraft
and strategic thinking.84It describes the process of teaching in the
operating room more accurately than mentoring, a term that has been
used by most authors of the aforementioned studies. Scaffolding as
a teaching strategy is a concept that is unconsciously or consciously
used by most surgical teachers as they provide individualized support
according to the trainee’s abilities.85This can also be reflected by an
expert’s careful choice of a suitable training case appropriate to the
trainees’ level of skill.86,87The meta-analysis of data from 3430 cases
for the risk of conversion, representing a predictive level of difficulty,
may aid this selection process. Vygotsky described the Zone of Prox-
imal Development as the range of tasks someone is able to learn. If
the teacher deploys the scaffolding instruction strategy, the trainee is
able to move up along this scale, and in this context our results can
be explained.88,89
Novel simulation techniques designed specifically for LCS
have been developed. These include a combination of virtual real-
ity simulators and box trainers, animal and human tissue, and syn-
thetic materials.65–70This systematic review revealed a notable lack
of available data on the educational value of simulated training in
LCS. Unlike for laparoscopic cholecystectomies and basic gyneco-
logical procedures, the transferability to clinical training has not been
investigated scientifically for these methods.90,91The only identifi-
able studies on LCS courses were satisfaction surveys that indicated
that cadaveric models seem to be superior, in terms of reality and
learning effect, to live animal (pig) models, despite the lack of tissue
perfusion.39,72,73Therecentstudiesonhybridsimulationandaninitial
report on virtual reality colonic models based on individual patient’s
imaging data may advance simulation training as they will not only
negate the availability and ethical issues surrounding cadavers but
also allow a stepwise increase in task difficulty similar to clinical
training.62,79
This review also has its limitations. First, a direct comparison
between the 3 groups (mentored, nonmentored, and expert) may be
problematic as there was no case control for surgical complexity
and patient morbidity. Expert surgeons are more likely to operate on
more complex cases that may explain why no significant difference
between mentored trainees and experts was found. Nevertheless, a
significant difference was found on comparing the mentored and
nonmentored subgroups. Second, difficulties exist when synthesizing
data from studies of different designs and methodological strength.
Thiswasaddressedbyapplyingthesignal-to-noiseratio,andalthough
the method is not validated, the concept represents both the quality
of methods and the relevance of the content to the review question
with an acceptable interrater reliability.34,35Analysis gave an average
signal-to-noise ratio of 0.43 and method score of 0.65, which implies
a lack of well-designed studies. Surprisingly, there was a paucity
of educational data on simulated training and clinical outcome. In
particular, no studies compared the effects of different simulation
methods on training. Also, there is no available study comparing the
cadaveric and animal models by the same trainees that would provide
more direct and reliable information.
The collated evidence provided by this review permits the sug-
gestion of several considerations when setting up a training program
for technically advanced procedures. First, supervised training and
appropriate case selection even at an advanced level is essential to
ensure safe clinical practice. Further studies need to evaluate the
training effect on the proficiency gain process. Second, integrated
simulation models are more likely to be effective when anatomically
accurate and realistic. From the trainee’s perspective, cadavers fulfill
these requirements more than animal models. Third, a training pro-
gram needs to be evaluated and have a governance structure. This
is particularly important in the context of the modern world where
patient safety is paramount. It is also relevant in terms of medicole-
gal indemnity issues, as a higher complication or conversion rate
as the result of a learning process is no longer acceptable. Ideally,
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
948 | www.annalsofsurgery.com
C ?2010 Lippincott Williams & Wilkins

Page 7

Annals of SurgeryrVolume 252, Number 6, December 2010Mentoring and Simulation in Laparoscopy
such an evaluation should encompass both clinical outcome data to
ensure quality and trainee performance data to monitor the training
progress.
Future research for advanced interventional and surgical train-
ing programs needs to focus on the educational value of different
simulation methods and the predictive value of assessment tools. Al-
though trainees seem to appreciate the benefits of cadaveric training,
the availability of such models is limited. Novel, computer-based
simulation needs to be assessed for training effectiveness. The ideal
feasible assessment tool is still to be developed.
ACKNOWLEDGMENTS
We thank Daniel Birch (Edmonton, Alberta, Canada),
Selim Dincler and Peter Buchmann (Zurich, Switzerland), Omar Faiz
(London, United Kingdom), In Ja Park (Daegu, South Korea), Nico-
las Rotholtz (Buenos Aires, Argentina), and Christopher Schlachta
(Toronto, Onatrio, Canada) for providing us with nonpublished data
oftheirseries.WealsothankElenaKulinskaya,DirectorofStatistical
AdvisoryService,ImperialCollege,London,forherexpertadviceand
review of the statistical methods. We are grateful to Matthias Egger,
Head of the Department of Epidemiology and Public Health Uni-
versity of Bern (Switzerland), for his helpful comments on an earlier
draft of this manuscript.
D.M., S.M.W., and G.B.H. designed the study and drafted the
manuscript. D.M., S.M.W., and M.N. acquired and analyzed data.
G.B.H. and A.W.D. critically revised the manuscript.
REFERENCES
1. Vickers JV, Savage CJ, Hruza M, et al. The surgical learning curve for la-
paroscopic radical prostatectomy: a retrospective cohort study. Lancet Oncol.
2009;10(5):475–480.
2. Tekkis PP, Senagore AJ, Delaney CP, et al. Evaluation of the learning curve
in laparoscopic colorectal surgery: comparison of right-sided and left-sided
resections. Ann Surg. 2005;242(1):83–91.
3. Biau DJ, Williams SM, Schlup MM, et al. Quantitative and individualized as-
sessmentofthelearningcurveusingLC-CUSUM.BrJSurg.2008;95(7):925–
929.
4. Toker A, Tanju S, Ziyade S, et al. Learning curve in videothoracoscopic
thymectomy: how many operations and in which situations? Eur J Cardio-
thorac Surg. 2008;34(1):155–158.
5. Guillou PJ, Quirke P, Thorpe H, et al. Short-term endpoints of conven-
tional versus laparoscopic-assisted surgery in patients with colorectal can-
cer (MRC CLASICC trial): multicentre, randomised controlled trial. Lancet.
2005;365(9472):1718–1726.
6. Clinical Outcomes of Surgical Therapy Study Group. A comparison of la-
paroscopically assisted and open colectomy for colon cancer. N Engl J Med.
2004;350(20):2050–2049.
7. Laparoscopically assisted colectomy is as safe and effective as open colectomy
in people with colon cancer. Abstracted from: Nelson H, Sargent D, Wie-
and HS, et al; for the Clinical Outcomes of Surgical Therapy Study Group.
A comparison of laparoscopically assisted and open colectomy for colon
cancer. N Engl J Med. 2004;350:2050–2059. Cancer Treat Rev. 2004;30(8):
707–709.
8. Buunen M, Veldkamp R, Hop WC, et al. Survival after laparoscopic surgery
versus open surgery for colon cancer: long-term outcome of a randomised
clinical trial. Lancet Oncol. 2009;10(1):44–52.
9. Veldkamp R, Kuhry E, Hop WC, et al. Laparoscopic surgery versus open
surgery for colon cancer: short-term outcomes of a randomised trial. Lancet
Oncol. 2005;6(7):477–484.
10. Faiz O, Warusavitarne J, Bottle A, et al. Laparoscopically-assisted vs open
elective colonic and rectal resection: a comparison of outcomes in English
National Health Service Trusts between 1996 and 2006. Dis Colon Rectum.
2009;52(10):1695–1704.
11. HedrickT,TurrentineF,SanfeyH,etal.Implicationsoflaparoscopyonsurgery
residency training. Am J Surg. 2009;197(1):73–75.
12. TichanskyDS,TaddeucciRJ,HarperJ,etal.Minimallyinvasivesurgeryfellows
wouldperformawidervarietyofcasesintheir“ideal”fellowship.SurgEndosc.
2008;22(3):650–654.
13. Bittner JGt, Coverdill JE, Imam T, et al. Do increased training requirements in
gastrointestinal endoscopy and advanced laparoscopy necessitate a paradigm
shift? A survey of program directors in surgery. J Surg Educ. 2008;65(6):418–
430.
14. Schwab KE, Dowson HM, Van Dellen J, et al. The uptake of laparoscopic
colorectal surgery in Great Britain and Ireland: a questionnaire survey of
consultant members of the ACPGBI. Colorectal Dis. 2009;11(3):318–322.
15. Fleshman J, Marcello P, Stamos MJ, et al. Focus Group on Laparoscopic
Colectomy Education as endorsed by The American Society of Colon and
Rectal Surgeons (ASCRS) and The Society of American Gastrointestinal and
Endoscopic Surgeons (SAGES). Dis Colon Rectum. 2006;49(7):945–949.
16. Royal Colleges of England Glasgow Edinburgh and Ireland. Intercol-
legiate Surgical Curriculum Programme. Available at: www.iscp.ac.uk.
Accessed January 29, 2010.
17. Schout BM, Hendrikx AJ, Scherpbier AJ, et al. Update on training models in
endourology: a qualitative systematic review of the literature between January
1980 and April 2008. Eur Urol. 2008;54(6):1247–1261.
18. LarsenCR,SoerensenJL,GrantcharovTP,etal.Effectofvirtualrealitytraining
on laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802.
19. Charron P, Campbell R, Dejesus S, et al. The gap in laparoscopic colorectal
experience between colon and rectal and general surgery residency training
programs. Dis Colon Rectum. 2007;50(12):2023-2031; discussion 2031.
20. Birch DW, Misra M, Farrokhyar F. The feasibility of introducing advanced
minimallyinvasivesurgeryintosurgicalpractice.CanJSurg.2007;50(4):256–
260.
21. Kirchhoff P, Dincler S, Buchmann P. A multivariate analysis of potential risk
factorsforintra-andpostoperativecomplicationsin1316electivelaparoscopic
colorectal procedures. Ann Surg. 2008;248(2):259–265.
22. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Inter-
ventions Version 5.0.1 (updated September 2008). Available at: The Cochrane
Collaboration, 2008. Available at: www.cochrane-handbook.org. Accessed
August 3, 2009.
23. vanMookWN,vanLuijkSJ,O’SullivanH,etal.Generalconsiderationsregard-
ing assessment of professional behaviour. Eur J Intern Med. 2009;20(4):e90–
e95.
24. FincherRE,LewisLA.Simulationusedtoteachclinicalskills.In:NormanGR,
Van der Veluten CPM, Newble DI, eds. International Handbook of Research
in Medical Education, Vol. 1. Dordrecht, Boston, London: Kluwer Academic
Publishers; 2002:499–535.
25. Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery:
where to begin? Surgery. 2004;135(1):6–20.
26. AskOxford.com (Oxford Dictionaries). Available at: www.askoxford.com.
2009. Accessed January 29, 2010.
27. Review Manager (RevMan) [computer program]. Version 4.2.10 for Win-
dows. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collabora-
tion; 2003.
28. DeeksJJ,HigginsJPT,AltmanDG.Chapter9:Analysingdataandundertaking
meta-analyses.In:HigginsJPT,GreenS,eds.CochraneHandbookforSystem-
atic Reviews of Interventions Version 5.0.1 (updated September 2008): The
Cochrane Collaboration; 2008. Available at: www.cochrane-handbook.org.
Accessed August 3, 2009.
29. Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-
analyses. BMJ. 2003;327(7414):557–560.
30. Rotholtz NA, Laporte M, Zanoni G, et al. Predictive factors for conversion in
laparoscopic colorectal surgery. Tech Coloproctol. 2008;12(1):27–31.
31. Birch DW, Asiri AH, de Gara CJ. The impact of a formal mentoring program
for minimally invasive surgery on surgeon practice and patient outcomes. Am
J Surg. 2007;193(5):589–591; discussion 591–592.
32. Schlachta CM, Mamazza J, Seshadri PA, et al. Determinants of outcomes in la-
paroscopiccolorectalsurgery:amultipleregressionanalysisof416resections.
Surg Endosc. 2000;14(3):258–263.
33. Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS)
for assessing the quality of nonrandomised studies in meta-analysis. Available
at: http://www.ohri.ca/programs/clinical epidemiology/oxford.htm. Accessed
August 3, 2009.
34. Edwards AG, Russell IT, Stott NC. Signal versus noise in the evidence
base for medicine: an alternative to hierarchies of evidence? Fam Pract.
1998;15(4):319–322.
35. Higginson IJ, Finlay I, Goodwin DM, et al. Do hospital-based palliative teams
improvecareforpatientsorfamiliesattheendoflife?JPainSymptomManage.
2002;23(2):96–106.
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
C ?2010 Lippincott Williams & Wilkins
www.annalsofsurgery.com | 949

Page 8

Miskovic et al
Annals of SurgeryrVolume 252, Number 6, December 2010
36. Bland JM, Altman DG. Statistical methods for assessing agreement be-
tween two methods of clinical measurement. Lancet. 1986;1(8476):307–
310.
37. Ito M, Sugito M, Kobayashi A, et al. Influence of learning curve on short-
term results after laparoscopic resection for rectal cancer. Surg Endosc.
2009;23(2):403-408.
38. Chikkappa MG, Jagger S, Griffith JP, et al. In-house colorectal laparoscopic
preceptorship: a model for changing a unit’s practice safely and efficiently. Int
J Colorectal Dis. 2009;24(7):771–776.
39. LinE,SzomsteinS,AddasiT,etal.Modelforteachinglaparoscopiccolectomy
to surgical residents. Am J Surg. 2003;186(1):45–48.
40. Mehall JR, Shroff S, Fassler SA, et al. Comparing results of residents and
attending surgeons to determine whether laparoscopic colectomy is safe. Am
J Surg. 2005;189(6):738–741.
41. Daetwiler S, Guller U, Schob O, et al. Early introduction of laparoscopic
sigmoid colectomy during residency. Br J Surg. 2007;94(5):634–641.
42. Pigazzi A, Anderson C, Mojica-Manosa P, et al. Impact of a full-time pre-
ceptor on the institutional outcome of laparoscopic colectomy. Surg Endosc.
2008;22(3):635–639.
43. Champagne BJ, Lee EC, Valerian B, et al. A novel end point to assess a
resident’s ability to perform hand-assisted versus straight laparoscopy for left
colectomy: is there really a difference? J Am Coll Surg. 2008;207(4):554–
549.
44. Schlachta CM, Kent SA, Lefebvre KL, et al. A model for longitudinal
mentoring and telementoring of laparoscopic colon surgery. Surg Endosc.
2009;23(7):1634–1638.
45. Sebajang H, Trudeau P, Dougall A, et al. The role of telementoring and teler-
obotic assistance in the provision of laparoscopic colorectal surgery in rural
areas. Surg Endosc. 2006;20(9):1389–1393.
46. Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected
by a simple, graphical test. BMJ. 1997;315(7109):629–634.
47. Schlachta CM, Mamazza J, Seshadri PA, et al. Defining a learning curve for
laparoscopic colorectal resections. Dis Colon Rectum. 2001;44(2):217–222.
48. Agha A, Furst A, Iesalnieks I, et al. Conversion rate in 300 laparoscopic
rectal resections and its influence on morbidity and oncological outcome. Int
J Colorectal Dis. 2008;23(4):409–417.
49. Tan PY, Stephens JH, Rieger NA, et al. Laparoscopically assisted colec-
tomy: a study of risk factors and predictors of open conversion. Surg Endosc.
2008;22(7):1708–1714.
50. Le Moine MC, Fabre JM, Vacher C, et al. Factors and consequences of con-
version in laparoscopic sigmoidectomy for diverticular disease. Br J Surg.
2003;90(2):232–236.
51. Hamel CT, Pikarsky AJ, Weiss E, et al. Do prior abdominal operations alter
the outcome of laparoscopically assisted right hemicolectomy? Surg Endosc.
2000;14(9):853–857.
52. SchwandnerO,SchiedeckTH,BruchH.Theroleofconversioninlaparoscopic
colorectal surgery: Do predictive factors exist? Surg Endosc. 1999;13(2):151-
156.
53. Kim J, Edwards E, Bowne W, et al. Medial-to-lateral laparoscopic colon resec-
tion: a view beyond the learning curve. Surg Endosc. 2007;21(9):1503–1507.
54. Reichenbach DJ, Tackett AD, Harris J, et al. Laparoscopic colon resec-
tion early in the learning curve: what is the appropriate setting? Ann Surg.
2006;243(6):730–735; discussion 735–737.
55. Arteaga Gonzalez I, Diaz Luis H, Martin Malagon A, et al. A comparative
clinical study of short-term results of laparoscopic surgery for rectal cancer
during the learning curve. Int J Colorectal Dis. 2006;21(6):590–595.
56. Lauter DM, Froines EJ. Initial experience with 150 cases of laparoscopic
assisted colectomy. Am J Surg. 2001;181(5):398–403.
57. Pandya S, Murray JJ, Coller JA, et al. Laparoscopic colectomy: indications for
conversion to laparotomy. Arch Surg. 1999;134(5):471–475.
58. Agachan F, Joo JS, Sher M, et al. Laparoscopic colorectal surgery. Do we get
faster? Surg Endosc. 1997;11(4):331–335.
59. Begos DG, Arsenault J, Ballantyne GH. Laparoscopic colon and rectal
surgery at a VA hospital. Analysis of the first 50 cases. Surg Endosc.
1996;10(11):1050–1056.
60. Reissman P, Cohen S, Weiss EG, et al. Laparoscopic colorectal surgery: as-
cending the learning curve. World J Surg. 1996;20(3):277–281; discussion
282.
61. Wishner JD, Baker JW, Jr., Hoffman GC, et al. Laparoscopic-assisted colec-
tomy. The learning curve. Surg Endosc. 1995;9(11):1179–1183.
62. Park IJ, Choi GS, Lim KH, et al. Multidimensional analysis of the learning
curve for laparoscopic colorectal surgery: lessons from 1,000 cases of laparo-
scopic colorectal surgery. Surg Endosc. 2009;23(4):839–846.
63. Jamali FR, Soweid AM, Dimassi H, et al. Evaluating the degree of difficulty of
laparoscopic colorectal surgery. Arch Surg. 2008;143(8):762–767; discussion
768.
64. Seki Y, Ohue M, Sekimoto M, et al. Evaluation of the technical difficulty
performing laparoscopic resection of a rectosigmoid carcinoma: visceral fat
reflects technical difficulty more accurately than body mass index. Surg En-
dosc. 2007;21(6):929–934.
65. Bohm B, Milsom JW. Animal models as educational tools in laparoscopic
colorectal surgery. Surg Endosc. 1994;8(6):707–713.
66. Mutter D, Rubino F, Temporal MS, et al. Surgical education and Internet-based
simulation:TheWorldVirtualUniversity.MinimInvasiveTherAlliedTechnol.
2005;14(4):267–274.
67. SuzukiS,EtoK,HattoriA,etal.Surgerysimulationusingpatient-specificmod-
els for laparoscopic colectomy. Stud Health Technol Inform. 2007;125:464–
466.
68. Waseda M, Inaki N, Mailaender L, et al. An innovative trainer for surgi-
cal procedures using animal organs. Minim Invasive Ther Allied Technol.
2005;14(4):262–266.
69. Ramshaw BJ, Young D, Garcha I, et al. The role of multimedia interactive pro-
grams in training for laparoscopic procedures. Surg Endosc. 2001;15(1):21–
27.
70. Neary PC, Boyle E, Delaney CP, et al. Construct validation of a novel hybrid
virtual-reality simulator for training and assessing laparoscopic colectomy:
results from the first course for experienced senior laparoscopic surgeons.
Surg Endosc. 2008;22(10):2301–2309.
71. Ross HM, Simmang CL, Fleshman JW, et al. Adoption of laparoscopic colec-
tomy: results and implications of ASCRS hands-on course participation. Surg
Innov. 2008;15(3):179–183.
72. Pattana-arun J, Udomsawaengsup S, Sahakitrungruang C, et al. The new la-
paroscopic proctocolectomy training (in soft cadaver). J Med Assoc Thai.
2005;88 (Suppl 4):S65–S69.
73. Giger U, Fresard I, Hafliger A, et al. Laparoscopic training on Thiel human
cadavers: a model to teach advanced laparoscopic procedures. Surg Endosc.
2008;22(4):901–906.
74. Wohaibi EM, Earle DB, Ansanitis FE, et al. A new web-based operative skills
assessmenttooleffectivelytracksprogressioninsurgicalresidentperformance.
J Surg Educ. 2007;64(6):333–341.
75. Sidhu RS, Vikis E, Cheifetz R, et al. Self-assessment during a 2-day laparo-
scopic colectomy course: can surgeons judge how well they are learning new
skills? Am J Surg. 2006;191(5):677–681.
76. Rijbroek A, Wisselink W, Rauwerda JA. The impact of training in unselected
patients on mortality and morbidity in carotid endarterectomy in a vascular
training center and the recommendations of the European Board of Surgery
QualificationinVascularSurgery.EurJVascEndovascSurg.2003;26(3):256–
261.
77. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool for
evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190(1):107–
113.
78. Tang B, Hanna GB, Joice P, et al. Identification and categorization of technical
errors by Observational Clinical Human Reliability Assessment (OCHRA)
during laparoscopic cholecystectomy. Arch Surg. 2004;139(11):1215–1220.
79. Gauba V, Tsangaris P, Tossounis C, et al. Human reliability analysis of cataract
surgery. Arch Ophthalmol. 2008;126(2):173–177.
80. Cox A, Dolan L, Macewen CJ. Human reliability analysis: a new method to
quantify errors in cataract surgery. Eye. 2008;22(3):394–397.
81. Tang B, Hanna GB, Carter F, et al. Competenceassessment of laparoscopic op-
erativeandcognitiveskills:ObjectiveStructuredClinicalExamination(OSCE)
or Observational Clinical Human Reliability Assessment (OCHRA). World J
Surg. 2006;30(4):527–534.
82. Dincler S, Koller MT, Steurer J, et al. Multidimensional analysis of learning
curvesinlaparoscopicsigmoidresection:eight-yearresults.DisColonRectum.
2003;46(10):1371–1378; discussion 1378–1379.
83. Tekkis PP, Fazio VW, Lavery IC, et al. Evaluation of the learning curve
in ileal pouch-anal anastomosis surgery. Ann Surg. 2005;241(2):262–
268.
84. Collins A, Brown, JS, & Newman, SE. Cognitive apprenticeship: Teaching the
craft of reading, writing and mathematics(Technical ReportNo. 403). January
1987. . Vol. 403: Centre for the Study of Reading, University of Illinois;
1987.
85. De Grave WS, Dolmans DH, van der Vleuten CP. Profiles of effective tu-
tors in problem-based learning: scaffolding student learning. Med Educ.
1999;33(12):901–906.
86. Abernethy B. Searching for the minimal essential information for skilled per-
ception and action. Psychol Res. 1993;55:131–138.
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
950 | www.annalsofsurgery.com
C ?2010 Lippincott Williams & Wilkins

Page 9

Annals of SurgeryrVolume 252, Number 6, December 2010Mentoring and Simulation in Laparoscopy
87. Custers EJFM, Boshuizen HPA. The psychology of learning. In: Norman GR,
Van der Vleuten CPM, Newble DI, eds. International Handbook of Research
in Medical Education, Vol. 1. Dordrecht: Kluwer Academic Publishers; 2002.
88. Vygotsky L. Mind in Society. Cambridge: Harvard University Press, 1978.
89. ZorgaS.Supervision:theprocessoflife-longlearninginsocialandeducational
professions. J Interprof Care. 2002;16(3):265–276.
90. Torkington J, Smith SG, Rees B, et al. The role of the basic surgical skills
course in the acquisition and retention of laparoscopic skill. Surg Endosc.
2001;15(10):1071–1075.
91. Sturm LP, Windsor JA, Cosman PH, et al. A systematic review of skills
transfer after surgical simulation training. Ann Surg. 2008;248(2):166–
179.
Copyright © 2010 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
C ?2010 Lippincott Williams & Wilkins
www.annalsofsurgery.com | 951