Assessment of an in vitro model for laparoscopic pelvic lymphadenectomy

Abstract

An in vitro model was developed for laparoscopic pelvic lymphadenectomy. A construct validity study was performed comparing experienced laparoscopic surgeons with inexperienced trainees. Outcome measures included global and task-orientated scoring and data from electromagnetic motion analysis of hands. Marking was performed by three independent surgeons using video playback. A significant difference was found in the time taken, number of movements made and total distance travelled between the groups. Both scoring systems discriminated between the expert and inexperienced group, demonstrating the model's construct validity.

Full text

Assessment of an in vitro model for laparoscopic
pelvic lymphadenectomy
J Bowring, JH Shepherd, TEJ Ind
Department of Gynaecological Oncology, The Royal Marsden Hospital, London, UK
Correspondence: Dr J Bowring, West Middlesex University Hospital, Iselworth, TW7 6AF UK. Email juliebowring@yahoo.co.uk
Accepted 19 April 2007. Published OnlineEarly 18 June 2007.
An in vitro model was developed for laparoscopic pelvic
lymphadenectomy. A construct validity study was performed
comparing experienced laparoscopic surgeons with inexperienced
trainees. Outcome measures included global and task-orientated
scoring and data from electromagnetic motion analysis of hands.
Marking was performed by three independent surgeons using
video playback. A significant difference was found in the time
taken, number of movements made and total distance travelled
between the groups. Both scoring systems discriminated between
the expert and inexperienced group, demonstrating the model’s
construct validity.
Keywords Laparoscopy, lymphadenectomy, minimal access
surgery.
Please cite this paper as: Bowring J, Shepherd J, Ind T. Assessment of an in vitro model for laparoscopic pelvic lymphadenectomy. BJOG 2007;114:964–969.
Introduction
Laparoscopic pelvic lymphadenectomy is commonly per-
formed for some women with cervix, uterine and ovarian
cancer. Potential perioperative complications include damage
to the major vessels in the pelvis, the ureter and to the obtu-
rator or genitofemoral nerves. Furthermore, care must be
taken to prevent perioperative rupture to potentially involved
lymph nodes.
Current surgical training is faced with the dilemma of
reduced training hours and operating exposure plus a raised
public awareness of medical errors.1This makes it difficult to
justify the learning of new techniques in a ‘trial and error’
fashion.1The surgical technique in laparoscopy differs con-
siderably from open surgery, as one loses direct contact and
tactile feedback from the tissues, and must convert direct
three dimensional vision into a two dimensional image seen
on the monitor.2Laparoscopic surgery has therefore received
a great deal of attention in terms of training, particularly away
from the operating room.
We have designed an in vitro model to simulate the move-
ments required for laparoscopic pelvic lymphadenectomy.
We have evaluated the construct validity of the model using
a combination of movement analysis and objective technical
skill and task-specific scoring systems based on video footage.
A questionnaire was handed out among the experienced par-
ticipants to test the model’s realism.
Methods
A model was created of the pelvic sidewall (Figures 1 and 2).
This used a sloped wooden block as a base. Penrose drains
filled with water represented the iliac vessels, and these were
laid out alongside two pieces of string representing the genito-
femoral and obturator nerves. Soaked chickpeas and butter
beans were embedded in cotton wool to replicate lymph
nodes and surrounding fatty tissue. The model was wrapped
in cling film to reproduce the peritoneal covering.
There were 20 participants in total, 9 were ‘experienced
laparoscopic surgeons’ and 11 were ‘inexperienced surgical
trainees’. The experienced group were defined as those
performing over 30 major laparoscopic procedures a year.
The inexperienced group were all surgical trainees at senior
house officer level; they had no experience of pelvic lymph-
adenectomy and were unable to insert a primary trocar
unsupervised.
The model was placed on a constant spot inside a box
trainer with a foam covering. A 10-mm, 0angle camera
was held manually in a central port, and laparoscopic scissors
and graspers were inserted into fixed sites. Each subject was
asked to dissect the cling film before removing all the chick-
peas and butter beans paying careful attention to the sur-
rounding structures. They used identical starting and end
positions. The camera was held in a fixed position by the same
operator for each candidate.
964 ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology
DOI: 10.1111/j.1471-0528.2007.01407.x
www.blackwellpublishing.com/bjog Surgical techniques

Movement analysis was performed by tracking the position
of two electromagnetic transducers (Fastrack, Polhemus,
VT, USA) 20 times every second. The transmitters were 1 cm2
in size and fixed to the back of the laparoscopic instruments.
The receiver was placed at a fixed distance away of 1 m. The
three dimensional positional data were analysed using an in-
house computer program that calculated the distance tra-
velled by the surgeon’s hands and the number of movements.
A movement was defined as an episode with a peak velocity.
The computer software used velocity and duration filters to
eliminate the detection of small and insignificant movements.
Previous experiments had demonstrated that a velocity
threshold of 1 m/s and duration of 0.05 seconds accurately
detected a set number of movements. Ambidexterity was
defined as the difference in number of movements between
the dominant and nondominant hand as a percentage of the
total. Five outcomes were calculated from both the dominant
and nondominant hands. This included time taken to com-
plete task, number of movements, distance travelled, distance
travelled per movement and movements per second.
The candidates were video recorded during the task. Three
independent markers examined the video footage and scored
each candidate. The markers could not identify who the sur-
geons were. A global rating scale, assessing technique and
skill, and a task-specific scoring system, assessing the quality
of the final product, were used (Appendices 1 and 2). A ques-
tionnaire was handed to the ‘experienced’ participants to gain
feedback on the model’s realism and suitability as a training
aid (Appendix 3).
Nonparametric data were compared using the Mann–
Whitney test, and dichotomous data were compared using
the Fisher’s exact test. Agreement between different observers
for the global scoring was analysed using a Bland Altman plot
and intraclass correlation.
Results
The experts were significantly quicker in completing the
task (median difference 260 seconds, P= 0.0016) (Table 1).
In addition, movement analysis demonstrated a reduced num-
ber of movements (median difference 842 movements, P=
0.0005) and distance travelled (median difference 5622 cm,
P= 0.0057) in the expert group compared with nonex-
perts (Table 1). There was no significant difference in the
distance travelled for each movement between the groups.
The four parameters were analysed for ambidexterity, and
no significant differences was demonstrated between the two
groups.
In the nonexpert group, vessel injury occurred on three
occasions and one candidate damaged the obturator nerve.
No one in the expert group caused damage to the model;
however, these findings did not reach statistical significance
(Table 2). Less people in the nonexpert group retrieved all
the lymph nodes compared with those in the expert group
(45.5 cf 100%, P= 0.0141) (Table 2). In addition, nonexperts
were less likely to complete the task correctly compared with
the experts (9.1 cf 88%, P= 0.0009) (Table 2).
The global scoring system demonstrated higher scores in
the expert group compared with nonexperts (37 versus 68,
U=99,P= 0.0001) (Figure 3). The plot of global scores for the
Figure 2. The pelvic sidewall model in cross-section to show the layers dissected during the task.
Figure 1. The pelvic sidewall model made from sponge-covered wood.
In vitro pelvic lymphadenectomy model
ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology 965

three markers is shown in Figure 4. The intraclass correlation
coefficient between the observers was 0.9.
Eight of nine (89%) questionnaires were returned. All
(100%) the eight experts rated the model from good to excel-
lent for replication of the required movements, two (25%)
experts felt that the anatomical accuracy of the model needed
improving.
Discussion
Our results show that the experienced group were distin-
guished from the inexperienced surgeons when comparing
motion analysis outcomes together with results from the
global and task-specific scores. The experts completed their
task in a significantly shorter time with fewer movements, but
the amplitude of the movements was the same in each group.
The experts were more likely to retrieve all the lymph nodes,
and a significantly larger number achieved full marks from the
task-specific checklist.
This study shows that a model for laparoscopic lymph node
dissection demonstrated good construct validity between
expert and inexperienced surgeons. This supports the previ-
ous literature relating to surgical skill and movement analysis.
Datta et al.3,4 found a significant relationship between the
time taken and number of movements made with increasing
seniority when carrying out two simulated tasks; suturing
a synthetic vein patch to an artery and performing a small
bowel anastomosis. It was noted that once time was con-
trolled, the number of movements made correlated signi-
ficantly with surgical expertise. Sokollik et al.5showed
significant differences between novices and experts in terms
of time to perform a laparoscopic task with significantly
shorter error times in the expert group. The virtual reality
simulator LapSim used by Sherman et al.6compared three
different levels of skill. They concluded that ‘time error’ scores
were of value in predicting performance, but ‘motion scores’,
relating to path length, correlated less with the level of ex-
perience. Timing candidates on simulated tasks appear to be
worthwhile to assess skill, as a quicker performance seems
to result from better economy of movement. A recent study
Table 2. Comparison of the task-specific scoring between experts and nonexperts
Experts n/N(%) Nonexperts n/N(%) Statistic
Total 9/9 (100.0) 11/11 (100.0) n520
No damage to vessels 9/9 (100.0) 8/11 (72.7) NS
No damage to obturator nerve 9/9 (100.0) 10/11 (90.9) NS
No damage to genitofemoral nerve 9/9 (100.0) 11/11 (100.0) NS
Retrieved all lymph nodes 9/9 (100.0) 5/11 (45.5) P50.0141 (FET)
Damaged no lymph nodes 8/9 (88.9) 8/11 (72.7) NS
Completed the task completely 8/9 (88.9) 1/11 (9.1) P50.0009 (FET)
FET, fisher exact test; NS, non significant.
Figure 3. Global rating scores in junior and senior surgeons.
Table 1. Results of movement analysis
Outcome Expert median (range) Nonexpert median (range) Statistic (Uvalue) P
Time (seconds) 360 (229–738) 620 (339–1152) 10 0.0016
Velocity peaks (n—both hands) 773 (486–1326) 1615 (700–2528) 7 0.0005
Total distance travelled (cm) 3882 (2173–14 475) 9504 (3920–15 744) 14 0.0057
Distance of each movement (cm) 5.1 (3.5–6.7) 5.6 (2.3–6.4) 54 0.7664
Movements per second 1.21 (1.02–1.29) 1.21 (1.03–1.32) 42 0.6207
Bowring et al.
966 ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology

by Aggarwal et al.7showed that a virtual reality laparoscopic
salpingectomy simulator could successfully distinguish be-
tween three levels of gynaecology surgeons when comparing
time taken and total instrument path length.
The two part Structured Technical Skills Assessment Form
was developed by Winckel et al.8to assess technical skill using
structured checklists. Their approach showed high levels of
interrater reliability, while effectively discriminating between
junior and experienced participants. If a simulator demon-
strates ‘construct validity’, it will correlate closely with ability
and skill in theatre. A trainee’s performance on the simu-
lator should therefore improve over time as experience is
gained.9,10 Our task-specific checklist was completed at the
end of the task, assessing the quality of the final product using
five components of the procedure. The second form consisted
of a 6-point global rating scale based on operative perfor-
mance. The scores obtained from the three independent
markers were able to discriminate between the two groups
using both forms. It has been proposed that for high stakes
decisions, which may lead to credentialing a candidate, inter-
rater reliability should be no less than 0.80.11 The method of
assessing video playback was used by Dath et al.,11 they found
their interrater reliability to be 0.74, while assessment times
were shortened by up to 80%. The intraclass correlation co-
efficient between the observers in our study was 0.90.
Practicing the necessary movements and techniques
required by the task, trainees should see an improvement in
their performance demonstrated by higher scores, while re-
taining the skills they have learnt. Promising results regarding
skill retention on laparoscopic trainers have been shown.
Stefanidis et al.12 went back to assess skill retention after pro-
ficiency-based training had occurred several months previ-
ously. Some skill loss was seen in the first few weeks after
training; however, beyond this time it remained the same.
This effect was seen at all levels of experience.
The introduction of simulators, similar to those found in
the aviation industry, allows trainees and their senior col-
leagues to practice skilled laparoscopic tasks together with
the art of decision-making, all in real time.13 Whether they
are the conventional laparoscopic simulators or the newer
virtual reality systems (Minimally Invasive Surgery Trainer-
Virtual Reality [Manchester, UK] and LapSim [Surgical
Science, Gothenburg, Sweden]), many have been shown to
be effective in teaching, assessment and improvement of basic
psychomotor skills required in laparoscopy.14–16 Aggarwal et al.7
demonstrated varying learning curves between experienced,
intermediate and novice groups attempting laparoscopic sal-
pingectomy with the inexperienced group obtaining similar
levels towards the end of training. Allowing trainees regular
access to simulators will serve to improve their laparoscopic
technique with the ultimate aim of improving performance
within the operating theatre.
Although yet to be tested, this pelvic sidewall model has the
potential to become an effective training aid if improved
technical skill is seen after repeated use. Two of the experts
felt that the anatomical accuracy could be improved, and so
a more realistic model is desirable. However, this must be
balanced with the need for a cost effective reproducible design
when virtual reality systems are not available. The incorpora-
tion of such training and education opportunities into every-
day practice will improve surgical outcomes and quality of
patient care.17
Conclusion
Quality assurance is of paramount importance in surgery.
Formal training and tuition has long been recognised to
improve the basic skills of trainees in all aspects of surgical
practice.18,19 We believe that this pelvic sidewall model pro-
vides a starting block on which trainees can attain the neces-
sary skills needed to carry out laparoscopic lymph node
dissections. If future studies can demonstrate improved per-
formance following repeated attempts on the simulator, the
model will undoubtedly provide an effective means of assess-
ing individual candidates and monitoring progress. The use of
similar surgical simulators should be encouraged, particularly
at a time when the need for structured, effective surgical
training has never been greater.
Acknowledgements
The authors would like to acknowledge and thank Dr John
Butler, Mr Barnaby Rufford and Miss Lisa Wong for their
time in reviewing and marking the video footage. A grateful
thanks is given to the generous contribution of the Sandy Fyfe
Research Fund in Gynaecological Oncology at St Bartholo-
mew’s Hospital for funding this work. j
Figure 4. Bland Altman plot demonstrating inter-observer variability for
the global scoring system.
In vitro pelvic lymphadenectomy model
ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology 967

References
1Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D,
Ghitulescu G, et al. Proving the value of simulation in laparoscopic
surgery. Ann Surg 2004;240:124–34.
2Gallagher AG, Cowie R, Crothers I, Jordan-Black JA, Satava RM. PicSOr:
an objective test of perceptual skill that predicts laparoscopic technical
skill in three initial studies of laparoscopic performance. Surg Endosc
2003;17:1468–71.
3Datta V, Chang A, Mackay S, Darzi A. The relationship between mo-
tion analysis and surgical technical assessments. Am J Surg 2002;184:
70–3.
4Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic
motion tracking analysis to objectively measure open surgical skill in
the laboratory-based model. J Am Coll Surg 2001;193:479–85.
5Sokollik C, Gross J, Buess G. New model for skills assessment and
training progress in minimally invasive surgery. Surg Endosc 2004;
18:495–500.
6Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM. Assessing
the learning curve for the acquisition of laparoscopic skills on a virtual
reality simulator. Surg Endosc 2005;19:678–82.
7Aggarwal R, Tully A, Grantcharov T, Larsen CR, Miskry T, Farthing A,
et al. Virtual reality simulation can improve technical skills during lap-
aroscopic salpingectomy for ectopic pregnancy. BJOG 2006;113:
1382–7.
8Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct
validity of a structured technical skills assessment form. Am J Surg
1994;167:423–7.
9Paisley AM, Baldwin PJ, Paterson-Brown S. Validity of surgical simula-
tion for the assessment of operative skill. Br J Surg 2001;88:1525–32.
10 Reznick RK. Surgical simulation: a vital part of our future. Ann Surg
2005;242:640–1.
11 Dath D, Regehr G, Birch D, Schlachta C, Poulin E, Mamazza J, et al.
Toward reliable operative assessment: the reliability and feasibility of
videotaped assessment of laparoscopic technical skills. Surg Endosc
2004;18:1800–4.
12 Stefanidis D, Korndorffer JR, Sierra R, Touchard C, Dunne JB, Scott DJ.
Skill retention following proficiency-based laparoscopic simulator train-
ing. Surgery 2005;138:165–70.
13 Macedonia CR, Gherman RB, Satin AJ. Simulation laboratories for
training in obstetrics and gynecology. Obstet Gynecol 2003;102:
388–92.
14 Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. MIST VR:
a virtual reality trainer for laparoscopic surgery assesses performance.
Ann R Coll Surg Engl 1997;79:403–4.
15 Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, et al.
Construct validity for the LAPSIM laparoscopic surgical simulator. Surg
Endosc 2005;19:401–5.
16 Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual
reality and box trainers: is one superior to the other? Surg Endosc
2004;18:485–94.
17 Butler J, Ind TEJ. Auditing morbidity and mortality in gynaecological
surgery. Curr Obstet Gynaecol 2004;14:426–33.
18 Ind TEJ, Shelton JC, Shepherd JH. Influence of training on reliability of
surgical knots. BJOG 2001;108:1013–6.
19 Ind TEJ. Surgeon’s knots: old skills, new training. Lancet 2002;
360:874.
Appendix 1. Global rating scale of operative performance
Bowring et al.
968 ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology

Appendix 1. Continued
Appendix 2. Task-specific scoring system
Appendix 3. Questionnaire given to the experts
In vitro pelvic lymphadenectomy model
ª2007 The Authors Journal compilation ªRCOG 2007 BJOG An International Journal of Obstetrics and Gynaecology 969