Skills Acquisition and Assessment after a
Microsurgical Skills Course for
Daniel G. Ezra, MA, MRCOphth,1Raj Aggarwal, PhD, MRCS,2Michel Michaelides, MD, MRCOphth,1
Narciss Okhravi, PhD, FRCOphth,1Seema Verma, MD, FRCOphth,1Larry Benjamin, FRCS, FRCOphth,3
Philip Bloom, FRCS, FRCOphth,4Ara Darzi, MD, FRCS,2Paul Sullivan, MD, FRCOphth1
validity of a video-based modified Objective Structured Assessment of Technical Skill (OSATS) assessment tool
that has not previously been applied to ophthalmic surgery.
Prospective longitudinal cohort study.
Fourteen residents were recruited from 20 attendees at the Moorfields Eye Hospital micro-
surgical skills course for residents.
Each resident performed a standardized microsurgical task consisting of the placement of a 10-0
nylon corneal suture into a model eye using an operating microscope with standardized equipment in a
standardized environment. Objective measurements were made using the Imperial College Surgical Assessment
Device (ICSAD). This is a motion-tracking device returning 3 parameters for economy of movement: total path
length, time, and number of individual hand movements. A concurrent video recording was made of each task
by 2 independent observers who were masked to the time of the recording relative to the course and the identity
of the resident. Video footage was marked in accordance with the OSATS video scoring template.
Main Outcome Measures:
Each resident had motion-tracking analysis performed during corneal suturing
before and after the course (total path length, time, and number of individual hand movements), along with
concurrent OSATS video scores.
Skills improvement after the course was found to be statistically significant for all 3 ICSAD economy
of movement parameters: path length, P ? 0.001; hand movements, P ? 0.012; and time, P ? 0.009. Differences
in the combined OSATS scores of the 2 raters before and after the course were found to be significant (P ?
0.039). Interrater reliability of OSATS scorers was 0.78 (alpha Cronbach). Correlations between the OSATS
scores and each of the ICSAD parameters were found to be significant (P?0.001).
A video-based OSATS scoring system has significant correlation with the ICSAD motion-
tracking parameters, demonstrating concurrent validity between the 2 assessment tools. These data also
demonstrate that surgical skill, as measured by a validated motion-tracking system, is significantly improved
after a 1-day microsurgical skills course.
The author(s) have no proprietary or commercial interest in any materials discussed
in this article. Ophthalmology 2009;116:257–262 © 2009 by the American Academy of Ophthalmology.
To assess the impact of a skills course on microsurgical skills acquisition and to investigate the
Contemporary postgraduate surgical training has developed
from the great traditions of the American and European
clinics of the 19th century. Throughout this evolution,
teaching approaches have been predicated on the “appren-
ticeship” model.1Although apprenticeship can be a power-
ful learning model, it has been criticized for being largely
opportunistic with limited provision to address deficiencies
in trainee skills or knowledge.2Furthermore, in the context
of the apprenticeship training model, patients often have
borne the bulk of the learning curve that trainee surgeons
experience, with complications being seen as inevitable.
However, recent years have seen changes in the expec-
tations of both patients and the profession, which have
converged to recognize that unnecessary complications are
not acceptable.3–5As a result of this, one of the most
fundamental changes in clinical training in the last decade
has been the move away from apprenticeship toward a more
systematic approach and the implementation of teaching
strategies to provide for the development of skills and
experience in a graduated process toward reality. This in-
volves trainees acquiring skills in a simulated patient envi-
ronment before or in parallel to exposure to patients.6This
transition has been established for some time in undergrad-
uate medical training7and is now being embraced by post-
graduate surgical training.8
This has been widely recognized in other specialties, and
a range of techniques outside of the operating theater have
been pioneered, including surgical skills courses, multime-
dia demonstrations, and virtual reality training techniques.9
Although ophthalmology has lagged somewhat behind these
trends in other specialties, there has been a recent emer-
gence of a variety of surgical training modalities to allow
© 2009 by the American Academy of Ophthalmology
Published by Elsevier Inc.
ISSN 0161-6420/09/$–see front matter
residents to improve their skills.10Microsurgical skills
courses have played a central role in skills development, and
the need for these skills courses is now widely accepted.
The Royal College of Ophthalmologists in the United King-
dom has made attendance at such a course a compulsory
requirement before the commencement of intraocular sur-
gery.11In the United States, the Accreditation Council for
Graduate Medical Education has made the availability of a
microsurgical skills laboratory a compulsory requirement
for retaining resident program accreditation,12and some
residency programs have integrated wet laboratory experi-
ence in the residency curriculum.13
Anecdotal evidence suggests that ophthalmology resi-
dents find microsurgical skills courses of great value. How-
ever, there is no evidence that microsurgical skills courses
have any impact on surgical skill acquisition. Although
many trainees will have had good background skills at
phacoemulsification, other skills (e.g., corneal suturing) are
more difficult to acquire in the current climate of reduced
training hours, increased expectations of surgical compe-
tence,10and advances in sutureless ophthalmic surgery.
Identifying the extent of skills acquisition after these
courses would inform our understanding of how best to
educate residents performing microsurgical techniques.
Parallel trends in postgraduate education have led to the
integration of competency elements into the postgraduate
curriculum by the Accreditation Council for Graduate Med-
ical Education,14and more recently by the Royal College of
Ophthalmologists in the United Kingdom.15However, there
has been no consensus on the methodology for assessing
surgical competence in ophthalmology despite an Accredi-
tation Council for Graduate Medical Education mandate for
all residency programs to implement an assessment of sur-
One widely used generic surgical skills assessment tool is the
Objective Structured Assessment of Technical Skill (OSATS)
scoring system,17which has been applied and validated in many
fields of surgery. A condensed modification of this, focusing on
the assessment of psychomotor skills rather than procedural or
anatomic knowledge, has been developed to be used specifically
for video-based assessment rather than a live format assessment.
This modified OSATS has been validated in a general surgical
The objectives of this study are to assess the impact of a
skills course on microsurgical skills acquisition and to in-
vestigate the validity of a video-based modified OSATS
system that has not previously been applied to ophthalmic
Materials and Methods
Moorfields Eye Hospital organizes an annual intermediate micro-
surgical skills course for 20 residents that is hosted at the Royal
College of Ophthalmologists microsurgical skills laboratory.
One week before the course, each resident was asked to perform a
standardized microsurgical task consisting of the placement of a 10-0
nylon corneal suture into a model eye using an operating microscope
with standardized equipment in a standardized environment. The
following equipment was provided: a micro-vitreoretinal blade, cor-
neal notched forceps, McPherson’s forceps, suture-tying forceps,
Troutman suture forceps, and a single-ended 10-0 nylon suture. Res-
idents were then asked to repeat the task 1 week after the course. The
course took place over 1 day and focused primarily on corneal
suturing and instrument handling. A detailed agenda for the course is
summarized in Table 1.
Table 1. Agenda for a Microsurgical Skills Course
Basic techniques on knot tying
and instrument handling
Making a corneal section
Making and suturing corneal
Repairing open globes
Repairing lid lacerations
Repairing corneal stellate
Practical using porcine eyes
Practical using porcine eyes
Table 2. Video-based Modified OSATS Scoring Criteria
Economy of movement (time
Confidence of movement
Many unnecessary movesEfficient time/motion but
some unnecessary moves
Competent use of instruments,
appeared stiff or awkward
Careful handling of tissue, but
Economy of movement and
Fluid moves with instruments with
Repeatedly makes tentative or
awkward moves with
Frequently used unnecessary force
on tissues or caused damage by
inappropriate use of
Imprecise, wrong technique in
approaching the operative
Respect for tissue Consistently handled tissues
appropriately with minimal
Precision of operative technique
(flow of operation)
Careful technique with
Fluent, secure, and correct technique
in all stages of the operative
OSATS ? Objective Structured Assessment of Technical Skill.
Volume 116, Number 2, February 2009
Objective measurements were made using the Imperial College
Surgical Assessment Device (ICSAD). This is a motion-tracking
device using an alternating current electromagnetic field with a
single passive receiver attached to the index finger of each hand at
the distal interphalangeal joint and held in place under latex
surgical gloves. A device to generate an electromagnetic field is
incorporated into the system. Each receiver contains 3 differen-
tially orientated electromagnetic coils that transduce motion
through the electromagnetic field in the x, y, and z planes. The
motion analysis is processed by a computer program that includes
a Gaussian filter to reduce background noise. The ICSAD returns
3 different parameters for economy of movement: total path
length, of both hands measured in 3 planes; time taken, from the
start to end of the procedure; and individual hand movements. The
ICSAD has been widely used as an assessment device in other
areas of surgery,19and an ophthalmologic application of the device
has confirmed good construct validity for assessing corneal sutur-
ing.20Recordings were started from the needle-mounted position
and ended once the corneal suture was turned. An independent
observer ensured the correct completion of the task.
A concurrent video recording was made of each task using a
tripod-mounted digital video camera, which was recorded into
Microsoft Windows .avi format (Microsoft Corporation, Red-
mond, WA). All of the video footage was labeled using ano-
nymised numeric codes. Video footage was marked in accordance
with the OSATS video scoring template (Table 2) by 2 indepen-
dent observers with experience in microsurgical skills tutoring
who were masked to the time of the recording relative to the course
and to the identity of the resident.
Data were analyzed using the Statistical Package for Social Sci-
ences, version 15.0 (SPSS, Chicago, IL). Intergroup comparisons
before and after the course were analyzed using the Wilcoxon
signed-rank test. Interrater reliability for video scoring was inves-
tigated using the Cronbach alpha score. Correlation between the
ordinal video-based scoring system and motion-tracking parame-
ters was explored using a Spearman’s rank correlation.
Fourteen residents were recruited into the study. The relevant
characteristics, including previous surgical experience, of this
group are summarized in Table 3. Summary statistics for the 3
ICSAD parameters (hand movement, path length, and time) are
represented as median and interquartile ranges in Table 4. The
dramatic improvement after the course in all 3 parameters is
evident. This improvement was found to be statistically significant
for all 3 parameters of path length (P ? .001), hand movements
(P ? .012), and time (P ? .009). In an attempt to stratify the
improvement by previous experience, the cohort ICSAD data were
divided into 2 groups on the basis of phacoemulsification proce-
dures performed. Seven participants, forming the more junior
group, had performed between 100 and 300 procedures, and 7
participants, forming the more senior group, had performed be-
tween 375 and 850 procedures. Differences in ICSAD parameters
before and after the course were then calculated independently for
each of these 2 groups. The results are summarized in Table 5.
Simultaneous video recordings, both before and after the
course, were taken. For technical reasons, 2 tasks did not have a
video recording because of a camera malfunction. The 26 remain-
ing videos were scored in accordance with the modified global
OSATS system by 2 designated independent scorers (PS and LB).
Figure 1 demonstrates the concordance between the 2 observers
with an interrater reliability of 0.78 (? Cronbach). Differences in
the combined OSATS scores of the 2 raters before and after the
course was found to be significant (pre-course: median 20.5
[range: 13–29], post-course: median 28.5 [range: 25–35]; P ?
The relationship between each of the motion-tracking param-
eters measured by the ICSAD (time, path length, and hand move-
ments) and the combined OSATS score of each task was investi-
gated using a Spearman’s rank correlation. Significant correlations
between the OSATS scores and each of the ICSAD parameters
were to be significant. The correlation data are summarized in
Contrary to popular belief, surgical proficiency is not a
direct result of intrinsic ability. Recent evidence has dem-
onstrated that repeating a particular technique can allow a
trainee to acquire and maintain a high level of skill.21,22The
recognition of the need for real and simulated training
environments operating in parallel is well recognized, and in
ophthalmology a variety of simulated teaching strategies
Table 3. Group Characteristics
Previous No. of phacoemulsification
Previous ECCE procedures
Estimated No. of corneal sutures
Mean 32.2 (SD 2.8)
Mean 378.2 (SD 224)
Mean 3.3 (SD ? 5.2)
ECCE ? extracapsular cataract extraction; SD ? standard deviation.
Table 4. Summary Statistics for Pre- and Post-course ICSAD Data
ICSAD ParameterPre-course MeasurementsPost-course Measurements Wilcoxon Signed-rank Test
Path length, median (interquartile range)/m
Hand movements, median (interquartile range)/No.
Time, median (interquartile range)/s
P ? 0.001
P ? 0.012
P ? 0.009
ICSAD ? Imperial College Surgical Assessment Device.
Ezra et al ? Motion Analysis Demonstrates Improvement in Surgical Skill
have surfaced, including in vitro human and models and
computer-based virtual reality models.10Probably the most
widely used teaching technique in recent years has been the
microsurgical skills course. Implicit in the widespread use
of surgical skills courses is the intention that the learning
curve is endured on simulated tissue rather than patients.23
In addition to patient safety improvements, this can also
bring cost benefits. A pertinent analogy is the airline indus-
try, in which the transfer-effectiveness ratio describes a
quantification of the amount of time and money that can be
saved by pilots using high-fidelity computer-based simula-
tors on the ground.24,25In surgery, one study has estimated
the cost of lost time during theater sessions as $47,970 over
a 4-year residency.26
These data have demonstrated that surgical skill, as mea-
sured by a validated motion-tracking system, is significantly
improved after a 1-day microsurgical skills course. Al-
though it has been established in the field of general surgery
that surgical skills courses can have a significant impact on
surgical skill,18these are the first data to show that a similar
improvement can occur after a microsurgical skills course
for ophthalmic surgery. This finding also may be of addi-
tional interest outside of the specialty as microsurgical
techniques are increasingly being applied to many other
The stratified analysis on the ICSAD data on the basis of
previous phacoemulsification experience, summarized in
Table 5, demonstrated unexpected results. Although both
groups achieved significantly different results in terms of
path length, only the more senior group achieved significant
differences in terms of time and hand movements. This
observation must be taken with caution because of the small
numbers and the approximation to significance of the P
values. However, it does suggest that there may be an
element to the learning curve that requires significant base-
line surgical experience to benefit maximally from further
Surgical Assessment Tool
Any surgical assessment tool must be feasible, objective,
valid, reliable, and cheap to deliver28(Table 7 describes the
ideal qualities of a surgical assessment tool). Although the
use of motion tracking has been demonstrated to have
construct validity for corneal suturing,20the equipment
needed is expensive, cumbersome, and costly. Other prom-
ising objective scoring systems have been developed re-
cently, including cataract-specific scoring templates29,30and
Figure 1. Interrater reliability of OSATS global rating score between observer 1 and observer 2 (Cronbach’s ? ? 0.78).
Table 5. Comparison Between Pre- and Post-course ICSAD
Data for the More Junior Group (100–300 Phacoemulsification
Procedures) and the More Senior Group (375–850
ICSAD ParameterJunior Group Senior Group
P ? 0.018
P ? 0.116
P ? 0.063
P ? 0.018
P ? 0.043
P ? 0.028
ICSAD ? Imperial College Surgical Assessment Device.
Table 6. Correlation Between OSATS Global Rating Score
and ICSAD Motion-Tracking Parameters, Calculated Using
Spearman Rank Correlation Test
Correlation with OSATS
Video Score (r Value)
Score (P Value)
r ? ?0.765
r ? ?0.55
r ? ?0.631
ICSAD ? Imperial College Surgical Assessment device; OSATS ? Ob-
jective Structured Assessment of Technical Skill.
Volume 116, Number 2, February 2009
other more generic scoring systems such as the Global
Rating Assessment of Skills in Intraocular Surgery31and
the Eye Surgical Skills Assessment Test.32,33
Although these tools have demonstrated face and content
validity, none of these generic scoring systems have been
shown to have construct validity. These tests therefore may
be effective in demonstrating a level of ability before en-
tering the operating theater,34but their ability to differenti-
ate between trainees of different levels and therefore to
monitor progress is questionable,35although some promis-
ing and ongoing work with the Eye Surgical Skills Assess-
ment Test tool has attempted to address this.32
These data demonstrate that a video-based modified OSATS
scoring system has significant correlation with the ICSAD
motion-tracking parameters, thus demonstrating concurrent
validity between the 2 assessment tools. These data also
demonstrate good interrater reliability of the tool. The use of
OSATS to assess suturing technique offers great advantages
in terms of feasibility over other tools. TheEyeSurgicalSkills
Assessment Test requires a multistation wet-laboratory “obstacle
course,” which is expensive and time-consuming. The
Global Rating Assessment of Skills in Intraocular Surgery
requires live observation of the resident and some of the
scoring domains, such as “interaction with assistants,” are
difficult to capture by video. Video assessment offers dis-
tinct advantages because it allows for masking of the trainee
and allows the rater to view the videos at their own conve-
nience, although live observation does have some advan-
tages over post hoc video assessment as more aspects of the
surgical procedure can be assessed.36
Furthermore, as well as being useful in trainee assess-
ment, these methods might also be of value in an objective
audit of improvements in participant technical skills after a
microsurgical skills course. This may also allow for quality
control of similar courses delivered at various times with
different faculty or at different centers. In addition, if mod-
ifications to courses are made in terms of equipment, fac-
ulty, or selection of participants, these methods can objec-
tively assess any change in technical skills acquisition after
These results demonstrate that a microsurgical skills
course is effective at improving the surgical skill of resident
ophthalmologists, at least in the short term. This study has
also described an applicable video-based modified OSATS
scoring system to be an effective tool at assessing corneal
suturing. Further research is needed to assess the impact of
microsurgical courses on other aspects of ophthalmic sur-
gery and to investigate the relationship between degree of
improvement and baseline surgical proficiency.
1. Wanzel KR, Ward M, Reznick RK. Teaching the surgical
craft: from selection to certification. Curr Probl Surg 2002;39:
2. Seabrook MA, Woodfield SJ, Papagrigoriadis S, et al. Con-
sistency of teaching in parallel surgical firms: an audit of student
experience at one medical school. Med Educ 2000;34:292–8.
3. Blomquist PH, Rugwani RM. Visual outcomes after vitreous
loss during cataract surgery performed by residents. J Cataract
Refract Surg 2002;28:847–52.
4. Elwyn G, Corrigan JM. The patient safety story. BMJ 2005;
5. Cuschieri A. Nature of human error: implications for surgical
practice. Ann Surg 2006;244:642–8.
6. Wong JA, Matsumoto ED. Primer: cognitive motor learning
for teaching surgical skill—how are surgical skills taught and
assessed? Nat Clin Pract Urol 2008;5:47–54.
7. Tomorrow’s doctors. Recommendations on Undergraduate
Medical Education. London: General Medical Council; 1993.
8. Accreditation Council for Graduate Medical Education
(ACGME) Outcome Project. Common Program Require-
ments: General Competencies. Available at: http://www.acgme.
Accessed August 28, 2008.
9. Kneebone RL, Nestel D, Vincent C, Darzi A. Complexity, risk
and simulation in learning procedural skills. Med Educ 2007;
10. Henderson BA, Ali R. Teaching and assessing competence in
cataract surgery. Curr Opin Ophthalmol 2007;18:27–31.
11. Royal College of Ophthalmologists. Microsurgical skills course.
Available at: http://www.rcophth.ac.uk/about/skillscentre/bms. Ac-
cessed February 17, 2007.
12. Accreditation Council for Graduate Medical Education. RRC
News for Ophthalmology. Winter 2005. Available at: http://
pdf. Accessed February 19, 2008.
13. Lee AG, Greenlee E, Oetting TA, et al. The Iowa ophthalmology
wet laboratory curriculum for teaching and assessing cataract
surgical competency. Ophthalmology 2007;114:e21–6.
14. Huddle TS, Heudebert GR. Taking apart the art: the risk of
anatomizing clinical competence. Acad Med 2007;82:536–
15. Royal College of Ophthalmologists. Guide for delivery of Ophthal-
OST_-__November_2006.pdf. Accessed February 12, 2008.
16. Lee AG, Carter KD. Managing the new mandate in resident
education: a blueprint for translating a national mandate into
local compliance. Ophthalmology 2004;111:1807–12.
17. Martin JA, Regehr G, Reznick R, et al. Objective structured
assessment of technical skill (OSATS) for surgical residents.
Br J Surg 1997;84:273–8.
Table 7. Qualities of the Ideal Surgical Assessment Tool
Extent to which the examination resembles real-life situations
Extent to which the item that is being assessed is measured by the assessment tool. For example, while trying to assess
technical skills we may actually be testing knowledge
Extent to which the assessment tool measures the trait that it purports to measure; one inference of construct validity is the
extent to which a test discriminates between different levels of expertise
Extent to which the results of the assessment tool correlate with the gold standard for that domain
Ability of the assessment to predict future performance
Ezra et al ? Motion Analysis Demonstrates Improvement in Surgical Skill
18. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized Download full-text
clinical trial of virtual reality simulation for laparoscopic skills
training. Br J Surg 2004;91:146–50.
19. Bann SD, Khan MS, Darzi AW. Measurement of surgical
dexterity using motion analysis of simple bench tasks. World
J Surg 2003;27:390–4.
20. Saleh GM, Voyatzis G, Hance J, et al. Evaluating surgical
dexterity during corneal suturing. Arch Ophthalmol 2006;124:
21. Ericsson KA. Deliberate practice and the acquisition and
maintenance of expert performance in medicine and related
domains. Acad Med 2004;79(suppl):S70–81.
22. Hamstra SJ, Dubrowski A, Backstein D. Teaching technical
skills to surgical residents: a survey of empirical research. Clin
Orthop Relat Res 2006;449:108–15.
23. Aggarwal R, Boza C, Hance J, et al. Skills acquisition for
laparoscopic gastric bypass in the training laboratory: an in-
novative approach. Obes Surg 2007;17:19–27.
24. Roscoe SN. Incremental transfer effectiveness. Hum Factors
25. Aggarwal R, Ward J, Balasundaram I, et al. Proving the
effectiveness of virtual reality simulation for training in lapa-
roscopic surgery. Ann Surg 2007;246:771–9.
26. Bridges M, Diamond DL. The financial impact of teaching surgical
residents in the operating room. Am J Surg 1999;177:28–32.
27. Klein I, Steger U, Timmermann W, et al. Microsurgical train-
ing course for clinicians and scientists at a German university
hospital: a 10-year experience. Microsurgery 2003;23:461–5.
28. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training
and assessment. Br J Surg 2004;91:1549–58.
29. Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson BA.
Objective Assessment of Skills in Intraocular Surgery (OASIS).
30. Saleh GM, Gauba V, Mitra A, et al. Objective structured
assessment of cataract surgical skill. Arch Ophthalmol 2007;
31. Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global Rating
Assessment of Skills in Intraocular Surgery (GRASIS). Oph-
32. Taylor JB, Binenbaum G, Tapino P, Volpe NJ. Microsur-
gical lab testing is a reliable method for assessing ophthal-
mology residents’ surgical skills. Br J Ophthalmol 2007;
33. Fisher JB, Binenbaum G, Tapino P, Volpe NJ. Development
and face and content validity of an eye surgical skills assess-
ment test for ophthalmology residents. Ophthalmology 2006;
34. Volpe NJ, Binenbaum G, Fisher JB, Tapino PJ. Author reply
[letter]. Ophthalmology 2007;114:1588–9.
35. Chung AK, Gauba V, Saleh GM. Assessing resident compe-
tency [letter]. Ophthalmology 2007;114:1587–8.
36. Aggarwal R, Grantcharov T, Moorthy K, et al. Toward
feasible, valid, and reliable video-based assessments of
technical surgical skills in the operating room. Ann Surg
Footnotes and Financial Disclosures
Originally received: May 12, 2008.
Final revision: May 12, 2008.
Accepted: September 23, 2008.
Available online: December 16, 2008.Manuscript no. 2008-571.
1Department of Education Moorfields Eye Hospital, London, United
2Department of Biosurgery and Surgical Technology, Imperial College
London, St. Mary’s Hospital, London, United Kingdom.
3Department of Ophthalmology Stoke Mandeville Hospital, Aylesbury,
4Western Eye Hospital, London, United Kingdom.
The authors have no proprietary or commercial interests in any materials
discussed in this article.
Hospital, London EC1V 2PD. E-mail: firstname.lastname@example.org.
Volume 116, Number 2, February 2009