The Need to Touch Medical Virtual Environments?

Abstract

Haptics technologies are frequently used in virtual environments to allow participants to touch virtual objects. Medical applications are no exception and a wide variety of commercial and bespoke haptics hardware solutions have been employed to aid in the simulation of medical procedures. Intuitively the use of haptics will improve the training of the task. However, little evidence has been published to prove that this is indeed the case. In the paper we summarise the available evidence and use a case study from interventional radiology to discuss the question of how important is it to touch medical virtual environments?

Full text

The Need to Touch Medical Virtual Environments?
Fernando Bello‡ Timothy R Coles* Derek A Gould† Christopher J Hughes*
Nigel W. John* Franck P Vidal§ Simon Watt*
*Bangor University, UK
‡Imperial College London, UK
†Royal Liverpool Hospital, UK
§University of California, San Diego

ABSTRACT
Haptics technologies are frequently used in virtual
environments to allow participants to touch virtual objects.
Medicalapplicationsarenoexceptionandawidevarietyof
commercial and bespoke haptics hardware solutions have
beenemployedtoaidinthesimulationofmedicalprocedures.
Intuitively the use ofhaptics w ill improve the training of the
task. However, little evidence has been published to prove
thatthisisindeedthecase.Inthepaperwesummarisethe
available evidence and use a case study from interventional
radiologytodiscussthequestionofhowimportantisitto
touchmedicalvirtualenvironments?
KEYWORDS:Haptics,virtualenvironments,touch.
INDEXTERMS: H.5.1[Information InterfacesAndPresentation]:
Multimedia Information Systems—Artificial, augmented, and
virtualrealities;J.3[ComputerApplications]:Lifeand Medical
Sciences
1 INTRODUCTION
Today, more than at any time in the history of medicine,
thereisunrelentingpressurefor changestoaccepted medical
practice, particularly as a consequence of legislation such as
theEuropeanworking timedirective,andtheCalmanreforms
intheUK[1][2].Someofthisisdrivingsurgicalmanagement
intonovelminimal access approaches, in turnraising further
challengesoftrainingtheincreasinglycomplexskillsthatare
a part of such innovative practices. Safe, effective training of
the next generation healthcare professional can benefit by
takingadvantageofnewtechnologiesandapproaches
emergingfromcomputerscience,engineering,psychologyand
otherrelateddisciplines.Theuseofhapticsdevicestoprovide
force and/or tactile feedback within a simulator is a prime
exampleofthisandthereareagrowingnumberofsimulators
fromawidescopeofmedicalspecialtiesthatusesomesortof
haptics device. A detailed survey of the  current state‐of‐the‐
artcanbefoundin[3].
A complete evaluation of transfer of skills to patients,
comparing training of a control group( notusing simulation)
againsttwo study groups(oneusingsimulation with, andone
without haptics feedback) has yet to be published. This has
partiallybeenaddressedbyMorrisetal[4],who
demonstrated that recall following visuohaptic training is
significantly more accurate than recall following visual or
haptictrainingalone,althoughhaptictrainingaloneisinferior
tovisualtrainingalone.Arecentreviewofhapticfeedbackin
conventional and robot‐assisted laparoscopic minimally
invasivesurgery(MIS)[5]concludedthatthereisnofirm
consensusontheimportanceofhapticfeedbackinperforming
MISinterventions.Whilst amajorityofthe studiesresultedin
a positive assessment of the inclusion of force feedback,
resultsareuncertainandinconsistent.Thisambivalencecould
wellbeduetothenatureoflaparoscopicMISproceduresthat
are carried out using thin elongated instruments inserted
through ports that may create frictional forces in excess of 3
Newtons [6] capable of interfering with more subtle haptic
cues.
Inadditiontoopensurgerywherethesurgeonlargelyrelies
onthesenseoftouchtoconducttheprocedure,thereare
severalmedical specialtieswherereacting tohapticscues isa
vitalpartofasuccessfulprocedure.Examplesoftheseinclude
interventional radiology, arthroscopy and internal
examinations(rectalandvaginal).Whethertheabovefindings
are true for any of these specialties has not yet been
investigated.Inthispaper wediscussfurtherifthere isa true
need for haptics devices within a medical simulator and use
ourworkindevelopingsimulationsforinterventional
radiologyasacasestudy[7][8].
2 INTERVENTIONAL RADIOLOGY
The practice of interventionalradiology(IR)usesimaging
(fluoroscopic, computed tomography, and ultrasound) to
guide catheters (tubes) and wires through organ systems
usinga small portal of entry (suchas a needle)into thebody
[7]. As a result, IR techniques generally have less risk, less
post‐operative pain and shorter recovery time compared to
opensurgery.IRextendstoavastrangeofimagingguided
proceduresthat are minimally invasive.Reactingcorrectly to
boththetactileandforcefeedbackelementsthatcomprisethe
practitioner’s ‘feel’ during a procedure is essential to avoid
complications.Wedescribebelowthreestageswheresuch
cues are important and have been implemented in our
simulatorsusinghapticsdevices.
2.1 Palpation
Palpation is the use of a clinician’s sense of touch to probe
deeply beneath the patient’s skin, seeking evidence of any
pathology in the underlying anatomical structures. Palpation
andgeneralhapticresponse commonly requires multi finger,
multi contact tactile manipulations. Such an effect is difficult
to achieve within a medical virtual environment and when
included,themanipulationisusuallygreatlysimplified.
 Itisnecessarytosimulatebothforcefeedbacktoconveythe
resistive force of the skin, organs and bones and, tactile
feedbacktoconveythefinerinformationsuchasthepulseand
small abnormalities felt at the surface of the fingertips.
Althoughoverthelast15yearstherehasbeenalotofactivity
in the development of force feedback devices, a lack of
understanding of the large number of different tactile
receptorsinourhandshasmeantamuchlowerrateof
developmentoftactiledeviceswithnosignificantproduct
available commercially. Possible technologies include use of
piezoelectric materials (as in Fig. 1), the vibrations from a

small audio speaker, pin arrays, and pneumatic solutions.
Practically, stimulating each of the fingertips’ receptors as
stimulated in a real palpation is infeasible, however, and an
approximationusingaforcefeedbackdevice combinedwitha
mannequinlike end effector appears to currently provide the
besttradeoffbetweencostandfidelity.

Figure 1. Palpation device combining force feedback from a
NovINT Falcon and tactile feedback at the fingertips from
piezoelectric materials.
2.2 Needle Insertion
Aneedle insertion is awidelyperformedprocedure which, in
thecontextofIR,isneededforabiopsyortointroduceaguide
wire into the femoral artery, liver, kidney, etc. The task
requires 6 DOF but force feedback can be realistically
simulatedwithonly5degreesofforcefeedbackbyneglecting
theforces involved inrotatingthe needle shaftalongwhichit
is inserted. Many needle insertion simulations, both
commercial and academic opt to simulate only 3 degrees of
force feedback to reduce simulation cost for example
Mediseus Epidural from MedicVision (Kensington, Australia)
andourownsimulator[7].
 UltrasoundimagesarecommonlyusedinIRtoguide
needles.Prior tothe needle insertion, an anaesthetic solution
isinjectednearthepuncturesite.Itisfollowedbytheincision
ofthepatient’sskinsothatnoresistancefromtheskincanbe
feltduring the needleinsertion.Thenthe ultrasound probe is
placedonthepatient’sskinattheincisionsitetolocatethe
needletarget.Finallytheneedlecanbeinserted.Figure2
showsa simulationofthisstage usingcommerciallyavailable
forcefeedbackdevices.Itshouldbevisible on the ultrasound
screenasmuchaspossible.Lookingattheultrasound
monitor,radiologistsoftenjiggletheneedletoidentifyitstip
location within the patient. Once the needle is visible on the
image,itisslowlyadvancedsthetarget.Althoughtheneedle
should be identified on the ultrasoundimagesusedfor
guidance,thisisnotalwaysthecase.Whenthevisualfeedback
doesnotprovideanyinformationaboutthelocationofthe
needle tip, the resistance on the needle during the insertion
will provide invaluable information. For example, at the
interface between two kinds of tissue, the resistance will
increase until the surface of the deeper tissue is punctured,
thenit willdecrease.Also,the force required ontheneedleis
greater for the kidney than fat, and it is greater for the
diseased, cirrhotic liver than the normal kidney. Therefore,
such features are also required when teaching image guided
needlepuncture using a VRtraining simulator. The response
of real tissues during an actual needle insertion can be
recordedandmodeledanalyticallytointegraterealistic
hapticsmodels[10].
Figure 2. Using PhanToM Omni Force feedback joysticks as an
ultrasound probe and virtual needle.
Althoughvalidatedneedleinsertionsimulationshavebeen
produced,noneasyetleaddirectlyontothenextstageofan
IRprocedure,guidewiremanipulation.
2.3 Guidewire and Catheter manipulation
Inlaparoscopicsurgery,rigidinstrumentsprovideaccessto
theabdominalcavity.Theseconveylimitedtactilecueingfrom
viscera,withkeycuesbeingvisual,from2Dimaging.InIR,the
longflexibletools(catheter,guidewire)involvedmayconvey
greater tactile feedback, which is highly relevant to avoiding
complications and maintaining safe practice. For this reason,
realistic IR catheterisatonsimulations cannotusejoystick like
interfaces. Instead, frictional sensors provide essential
rotationalandtranslationalhaptics,suchaswouldbeneeded
forEndoscopysimulations.
Replication of haptics in simulations is necessarily an
approximation,butcanbeenhancedbyseekingevidencefrom
procedures performed in the real world. This in itself is
challengingwork, whichmustuse unobtrusive, novel sensors
thatavoidinterferencewithcorrectperformanceofa
procedure in a patient. At the same time, collection of
procedural force data might indicate the basis of haptics
cueingforaproceduralstep,distinguishingbetweencuesthat
useabsoluteforce,andthosethatarebasedonrelativevalues
during one or more actions. Such data can then be used as a
basis for refining algorithms, and identifying the actual
resolutionneededforaninterfacedevicetomeetaspecific
leveloffidelity.
Figure 3. Simulator for the Seldinger Technique. Real guide wires
and catheters can be fed through a needle portal into the virtual
patient. Custom hardware has been designed to orientate the
needle (inset picture) and provide force feedback.
 A major goal fo r a successful outco me is not to penetra te a
vesselwallwiththeguidewire.Insomecasesaverylowforce
canresult in penetration (particularly incompleteocclusions,
where the force to continue along the lumen can be very
similar to that which allows wall perforation). Responding
correctlytotheperceivedhapticscuesisthenvital.

3 DISCUSSION

Agoodoverviewoftheissuesthatneedtobeconsidered
whenassessingasurgicalsimulatorcanbefoundin[11].The
need for multidisciplinary collaboration to build an effective
simulator is advocated. Other important points made are the
advantagesfortrainingindeconstructingtasksintosimple
steps, unlimited deliberate practice with th is repeatability  of
procedures facilitating learning from mistakes, the provision
ofobjective feedback, andtheneed to integratethesimulator
intotheeducationcurriculum.Thereisoftenatradeoff
betweenthefidelityofthesimulationanditscost,anditisnot
always necessary to achieve ultra high fidelity in order to
provideatrainingbenefit.
 Basicresearchoncross‐modalperceptiondoes indicate the
likelyimportanceofhaptics in virtual medical environments.
Its potential contribution is most readily apparent when
visionand haptics providequalitativelydifferent information.
Thisarisesfrequentlybecausethetwosensorysystemshave
verydifferentcapabilities:visionprovidespreciseinformation
about spatial properties, whereas haptics provides
information about texture, and material properties such as
stiffness/compliance [12]. Recent research shows, however,
thathapticperceptionalsoplaysanimportantroleevenwhen
equivalentinformationisavailable,simultaneously,from
vision.In this case perceptionis not dominated by onesense
(i.e. vision). Instead, information from vision and haptics is
unconsciously and automatically integrated, with each signal
weighted according to how reliable it is in a given instance
[13].Thebenefitofthisisthatobjectpropertiesareestimated
morepreciselythanispossible from either sense alone. If
sensory integration occurs in real‐world bimodal tasks, it
shouldpresumablyoccurinmedicalsimulatorstoo.The
process of sensory integration only operates appropriately,
however, if the brain can determine which visual and haptic
signals'belongtogether'‐thatis,whichsignalsprovide
informationaboutthesameobjectand whichdo not[14][15].
Thisprocessappearstobedynamicandflexible:visual‐haptic
integrationhas been observed whenusingtools,forexample,
which systematically alter the normal spatial relationships
betweenvisual and haptic signals [15].However,spatial(and
temporal) congruency between movements of the hand
and/or surgical instruments and the visual consequences of
thosemovementsseemstobecrucialforsensoryintegration
to occur [14][15]. This suggests that it is critical to achieve
spatial and temporal co‐registration of visual and haptic
signalsinvirtualenvironments.Moreover,wecurrentlyknow
littleabouttheimportanceofsensoryintegrationforlearning
bimodaltaskssuch as surgical procedures. There is evidence
thatpeopleform'amodal'spatialrepresentationswhenvisual
andhaptic information is presented simultaneously(asisthe
casewhencarryingoutataskintherealworld),butnotwhen
they are temporally separated [16]. It seems plausible, then,
thattraining inthesame(bimodal) sensory conditionsasthe
realtask could lead bothtobetterlearningandbetteroverall
performance.
 The current published evidence clearly demonstrates that
VR simulation can improve intra‐operative performance. We
advocate that good use of haptics (e.g. force responses are
accurately modeled, the resolution of the haptics device is
appropriate,onetooneregistrationbetweenthehaptics
hardwareandthevirtualtoolisachieved,thehapticscue
appearsatthecorrectlocationintheanatomy,etc.)hasan
importantroletoplayinachievingthisgoal.Thiswillbe
particularlyimportantinmanymedicalspecialitieswhere
reacting to haptics cues is a vital part of a successful
procedure,suchastheIRtasksdescribedabove.Weplanto
use the needle puncture simulator to set up a test study to
provethishypothesis.

4 REFERENCES
[1] K. Calman, “Hospital doctors: training for the future”, British
JournalofObstetricsandGynaecology,vol.102(5),1995,pp.354‐
356.
[2] F.D.Skidmore,“Juniorsurgeonsarebecomingdeskilledasresult
ofcalmanproposals”, Britishmedicaljournal, vol.327,1997, pp.
1032‐1037.
[3] T.Coles,DMeglanandN.W.John,“TheRoleofHapticsinMedical
TrainingSimulators:ASurveyoftheState‐of‐the‐art”,IEEE
TransactionsonHaptics.InSubmission.
[4] D.Morris,H.Tan,F.Barbagli,T.Chang,andK.Salisbury,"Haptic
Feedback Enhances Force Skill Learning", SecondJoint
EuroHapticsConferenceandSymposiumonHapticInterfacesfor
VirtualEnvironmentandTeleoperatorSystems(WHC'07), 2007,
pp.21‐26.
[5] O.A.J.vanderMeijdenandM.P.Schijven,“Thevalueofhaptic
feedback in conventional and robot‐assisted minimal invasive
surgery and virtual reality training: a current review”, Surg
Endosc(2009)23:1180–1190.
[6] PicodG,JambonAC,VinatierD,DuboisP,“Whatcantheoperator
actuallyfeelwhenperformingalaparoscopy?”(2005)Surg
Endosc19:95–100.
[7] P.F.Villard,F.P.Vidal,C.Hunt,F.Bello,N.W.John,S.Johnson,and
D.A.Gould,"Aprototypepercutaneoustranshepatic
cholangiography training simulator with real‐time breathing
motion," InternationalJournalofComputerAssistedRadiology
andSurgery,vol.4,2009,pp.571‐578
[8] V.Luboz, C.Hughes,D. Gould,N.W.John,and F.Bello,"Real‐time
Seldinger technique simulation in complex vascular models,"
InternationalJournalofComputerAssistedRadiologyandSurgery,
vol.4,2009,pp.589‐596.
[9] S.E. Lakhan, A. Kaplan, C. Laird, and Y. Leiter Y, "The
interventionalism of medicine: interventional radiology, cardiology,
and neuroradiology". International Archives of Medicine 2 (27).
2009.
[10] F.P. Vidal, A.E. Healey, N.W. John and D.A. Gould, “Force
PenetrationofChibaNeedlesforHapticRenderinginUltrasound
Guided Needle Puncture Training Simulator”, MICCAI2008–
WorkshoponNeedleSteering:RecentResultsandFuture
Opportunities,2008
[11] A.G.GallagherandE.M.Ritter,“VirtualReality:Objective
Assessment, Education, and Training”, EmergingTechnologiesin
Surgery,M.Satava,A.GaspariandN.DiLorenzo(editors), ISBN
978‐3‐540‐39599‐7SpringerBerlinHeidelberg2007,pp27‐32
[12] S.J.LedermanandR.L.Klatzky,“Hapticperception:atutorial,”
Attention,Perception,andPsychophysics,vol.71,pp.1439‐1459,
2009.
[13] M.O.ErnstandM.S.Banks,“Humansintegratevisualandhaptic
informationinastatisticallyoptimalfashion,”Nature,vol.415,
pp.429‐433,2002.
[14] H.B.HelbigandM.O.Ernst.“Knowledgeaboutacommonsource
canpromotevisual‐hapticintegration,”Perception,vol.36,1523‐
1533,2007.
[15] C.Takahashi,J.Diedrichsen,andS.J.Watt,“Integrationofvision
andhapticsduringtooluse,”JournalofVision,vol.9:3,1‐13,
2009.
[16] N.A.Giudice,R.L.Klatzky,andJ.M.Loomis,J.M.,“Evidencefor
amodalrepresentationsafterbimodallearning:integrationof
haptic‐visuallayoutsintoacommonspatialimage,”Spatial
CognitionandComputation,vol.9,287–304,2009.