Haptic simulation of breast cancer palpation: A case study of haptic augmented reality.

Full text

Page 1

Haptic Simulation of Breast Cancer Palpation:
A Case Study of Haptic Augmented Reality
Seokhee Jeon∗
Haptics and Virtual Reality
Laboratory, POSTECH, Korea
Benjamin Knoerlein†
Computer Vision Laboratory
ETH Zurich, Switzerland
Matthias Harders‡
Computer Vision Laboratory
ETH Zurich, Switzerland
Seungmoon Choi§
Haptics and Virtual Reality
Laboratory, POSTECH, Korea
ABSTRACT
Haptic augmented reality (AR) allows to modulate the haptic prop-
erties of a real object by providing virtual haptic feedback. We
previously developed a haptic AR system wherein the stiffness of a
real object can be augmented with the aid of a haptic interface. To
demonstrate its potential, this paper presents a case study for med-
ical training of breast cancer palpation. A real breast model made
of soft silicone is augmented with a virtual tumor rendered inside.
Haptic stimuli for the virtual tumor are generated based on a con-
tact dynamics model identified via real measurements, without the
need of geometric information on the breast. A subjective evalua-
tion confirmed the realism and fidelity of our palpation system.
Index Terms:
H.5.2 [Information Interfaces and Presentation]:
User Interfaces—Haptic I/O; H.5.1 [Information Interfaces and
Presentation]: Multimedia Information Systems—Artificial, aug-
mented, and virtual realities
1
In haptic augmented reality (AR), haptic signals of real environ-
ments are modulated or augmented with virtual touch feedback. In
recent research we have focused on integrating haptic feedback into
visual AR systems [2] as well as on modulating the stiffness of real
objects with the aid of a haptic interface [4]; the latter system allows
to make the stiffness of a real object virtually harder or softer.
This paper examines the application of this technology in the
context of a medical training environment. We introduce AR-based
simulation methods for the haptic response of a tumor surrounded
by soft tissues, as a case study for breast cancer palpation training.
To achieve high-fidelity touch feedback, a real breast model made
of soft silicone is augmented with a harder virtual tumor rendered
inside. The real silicone model produces natural haptic feedback of
the breast tissue deformation, while our AR system is responsible
for the tumor simulation. For the recreation of the tumor response,
weuseacontactdynamicsmodelidentifiedusingpositionandforce
data measured from a real breast model containing an actual tumor
lump. In particular, our framework requires no preprocessing for
the geometric model of the breast, preserving a crucial advantage
of AR. An initial subjective evaluation confirmed that our system
can provide realistic behavior close to the real counterparts.
INTRODUCTION
2
The goal of the present system is to modulate the stiffness of a real
breast model as if a stiffer tumor were placed inside. The behav-
ior of the breast model silicone is highly homogeneous, thus facil-
itating the model-based estimation of the dynamic response of the
tumor.
INTERACTION MODEL
∗e-mail: yeager@postech.ac.kr
†e-mail:knoerleb@vision.ee.ethz.ch
‡e-mail:mharders@vision.ee.ethz.ch
§e-mail:choism@postech.ac.kr
Our system is configured as shown in Figure 1(a). The response
force from the real breast model at time t, fr(t), is what the user
perceives if no virtual tumor is rendered. The goal is to alter the
force delivered to the user’s hand, fh(t), from fr(t) to
fh(t) = fr(t)+ft(t),
(1)
where ft(t) is the force that the haptic interface produces to repre-
sent the virtual tumor. The realism of the tumor simulation relies on
the recreation accuracy of ft(t) according to the user’s interaction.
A key idea of our approach is to derive ft(t) based on a nonlin-
ear dynamics model identified using data measured from a breast
mock-up containing a real tumor. This allows us to minimize the
preprocessing for the breast geometric model and the tumor re-
sponse while preserving plausible simulation realism. We use the
Hunt-Crossley model, which can account for the nonlinear vis-
coelastic contact dynamics of a deformable object such as human
tissues [3], to describe the responses of the tumor and silicone mod-
els. It has the form of
f(t) = Ke{x(t)}m+Be{x(t)}m˙ x(t),
where x(t) and ˙ x(t) are the displacement and velocity of the haptic
device tip, respectively, Keis object stiffness, and m is a constant
exponent (usually between 1 and 2).
Variables necessary to derive ft(t) are defined in Figure 1(c). In
our current model we assume that the tumor has a spherical shape.
ptis the position of the tumor sphere, and ptsis the closest point on
the original non-deformed breast surface from pt. Both values are
known at the start, and our algorithm assumes that they are constant
over time. The effect of tumor movements on ft(t) is, however, still
captured in the response model obtained in the preprocessing step
and is thus included in ft(t). Let the line segment ptsptbe l0. We
first identify the Hunt-Crossley model that describes the force re-
sponse of the tumor along l0in the preprocessing (see Section 3).
This is the only information that our algorithm needs in advance.
Then, using this identified information we approximate ft(t) at po-
sitions not on l0and render the virtual tumor based on this approx-
imation (see Section 4).
(2)
3
To identify the Hunt-Crossley model describing the tumor’s re-
sponse along l0, we use data collected from two real breast models;
one with a real tumor model of higher stiffness included and one
without. The two breast models were made by casting a mixture of
Ecoflex 0030 (SmoothOn Inc.) and silicone thinner into a breast-
shaped mold (half sphere of 55 mm radius). The no-tumor model
had uniform elasticity, and its linear stiffness measured at 10 mm
displacement was 0.13 N/mm. The tumor-embedded model had the
same stiffness except for a 12.5 mm-radius, harder tumor (stiffness
of 0.54 N/mm) at 25 mm below the surface.
The hardware configuration is shown in Figure 1(b). We use
a PHANToM 1.5 high force model (Sensable Technology) for the
haptic interface, which is capable of 3DOF force feedback and
6DOF pose sensing. A 6D NANO17 force sensor (ATI Automa-
tion) is attached at the end of the interaction tool to measure the
reaction force from a real object.
PREPROCESSING TUMOR RESPONSE
237
IEEE International Symposium on Mixed and Augmented Reality 2010
Science and Technolgy Proceedings
13 -16 October, Seoul, Korea
978-1-4244-9345-6/10/$26.00 ©2010 IEEE

Page 2

(a) System configuration(b) Hardware
(c) Definitions of variables
Figure 1: System configuration and variables definition.
Using this setup, we palpated the two models and collected a
set of data triples (reaction force, deformation displacement, and
velocity) for each model. We denote the data triple for the no-
tumor model as (f1,x1, ˙ x1) and that for the tumor-embedded model
as (f2,x2, ˙ x2). When palpating the tumor-embedded model, spe-
cial care was taken to press along l0by carefully selecting the
contact point and the pressing direction. Then, we estimated the
Hunt-Crossley model parameters for the no-tumor model using
(f1,x1, ˙ x1), which is denoted by H1(x, ˙ x). This represents the mag-
nitude of fr(t) in (1). Since f2values measured from the tumor-
embedded model include both fr(t) and ft(t), the magnitude of ft(t)
can be extracted by subtracting f1from f2. To this end, we passed
all data pairs of (x2, ˙ x2) to H1(x, ˙ x) and computed the differences by
ft(x2, ˙ x2) = f2−H1(x2, ˙ x2).
(3)
By identifying the Hunt-Crossley model again using the data of
(ft,x2, ˙ x2), the response of only the tumor along l0was derived.
This model is denoted by Ht(x, ˙ x). The parameters of the Hunt-
Crossley model were identified using the recursive least-square es-
timation proposed in [1].
4
The palpation begins with touching the breast model using the hap-
tic tool. The time instance when the tool collides with the breast
surface is detected by our algorithm in [4]. After the contact, the
haptic interface exerts forces for virtual tumor rendering.
Suppose that a user makes a deformation of d(t) at time t in
Figure 1(c). d(t) is directed from phs(t) to ph(t), where ph(t) is
the haptic tool position, and phs(t) is the closest point from ph(t)
on the non-deformed breast surface. To determine phs(t), we use
an estimation method that uses dynamics models based on reaction
force measurements [4], instead of a geometric model of the breast.
Then, the tumor response force ft(t) is determined by
RENDERING
ft(t) = ft(t)ph(t)−pt
|ph(t)−pt|.
(4)
ft(t) is directed from pt(tumor position) to ph(t) (tool tip position)
with magnitude ft(t).
To estimate ft(t), we use the following algorithm. Let lt(t) be a
line segment phs(t)pt. Then, we can project the tool position ph(t)
to lt(t) as
xlt(t) = d(t)·ut(t),
(5)
where ut(t) is a unit vector from phs(t) to pt. xlt(t) represents the
deformation caused by the virtual tumor reflected in d(t).
From xlt(t), we determine ft(t) using the Hunt-Crossley model
of the tumor obtained by the preprocessing, Ht(x, ˙ x). Ht(x, ˙ x) repre-
sents the exact response dynamics of the tumor when a user presses
along l0. Under the homogeneity assumption, the tumor response
along lt(t) can be described by Ht(x, ˙ x) if the length of l0is identi-
cal to the length of lt(t). But in general, |l0| ≤ |lt(t)|, thus we use
the following approximation:
x(t) = xlt(t)
|l0|
|lt(t)|,
(6)
where x(t) is a linearly-normalized deformation magnitude in rela-
tion to the reference deformation along l0. Then, the force magni-
tude due to the virtual tumor is estimated as
ft(t) = Ht(x(t), ˙ x(t)).
This algorithm is a plausible approximation to the real physical
responses, designed for real-time rendering while avoiding the need
of geometric models of real objects. We confirmed in an initial sub-
jective evaluation reported in Section 5 that virtual tumors rendered
using this algorithm are perceptually similar to real cases. We note
that the algorithm may not be applicable to the cases where body
parts surrounding a tumor are highly inhomogeneous.
(7)
5
The performance of our haptic AR algorithm was verified through
a user experiment. Twelve subjects were asked to freely explore
two breast models—one with a real tumor and the other with an
augmented tumor—placed side by side, and to rate the perceptual
similarity of tumor palpation in a 7-point Likert scale. Point 1 rep-
resented that the two breasts were completely different, and point
7 that the two breasts were exactly the same. The mean and stan-
dard deviation of the similarity scores were 6.1 and 0.79, respec-
tively. This initial result indicates that the haptic feedback of the
augmented tumor is comparable to the real tumor mock-up. How-
ever, more detailed studies will be carried out in the future.
SUBJECTIVE ASSESSMENT
6
We developed a promising AR-based training system for breast tu-
mor palpation. The system achieves excellent realism without the
need of a real object geometry model, which is an advantage of our
system for practical applications. We hope that this work would
prompt more attention to the field of haptic AR and its application.
CONCLUSIONS
ACKNOWLEDGEMENTS
This work was supported in parts by a Korean-Swiss Cooperative
program 2009-00539, by a NRL program R0A-2008-000-20087-0
both from NRF, and by an ITRC program NIPA-2010-C1090-1031-
0006 from NIPA, all funded by the Korean government.
REFERENCES
[1] A.HaddadiandK.Hashtrudi-Zaad. Anewmethodforonlineparameter
estimation of Hunt-Crossley environment dynamic models. In IEEE
Intl. Conf. on Intelligent Robots and Systems, pages 981–986, 2008.
[2] M. Harders, G. Bianchi, B. Knoerlein, and G. Szekely. Calibration,
registration, and synchronization for high precision augmented reality
haptics. IEEE Transactions on Visualization and Computer Graphics,
15(1):138–149, 2009.
[3] K. Hunt and F. Crossley. Coefficient of restitution interpreted as damp-
ing in vibroimpact. ASME J. of Applied Mechanics, 42:440–445, 1975.
[4] S. Jeon and S. Choi. Stiffness modulation for haptic augmented reality:
Extension to 3D interaction. In Proceedings of the Haptics Symposium,
pages 273–280, 2010.
238