Content uploaded by Soroush Sadeghnejad
Author content
All content in this area was uploaded by Soroush Sadeghnejad on Jul 22, 2018
Content may be subject to copyright.
Phenomenological Contact Model Characterization
and Haptic Simulation of an Endoscopic Sinus and
Skull Base Surgery Virtual System
Soroush Sadeghnejad
1
, Mojtaba Esfandiari
1
, Farzam Farahmand
2
and Gholamreza Vossoughi
2
School of Mechanical Engineering
Sharif University of Technology
Tehran, Iran
1
{s_sadeghnejad, esfandiari_m}@mech.sharif.edu,
2
{farahmand, vossough}@sharif.edu
Abstract— During the endoscopic sinus and skull base
training surgeries, the haptic perception of tool-tissue
interaction and even transitions and ruptures in the tissues are
fundamental which should be taken into account in a robotic
control scheme. However, this problem is extremely complex
given the nature and the variety of tissues involved in an ESS
procedures. In this article, ex-vivo indentation and relaxation
experiments associated with an offline model estimation of the
interaction between tissues and a surgical tool are presented.
The estimated parameters of the modified Kelvin-Voigt model
are then used to provide a realistic tool-tissue interaction
dynamic model. Finally, the principle of a virtual reality
simulation scheme that would allow better haptic discrimination
of tool-tissue interaction is proposed and illustrated.
Keywords— virtual reality; haptic; endoscopic sinus surgery;
modified Kelvin-Voigt model.
I.
I
NTRODUCTION
Use of virtual reality and integrating the haptic systems in
the surgical simulators, which contains two main human
sensorial modalities as visual and touch, allow surgeons to
practice various types of procedures to minimize any risk of
patient’s life during a real surgery [1]. The ideal simulator for
Endoscopic Sinus Surgery (ESS) must be supported by a
physical model and should have the ability to provide
repetitive practice under a controlled environment, which
includes the rendering of temporary and permanent changes
[2]. While the interaction of tools with the variety of tissues
that are soft, semi soft, and rigid, they should look and feel
like real [3]. The most significant challenge in the VR-based
surgical simulation are two folds, first, the development of
realistic tissue models and second, dynamic simulation,
containing appropriate control strategy [4, 5]. Acquiring data
from biological tissues and developing models appropriate
for application in ESS simulation or robot-assisted surgery,
are difficult due to different reasons. First, the different
layers, deformed by a surgical tool, are such that both soft and
hard tissues exist. Second, these parameters values may be
patient dependent. Finally, since the tool makes a large
deformation while interaction with the tissues, the interaction
is clearly nonlinear. For all these reasons, haptic modeling,
which states the relations between motions and interaction
forces, is a difficult task in the case of ESS operation. The
essence of haptics, in virtual reality-based training systems,
is to propose a condition in which the boundaries between
reality and virtuality will be removed and it will make a
system transparency achievable. Due to the ever-increasing
demand for medical simulators within the field of rhinology,
the role of control methods is to provide an interconnection
between the dynamic simulation and the haptic interface [6].
Indeed, tissue dynamic properties are important to design the
system’s control strategy, particularly in the case of
impedance or admittance types virtual environments. The
admittance-type environments compute a displacement in
response to the measurement of the haptic interface force
while the impedance-type environments calculate a reaction
force in response to the users’ position. In many VR system
applications, especially those with high force output
requirements, admittance-type controllers are widely
employed in the control of haptic devices [7].
To address the haptic or force feedback as a very
supplementary information for the surgeons in most surgical
robotic training systems, in this research, a considerable
efforts dedicated to study the interactions between surgical
tools and simultaneous soft and hard tissue deformation in the
sinonasal region for proposing a phenomenological contact
model for further use in an Endoscopic Sinus and Skull Base
Surgery Haptic Simulation System. A number of
experimental surveys have been performed to study the
mechanical behavior mechanisms of sino-nasal region
surface under indentation and relaxation loading. We used the
classical standard nonlinear viscoelastic model (modified
Kelvin-Voigt model), in order to estimate the deformation of
the tissues prior to the rupture point. To improve the stability
and performance of haptic interaction with impedance haptic
devices, we use adaptive nonlinear controllers, proposed in
[6], and we simulated a simple haptic system without
considering any delay, in order to be used in an endoscopic
sinus and skull base surgery virtual-based simulation system.
The rest of this paper is as follows. The methodology for
characterization of tool-tissue interaction forces and design
and implementation of an adaptive controller for a haptic
interface device, interacting with admittance-type virtual
environments are presented in Section 2. Section 3 presents
the tissue modeling results and also simulation studies which
conducted to illustrate the performance of the proposed
robust adaptive controller and the concluding remarks are
presented in Section 4.
II.
M
ETHODOLOGY
A. Tool-tissue interaction experiments and preparation
Characterization of the tool-tissue interaction for accurate
modeling of the tissue deformation phenomena is necessary
to use the force data associated with the tool interaction. To
measure and analyze the tissue behaviors of the coronal
orbital floor, there are several types of experiments that can
be performed.
Amongst usual experiments, indentation and
relaxation loadings can be used to study the mechanical
behavior mechanisms of sinonasal regions. Indentation and
relaxation experiments were performed ex-vivo and the tool
insertions, in the sheep sino-nasal region, were used as a
benchmark for the reason that the sinonasal tissues of a sheep
head are rather similar to human tissues. In order to have deep
insight in the sinonasal anatomy of sheep head, we asked an
ENT surgeon, from Tehran University of Medical Science, to
conduct animal experiments. Performing the surgery
procedures, the surgeon can identify the relevant areas, where
are particularly important and the measurement tasks can be
performed and the data can be acquired. Prior to starting the
surgery procedures, a high-resolution CT scan had been
performed on the sheep head for further navigation purpose
and data processing. Force-displacement data were acquired
using a 1-degree-of-freedom translation INSTRON 5560
Series Table Model Testing Systems. The sheep heads were
all tested at room temperature. The indentation and relaxation
experiments were conducted with 17 adult sheep head to
characterize the material properties of the sino-nasal region.
We developed an indenter based on the surgery tool shape
with the rounded tip of 4mm diameter. The specimens were
intended three times in six different rates of 10, 70, 100, 150,
250 and 500mm/min.
Although there are other experiments to measure the tissue
behavior [9, 10], in this research, the relaxation tests
preformed to evaluate the mechanical properties of the
tissues, as well. A position step input preformed on the
specimens and the exerted forces have been measured. In
these series tests, the coronal orbital floor of the sheep heads
was first indented to pre-defined depths of 2, 4, 6 and 8mms
with the indenter speed of 250mm/min and held there for the
30s to record the force relaxation response of the tool with
respect to time [11]. By acquiring the tissue viscoelastic
parameters from the indentation experiments, one can easily
model the phenomena in order to study the tool-tissue
interaction in the endoscopic endonasal sinus surgery.
B. Tissue mechanical model
To come to a phenomenological model, the schematic
representation of this standard nonlinear solid model is
represented in Fig. 1 [8]. In many tool-tissue interaction
applications, Kelvin-Voigt contact models are suitable for
modeling the nonlinear viscoelastic effects. As presented in
[8], the components of the model is dependent on the tool
deformation, in order to contain nonlinear behavior of the
material during any large deformation. This issue will
become very important in the case of rupture modeling since
definitely large deformation will occur prior to any rupture.
In the proposed model, there are two definite functions. The
static component of the exerted tool force, which composed
of nonlinear force-deformation function and the series
connection of nonlinear spring and damper, calculate the
dynamic component of the acted forces. Assuming the
standard nonlinear viscoelastic model, as what is revealed in
[8], the contact force can be given by:
(1)
Where
is the total force exerted by the tool during any
interaction with any tissue,
is the nonlinear part of the
static force which is dependent to just deformation and
is
the dynamic part of the exerted force which is both dependent
on tool deformation and time. The dynamic part of the
exerted force is calculated, using:
(2)
and
are the stiffness and damping functions, which
are defining the tissue dynamic behaviors.
The relation
between the position of spring and damper can be concluded
in:
(3)
Combining
equations 1 with 2, we can have:
(4)
Taking a time derivative of equation (4) and using again
the aforementioned equation and equation (2), the general
force-displacement relationship is given by the following
equation:
(5)
Supposing the relaxation time
(independent of due to
the dependency of both stiffness and damping parameters to
a same function of deformation with the different ratio), we
can define different relations, based on the proposed model
and basic concept of the modified Kelvin model. Using
equation (2) and defining
, the spring
deformation will yield as:
(6)
Taking the time derivative of (3) and combining the result
with (6) yields the linear differential equation of spring
displacement. The general solution for an aforementioned
equation with respect to a constant tool velocity can be
obtained by a convolution integral. However, the total force–
deformation response of the model can be defined as:
(7)
Fig. 1 Modified standard nonlinear viscoelastic model
For a given deformation, increasing the indentation
velocity always increases the tool-tissue force interaction.
Also, supposing the
and as
increasing functions,
equation (7) reveals that the maximum force of the interaction
is a decreasing function of insertion velocity [8].
C. Tissue mechanical model identification
The work described here is the estimation of the model
parameters in the structure shown in Fig. 1, from input–
output data, which can properly estimate a general behavior
of the tissue prior to a rupture event. However, the focus will
be on estimation algorithms for the proposed model structures
proposed in (5). To evaluate the modified contact model for
this phase, we should identify the parameters of modified
Kelvin model. The two parameters of the model, which are
necessary to be identified, are the static and dynamic stiffness
parameters. In the very low-velocity indentation experiment,
the acquired data is the best fit by a third-order polynomial
equation form. So, for the static part, we will assume a third
order polynomial function of the following form:
(8)
Because all of the experimental tests have been performed
at constant speeds, therefore the velocity can be treated as
a constant coefficient. We can write the general form of the
total interaction force equation, ((5)), in the new form as:
(9)
In which,
and
. To identify the tissue
viscoelastic parameters in equation (5), the Sum Square Error
(SSE) criteria is defined to evaluate the training process. For
all training patterns and model outputs, the SSE is calculated
by:
(10)
In which, P is the index of patterns, from 1 to , where is
the number of patterns, is the index of outputs, from 1 to
, where
is the number of outputs in the pattern. is
the input vector, is the unknown parameter vector and
is the training error at output , when applying pattern p and
it is defined as:
(11)
Where,
is experimental force related to the output
at pattern .
is the calculated force from the proposed
model related to the output at pattern . The Jacobian
matrix is introduced as:
(12)
To optimize the equation (10), we used the iterative
method update rule based on the Levenberg-Marquardt
algorithm, as follow:
(13)
For sufficiently large values of , the matrix
is positive definite, and a descent direction is guaranteed.
D. Adaptive control Design of a haptic interface device
interacting with an admittance-type virtual Environments
As underlined in the introduction, the haptic rendering and
having a reasonable insight of the tool-tissue interaction that
the practitioner feels during any surgery operation are very
important. However, it can sometimes be difficult to detect a
real haptic interaction, in particular for surgery simulation
systems with the lack of an appropriate physical model.
Using the physical interaction model presented in section
2, the haptic perception of force interaction can be enhanced.
To achieve this purpose, the simplest solution would be to
provide an admittance-type virtual environment controller,
which has a good performance in rendering the rigid contacts.
Fig. 6 reveals the structure of the haptic control system in
interaction with admittance-type virtual environments. In this
case, the environment force
is the input to the virtual
dynamic environment and the environment acceleration
,
velocity
, and position
are calculated based on this input
force. The dynamics of the haptic device and the user in the
workspace coordinates can be written as:
(14)
In which,
is the human exogenous force input which
passes through the hand dynamics,
denotes the control
input force, is the un-modeled dynamics and
uncertainties:
(15)
Considering a linear MSD model for the dynamic of the
operator’s hand, where is a symmetric and uniformly
positive-definite function and is upper bounded and
satisfies skew-symmetry property. Considering an adaptive
controller, designed in [6], which is based on Slotine’s
passivity-based position controller and the adaptive
motion/force controller, if we define the velocity
errorbetween the operator’s hand and the virtual tool as:
(16)
and if the reference signal
in (16) is defined as:
(17)
We can combine the system dynamic equation in (14) with
the equations (16) and (17) and simplify the system dynamic
equation to have:
(18)
In which, is a dynamic regression matrix and are the
unknown parameters which should be estimated, properly.
(19)
Choosing the
control
signal as:
(20)
Where
is the control law,
is the estimate of unknown
parameters,
is a positive constant and
is the robust
controller term.
(21)
are positive coefficients and also
,
is a continuous
differentiable function which satisfies the following
condition:
(22)
Defining the parameters estimation errors as
, and
using
as the parameters estimation law, one can
easily design a stable adaptive controller for the haptic system
with an admittance virtual environment (Appendix I).
III.R
ESULTS
A. Tissue mechanical modelling results
As previously underlined, the biological tissue linearly
behaves for small deformation, but from our acquired data, it
nonlinearly behaves due to the existence of large
deformation, so a nonlinear viscoelastic model should be
used in order to model the force response of the tissue for this
step. The relaxation time
can be determined using the
acquired data from the relaxation experiments. The total of
four probing measurements of the coronal orbital floor of
sheep sinonasal regions was taken and the raw data, which
have been collected from different indentation depths are
shown in Fig. 2. We implemented an exponential force-time
response to the part of the relaxation data where the
acceleration and inertia force have less effect on the recorded
data. Table 1 gives the relaxation time, resulted from four
different relaxation experiments. From the presented result,
we can conclude that the average relaxation time for the
coronal orbital floors can result in a time constant of
. Fig. 3 displays the typical force-displacement
response of an insertion for a surgical tool into the coronal
orbital floors for various velocities. Due to the fact of
differences in race, gender and age of each specimen and
place of indentation on the coronal orbital floors of sheep
sino-nasal areas, the results will show some deviations and
the maximum force significantly varies among different trials
of same velocities. The results show that tissue will show a
nonlinear behavior, especially in low-velocity indentation
experiments. As it is clear from Fig. 3, in a given finite tool
displacement, while one increases the insertion velocity, the
tool-tissue interaction force will increase, significantly. We
also measured the maximum force prior to the tissue rupture,
based on 18 measurements of tool-tissue interaction with the
coronal orbital floor of the sinonasal region. Fig. 4 displays
the mean maximum force prior to rupture over the three trials
at each velocity versus velocity for the surgical tool
indentation into the tissue. Results show that average
maximum force is a decreasing function of velocity, as
predicted by (7). To evaluate the modified Kelvin model in
(5), the nonlinear static and dynamic parts of the exerted force
are estimated, considering the Levenberg-Marquardt
algorithm proposed in (13).
Table 1 Relaxation time parameter acquired from the experiments
Test Depth (mm) Max force (N) Relaxation time (s)
12 86.64 0.0214
24 122.32 0.0212
36 176.76 0.0206
48 221.37 0.0209
Fig. 2
The force-time curves generated from raw stress relaxation data for
the indentation depths
Fig. 3 The force-displacement for surgical tool insertion into the coronal
orbital floor of sheep head prior to rupture with different velocities
Fig. 4 Mean peak force and standard deviation prior to rupture versus
velocity for coronal orbital floor indentation
0 5 10 15 20 25 30
0
50
100
150
200
250
Time (Sec)
Force (N )
2 mm
4 mm
6 mm
8 mm
Di splacemnt ( mm)
For ce ( N)
Velosi ty (mm/ min )
For ce ( N)
Table 2 Estimated modified Kelvin model parameters for coronal orbital
floor region of tool-tissue interaction
63.62 0.021
Fig. 5 The tool-tissue interaction model is compared to the three different
insertions on real coronal orbital floor regions
Taking into account several various experiments with
various velocities, Table 2 shows the identified parameters for
sheep head coronal orbital floor region prior to the rupture
point. The estimated model parameters were validated by
direct comparison of typical simulated and real data in
constant velocities. Based on the parameter values reported in
Table 2, a complete model of the tool-tissue interaction force
profile was created which can be compared in Fig. 5. Although
the model cannot exactly match the data because of the wide
variation in different specimens and collisions with
unmodeled internal structures, there are some deviations in the
total behavior of simulated data in comparison to real data but,
the overall shape of the model is similar to the data collected
through experiment. Despite these modeling deviations, the
proposed model may be still used for event haptic simulations.
B. Haptic simulation results
Simulations are brought to show the performance of the
proposed controller. Results have been revealed in the
presence of new proposed environment model. The haptic
interface robot parameters and also, human properties are
revealed in Table 3. The inputs for simulation study are as
follow:
,
We considered a virtual tool with a simple dynamic as
. To implement this simple dynamic equation, we
chose
. The value of the mass has been
selected to maintain the closed-loop stability.
Fig. 6 Adaptive control for the tool-tissue interaction in an admittance-type
virtual environment
Table 3 parameters of the haptic interface robot and operator model
Operator model parameters Haptic device parameters
1.25 2 30 1.158
115.4
31.46
(a)
(b)
Fig. 7 Contact simulation with adaptive controller of an admittance
environment (a) position tracking and referred error (b) velocity tracking
To show the system stability in regard to environment
dynamics changes, the stiffness of the tissue could be changed
to larger values, (100 times of estimated values) to check the
system stability. Fig. 7 shows the position and velocity
tracking and referred error.
IV. C
ONCLUSION AND
F
UTURE
W
ORKS
In this article, a qualitative dynamic model was used to
relate the tool-tissue interaction force prior to the rupture of
the coronal orbital floor of the sinonasal region in an ESS. We
used a modified Kelvin model to relate the force of interaction
to tissue deformation. Although the NRMSD (normalized root
mean square error deviation) of acquired data in Fig. 5 show a
variation of 4.23% to 10.95%, but the overall model
representation shows a non-positive dependency of maximum
force on tool insertion velocity which means employing a
faster tool insertion velocity will decrease the maximum
acquired force values or final rupture force amount. However,
we concluded that the overall shape of the model is similar to
the data collected through experiment. Despite these modeling
deviations, the proposed model may still be used for event
haptic simulations. To achieve a VR-based haptic system with
Di splacement (mm)
Force ( N)
Time(sec)
Posit i on (m)
Time(sec)
End e
,
ectorpositionerror(m)
Time(sec)
End e
,
ect o r v el osi t y ( m/ s )
a reasonable performance, adaptive nonlinear control schemes
for the haptic interaction of an impedance-type device with
admittance-type virtual environments were proposed.
Implementing an uncertain mass-spring-damper model of the
user, the controllers can accommodate the proposed nonlinear
model of the environment. We used a Lyapunov-based
approach to analyzing the performance and stability of haptic
simulation. Using an admittance-type environments control
method, the simulations were conducted to investigate the
effectiveness of the proposed controllers. The results
demonstrated that the adaptive controller with an admittance-
type virtual environment shows a reasonable performance in
terms of rendering rigid contacts whereas the changes in the
dynamics of the environment guaranteed the effectiveness of
the proposed method. In summary, it can be concluded that
the adaptive controller with the admittance-type virtual
environment can easily render a stable high impedance contact
for high-performance haptics applications. In the future, we
are focusing on characterizing a complete model for
estimating the rupture phenomenon which is necessary for
accurate surgical simulation, preoperative planning, and
intelligent robotic assistance.
A
PPENDIX
I
Proof. Consider Lyapunov condidate function as :
(I-1)
Since and
are positive defenite, V is positive defenite. By
differentiating V with respect to time we get :
(I-2)
Substituting (20) into (I-2) will result in:
(I-3)
Since
, using the adaptation law and considering the
as
a positive constant, will be as:
(I-4)
With respect to , equation (I-4) will result as follow:
(I-5)
Since
, then:
(I-6)
Because is always a positive function inside the circles with radii
, we have . Hence, this value seems to be ultimate bounds
for , which is inaccessible for to reach. But relating to the second
property of the function , it converges to zero as time goes to infinity
(
). Therefore, the radii of these circles tend to be zero and the
inaccessible bound for converge to zero, accordingly and outside of this
circles (origins). Integrating in (I-6), the final result will be as:
(I-7)
Considering and as the functions defined before:
(I-8)
As we know, () it is also reasonable to state that
and is bounded (i.e., ,
). For a robot manipulator, we
can feasibly deduce that is bounded. Therefore, and are also bounded
(i.e., ,
). As a result, by the use of Barbalat’s lemma [12], we can
conclude that:
(I-9)
R
EFERENCES
[1] B. Perez-Gutierrez, D. M. Martinez, and O. E. Rojas, "Endoscopic
endonasal haptic surgery simulator prototype: A rigid endoscope
model," in Virtual Reality Conference (VR), 2010 IEEE, 2010, pp. 297-
298.
[2] P. Piromchai, "Virtual reality surgical training in ear, nose and throat
surgery," International Journal of Clinical Medicine, vol. 2014, 2014.
[3] S. Delorme, D. Laroche, R. DiRaddo, and R. F. Del Maestro,
"NeuroTouch: a physics-based virtual simulator for cranial
microneurosurgery training," Neurosurgery, vol. 71, pp. ons32-ons42,
2012.
[4] M. P. Fried, J. I. Uribe, and B. Sadoughi, "The role of virtual reality in
surgical training in otorhinolaryngology," Current opinion in
otolaryngology & head and neck surgery, vol. 15, pp. 163-169, 2007.
[5] M. Esfandiari, S. Sadeghnejad, F. Farahmand, and G. Vosoughi,
"Adaptive characterisation of a human hand model during intercations
with a telemanipulation system," in Robotics and Mechatronics
(ICROM), 2015 3rd RSI International Conference on, 2015, pp. 688-
693.
[6] A. Abdossalami and S. Sirouspour, "Adaptive control of haptic
interaction with impedance and admittance type virtual environments,"
in Haptic interfaces for virtual environment and teleoperator systems,
2008. haptics 2008. symposium on, 2008, pp. 145-152.
[7] F. H. Khabbaz, A. Goldenberg, and J. Drake, "An adaptive force
reflective teleoperation control method using online environment
impedance estimation," in Robotic Intelligence In Informationally
Structured Space (RiiSS), 2014 IEEE Symposium on, 2014, pp. 1-8.
[8] M. Mahvash and P. E. Dupont, "Mechanics of dynamic needle insertion
into a biological material," Biomedical Engineering, IEEE
Transactions on, vol. 57, pp. 934-943, 2010.
[9] T. Yamamoto, Applying tissue models in teleoperated robot-assisted
surgery: THE JOHNS HOPKINS UNIVERSITY, 2011.
[10] Y. Kobayashi, A. Kato, H. Watanabe, T. Hoshi, K. Kawamura, and M.
G. Fujie, "Modeling of viscoelastic and nonlinear material properties
of liver tissue using fractional calculations," Journal of Biomechanical
Science and Engineering, vol. 7, pp. 177-187, 2012.
[11] E. Samur, M. Sedef, C. Basdogan, L. Avtan, and O. Duzgun, "A robotic
indenter for minimally invasive measurement and characterization of
soft tissue response," Medical Image Analysis, vol. 11, pp. 361-373,
2007.
[12] J.-J. E. Slotine and W. Li, Applied nonlinear control vol. 199: Prentice-
Hall Englewood Cliffs, NJ, 1991.
