Content uploaded by Soroush Sadeghnejad

Author content

All content in this area was uploaded by Soroush Sadeghnejad on Jul 22, 2018

Content may be subject to copyright.

Phenomenological Contact Model Characterization

and Haptic Simulation of an Endoscopic Sinus and

Skull Base Surgery Virtual System

Soroush Sadeghnejad

1

, Mojtaba Esfandiari

1

, Farzam Farahmand

2

and Gholamreza Vossoughi

2

School of Mechanical Engineering

Sharif University of Technology

Tehran, Iran

1

{s_sadeghnejad, esfandiari_m}@mech.sharif.edu,

2

{farahmand, vossough}@sharif.edu

Abstract— During the endoscopic sinus and skull base

training surgeries, the haptic perception of tool-tissue

interaction and even transitions and ruptures in the tissues are

fundamental which should be taken into account in a robotic

control scheme. However, this problem is extremely complex

given the nature and the variety of tissues involved in an ESS

procedures. In this article, ex-vivo indentation and relaxation

experiments associated with an offline model estimation of the

interaction between tissues and a surgical tool are presented.

The estimated parameters of the modified Kelvin-Voigt model

are then used to provide a realistic tool-tissue interaction

dynamic model. Finally, the principle of a virtual reality

simulation scheme that would allow better haptic discrimination

of tool-tissue interaction is proposed and illustrated.

Keywords— virtual reality; haptic; endoscopic sinus surgery;

modified Kelvin-Voigt model.

I.

I

NTRODUCTION

Use of virtual reality and integrating the haptic systems in

the surgical simulators, which contains two main human

sensorial modalities as visual and touch, allow surgeons to

practice various types of procedures to minimize any risk of

patient’s life during a real surgery [1]. The ideal simulator for

Endoscopic Sinus Surgery (ESS) must be supported by a

physical model and should have the ability to provide

repetitive practice under a controlled environment, which

includes the rendering of temporary and permanent changes

[2]. While the interaction of tools with the variety of tissues

that are soft, semi soft, and rigid, they should look and feel

like real [3]. The most significant challenge in the VR-based

surgical simulation are two folds, first, the development of

realistic tissue models and second, dynamic simulation,

containing appropriate control strategy [4, 5]. Acquiring data

from biological tissues and developing models appropriate

for application in ESS simulation or robot-assisted surgery,

are difficult due to different reasons. First, the different

layers, deformed by a surgical tool, are such that both soft and

hard tissues exist. Second, these parameters values may be

patient dependent. Finally, since the tool makes a large

deformation while interaction with the tissues, the interaction

is clearly nonlinear. For all these reasons, haptic modeling,

which states the relations between motions and interaction

forces, is a difficult task in the case of ESS operation. The

essence of haptics, in virtual reality-based training systems,

is to propose a condition in which the boundaries between

reality and virtuality will be removed and it will make a

system transparency achievable. Due to the ever-increasing

demand for medical simulators within the field of rhinology,

the role of control methods is to provide an interconnection

between the dynamic simulation and the haptic interface [6].

Indeed, tissue dynamic properties are important to design the

system’s control strategy, particularly in the case of

impedance or admittance types virtual environments. The

admittance-type environments compute a displacement in

response to the measurement of the haptic interface force

while the impedance-type environments calculate a reaction

force in response to the users’ position. In many VR system

applications, especially those with high force output

requirements, admittance-type controllers are widely

employed in the control of haptic devices [7].

To address the haptic or force feedback as a very

supplementary information for the surgeons in most surgical

robotic training systems, in this research, a considerable

efforts dedicated to study the interactions between surgical

tools and simultaneous soft and hard tissue deformation in the

sinonasal region for proposing a phenomenological contact

model for further use in an Endoscopic Sinus and Skull Base

Surgery Haptic Simulation System. A number of

experimental surveys have been performed to study the

mechanical behavior mechanisms of sino-nasal region

surface under indentation and relaxation loading. We used the

classical standard nonlinear viscoelastic model (modified

Kelvin-Voigt model), in order to estimate the deformation of

the tissues prior to the rupture point. To improve the stability

and performance of haptic interaction with impedance haptic

devices, we use adaptive nonlinear controllers, proposed in

[6], and we simulated a simple haptic system without

considering any delay, in order to be used in an endoscopic

sinus and skull base surgery virtual-based simulation system.

The rest of this paper is as follows. The methodology for

characterization of tool-tissue interaction forces and design

and implementation of an adaptive controller for a haptic

interface device, interacting with admittance-type virtual

environments are presented in Section 2. Section 3 presents

the tissue modeling results and also simulation studies which

conducted to illustrate the performance of the proposed

robust adaptive controller and the concluding remarks are

presented in Section 4.

II.

M

ETHODOLOGY

A. Tool-tissue interaction experiments and preparation

Characterization of the tool-tissue interaction for accurate

modeling of the tissue deformation phenomena is necessary

to use the force data associated with the tool interaction. To

measure and analyze the tissue behaviors of the coronal

orbital floor, there are several types of experiments that can

be performed.

Amongst usual experiments, indentation and

relaxation loadings can be used to study the mechanical

behavior mechanisms of sinonasal regions. Indentation and

relaxation experiments were performed ex-vivo and the tool

insertions, in the sheep sino-nasal region, were used as a

benchmark for the reason that the sinonasal tissues of a sheep

head are rather similar to human tissues. In order to have deep

insight in the sinonasal anatomy of sheep head, we asked an

ENT surgeon, from Tehran University of Medical Science, to

conduct animal experiments. Performing the surgery

procedures, the surgeon can identify the relevant areas, where

are particularly important and the measurement tasks can be

performed and the data can be acquired. Prior to starting the

surgery procedures, a high-resolution CT scan had been

performed on the sheep head for further navigation purpose

and data processing. Force-displacement data were acquired

using a 1-degree-of-freedom translation INSTRON 5560

Series Table Model Testing Systems. The sheep heads were

all tested at room temperature. The indentation and relaxation

experiments were conducted with 17 adult sheep head to

characterize the material properties of the sino-nasal region.

We developed an indenter based on the surgery tool shape

with the rounded tip of 4mm diameter. The specimens were

intended three times in six different rates of 10, 70, 100, 150,

250 and 500mm/min.

Although there are other experiments to measure the tissue

behavior [9, 10], in this research, the relaxation tests

preformed to evaluate the mechanical properties of the

tissues, as well. A position step input preformed on the

specimens and the exerted forces have been measured. In

these series tests, the coronal orbital floor of the sheep heads

was first indented to pre-defined depths of 2, 4, 6 and 8mms

with the indenter speed of 250mm/min and held there for the

30s to record the force relaxation response of the tool with

respect to time [11]. By acquiring the tissue viscoelastic

parameters from the indentation experiments, one can easily

model the phenomena in order to study the tool-tissue

interaction in the endoscopic endonasal sinus surgery.

B. Tissue mechanical model

To come to a phenomenological model, the schematic

representation of this standard nonlinear solid model is

represented in Fig. 1 [8]. In many tool-tissue interaction

applications, Kelvin-Voigt contact models are suitable for

modeling the nonlinear viscoelastic effects. As presented in

[8], the components of the model is dependent on the tool

deformation, in order to contain nonlinear behavior of the

material during any large deformation. This issue will

become very important in the case of rupture modeling since

definitely large deformation will occur prior to any rupture.

In the proposed model, there are two definite functions. The

static component of the exerted tool force, which composed

of nonlinear force-deformation function and the series

connection of nonlinear spring and damper, calculate the

dynamic component of the acted forces. Assuming the

standard nonlinear viscoelastic model, as what is revealed in

[8], the contact force can be given by:

(1)

Where

is the total force exerted by the tool during any

interaction with any tissue,

is the nonlinear part of the

static force which is dependent to just deformation and

is

the dynamic part of the exerted force which is both dependent

on tool deformation and time. The dynamic part of the

exerted force is calculated, using:

(2)

and

are the stiffness and damping functions, which

are defining the tissue dynamic behaviors.

The relation

between the position of spring and damper can be concluded

in:

(3)

Combining

equations 1 with 2, we can have:

(4)

Taking a time derivative of equation (4) and using again

the aforementioned equation and equation (2), the general

force-displacement relationship is given by the following

equation:

(5)

Supposing the relaxation time

(independent of due to

the dependency of both stiffness and damping parameters to

a same function of deformation with the different ratio), we

can define different relations, based on the proposed model

and basic concept of the modified Kelvin model. Using

equation (2) and defining

, the spring

deformation will yield as:

(6)

Taking the time derivative of (3) and combining the result

with (6) yields the linear differential equation of spring

displacement. The general solution for an aforementioned

equation with respect to a constant tool velocity can be

obtained by a convolution integral. However, the total force–

deformation response of the model can be defined as:

(7)

Fig. 1 Modified standard nonlinear viscoelastic model

For a given deformation, increasing the indentation

velocity always increases the tool-tissue force interaction.

Also, supposing the

and as

increasing functions,

equation (7) reveals that the maximum force of the interaction

is a decreasing function of insertion velocity [8].

C. Tissue mechanical model identification

The work described here is the estimation of the model

parameters in the structure shown in Fig. 1, from input–

output data, which can properly estimate a general behavior

of the tissue prior to a rupture event. However, the focus will

be on estimation algorithms for the proposed model structures

proposed in (5). To evaluate the modified contact model for

this phase, we should identify the parameters of modified

Kelvin model. The two parameters of the model, which are

necessary to be identified, are the static and dynamic stiffness

parameters. In the very low-velocity indentation experiment,

the acquired data is the best fit by a third-order polynomial

equation form. So, for the static part, we will assume a third

order polynomial function of the following form:

(8)

Because all of the experimental tests have been performed

at constant speeds, therefore the velocity can be treated as

a constant coefficient. We can write the general form of the

total interaction force equation, ((5)), in the new form as:

(9)

In which,

and

. To identify the tissue

viscoelastic parameters in equation (5), the Sum Square Error

(SSE) criteria is defined to evaluate the training process. For

all training patterns and model outputs, the SSE is calculated

by:

(10)

In which, P is the index of patterns, from 1 to , where is

the number of patterns, is the index of outputs, from 1 to

, where

is the number of outputs in the pattern. is

the input vector, is the unknown parameter vector and

is the training error at output , when applying pattern p and

it is defined as:

(11)

Where,

is experimental force related to the output

at pattern .

is the calculated force from the proposed

model related to the output at pattern . The Jacobian

matrix is introduced as:

(12)

To optimize the equation (10), we used the iterative

method update rule based on the Levenberg-Marquardt

algorithm, as follow:

(13)

For sufficiently large values of , the matrix

is positive definite, and a descent direction is guaranteed.

D. Adaptive control Design of a haptic interface device

interacting with an admittance-type virtual Environments

As underlined in the introduction, the haptic rendering and

having a reasonable insight of the tool-tissue interaction that

the practitioner feels during any surgery operation are very

important. However, it can sometimes be difficult to detect a

real haptic interaction, in particular for surgery simulation

systems with the lack of an appropriate physical model.

Using the physical interaction model presented in section

2, the haptic perception of force interaction can be enhanced.

To achieve this purpose, the simplest solution would be to

provide an admittance-type virtual environment controller,

which has a good performance in rendering the rigid contacts.

Fig. 6 reveals the structure of the haptic control system in

interaction with admittance-type virtual environments. In this

case, the environment force

is the input to the virtual

dynamic environment and the environment acceleration

,

velocity

, and position

are calculated based on this input

force. The dynamics of the haptic device and the user in the

workspace coordinates can be written as:

(14)

In which,

is the human exogenous force input which

passes through the hand dynamics,

denotes the control

input force, is the un-modeled dynamics and

uncertainties:

(15)

Considering a linear MSD model for the dynamic of the

operator’s hand, where is a symmetric and uniformly

positive-definite function and is upper bounded and

satisfies skew-symmetry property. Considering an adaptive

controller, designed in [6], which is based on Slotine’s

passivity-based position controller and the adaptive

motion/force controller, if we define the velocity

errorbetween the operator’s hand and the virtual tool as:

(16)

and if the reference signal

in (16) is defined as:

(17)

We can combine the system dynamic equation in (14) with

the equations (16) and (17) and simplify the system dynamic

equation to have:

(18)

In which, is a dynamic regression matrix and are the

unknown parameters which should be estimated, properly.

(19)

Choosing the

control

signal as:

(20)

Where

is the control law,

is the estimate of unknown

parameters,

is a positive constant and

is the robust

controller term.

(21)

are positive coefficients and also

,

is a continuous

differentiable function which satisfies the following

condition:

(22)

Defining the parameters estimation errors as

, and

using

as the parameters estimation law, one can

easily design a stable adaptive controller for the haptic system

with an admittance virtual environment (Appendix I).

III.R

ESULTS

A. Tissue mechanical modelling results

As previously underlined, the biological tissue linearly

behaves for small deformation, but from our acquired data, it

nonlinearly behaves due to the existence of large

deformation, so a nonlinear viscoelastic model should be

used in order to model the force response of the tissue for this

step. The relaxation time

can be determined using the

acquired data from the relaxation experiments. The total of

four probing measurements of the coronal orbital floor of

sheep sinonasal regions was taken and the raw data, which

have been collected from different indentation depths are

shown in Fig. 2. We implemented an exponential force-time

response to the part of the relaxation data where the

acceleration and inertia force have less effect on the recorded

data. Table 1 gives the relaxation time, resulted from four

different relaxation experiments. From the presented result,

we can conclude that the average relaxation time for the

coronal orbital floors can result in a time constant of

. Fig. 3 displays the typical force-displacement

response of an insertion for a surgical tool into the coronal

orbital floors for various velocities. Due to the fact of

differences in race, gender and age of each specimen and

place of indentation on the coronal orbital floors of sheep

sino-nasal areas, the results will show some deviations and

the maximum force significantly varies among different trials

of same velocities. The results show that tissue will show a

nonlinear behavior, especially in low-velocity indentation

experiments. As it is clear from Fig. 3, in a given finite tool

displacement, while one increases the insertion velocity, the

tool-tissue interaction force will increase, significantly. We

also measured the maximum force prior to the tissue rupture,

based on 18 measurements of tool-tissue interaction with the

coronal orbital floor of the sinonasal region. Fig. 4 displays

the mean maximum force prior to rupture over the three trials

at each velocity versus velocity for the surgical tool

indentation into the tissue. Results show that average

maximum force is a decreasing function of velocity, as

predicted by (7). To evaluate the modified Kelvin model in

(5), the nonlinear static and dynamic parts of the exerted force

are estimated, considering the Levenberg-Marquardt

algorithm proposed in (13).

Table 1 Relaxation time parameter acquired from the experiments

Test Depth (mm) Max force (N) Relaxation time (s)

12 86.64 0.0214

24 122.32 0.0212

36 176.76 0.0206

48 221.37 0.0209

Fig. 2

The force-time curves generated from raw stress relaxation data for

the indentation depths

Fig. 3 The force-displacement for surgical tool insertion into the coronal

orbital floor of sheep head prior to rupture with different velocities

Fig. 4 Mean peak force and standard deviation prior to rupture versus

velocity for coronal orbital floor indentation

0 5 10 15 20 25 30

0

50

100

150

200

250

Time (Sec)

Force (N )

2 mm

4 mm

6 mm

8 mm

Di splacemnt ( mm)

For ce ( N)

Velosi ty (mm/ min )

For ce ( N)

Table 2 Estimated modified Kelvin model parameters for coronal orbital

floor region of tool-tissue interaction

63.62 0.021

Fig. 5 The tool-tissue interaction model is compared to the three different

insertions on real coronal orbital floor regions

Taking into account several various experiments with

various velocities, Table 2 shows the identified parameters for

sheep head coronal orbital floor region prior to the rupture

point. The estimated model parameters were validated by

direct comparison of typical simulated and real data in

constant velocities. Based on the parameter values reported in

Table 2, a complete model of the tool-tissue interaction force

profile was created which can be compared in Fig. 5. Although

the model cannot exactly match the data because of the wide

variation in different specimens and collisions with

unmodeled internal structures, there are some deviations in the

total behavior of simulated data in comparison to real data but,

the overall shape of the model is similar to the data collected

through experiment. Despite these modeling deviations, the

proposed model may be still used for event haptic simulations.

B. Haptic simulation results

Simulations are brought to show the performance of the

proposed controller. Results have been revealed in the

presence of new proposed environment model. The haptic

interface robot parameters and also, human properties are

revealed in Table 3. The inputs for simulation study are as

follow:

,

We considered a virtual tool with a simple dynamic as

. To implement this simple dynamic equation, we

chose

. The value of the mass has been

selected to maintain the closed-loop stability.

Fig. 6 Adaptive control for the tool-tissue interaction in an admittance-type

virtual environment

Table 3 parameters of the haptic interface robot and operator model

Operator model parameters Haptic device parameters

1.25 2 30 1.158

115.4

31.46

(a)

(b)

Fig. 7 Contact simulation with adaptive controller of an admittance

environment (a) position tracking and referred error (b) velocity tracking

To show the system stability in regard to environment

dynamics changes, the stiffness of the tissue could be changed

to larger values, (100 times of estimated values) to check the

system stability. Fig. 7 shows the position and velocity

tracking and referred error.

IV. C

ONCLUSION AND

F

UTURE

W

ORKS

In this article, a qualitative dynamic model was used to

relate the tool-tissue interaction force prior to the rupture of

the coronal orbital floor of the sinonasal region in an ESS. We

used a modified Kelvin model to relate the force of interaction

to tissue deformation. Although the NRMSD (normalized root

mean square error deviation) of acquired data in Fig. 5 show a

variation of 4.23% to 10.95%, but the overall model

representation shows a non-positive dependency of maximum

force on tool insertion velocity which means employing a

faster tool insertion velocity will decrease the maximum

acquired force values or final rupture force amount. However,

we concluded that the overall shape of the model is similar to

the data collected through experiment. Despite these modeling

deviations, the proposed model may still be used for event

haptic simulations. To achieve a VR-based haptic system with

Di splacement (mm)

Force ( N)

Time(sec)

Posit i on (m)

Time(sec)

End e

,

ectorpositionerror(m)

Time(sec)

End e

,

ect o r v el osi t y ( m/ s )

a reasonable performance, adaptive nonlinear control schemes

for the haptic interaction of an impedance-type device with

admittance-type virtual environments were proposed.

Implementing an uncertain mass-spring-damper model of the

user, the controllers can accommodate the proposed nonlinear

model of the environment. We used a Lyapunov-based

approach to analyzing the performance and stability of haptic

simulation. Using an admittance-type environments control

method, the simulations were conducted to investigate the

effectiveness of the proposed controllers. The results

demonstrated that the adaptive controller with an admittance-

type virtual environment shows a reasonable performance in

terms of rendering rigid contacts whereas the changes in the

dynamics of the environment guaranteed the effectiveness of

the proposed method. In summary, it can be concluded that

the adaptive controller with the admittance-type virtual

environment can easily render a stable high impedance contact

for high-performance haptics applications. In the future, we

are focusing on characterizing a complete model for

estimating the rupture phenomenon which is necessary for

accurate surgical simulation, preoperative planning, and

intelligent robotic assistance.

A

PPENDIX

I

Proof. Consider Lyapunov condidate function as :

(I-1)

Since and

are positive defenite, V is positive defenite. By

differentiating V with respect to time we get :

(I-2)

Substituting (20) into (I-2) will result in:

(I-3)

Since

, using the adaptation law and considering the

as

a positive constant, will be as:

(I-4)

With respect to , equation (I-4) will result as follow:

(I-5)

Since

, then:

(I-6)

Because is always a positive function inside the circles with radii

, we have . Hence, this value seems to be ultimate bounds

for , which is inaccessible for to reach. But relating to the second

property of the function , it converges to zero as time goes to infinity

(

). Therefore, the radii of these circles tend to be zero and the

inaccessible bound for converge to zero, accordingly and outside of this

circles (origins). Integrating in (I-6), the final result will be as:

(I-7)

Considering and as the functions defined before:

(I-8)

As we know, () it is also reasonable to state that

and is bounded (i.e., ,

). For a robot manipulator, we

can feasibly deduce that is bounded. Therefore, and are also bounded

(i.e., ,

). As a result, by the use of Barbalat’s lemma [12], we can

conclude that:

(I-9)

R

EFERENCES

[1] B. Perez-Gutierrez, D. M. Martinez, and O. E. Rojas, "Endoscopic

endonasal haptic surgery simulator prototype: A rigid endoscope

model," in Virtual Reality Conference (VR), 2010 IEEE, 2010, pp. 297-

298.

[2] P. Piromchai, "Virtual reality surgical training in ear, nose and throat

surgery," International Journal of Clinical Medicine, vol. 2014, 2014.

[3] S. Delorme, D. Laroche, R. DiRaddo, and R. F. Del Maestro,

"NeuroTouch: a physics-based virtual simulator for cranial

microneurosurgery training," Neurosurgery, vol. 71, pp. ons32-ons42,

2012.

[4] M. P. Fried, J. I. Uribe, and B. Sadoughi, "The role of virtual reality in

surgical training in otorhinolaryngology," Current opinion in

otolaryngology & head and neck surgery, vol. 15, pp. 163-169, 2007.

[5] M. Esfandiari, S. Sadeghnejad, F. Farahmand, and G. Vosoughi,

"Adaptive characterisation of a human hand model during intercations

with a telemanipulation system," in Robotics and Mechatronics

(ICROM), 2015 3rd RSI International Conference on, 2015, pp. 688-

693.

[6] A. Abdossalami and S. Sirouspour, "Adaptive control of haptic

interaction with impedance and admittance type virtual environments,"

in Haptic interfaces for virtual environment and teleoperator systems,

2008. haptics 2008. symposium on, 2008, pp. 145-152.

[7] F. H. Khabbaz, A. Goldenberg, and J. Drake, "An adaptive force

reflective teleoperation control method using online environment

impedance estimation," in Robotic Intelligence In Informationally

Structured Space (RiiSS), 2014 IEEE Symposium on, 2014, pp. 1-8.

[8] M. Mahvash and P. E. Dupont, "Mechanics of dynamic needle insertion

into a biological material," Biomedical Engineering, IEEE

Transactions on, vol. 57, pp. 934-943, 2010.

[9] T. Yamamoto, Applying tissue models in teleoperated robot-assisted

surgery: THE JOHNS HOPKINS UNIVERSITY, 2011.

[10] Y. Kobayashi, A. Kato, H. Watanabe, T. Hoshi, K. Kawamura, and M.

G. Fujie, "Modeling of viscoelastic and nonlinear material properties

of liver tissue using fractional calculations," Journal of Biomechanical

Science and Engineering, vol. 7, pp. 177-187, 2012.

[11] E. Samur, M. Sedef, C. Basdogan, L. Avtan, and O. Duzgun, "A robotic

indenter for minimally invasive measurement and characterization of

soft tissue response," Medical Image Analysis, vol. 11, pp. 361-373,

2007.

[12] J.-J. E. Slotine and W. Li, Applied nonlinear control vol. 199: Prentice-

Hall Englewood Cliffs, NJ, 1991.