Conference PaperPDF Available

Phenomenological Contact Model Characterization and Haptic Simulation of an Endoscopic Sinus and Skull Base Surgery Virtual System

Authors:

Abstract and Figures

During the endoscopic sinus and skull base training surgeries, the haptic perception of tool-tissue interaction and even transitions and ruptures in the tissues are fundamental which should be taken into account in a robotic control scheme. However, this problem is extremely complex given the nature and the variety of tissues involved in an ESS procedures. In this article, ex-vivo indentation and relaxation experiments associated with an offline model estimation of the interaction between tissues and a surgical tool are presented. The estimated parameters of the modified Kelvin-Voigt model are then used to provide a realistic tool-tissue interaction dynamic model. Finally, the principle of a virtual reality simulation scheme that would allow better haptic discrimination of tool-tissue interaction is proposed and illustrated.
Content may be subject to copyright.
Phenomenological Contact Model Characterization
and Haptic Simulation of an Endoscopic Sinus and
Skull Base Surgery Virtual System
Soroush Sadeghnejad
1
, Mojtaba Esfandiari
1
, Farzam Farahmand
2
and Gholamreza Vossoughi
2
School of Mechanical Engineering
Sharif University of Technology
Tehran, Iran
1
{s_sadeghnejad, esfandiari_m}@mech.sharif.edu,
2
{farahmand, vossough}@sharif.edu
Abstract— During the endoscopic sinus and skull base
training surgeries, the haptic perception of tool-tissue
interaction and even transitions and ruptures in the tissues are
fundamental which should be taken into account in a robotic
control scheme. However, this problem is extremely complex
given the nature and the variety of tissues involved in an ESS
procedures. In this article, ex-vivo indentation and relaxation
experiments associated with an offline model estimation of the
interaction between tissues and a surgical tool are presented.
The estimated parameters of the modified Kelvin-Voigt model
are then used to provide a realistic tool-tissue interaction
dynamic model. Finally, the principle of a virtual reality
simulation scheme that would allow better haptic discrimination
of tool-tissue interaction is proposed and illustrated.
Keywords— virtual reality; haptic; endoscopic sinus surgery;
modified Kelvin-Voigt model.
I.
I
NTRODUCTION
Use of virtual reality and integrating the haptic systems in
the surgical simulators, which contains two main human
sensorial modalities as visual and touch, allow surgeons to
practice various types of procedures to minimize any risk of
patient’s life during a real surgery [1]. The ideal simulator for
Endoscopic Sinus Surgery (ESS) must be supported by a
physical model and should have the ability to provide
repetitive practice under a controlled environment, which
includes the rendering of temporary and permanent changes
[2]. While the interaction of tools with the variety of tissues
that are soft, semi soft, and rigid, they should look and feel
like real [3]. The most significant challenge in the VR-based
surgical simulation are two folds, first, the development of
realistic tissue models and second, dynamic simulation,
containing appropriate control strategy [4, 5]. Acquiring data
from biological tissues and developing models appropriate
for application in ESS simulation or robot-assisted surgery,
are difficult due to different reasons. First, the different
layers, deformed by a surgical tool, are such that both soft and
hard tissues exist. Second, these parameters values may be
patient dependent. Finally, since the tool makes a large
deformation while interaction with the tissues, the interaction
is clearly nonlinear. For all these reasons, haptic modeling,
which states the relations between motions and interaction
forces, is a difficult task in the case of ESS operation. The
essence of haptics, in virtual reality-based training systems,
is to propose a condition in which the boundaries between
reality and virtuality will be removed and it will make a
system transparency achievable. Due to the ever-increasing
demand for medical simulators within the field of rhinology,
the role of control methods is to provide an interconnection
between the dynamic simulation and the haptic interface [6].
Indeed, tissue dynamic properties are important to design the
system’s control strategy, particularly in the case of
impedance or admittance types virtual environments. The
admittance-type environments compute a displacement in
response to the measurement of the haptic interface force
while the impedance-type environments calculate a reaction
force in response to the users’ position. In many VR system
applications, especially those with high force output
requirements, admittance-type controllers are widely
employed in the control of haptic devices [7].
To address the haptic or force feedback as a very
supplementary information for the surgeons in most surgical
robotic training systems, in this research, a considerable
efforts dedicated to study the interactions between surgical
tools and simultaneous soft and hard tissue deformation in the
sinonasal region for proposing a phenomenological contact
model for further use in an Endoscopic Sinus and Skull Base
Surgery Haptic Simulation System. A number of
experimental surveys have been performed to study the
mechanical behavior mechanisms of sino-nasal region
surface under indentation and relaxation loading. We used the
classical standard nonlinear viscoelastic model (modified
Kelvin-Voigt model), in order to estimate the deformation of
the tissues prior to the rupture point. To improve the stability
and performance of haptic interaction with impedance haptic
devices, we use adaptive nonlinear controllers, proposed in
[6], and we simulated a simple haptic system without
considering any delay, in order to be used in an endoscopic
sinus and skull base surgery virtual-based simulation system.
The rest of this paper is as follows. The methodology for
characterization of tool-tissue interaction forces and design
and implementation of an adaptive controller for a haptic
interface device, interacting with admittance-type virtual
environments are presented in Section 2. Section 3 presents
the tissue modeling results and also simulation studies which
conducted to illustrate the performance of the proposed
robust adaptive controller and the concluding remarks are
presented in Section 4.
II.
M
ETHODOLOGY
A. Tool-tissue interaction experiments and preparation
Characterization of the tool-tissue interaction for accurate
modeling of the tissue deformation phenomena is necessary
to use the force data associated with the tool interaction. To
measure and analyze the tissue behaviors of the coronal
orbital floor, there are several types of experiments that can
be performed.
Amongst usual experiments, indentation and
relaxation loadings can be used to study the mechanical
behavior mechanisms of sinonasal regions. Indentation and
relaxation experiments were performed ex-vivo and the tool
insertions, in the sheep sino-nasal region, were used as a
benchmark for the reason that the sinonasal tissues of a sheep
head are rather similar to human tissues. In order to have deep
insight in the sinonasal anatomy of sheep head, we asked an
ENT surgeon, from Tehran University of Medical Science, to
conduct animal experiments. Performing the surgery
procedures, the surgeon can identify the relevant areas, where
are particularly important and the measurement tasks can be
performed and the data can be acquired. Prior to starting the
surgery procedures, a high-resolution CT scan had been
performed on the sheep head for further navigation purpose
and data processing. Force-displacement data were acquired
using a 1-degree-of-freedom translation INSTRON 5560
Series Table Model Testing Systems. The sheep heads were
all tested at room temperature. The indentation and relaxation
experiments were conducted with 17 adult sheep head to
characterize the material properties of the sino-nasal region.
We developed an indenter based on the surgery tool shape
with the rounded tip of 4mm diameter. The specimens were
intended three times in six different rates of 10, 70, 100, 150,
250 and 500mm/min.
Although there are other experiments to measure the tissue
behavior [9, 10], in this research, the relaxation tests
preformed to evaluate the mechanical properties of the
tissues, as well. A position step input preformed on the
specimens and the exerted forces have been measured. In
these series tests, the coronal orbital floor of the sheep heads
was first indented to pre-defined depths of 2, 4, 6 and 8mms
with the indenter speed of 250mm/min and held there for the
30s to record the force relaxation response of the tool with
respect to time [11]. By acquiring the tissue viscoelastic
parameters from the indentation experiments, one can easily
model the phenomena in order to study the tool-tissue
interaction in the endoscopic endonasal sinus surgery.
B. Tissue mechanical model
To come to a phenomenological model, the schematic
representation of this standard nonlinear solid model is
represented in Fig. 1 [8]. In many tool-tissue interaction
applications, Kelvin-Voigt contact models are suitable for
modeling the nonlinear viscoelastic effects. As presented in
[8], the components of the model is dependent on the tool
deformation, in order to contain nonlinear behavior of the
material during any large deformation. This issue will
become very important in the case of rupture modeling since
definitely large deformation will occur prior to any rupture.
In the proposed model, there are two definite functions. The
static component of the exerted tool force, which composed
of nonlinear force-deformation function and the series
connection of nonlinear spring and damper, calculate the
dynamic component of the acted forces. Assuming the
standard nonlinear viscoelastic model, as what is revealed in
[8], the contact force can be given by:


 (1)
Where
is the total force exerted by the tool during any
interaction with any tissue,
is the nonlinear part of the
static force which is dependent to just deformation and
is
the dynamic part of the exerted force which is both dependent
on tool deformation and time. The dynamic part of the
exerted force is calculated, using:



(2)

and
are the stiffness and damping functions, which
are defining the tissue dynamic behaviors.
The relation
between the position of spring and damper can be concluded
in:

 (3)
Combining
equations 1 with 2, we can have:



(4)
Taking a time derivative of equation (4) and using again
the aforementioned equation and equation (2), the general
force-displacement relationship is given by the following
equation:





(5)
Supposing the relaxation time
(independent of due to
the dependency of both stiffness and damping parameters to
a same function of deformation with the different ratio), we
can define different relations, based on the proposed model
and basic concept of the modified Kelvin model. Using
equation (2) and defining
, the spring
deformation will yield as:

(6)
Taking the time derivative of (3) and combining the result
with (6) yields the linear differential equation of spring
displacement. The general solution for an aforementioned
equation with respect to a constant tool velocity can be
obtained by a convolution integral. However, the total force–
deformation response of the model can be defined as:




 (7)
Fig. 1 Modified standard nonlinear viscoelastic model
For a given deformation, increasing the indentation
velocity always increases the tool-tissue force interaction.
Also, supposing the
and as
increasing functions,
equation (7) reveals that the maximum force of the interaction
is a decreasing function of insertion velocity [8].
C. Tissue mechanical model identification
The work described here is the estimation of the model
parameters in the structure shown in Fig. 1, from input–
output data, which can properly estimate a general behavior
of the tissue prior to a rupture event. However, the focus will
be on estimation algorithms for the proposed model structures
proposed in (5). To evaluate the modified contact model for
this phase, we should identify the parameters of modified
Kelvin model. The two parameters of the model, which are
necessary to be identified, are the static and dynamic stiffness
parameters. In the very low-velocity indentation experiment,
the acquired data is the best fit by a third-order polynomial
equation form. So, for the static part, we will assume a third
order polynomial function of the following form:



(8)
Because all of the experimental tests have been performed
at constant speeds, therefore the velocity can be treated as
a constant coefficient. We can write the general form of the
total interaction force equation, ((5)), in the new form as:

(9)
In which,



and

. To identify the tissue
viscoelastic parameters in equation (5), the Sum Square Error
(SSE) criteria is defined to evaluate the training process. For
all training patterns and model outputs, the SSE is calculated
by:





(10)
In which, P is the index of patterns, from 1 to , where is
the number of patterns, is the index of outputs, from 1 to
, where
is the number of outputs in the pattern. is
the input vector, is the unknown parameter vector and

is the training error at output , when applying pattern p and
it is defined as:






(11)
Where,


is experimental force related to the output
at pattern .

is the calculated force from the proposed
model related to the output at pattern . The Jacobian
matrix is introduced as:






(12)
To optimize the equation (10), we used the iterative
method update rule based on the Levenberg-Marquardt
algorithm, as follow:





(13)
For sufficiently large values of , the matrix 

is positive definite, and a descent direction is guaranteed.
D. Adaptive control Design of a haptic interface device
interacting with an admittance-type virtual Environments
As underlined in the introduction, the haptic rendering and
having a reasonable insight of the tool-tissue interaction that
the practitioner feels during any surgery operation are very
important. However, it can sometimes be difficult to detect a
real haptic interaction, in particular for surgery simulation
systems with the lack of an appropriate physical model.
Using the physical interaction model presented in section
2, the haptic perception of force interaction can be enhanced.
To achieve this purpose, the simplest solution would be to
provide an admittance-type virtual environment controller,
which has a good performance in rendering the rigid contacts.
Fig. 6 reveals the structure of the haptic control system in
interaction with admittance-type virtual environments. In this
case, the environment force
is the input to the virtual
dynamic environment and the environment acceleration
,
velocity
, and position
are calculated based on this input
force. The dynamics of the haptic device and the user in the
workspace coordinates can be written as:



(14)
In which,
is the human exogenous force input which
passes through the hand dynamics,
denotes the control
input force,  is the un-modeled dynamics and
uncertainties:












(15)
Considering a linear MSD model for the dynamic of the
operator’s hand, where is a symmetric and uniformly
positive-definite function and  is upper bounded and
satisfies skew-symmetry property. Considering an adaptive
controller, designed in [6], which is based on Slotine’s
passivity-based position controller and the adaptive
motion/force controller, if we define the velocity
errorbetween the operator’s hand and the virtual tool as:


(16)
and if the reference signal

in (16) is defined as:


(17)
We can combine the system dynamic equation in (14) with
the equations (16) and (17) and simplify the system dynamic
equation to have:




(18)
In which, is a dynamic regression matrix and are the
unknown parameters which should be estimated, properly.






(19)
Choosing the
control
signal as:



(20)
Where
is the control law,
is the estimate of unknown
parameters,
is a positive constant and
is the robust
controller term.


(21)
are positive coefficients and also
,
 is a continuous
differentiable function which satisfies the following
condition:




(22)
Defining the parameters estimation errors as 
, and
using

as the parameters estimation law, one can
easily design a stable adaptive controller for the haptic system
with an admittance virtual environment (Appendix I).
III.R
ESULTS
A. Tissue mechanical modelling results
As previously underlined, the biological tissue linearly
behaves for small deformation, but from our acquired data, it
nonlinearly behaves due to the existence of large
deformation, so a nonlinear viscoelastic model should be
used in order to model the force response of the tissue for this
step. The relaxation time
can be determined using the
acquired data from the relaxation experiments. The total of
four probing measurements of the coronal orbital floor of
sheep sinonasal regions was taken and the raw data, which
have been collected from different indentation depths are
shown in Fig. 2. We implemented an exponential force-time
response to the part of the relaxation data where the
acceleration and inertia force have less effect on the recorded
data. Table 1 gives the relaxation time, resulted from four
different relaxation experiments. From the presented result,
we can conclude that the average relaxation time for the
coronal orbital floors can result in a time constant of

. Fig. 3 displays the typical force-displacement
response of an insertion for a surgical tool into the coronal
orbital floors for various velocities. Due to the fact of
differences in race, gender and age of each specimen and
place of indentation on the coronal orbital floors of sheep
sino-nasal areas, the results will show some deviations and
the maximum force significantly varies among different trials
of same velocities. The results show that tissue will show a
nonlinear behavior, especially in low-velocity indentation
experiments. As it is clear from Fig. 3, in a given finite tool
displacement, while one increases the insertion velocity, the
tool-tissue interaction force will increase, significantly. We
also measured the maximum force prior to the tissue rupture,
based on 18 measurements of tool-tissue interaction with the
coronal orbital floor of the sinonasal region. Fig. 4 displays
the mean maximum force prior to rupture over the three trials
at each velocity versus velocity for the surgical tool
indentation into the tissue. Results show that average
maximum force is a decreasing function of velocity, as
predicted by (7). To evaluate the modified Kelvin model in
(5), the nonlinear static and dynamic parts of the exerted force
are estimated, considering the Levenberg-Marquardt
algorithm proposed in (13).
Table 1 Relaxation time parameter acquired from the experiments
Test Depth (mm) Max force (N) Relaxation time (s)
12 86.64 0.0214
24 122.32 0.0212
36 176.76 0.0206
48 221.37 0.0209
Fig. 2
The force-time curves generated from raw stress relaxation data for
the indentation depths
Fig. 3 The force-displacement for surgical tool insertion into the coronal
orbital floor of sheep head prior to rupture with different velocities
Fig. 4 Mean peak force and standard deviation prior to rupture versus
velocity for coronal orbital floor indentation
0 5 10 15 20 25 30
0
50
100
150
200
250
Time (Sec)
Force (N )
2 mm
4 mm
6 mm
8 mm
  








Di splacemnt ( mm)
For ce ( N)






    










Velosi ty (mm/ min )
For ce ( N)
Table 2 Estimated modified Kelvin model parameters for coronal orbital
floor region of tool-tissue interaction
 


 63.62 0.021
Fig. 5 The tool-tissue interaction model is compared to the three different
insertions on real coronal orbital floor regions
Taking into account several various experiments with
various velocities, Table 2 shows the identified parameters for
sheep head coronal orbital floor region prior to the rupture
point. The estimated model parameters were validated by
direct comparison of typical simulated and real data in
constant velocities. Based on the parameter values reported in
Table 2, a complete model of the tool-tissue interaction force
profile was created which can be compared in Fig. 5. Although
the model cannot exactly match the data because of the wide
variation in different specimens and collisions with
unmodeled internal structures, there are some deviations in the
total behavior of simulated data in comparison to real data but,
the overall shape of the model is similar to the data collected
through experiment. Despite these modeling deviations, the
proposed model may be still used for event haptic simulations.
B. Haptic simulation results
Simulations are brought to show the performance of the
proposed controller. Results have been revealed in the
presence of new proposed environment model. The haptic
interface robot parameters and also, human properties are
revealed in Table 3. The inputs for simulation study are as
follow:


,

We considered a virtual tool with a simple dynamic as



. To implement this simple dynamic equation, we
chose


. The value of the mass has been
selected to maintain the closed-loop stability.
Fig. 6 Adaptive control for the tool-tissue interaction in an admittance-type
virtual environment
Table 3 parameters of the haptic interface robot and operator model
Operator model parameters Haptic device parameters



 

 



 

 
1.25 2 30 1.158
115.4
31.46
(a)
(b)
Fig. 7 Contact simulation with adaptive controller of an admittance
environment (a) position tracking and referred error (b) velocity tracking
To show the system stability in regard to environment
dynamics changes, the stiffness of the tissue could be changed
to larger values, (100 times of estimated values) to check the
system stability. Fig. 7 shows the position and velocity
tracking and referred error.
IV. C
ONCLUSION AND
F
UTURE
W
ORKS
In this article, a qualitative dynamic model was used to
relate the tool-tissue interaction force prior to the rupture of
the coronal orbital floor of the sinonasal region in an ESS. We
used a modified Kelvin model to relate the force of interaction
to tissue deformation. Although the NRMSD (normalized root
mean square error deviation) of acquired data in Fig. 5 show a
variation of 4.23% to 10.95%, but the overall model
representation shows a non-positive dependency of maximum
force on tool insertion velocity which means employing a
faster tool insertion velocity will decrease the maximum
acquired force values or final rupture force amount. However,
we concluded that the overall shape of the model is similar to
the data collected through experiment. Despite these modeling
deviations, the proposed model may still be used for event
haptic simulations. To achieve a VR-based haptic system with
  




Di splacement (mm)
Force ( N)






  





Time(sec)
Posit i on (m)


  



Time(sec)
End e
,
ectorpositionerror(m)
  





Time(sec)
End e
,
ect o r v el osi t y ( m/ s )


a reasonable performance, adaptive nonlinear control schemes
for the haptic interaction of an impedance-type device with
admittance-type virtual environments were proposed.
Implementing an uncertain mass-spring-damper model of the
user, the controllers can accommodate the proposed nonlinear
model of the environment. We used a Lyapunov-based
approach to analyzing the performance and stability of haptic
simulation. Using an admittance-type environments control
method, the simulations were conducted to investigate the
effectiveness of the proposed controllers. The results
demonstrated that the adaptive controller with an admittance-
type virtual environment shows a reasonable performance in
terms of rendering rigid contacts whereas the changes in the
dynamics of the environment guaranteed the effectiveness of
the proposed method. In summary, it can be concluded that
the adaptive controller with the admittance-type virtual
environment can easily render a stable high impedance contact
for high-performance haptics applications. In the future, we
are focusing on characterizing a complete model for
estimating the rupture phenomenon which is necessary for
accurate surgical simulation, preoperative planning, and
intelligent robotic assistance.
A
PPENDIX
I
Proof. Consider Lyapunov condidate function as :



(I-1)
Since and

are positive defenite, V is positive defenite. By
differentiating V with respect to time we get :




(I-2)
Substituting (20) into (I-2) will result in:














(I-3)
Since 
, using the adaptation law and considering the
 as
a positive constant, will be as:







(I-4)
With respect to , equation (I-4) will result as follow:





(I-5)
Since


, then:

 (I-6)
Because is always a positive function inside the circles with radii

, we have . Hence, this value seems to be ultimate bounds
for , which is inaccessible for to reach. But relating to the second
property of the function , it converges to zero as time goes to infinity
(

). Therefore, the radii of these circles tend to be zero and the
inaccessible bound for converge to zero, accordingly and outside of this
circles (origins). Integrating in (I-6), the final result will be as:



 







(I-7)
Considering and as the functions defined before:


 (I-8)
As we know, () it is also reasonable to state that


and is bounded (i.e., ,
). For a robot manipulator, we
can feasibly deduce that is bounded. Therefore,  and  are also bounded
(i.e., ,
). As a result, by the use of Barbalat’s lemma [12], we can
conclude that: 




 (I-9)
R
EFERENCES
[1] B. Perez-Gutierrez, D. M. Martinez, and O. E. Rojas, "Endoscopic
endonasal haptic surgery simulator prototype: A rigid endoscope
model," in Virtual Reality Conference (VR), 2010 IEEE, 2010, pp. 297-
298.
[2] P. Piromchai, "Virtual reality surgical training in ear, nose and throat
surgery," International Journal of Clinical Medicine, vol. 2014, 2014.
[3] S. Delorme, D. Laroche, R. DiRaddo, and R. F. Del Maestro,
"NeuroTouch: a physics-based virtual simulator for cranial
microneurosurgery training," Neurosurgery, vol. 71, pp. ons32-ons42,
2012.
[4] M. P. Fried, J. I. Uribe, and B. Sadoughi, "The role of virtual reality in
surgical training in otorhinolaryngology," Current opinion in
otolaryngology & head and neck surgery, vol. 15, pp. 163-169, 2007.
[5] M. Esfandiari, S. Sadeghnejad, F. Farahmand, and G. Vosoughi,
"Adaptive characterisation of a human hand model during intercations
with a telemanipulation system," in Robotics and Mechatronics
(ICROM), 2015 3rd RSI International Conference on, 2015, pp. 688-
693.
[6] A. Abdossalami and S. Sirouspour, "Adaptive control of haptic
interaction with impedance and admittance type virtual environments,"
in Haptic interfaces for virtual environment and teleoperator systems,
2008. haptics 2008. symposium on, 2008, pp. 145-152.
[7] F. H. Khabbaz, A. Goldenberg, and J. Drake, "An adaptive force
reflective teleoperation control method using online environment
impedance estimation," in Robotic Intelligence In Informationally
Structured Space (RiiSS), 2014 IEEE Symposium on, 2014, pp. 1-8.
[8] M. Mahvash and P. E. Dupont, "Mechanics of dynamic needle insertion
into a biological material," Biomedical Engineering, IEEE
Transactions on, vol. 57, pp. 934-943, 2010.
[9] T. Yamamoto, Applying tissue models in teleoperated robot-assisted
surgery: THE JOHNS HOPKINS UNIVERSITY, 2011.
[10] Y. Kobayashi, A. Kato, H. Watanabe, T. Hoshi, K. Kawamura, and M.
G. Fujie, "Modeling of viscoelastic and nonlinear material properties
of liver tissue using fractional calculations," Journal of Biomechanical
Science and Engineering, vol. 7, pp. 177-187, 2012.
[11] E. Samur, M. Sedef, C. Basdogan, L. Avtan, and O. Duzgun, "A robotic
indenter for minimally invasive measurement and characterization of
soft tissue response," Medical Image Analysis, vol. 11, pp. 361-373,
2007.
[12] J.-J. E. Slotine and W. Li, Applied nonlinear control vol. 199: Prentice-
Hall Englewood Cliffs, NJ, 1991.
... Very recently, El-Monajjed et al. [8] successfully presented an algorithm that operated on a haptic device to be used in minimally invasive spine surgery practice, but they used a second-order polynomial for their force feedback model, and a proper material model was not identified. However, Sadeghnejad et al. [14] developed a nonlinear viscoelastic modified nonlinear Kelvin-Voigt model for soft tissue modeling to obtain the tissue deformation in the sino-nasal regions prior to haptic interaction simulation. They employed an adaptive controller for this simulation with an admittance-type virtual environment. ...
... This paper makes a similar contribution to those of Sadeghnejad et al. [14] and El-Monajjed et al. [8] and serves to expand the database of VR-based simulations on the IVD tissue. The exact research questions would be whether the model characterization and haptic setup proposed by Sadeghnejad et al. [14] are applicable to the IVD tissue. ...
... This paper makes a similar contribution to those of Sadeghnejad et al. [14] and El-Monajjed et al. [8] and serves to expand the database of VR-based simulations on the IVD tissue. The exact research questions would be whether the model characterization and haptic setup proposed by Sadeghnejad et al. [14] are applicable to the IVD tissue. Furthermore, does the suggested adaptive controller with an admittance-type environment yield a stable system for haptic applications regarding the IVD-related surgery practice? ...
Conference Paper
Full-text available
Haptic systems provide a sense of touch by applying forces and motions in order to create a real-world experience using computer simulations for educational and surgical purposes. Identification of realistic physical models and the advancement of simulators are essential for the effectiveness of haptic systems. In this study, a modified nonlinear Kelvin-Voigt model was developed to represent ovine intervertebral disc tissue as an approach to make the cervical spine surgical simulations and training practices more realistic. The proposed mechanical model was identified by three relaxation tests and three compressive tests with different constant velocities. A third-order polynomial function was employed to provide a tool-tissue interaction dynamic model. Finally, the closed-loop stability of the proposed model in an admittance-type virtual reality system suggested by previous literature was shown by simulation. Results approved that the adaptive controller using the admittance-type virtual environment could offer a high impedance contact for high-performance haptic applications.
... In previous researches, our group proposed a phenomenological model for the mechanical characterizations of the coronal orbital floor (COF) for the heads of the sheep. [20][21][22] In the other studies, we designed and evaluated a new model predictive control approach for improving the stability and the transparency of the haptic interface in the virtual-based environment. 20,21,23 However, our previous work required defining appropriate training exercises that could serve to teach the relevant skill set and evaluate the performance of the trainees. ...
... [20][21][22] In the other studies, we designed and evaluated a new model predictive control approach for improving the stability and the transparency of the haptic interface in the virtual-based environment. 20,21,23 However, our previous work required defining appropriate training exercises that could serve to teach the relevant skill set and evaluate the performance of the trainees. ...
Article
Full-text available
Background: The development of Endoscopic Sinus Surgery (ESS) training simulators for clinical environment applications have reduced the existing shortcomings in conventional teaching methods, creating a standard environment for trainers and trainees in a more accurate and repeatable fashion. Materials and methods: In this research, the validation study of an ESS training simulator has been addressed. It is important to consider components which guide trainees to improve their hand movements control in the orbital floor removal in an ESS operation. Therefore, we defined three tasks to perform: pre-experiment learning, training, and evaluation. In these tasks, the critical regions introduced in the virtual training environment are forbidden to be touched. Recruiting 20 participants, divided into two groups, we investigated the performance metrics: quality (the percentage of the realism for the generated force for orbital floor removal and the usefulness of the proposed training system for the surgical educational curricula.), efficiency (time, path length) and safety (touching the goal and forbidden wall). Results: All recruited participants answered a post-evaluation questionnaire regarding their perceptions of training system realism, potential educational benefits, and practiced skills. We investigate the differences between groups' performance metrics by utilizing the analysis of variance - Kruskal Wallis Test. Acquired results indicate that training before the actual process of the surgery has a significant effect on the accuracy and validity of the process for surgeons. Conclusions: Utilizing a standardized environment, trainers and trainees are able to carry out a process with regular features. In addition to traditional education methods, trainees can learn the risk of surgical operations. The training simulators can, also, provide a standard method for assessing the skills of surgical and medical students.
... In this work, we intended to design an online robust model predictive control structure for the VR haptic simulator, therefore, we tried to assign appropriate dynamic models to the each of the aforementioned three major components of such systems. To that end, we proposed a phenomenological tissue fracture model to represent the dynamics of the virtual environment, that is, endoscopic sinus and skull base surgery [26][27][28]. The haptic interface, Novint Falcon (Novint Technologies, Inc.), is a 3-DOF parallel robot manipulator used in the experimental set-up in this research. ...
Preprint
Haptic training simulators generally consist of three major components, namely a human operator, a haptic interface, and a virtual environment. Appropriate dynamic modeling of each of these components can have far-reaching implications for the whole system performance improvement in terms of transparency, the analogy to the real environment, and sense of touch to name a few. In this paper, we developed an Endoscopic Sinus Surgery (ESS) virtual-based haptic training simulator by doing a dynamic characterization of the phenomenological tissue fracture of the sinus in the virtual environment, using an input constrained linear parametric variable model. A parallel robot manipulator equipped with a calibrated force sensor is employed as a haptic interface. A lumped five-parameter single-degree-of-freedom mass-stiffness-damping impedance model is assigned to the operator's arm dynamic. A novel robust online output feedback quasi-min-max model predictive control (MPC) framework is proposed. The simulations and the experimental results demonstrate the effectiveness of the proposed control algorithm in terms of the robust stability as well as the convergence to the desired impedance quantities; In addition, the unwanted disturbances of the control efforts declined as a result of the switching in the piecewise linear dynamics of the virtual environment. Additional Information: Question Response Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation ABSTRACT Haptic training simulators generally consist of three major components, namely a human operator, a haptic interface, and a virtual environment. Appropriate dynamic modeling of each of these components can have far-reaching implications for the whole system performance improvement in terms of transparency, the analogy to the real environment, and sense of touch to name a few. In this paper, we developed an Endoscopic Sinus Surgery (ESS) virtual-based haptic training simulator by doing a dynamic characterization of the phenomenological tissue fracture of the sinus in the virtual environment, using an input constrained linear parametric variable model. A parallel robot manipulator equipped with a calibrated force sensor is employed as a haptic interface. A lumped five-parameter single-degree-of-freedom mass-stiffness-damping impedance model is assigned to the operator's arm dynamic. A novel robust online output feedback quasi-min-max model predictive control (MPC) framework is proposed. The simulations and the experimental results demonstrate the effectiveness of the proposed control algorithm in terms of the robust stability as well as the convergence to the desired impedance quantities; In addition, the unwanted disturbances of the control efforts declined as a result of the switching in the piecewise linear dynamics of the virtual environment.
... Figure 6 depicts the components, communications, and the schematic of the overall system. To characterize the force-displacement behavior, and mechanical properties of the virtual environment, a nonlinear rate-dependent model is used [33][34][35]. The proposed model is dependent on the tool's velocity and displacement while interacting with the virtual environment. ...
Article
Full-text available
Haptic interfaces, a kinesthetic link between a virtual environment and a human operator, is a significant element in the development of virtual reality-based simulators and play a key role in the reproduction of realistic haptic force feedback. Since most of the practical control theories are model-based, the identification of the robot's dynamics, for precise modeling of the system dynamic, is a process of high significance and usage. This research addresses dynamic characterization, performance issues, and structural stability, associated with a parallel haptic device interaction with an admittance type virtual environment. In this regard, considering the Lion identification scheme we characterized the dynamics of a robot nonlinearities in different operational points in the workspace, as a piecewise linear functions. Besides, utilizing the Lyapunov stability theory, we guaranteed the stability of the error dynamic of estimations. Next, by proposing a modified robust model predictive control scheme, one can guarantee the robust output feedback stability for a linear parametric variable system based on a quasi-min–max algorithm, subjected to the input constraints; besides, reducing the unwanted disturbances of the control efforts' signals caused by switching in the piecewise linear dynamics. The simulations and the experimental implementation of a single-degree-of-freedom virtual-based haptic system are carried out. Results indicate that the identification is of great accuracy and the appropriate convergence of parameters to the specific values will be outlined. Besides, the output tracking error and its derivative behave in a desirable manner. The proposed method guarantees the robust stability of the system with output feedback subjected to input constraints. Uncertainties are well addressed in the prediction dynamics, thereby improving the output signals' convergence to the desired output, is well suited for experimental practices of the haptic system.
... ESSS training simulators have therefore been developed for effective residency training [2] . The ideal simulator for ESSS training must be supported by a physical model [3,4] and provide repetitive behavior in a controlled environment [5] . Such models should represent both the instruments and the related anatomical structures and realistically render the interaction of tools with a variety of soft, semi-soft, and rigid tissues [6,7] . ...
Article
Full-text available
Endoscopic Sinus and Skull Base Surgery (ESSS) training simulators were developed as a way to improve the training process, due to the complex modern surgical techniques. The ideal simulator for ESSS not only must be supported by a physical model but also should have the ability to provide repetitive behavior in a controlled environment. Thus, development of the realistic tissue models is a key part of the ESSS virtual reality (VR) based surgical simulation. In this research, to address the haptic or force feedback, considerable effort is made to propose a phenomenological tissue fracture model for the sino-nasal tissue during the indentation of surgical tools. For this purpose, mechanical properties of the specific sino-nasal regions of sheep head, under various indentation and relaxation experiments have been studied. The process of the tool insertion into the coronal orbital floor (COF) tissue, under different indentation rates, is modeled as a sequence of three events: deformation, fracture, and cutting. For the deformation phase, the behavior can be characterized using a non-linear rate-dependent modified Kelvin-Voigt model. To complete the study, a nonlinear model for the tissue behavior prior to the fracture point is presented. The overall model shows a non-positive dependency of maximum force on tool indentation rate-that is-implying a faster tool insertion velocity will decrease the amount of maximum final fracture force. The tissue cutting phase has been modeled to characterize the necessary force for slicing through the COF. The proposed model in this study can help to develop VR-based ESSS base simulators in otolaryngology and ophthalmology surgeries; useful in preoperative planning, accurate surgical simulation, intelligent robotic assistance, and treatment applications.
Chapter
Full-text available
Robotics has seen increasing applications in the medical field, but its application is often limited by size and rigidity. By employing non-rigid mechanical structures, soft robotics allows for safer human interactions and is more readily adaptable with wearable medical devices. In addition, soft robotic devices provide greater freedom of movement and facilitate navigation in tight spaces and less deterministic geometries such as inside the human body. This chapter reviews soft robotics applications in the medical field as well as recent and prospective challenges. We focus on recent advances and state-of-the-art applications, namely rehabilitation, assistive devices, cardiac devices, and surgical robots. We point to general challenges such as modelling, implementation, simulation, and application-specific challenges that we will examine in the respective sections.
Chapter
Full-text available
Simulated training platforms offer a suitable avenue for surgical students and professionals to build and improve upon their skills, without the hassle of traditional training methods. To enhance the degree of realistic interaction paradigms of training simulators, great work has been done to both model simulated anatomy in more realistic fashion, as well as providing appropriate haptic feedback to the trainee. As such, this chapter seeks to discuss the ongoing research being conducted on haptic feedback incorporated simulators specifically for Endoscopic Sinus Surgery (ESS). This chapter offers a brief comparative analysis of some EES simulators, in addition to a deeper quantitative and qualitative look into our approach to designing and prototyping a complete virtual-based haptic EES training platform.
Thesis
Performing a safe and effective endoscopic sinus and skull-based surgery (ESSS) requires special training programs to gain sufficient hand-eye coordination and instrument manipulation skills. In this regard, virtual-based haptic surgical training systems have been considered among the medical students and residents as an effective approach for training practices. Therefore, in this study, the development of a haptic training system for ESSS, based on an animal model has been addressed. By conducting various indentation and relaxation experiments, mechanical properties of the specific sino-nasal regions of sheep head, as a function of force, displacement and tool insertion rate, for three phases of deformation, fracture, and cutting are presented by considering the modified Kelvin-Voigt and the data fitting models. To develop a virtual-based haptic ESSS training system, it is required to exploit the dynamic model for every three components of the haptic system, including the haptic interface, the human user, and the virtual environment. The dynamics of the sino-nasal virtual environment is characterized as a linear parametric variable problem with input constraints, based on a phenomenological tissue fracture model, and the Novint Falcon haptic device, equipped with a calibrated force sensor, is considered as a haptic interface. Redefining the type of cost function and deriving the one-step-ahead prediction, a novel modified online robust approach for the quasi-min-max output feedback model predictive control scheme is proposed. Employing a five-parameter dynamic model for the dynamic model of the user’s hand, the simulations and the experimental results of a one-degree-of-freedom virtual-based haptic system indicated the robust stability and the convergence of the output signals to the desired impedance values; besides, a reduction in the unwanted disturbances of the control efforts’ signals caused by switching in the piecewise linear dynamics. It is important to consider components in this research which guide trainees to improve their hand movements control in the orbital floor removal in an ESSS operation. Therefore, we defined three tasks to perform: pre-experiment learning, training, and evaluation. In these tasks, the critical regions introduced in the virtual training environment are forbidden to be touched. Recruiting 20 participants, divided in two groups, we investigated the performance metrics: quality (the percentage of the realism for the generated force for orbital floor removal and the usefulness of the proposed training system for the surgical educational curricula.), efficiency (time, path length) and safety (touching the goal and forbidden wall). All recruited participants answered a post-evaluation questionnaire regarding their perceptions of training system realism, potential educational benefits, and practiced skills. The acquired results showed that the proposed method is promising for practical use in ESSS training, considering its capability of providing a proper tradeoff between performance and stability. Utilizing a standardized environment, trainers and trainees are able to carry out a process with the regular features. In addition to traditional education methods, they can learn to the risk of surgical treatment. They can, also, provide a standard method for assessing the skills of surgical and medical students.
Conference Paper
Full-text available
Proper modeling of the human arm dynamic, as it interacts with telemanipulation and haptic systems, is important in enhancing the transparency of these systems. In this article, we introduced an adaptive identifier to estimate the impedance characteristic of a human operator as it interacts with a single translational degree of freedom mechanism. The five parameter model, including an extra spring and damper for a better approximation of the dynamic behavior of human arm, has been used. Since the impedance characteristic of human arm differs from one individual to another, it is important to estimate these parameters for each individual and update the controller to enhance the transparency and stability of the system. An adaptive architecture with normalized least square method with forgetting factor has been used for on-line identification of these five unknown impedance parameters of the human arm. The results show that the estimated parameters properly converge to actual parameters if the richness condition (richness of input data) is satisfied. It is shown that the suggested adaptive architecture has the ability to identify the impedance characteristics of different operators. Keywords—human arm; haptic interface; adaptive identifier; least square method;
Article
Full-text available
Virtual reality has been introduced to the medical field and it is now used in medical education as an alternative high fidelity simulator. The major components to successful learning that has contributed to the efficacy of the virtual reality system are the ability to provide repetitive practice under controlled environment, self-directed learning and proved construct validity. This review evaluated the validity and the efficacy of current virtual reality systems in ear, nose and throat surgery.
Article
Full-text available
During needle-based procedures, transitions between tissue layers often lead to rupture events that involve large forces and tissue deformations and produce uncontrollable crack extensions. In this paper, the mechanics of these rupture events is described, and the effect of insertion velocity on needle force, tissue deformation, and needle work is analyzed. Using the J integral method from fracture mechanics, rupture events are modeled as sudden crack extensions that occur when the release rate J of strain energy concentrated at the tip of the crack exceeds the fracture toughness of the material. It is shown that increasing the velocity of needle insertion will reduce the force of the rupture event when it increases the energy release rate. A nonlinear viscoelastic Kelvin model is then used to predict the relationship between the deformation of tissue and the rupture force at different velocities. The model predicts that rupture deformation and work asymptotically approach minimum values as needle velocity increases. Consequently, most of the benefit of using a higher needle velocity can be achieved using a finite velocity that is inversely proportional to the relaxation time of the tissue. Experiments confirm the analytical predictions with multilayered porcine cardiac tissue.
Article
Full-text available
The lack of experimental data in current literature on material properties of soft tissues in living condition has been a significant obstacle in the development of realistic soft tissue models for virtual reality based surgical simulators used in medical training. A robotic indenter was developed for minimally invasive measurement of soft tissue properties in abdominal region during a laparoscopic surgery. Using the robotic indenter, force versus displacement and force versus time responses of pig liver under static and dynamic loading conditions were successfully measured to characterize its material properties in three consecutive steps. First, the effective elastic modulus of pig liver was estimated as 10-15 kPa from the force versus displacement data of static indentations based on the small deformation assumption. Then, the stress relaxation function, relating the variation of stress with respect to time, was determined from the force versus time response data via curve fitting. Finally, an inverse finite element solution was developed using ANSYS finite element package to estimate the optimum values of viscoelastic and nonlinear hyperelastic material properties of pig liver through iterations. The initial estimates of the material properties for the iterations were extracted from the experimental data for faster convergence of the solutions.
Conference Paper
This paper proposes a new adaptive method for two-channel bilateral teleoperation systems control; the control method consists of adaptive force feedback and motion command scaling factors that ensure stable teleoperation with maximum achievable transparency at every moment of operation. The method is based on the integration of the real time estimation of the robot's environment impedance with the adaptive force and motion scaling factors generator. This paper formulates the adaptive scaling factors for stable teleoperation based on the impedance models of master, slave and estimated impedance of the environment. Feasibility and accuracy of an online environment impedance estimation method are analyzed through simulations and experiments. Then the proposed adaptive bilateral control method is verified through simulation studies. Results show stable interactions with maximum transparency for the simulated teleoperation system.
Article
We present a material model to represent the viscoelastic and material, nonlinear properties of liver tissue for needle insertion simulation. Material properties of liver tissue were measured using a rheometer and modeled from the measured data. The liver viscoelastic characteristics were represented by differential equations, including the fractional derivative term. Next, nonlinearity with respect to the fractional derivative was measured, and the stress-strain relationship using a cubic function was modeled. The material properties of liver tissue were represented by a simple equation with only a few parameters. We evaluated the variety of each stiffness parameter from measurements of 50 samples. The results showed a high degree of variation in each stiffness parameter, especially with respect to nonlinearity. Moreover, each parameter had a low correlation coefficient. We also modeled the probability of variation in material properties from these results to provide a basis for deformation simulations considering individual patient differences.
Conference Paper
The use of surgical simulators with virtual reality allows surgeons to practice diverse types of procedures for minimizing any risk on patients during the real surgery. This paper describes a prototype for a Minimal Invasive Surgery (MIS) simulator with haptic feedback applied to an endoscopic endonasal surgery. The research is divided in three parts: the simulation of a rigid endoscope device with four degrees of freedom (pitch, yaw, roll and dolly), a simplified model of the nasal tissue for fast haptic rendering and the integration with a virtual reality simulation system with collision detection. The main contribution of the paper is the rigid endoscope model using a simple lever acting as a force transformer pivoting in the nostril. The obtained results show a suitable response of the system for an interactive simulation.
This article reviews the rationale, current status and future directions for the development and implementation of virtual reality surgical simulators as training tools. The complexity of modern surgical techniques, which utilize advanced technology, presents a dilemma for surgical training. Hands-on patient experience - the traditional apprenticeship method for teaching operations - may not apply because of the learning curve for skill acquisition and patient safety expectation. The paranasal sinuses and temporal bone have intricate anatomy with a significant amount of vital structures either within the surgical field or in close proximity. The current standard of surgical care in these areas involves the use of endoscopes, cameras and microscopes, requiring additional hand-eye coordination, an accurate command of fine motor skills, and a thorough knowledge of the anatomy under magnified vision. A surgeon's disorientation or loss of perspective can lead to complications, often catastrophic and occasionally lethal. These considerations define the ideal environment for surgical simulation; not surprisingly, significant research and validation of simulators in these areas have occurred. Virtual reality simulators are demonstrating validity as training and skills assessment tools. Future prototypes will find application for routine use in teaching, surgical planning and the development of new instruments and computer-assisted devices.