Conference PaperPDF Available

Abstract and Figures

Accurate needle placement is highly relevant for puncture of anatomical structures. The clinician’s experience and medical imaging are essential to complete these procedures safely. However, imaging may come with inaccuracies due to image artifacts. Sensor-based solutions have been proposed for acquiring additional guidance information. These sensors typically require to be embedded in the instrument tip, leading to direct tissue contact, sterilization issues, and added device complexity, risk, and cost. Recently, an audio-based technique has been proposed for “listening” to needle tip-tissue interactions by an externally placed sensor. This technique has shown promising results for different applications. But the relation between the interaction event and the generated audio excitation is still not fully understood. This work aims to study this relationship, using a force sensor as a reference, by relating events and dynamical characteristics occurring in the audio signal with those occurring in the force signal. We want to show that dynamical information that a well-known sensor as force can provide could also be extracted from a low-cost and simple sensor such as audio. In this aim, the Pearson coefficient was used for signal-to-signal correlation between extracted audio and force indicators. Also, an event-to-event correlation between audio and force was performed by computing features from the indicators. Results show high values of correlation between audio and force indicators in the range of 0.53 to 0.72. These promising results demonstrate the usability of audio sensing for tissue-tool interaction and its potential to improve telemanipulated and robotic surgery in the future.
Content may be subject to copyright.
Feasibility check: can audio be a simple
alternative to force-based feedback for needle
guidance?
Anonymous12[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗], Anonymous2 [∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗],
Anonymous2[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗], and Anonymous1[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗]
1Anonymous Organization
2Anonymous Organization
Abstract. Accurate needle placement is highly relevant for puncture
of anatomical structures. The clinician’s experience and medical imag-
ing are essential to complete these procedures safely. However, imaging
may come with inaccuracies due to image artifacts. Sensor-based solu-
tions have been proposed for acquiring additional guidance information.
These sensors typically require to be embedded in the instrument tip,
leading to direct tissue contact, sterilization issues, and added device
complexity, risk, and cost. Recently, an audio-based technique has been
proposed for ”listening” to needle tip-tissue interactions by an externally
placed sensor. This technique has shown promising results for different
applications. But the relation between the interaction event and the gen-
erated audio excitation is still not fully understood. This work aims to
study this relationship, using a force sensor as a reference, by relating
events and dynamical characteristics occurring in the audio signal with
those occurring in the force signal. We want to show that dynamical in-
formation that a well-known sensor as force can provide could also be
extracted from a low-cost and simple sensor such as audio. In this aim,
the Pearson coefficient was used for signal-to-signal correlation between
extracted audio and force indicators. Also, an event-to-event correlation
between audio and force was performed by computing features from the
indicators. Results show high values of correlation between audio and
force indicators in the range of 0.53 to 0.72. These promising results
demonstrate the usability of audio sensing for tissue-tool interaction and
its potential to improve telemanipulated and robotic surgery in the fu-
ture.
Keywords: Audio guidance ·Force feedback ·Needle interventions.
1 Introduction
Percutaneous needle insertion is one of the most common minimally invasive pro-
cedures. The experience of the clinician is an important requirement for accurate
placement of needles, given the reduced visual and tactile information transmit-
ted to the clinician via the instruments. Imaging techniques such as magnetic
2 Anonymous et al.
resonance, computed tomography, or ultrasound can support clinicians in this
type of procedure, but the accuracy can still not be fully assured because of
artifacts present in the images [20, 14].
Sensor-based solutions have been proposed for providing haptic feedback dur-
ing the procedure [11, 10, 18, 7, 8, 1, 5, 12, 23, 19, 3]. However, most of these solu-
tions require sophisticated sensors that sometimes need to be embedded in the
instrument tip or shaft, leading to direct contact with human organs, steril-
ization issues, the use of non-standard and quality-reduced tools, added device
complexity, risk and cost. This imposes serious design limitations, and therefore
they have encountered difficulties in being adopted for regular clinical use.
Recently, an audio-based technique has been proposed in [9] for listening to
the needle tip-tissue interaction dynamics using a sensor placed at the proximal
end of the tool. The authors of this work has shown promising preliminary results
for monitoring medical interventional devices such as needles [9], guide wires
[15], and laparoscopic tools [4]. However, even if audio has proved to be a tool
with potential for providing guidance information such as tissue-tissue passage,
puncture and perforation events or palpation information, the generated audio
dynamics are still not fully understood.
The aim of this work is to investigate the audio dynamics generated from
needle-tissue interactions during needle insertion to have a better understand-
ing of the generation of the audio excitation using the audio-based guidance
technique. In this purpose, force is used as a reference since it has been widely
employed in the literature to understand interactions between needles and tis-
sue. The main idea is to relate events and dynamical characteristics extracted
from the audio signal with those extracted from the force signal through indica-
tors and event features computed by processing both signals. The audio signal
is processed by extracting its homomorphic envelope. Indicators related to local
event intensity, derivative, and curvature are computed from the force signal.
The Pearson coefficient is used for signal-to-signal correlation between audio
and force indicators. Then, event-to-event correlation between audio and force
events is performed by computing features from the indicators.
Results show values of Pearson coefficient between audio and force indicators
in the range of 0.53 to 0.72, being the highest one the correlation of audio
with force curvature. Additionally, events of high correlated indicators exhibit
a clear relationship that can be important for understanding audio behavior.
Both analyses show that audio, acquired non invasively with a simple and low-
cost sensor, can contain significant information that can be used as additional
feedback to clinicians.
2 Method
Needle insertion and its interaction with soft tissue has been widely studied
using force sensors, being possible to distinguish three phases of interaction [17,
2, 6]. During the first phase or pre-puncture phase, the needle tip deforms the
surface in contact with the tissue producing an increase in the force. The second
Title Suppressed Due to Excessive Length 3
phase starts with the puncture event or tissue breakage, characterized by a peak
in the force, followed by a sharp decrease. The third phase corresponds to the
post-puncture phase, where the force can vary due to friction, collision with
interior structures, or due to the puncture of a new tissue boundary. During the
first phase, when audio is acquired, no audio excitation occurs since there is no
tissue breakage or structure collision. The puncture during the second phase and
the collisions, friction, and new punctures during the second and third phases
can produce significant and complex audio excitation dynamics. However, even
if an audio response is complicated, its dynamics should be related to dynamical
characteristics of the force during the second and third phases. This is what we
want to explore in this work.
Our aim is to extract characteristics or feature indicators from the force that
can be related to dynamical characteristics of the audio excitation. The first
indicator that we want to explore is the local intensity of the force or detrended
force, which aims to emphasize the increase of force from a local deflection (con-
tact of needle tip with the tissue) passing through its peak (puncture event) and
coming back to a steady stage. We also believe that the cumulative energy stored
during the boundary displacement and the fast drop in force after the puncture
also influences the audio excitation, and this is why derivative and curvature
indicators are also extracted.
The idea of this work is not to explain mechanical properties and fundamen-
tals of needle insertion in soft tissue, but to demonstrate that characteristics of
audio and force, even resulting from sensors of entirely different nature, can be
strongly related through a sort of transfer function between both sensor modal-
ities. Through this relationship, we also want to show the wealth of information
that an audio signal can contain concerning tip-tissue interaction dynamics.
Fig. 1 displays a block diagram with the main steps to relate audio and force
characteristics. First, the audio signal and the force signal are processed in order
to compute the different indicators extracted for enhancing the signal features
that want to be compared: one audio indicator (IA), and four force indicators,
related to the local intensity (IFint ), to the curvature IFc), to the derivative
(IFd), and one indicator that integrates curvature and intensity (IFci ). Then,
a signal-to-signal correlation is performed between audio and force indicators in
order to assess similarity between features and also for optimizing the parameters
of the processing algorithms. Finally, an event-to-event correlation is performed
by computing features from the extracted indicators.
2.1 Audio indicator extraction
The audio signal is first pre-processed using a 7th-order Butterworth bandpass
filter (36 KHz) in order to enhance the needle tip tissue interaction information,
as shown in [9].
Fig. 2 displays the filtered audio signal together with the synchronized force
signal during needle insertion into soft tissue, using a recording from the dataset
presented in [9], which will be introduced in Section 3. It is possible to observe
from the three marked events (denoted by 1, 2 and 3) in Fig. 2(a) and (b) and
4 Anonymous et al.
MA
Smoothing
Parabolic fitting
curvature
Derivative
estimation
Homomorphic
envelope
( . )
Baseline
estimation and
force detrending
Positive part
Bandpass
filter
Homomorphic
envelope
Signal to signal
correlation
Event to
event
correlation
Audio processing
Force processing
audio
signal
force
signal
Fig. 1. General block diagram with the methodology to relate force and audio signals.
the zooms in Fig. 2(c), that a main puncture event, identifiable in the force as a
significant peak, exhibits a complicated succession of events in the audio signal.
This set of events denotes the accumulation of energy through modulation of the
signal amplitude at the time interval just after the rupture of the tissue. To rep-
resent this energy event accumulation, we compute the homomorphic envelope
of the audio signal, which represents the amplitude modulation of the signal and
that it is obtained using homomorphic filtering in the complex domain [21].
1 2 3 123
a)
b)
c)
d)
Time [s]
Force [N]
Fig. 2. (a) Bandpassed audio signal with labeled main puncture events. (b) force signal
with labeled main puncture events. (c) Audio zoom over the puncture events. (d)
Homomorphic envelope of the audio signal.
Title Suppressed Due to Excessive Length 5
2.2 Force processing for indicators extraction
As explained above, four indicators are extracted from the force signal by en-
hancing information related to characteristics such as intensity, derivative, and
curvature. The diagram of Fig. 1 shows the main steps for the computation of
the force indicators. The first step that is common to all the indicators is the
force signal smoothing, consisting of the application of a moving average filter
in order to reduce the high-frequency ripple dynamics from the force.
The force indicator IFiintends to extract the information concerning the
local intensity of the force. In this aim, a force detrending has to be performed
to attenuate the very low-frequency progression present in the force signal during
needle insertion. For that, the signal baseline is first estimated using a two-stage
median filter [22] that is then subtracted to the smoothed force signal. Finally,
the positive part of the resulting signal is extracted.
For the computation of the force indicator related to the derivative IFd, a
derivative filter in series with a smoothing filter is first applied following [13].
To only keep the most important positive characteristics of the derivative, the
homomorphic envelope is extracted.
The curvature is estimated with the algorithm proposed in [16] that enhances
curvature and intensity in signals. A 2nd-degree polynomial is used to fit the
force signal inside a sliding window. The 2nd-degree coefficient is related to the
curvature of the signal. The force indicator IFcis finally computed by applying
a homomorphic envelope to the resulting curvature.
As explained in [16], an indicator that enhance intensity and curvature jointly
can be extracted by computing the product between the constant polynomial
coefficient and the 2nd-degree coefficient of the polynomial. IFci is computed as
the square root of the joint indicator.
Fig. 3 shows the four extracted indicators, including also the original force
signal and the baseline estimation (in red).
Force and baseline Curvature indicator Derivative indicator
Detrended force Curvature + intensity
Time [s]
arb. unit
Fig. 3. Four indicators extracted from the force signal.
2.3 Force and audio information correlation methodology
Two approaches are applied for putting in relation the indicators extracted from
the force with the one extracted from the audio. A signal-to-signal correlation
using the Pearson coefficient is first performed. This step allows assessing the
similarity between both types of indicators and also to optimize the parameters
6 Anonymous et al.
involved in the computation of the indicators. In fact, the extraction of the force
and audio indicators requires to set some parameters:
The frequency cut-off of the low-pass filter to be used in the envelope extrac-
tion for the audio and force signals, denoted as lpfaand lpff, respectively.
The length of the sliding window used for computing the polynomial fitting
of the force signal, denoted as hwin.
The first and second stage averaging window length of the median filter
applied for the force detrending, denoted by L1and L2, respectively.
Each parameter is optimized by maximizing the Pearson coefficient between the
force indicator and the audio indicator.
Using the optimized parameters, we perform an event-to-event correlation
analysis. Significant puncture events in the force are first detected using a stan-
dard peak detector algorithm. Then, a window Wis defined around the detected
peak instant in the audio signal. Finally, for each detected event, we relate the
maximal value of the force indicator inside Wto the energy of the audio indica-
tor, also inside W.
3 Results
For evaluating the presented approach, the dataset generated in [9] was used.
This dataset includes 80 audio recordings acquired during the insertion of an 18G
200 mm length biopsy needle into ex-vivo porcine tissue phantom. The insertion
was done automatically using a testing machine (Zwicki, Zwick GmbH & Co.KG,
Ulm) at an insertion velocity of 3 mm/s that also recorded the axial insertion
force. The audio frequency sampling was 44100 Hz, and one force sample was
acquired every 0.03 mm. If we assume the velocity nearly constant, the force
sampling frequency can be estimated at around 100 Hz.
Table 1 shows the optimized results of the correlation between the four force
indicators and the audio indicator. The table also shows the values of the pa-
rameters for the extraction of the indicators. In bold are marked the parameters
that required optimization (lpfa,L1,L2,hwin , and lpff). The span of the MA
smoother used for the computation of the force indicators was set to 10 samples,
equivalent to 0.1 seconds for a sampling frequency of 100 Hz. The last column of
the table shows the average Pearson coefficient value (ρ) over the 80 recordings
of the dataset. It is possible to observe that the best correlation between audio
and force is obtained with the indicator related to the force curvature.
Fig. 4 displays a further analysis concerning the obtained correlation Pearson
coefficients. In Fig. 4(a), the histograms of the Pearson coefficients between the
four force indicators and the audio indicator are displayed. It is possible to verify
that the correlation values range in general between 0.3 and 0.9, but that for
curvature and the joint curvature and intensity indicators, 50% of the record-
ing has a correlation value over 0.6, which is high considering the completely
different nature between audio and force sensors. Fig. 4(b), which shows the
accumulative histogram of the Pearson coefficient, confirms the analysis made
Title Suppressed Due to Excessive Length 7
Table 1. Average Pearson coefficients for the correlation between the optimized four
force indicators and the optimized audio indicator for the 80 needle insertion recordings.
Comparison Audio Parameters Force parameter ρ
lpfaMAspan L1L2hwin lpff
IA vs I Fi4 10 100 100 n/a n/a 0.531
IA vs I Fc1 10 n/a n/a 7 1 0.717
IA vs I Fd1 10 n/a n/a n/a 1 0.664
IA vs I F ci 1 10 n/a n/a 13 1 0.672
previously. It is possible to observe that the best correlation between audio and
force is obtained with the curvature indicator IFc, followed by the joint curva-
ture and intensity indicator IFci. The derivative of the force IFdalso provides
high values of correlation, while the local intensity indicator I Fiis the weakest
indicator influencing the audio.
Fig. 5 shows examples of high correlations between each force indicator and
the audio indicator for different recordings belonging to the dataset. It is possible
to see how the main dynamics extracted from the audio indicator can follow the
dynamics obtained from the force indicator, i.e., many of the information involve
in force it is somehow visible in the audio indicator signal.
Pearson coefficent Pearson coefficent
nb. of recordingf
nb. of recordingf
(a) (b)
vs
vs
vs
vs
vs
vs
vs
vs
Fig. 4. Standard and cumulative histograms of the Pearson coefficients of the four force
indicators with audio, for the 80 recordings.
Fig. 6 shows the results of the event-to-event correlation of the two best
correlations obtained in the signal-to-signal analysis: curvature IFcindicator
versus the audio indicator IA and the joint curvature and intensity indicator
IFci and IA. We explore two scenarios, the first one (Fig. 6(a) and (b)) by taking
into account a large number of puncture events and the second one (Fig. 6(c) and
(d)) by taking into account only the most important puncture events occurring
during the needle insertion process. This is done by only modifying a simple
threshold in the peak detector; higher is the threshold more events will be taken
into account. For both force indicators, a clear event correlation can be observed
using both types of event thresholding. When a large number of events are
8 Anonymous et al.
10 15 20 25 10 15 20 25510 15 20510 15 20 255
Time [s] Time [s]Time [s]Time [s]
Fig. 5. Examples of force and audio indicators where high correlation were obtained.
taken into account, we can see that in the range of force events presenting low
intensities, it is not exactly clear that a high-intensity event will produce a high
audio excitation. However, when the number of events is reduced, it is possible
to observe an evident linear correlation between force and audio events, and this
is even more evident with IFci in Fig. 6(c).
Fig. 6. Event-to-event correlation for (a) IFcvs. I A with a low intensity-event thresh-
old, (b) IFci vs. I A with a low intensity-event threshold, (c) I Fcvs. IA with a high
intensity-event threshold, and (d) IFci vs. IA with a high intensity-event threshold.
4 Conclusion
In this work, we explored the audio dynamics generated from the tip/tissue in-
teraction during needle insertion into soft tissue using a recently proposed audio
guidance technique. The main idea was to observe the effect of different char-
acteristics of force measurement on the audio excitation. We have shown that
information resulting from sensors of entirely different nature, such audio and
force, can be strongly related during needle insertion. This suggests that audio
can contain valuable information for monitoring tip/tissue interaction dynamics.
Signal-to-signal and event-to-event correlation analysis showed that the curva-
ture of the force and the integration of force curvature and intensity strongly
affect the behavior of the audio excitation. The obtained results are highly signif-
icant since they show that audio contains considerable information on the needle
insertion process, and that it can be a low-cost and simple complementary tool
for guiding needles.
Title Suppressed Due to Excessive Length 9
References
1. Abayazid, M., Kemp, M., Misra, S.: 3d flexible needle steering in soft-tissue phan-
toms using fiber bragg grating sensors. In: 2013 IEEE International Conference on
Robotics and Automation. pp. 5843–5849. IEEE (2013)
2. Abolhassani, N., Patel, R., Moallem, M.: Needle insertion into soft tissue: A survey.
Medical engineering & physics 29(4), 413–431 (2007)
3. Chadda, R., Wismath, S., Hessinger, M., Sch¨afer, N., Schlaefer, A., Kupnik, M.:
Needle tip force sensor for medical applications. In: 2019 IEEE SENSORS. pp. 1–4.
IEEE (2019)
4. Chen, C., S¨uhn, T., Kalmar, M., Maldonado, I., Wex, C., Croner, R., Boese,
A., Friebe, M., Illanes, A.: Texture differentiation using audio signal analysis
with robotic interventional instruments. Computers in biology and medicine 112,
103370 (2019)
5. Elayaperumal, S., Bae, J.H., Daniel, B.L., Cutkosky, M.R.: Detection of membrane
puncture with haptic feedback using a tip-force sensing needle. In: 2014 IEEE/RSJ
International Conference on Intelligent Robots and Systems. pp. 3975–3981. IEEE
(2014)
6. van Gerwen, D.J., Dankelman, J., van den Dobbelsteen, J.J.: Needle–tissue inter-
action forces–a survey of experimental data. Medical engineering & physics 34(6),
665–680 (2012)
7. Henken, K., Van Gerwen, D., Dankelman, J., Van Den Dobbelsteen, J.: Accuracy
of needle position measurements using fiber bragg gratings. Minimally Invasive
Therapy & Allied Technologies 21(6), 408–414 (2012)
8. Ho, S.C.M., Razavi, M., Nazeri, A., Song, G.: Fbg sensor for contact level moni-
toring and prediction of perforation in cardiac ablation. Sensors 12(1), 1002–1013
(2012)
9. Illanes, A., Boese, A., Maldonado, I., Pashazadeh, A., Schaufler, A., Navab, N.,
Friebe, M.: Novel clinical device tracking and tissue event characterization using
proximally placed audio signal acquisition and processing. Scientific reports 8(1),
1–11 (2018)
10. Iordachita, I., Sun, Z., Balicki, M., Kang, J.U., Phee, S.J., Handa, J., Gehlbach, P.,
Taylor, R.: A sub-millimetric, 0.25 mn resolution fully integrated fiber-optic force-
sensing tool for retinal microsurgery. International journal of computer assisted
radiology and surgery 4(4), 383–390 (2009)
11. Kalvøy, H., Frich, L., Grimnes, S., Martinsen, Ø.G., Hol, P.K., Stubhaug, A.:
Impedance-based tissue discrimination for needle guidance. Physiological measure-
ment 30(2), 129 (2009)
12. Kumar, S., Shrikanth, V., Amrutur, B., Asokan, S., Bobji, M.S.: Detecting stages
of needle penetration into tissues through force estimation at needle tip using fiber
bragg grating sensors. Journal of biomedical optics 21(12), 127009 (2016)
13. Laguna, P., Thakor, N.V., Caminal, P., Jane, R., Yoon, H.R., de Luna, A.B.,
Marti, V., Guindo, J.: New algorithm for qt interval analysis in 24-hour holter ecg:
performance and applications. Medical and Biological Engineering and Computing
28(1), 67–73 (1990)
14. Lal, H., Neyaz, Z., Nath, A., Borah, S.: Ct-guided percutaneous biopsy of intratho-
racic lesions. Korean journal of radiology 13(2), 210–226 (2012)
15. Mahmoodian, N., Schaufler, A., Pashazadeh, A., Boese, A., Friebe, M., Illanes,
A.: Proximal detection of guide wire perforation using feature extraction from
bispectral audio signal analysis combined with machine learning. Computers in
biology and medicine 107, 10–17 (2019)
10 Anonymous et al.
16. Manriquez, A.I., Zhang, Q.: An algorithm for qrs onset and offset detection in single
lead electrocardiogram records. In: 2007 29th Annual International Conference of
the IEEE Engineering in Medicine and Biology Society. pp. 541–544. IEEE (2007)
17. Okamura, A.M., Simone, C., O’leary, M.D.: Force modeling for needle insertion
into soft tissue. IEEE transactions on biomedical engineering 51(10), 1707–1716
(2004)
18. Park, Y.L., Elayaperumal, S., Daniel, B., Ryu, S.C., Shin, M., Savall, J., Black,
R.J., Moslehi, B., Cutkosky, M.R.: Real-time estimation of 3-d needle shape and
deflection for mri-guided interventions. IEEE/ASME Transactions On Mechatron-
ics 15(6), 906–915 (2010)
19. Ravali, G., Manivannan, M.: Haptic feedback in needle insertion modeling and
simulation. IEEE reviews in biomedical engineering 10, 63–77 (2017)
20. Reusz, G., Sarkany, P., Gal, J., Csomos, A.: Needle-related ultrasound artifacts
and their importance in anaesthetic practice. British journal of anaesthesia 112(5),
794–802 (2014)
21. Rezek, I., Roberts, S.J.: Envelope extraction via complex homomorphic filtering.
Technical Report TR-98-9 Technical report (1998)
22. Sameni, R., Shamsollahi, M.B., Jutten, C.: Model-based bayesian filtering of car-
diac contaminants from biomedical recordings. Physiological Measurement 29(5),
595 (2008)
23. Xu, R., Yurkewich, A., Patel, R.V.: Curvature, torsion, and force sensing in contin-
uum robots using helically wrapped fbg sensors. IEEE Robotics and Automation
Letters 1(2), 1052–1059 (2016)
... This novel method showed promising results for needle guidance applications such as an indication of tissue passage [10] or for characterizing and classifying a needle perforation/puncture [9,10,19,22]. ...
... Preliminary results of this work have been presented in [9], where 80 needle insertion recordings were performed at an insertion velocity of 3 mm/s, showing high signal-to-signal and event-to-event correlations. However, the comparison was made by performing experiments with only one insertion velocity. ...
... The main contribution of this new work is to demonstrate that the dynamical relations between audio and force are independent of the velocity with which the needle is inserted and that the previously observed dependencies can be generalized. The experiment performed in [9] was repeated with three additional insertion velocities, and a slightly different signal processing algorithm than [9] was used for computing the audio and force indicators. ...
Article
Full-text available
Purpose Percutaneous needle insertion is one of the most common minimally invasive procedures. The clinician’s experience and medical imaging support are essential to the procedure’s safety. However, imaging comes with inaccuracies due to artifacts, and therefore sensor-based solutions were proposed to improve accuracy. However, sensors are usually embedded in the needle tip, leading to design limitations. A novel concept was proposed for capturing tip–tissue interaction information through audio sensing, showing promising results for needle guidance. This work demonstrates that this audio approach can provide important puncture information by comparing audio and force signal dynamics during insertion. Methods An experimental setup for inserting a needle into soft tissue was prepared. Audio and force signals were synchronously recorded at four different insertion velocities, and a dataset of 200 recordings was acquired. Indicators related to different aspects of the force and audio were compared through signal-to-signal and event-to-event correlation analysis. Results High signal-to-signal correlations between force and audio indicators regardless of the insertion velocity were obtained. The force curvature indicator obtained the best correlation performances to audio with more than $$70\%$$ 70 % of the correlations higher than 0.6. The event-to-event correlation analysis shows that a puncture event in the force is generally identifiable in audio and that their intensities firmly related. Conclusions Audio contains valuable information for monitoring needle tip/tissue interaction. Significant dynamics obtained from a well-known sensor as force can also be extracted from audio, regardless of insertion velocities.
Conference Paper
Full-text available
We present a 10 N nominal needle tip force sensor based on silicon-based semiconductor strain gauges that can be used in brachytherapy or biopsies. As deformation body the guiding rod of such a surgical needle with a diameter of 1.2 mm is used. In a thinned area, near the tip of the needle, the strain gauge, as sensing element, is applied with a flexible PCB for electrical connection. Two needles are built and characterized using a testing machine. The sensors provide sensitivities of 6.1 mV/N and 6.4 mV/N for only 1 mA supply current of the bridge, respectively. The linearity and hysteresis errors are in the range of 0.6 % to 4.4 %, which allows measuring interaction forces at the needle tip. Our results prove that the force sensor will provide new measurement means or applications with haptic feedback or for robotic insertion tools.
Article
Full-text available
Robotic minimally invasive surgery (RMIS) has played an important role in the last decades. In traditional surgery, surgeons rely on palpation using their hands. However, during RMIS, surgeons use the visual-haptics technique to compensate the missing sense of touch. Various sensors have been widely used to retrieve this natural sense, but there are still issues like integration, costs, sterilization and the small sensing area that prevent such approaches from being applied. A new method based on acoustic emission has been recently proposed for acquiring audio information from tool-tissue interaction during minimally invasive procedures that provide user guidance feedback. In this work the concept was adapted for acquiring audio information from a RMIS grasper and a first proof of concept is presented. Interactions of the grasper with various artificial and biological texture samples were recorded and analyzed using advanced signal processing and a clear correlation between audio spectral components and the tested texture were identified. https://doi.org/10.1016/j.compbiomed.2019.103370
Article
Full-text available
We propose a new and complementary approach to image guidance for monitoring medical interventional devices (MID) with human tissue interaction and surgery augmentation by acquiring acoustic emission data from the proximal end of the MID outside the patient to extract dynamical characteristics of the interaction between the distal tip and the tissue touched or penetrated by the MID. We conducted phantom based experiments (n = 955) to show dynamic tool/tissue interaction during tissue needle passage (a) and vessel perforation caused by guide wire artery perforation (b). We use time-varying auto-regressive (TV-AR) modelling to characterize the dynamic changes and time-varying maximal energy pole (TV-MEP) to compute subsequent analysis of MID/tissue interaction characterization patterns. Qualitative and quantitative analysis showed that the TV-AR spectrum and the TV-MEP indicated the time instants of the needle path through different phantom objects (a) and clearly showed a perforation versus other generated artefacts (b). We demonstrated that audio signals acquired from the proximal part of an MID could provide valuable additional information to surgeons during minimally invasive procedures.
Article
Full-text available
Needle insertion being the most basic skill in medical care, training has to be imparted not only for physicians but also for nurses and paramedics. In most needle insertion procedures haptic feedback from the needle is the main stimulus that novices are to be trained in. For better patient safety, the classical methods of training the haptic skills have to be replaced with simulators based on new robotic and graphics technologies. This paper reviews the current advances in needle insertion modeling. It is classified into three sections: needle insertion models, tissue deformation models, and needle-tissue interaction models. Although understated in the literature, the classical and dynamic friction models which are critical for needle insertion modeling are also discussed. The experimental setup or the needle simulators which have been developed to validate the models are described. The need of psychophysics for needle simulators and psychophysical parameter analysis of human perception in needle insertion are discussed, which are completely ignored in the literature.
Article
Full-text available
Real-time ultrasound guidance for any intervention relies on visualization of needle advancement towards a target. Unfortunately, correct identification of the needle tip is not straightforward, as artifacts always distort the image. The ultrasonic appearance of the needle is often degraded by reverberation, comet tail, side-lobe, beam-width, or bayonet artifacts, which can easily confuse an unprepared operator. Furthermore, the typical needle image, that is, a dot or a straight line (out-of-plane and in-plane approaches, respectively), is also a result of artifacts that hide the real dimensions of the needle. Knowledge and correct interpretation of these artifacts is important for safe practice and is paramount to success when precise needle manipulation is mandatory, for example, when the target is small. In this review, authors discuss the most important needle-related artifacts and provide a physical explanation focusing on implications for everyday practice. Recent advances that allow increased needle visualization and reduction of artifacts are also discussed.
Article
Artery perforation during a vascular catheterization procedure is a potentially life threatening event. It is of particular importance for the surgeons to be aware of hidden or non-obvious events. To minimize the impact it is crucial for the surgeon to detect such a perforation very early. We propose a novel approach to identify perforations based on the acquisition and analysis of audio signals on the outside proximal end of a guide wire. The signals were acquired using a stethoscope equipped with a microphone and attached to the proximal end of the guide wire via a 3D printed adapter. Bispectral analysis was employed to extract acoustic signatures in the signal and several features were extracted from the bispectrum of the signal. Finally, three machine learning algorithms - K-nearest Neighbor, Support Vector Machine (SVM), and Artificial Neural Network (ANN)- were used to classify a signal as a perforation or as an artifact. The bispectrum-based features resulted in valuable features allowing a perforation to be clearly identifiable from other occurring events. A perforation leaves a clear audio signal trace in the time-frequency domain. The recordings were classified as perforation, friction or guide wire bump using SVM with 97% (polykernel) and 98.62% (RBF) accuracy, k-nearest Neighbor an accuracy of 98.28% and ANN with accuracy of 98.73% was obtained. The presented approach shows that interactions starting at the tip of a guide wire can be picked up at its proximal end providing a valuable additional information that could be used during a guide wire procedure. Temporary Share Link: https://authors.elsevier.com/c/1YYm32OYcrjZu
Article
Several medical procedures involve the use of needles. The advent of robotic and robot assisted procedures requires dynamic estimation of the needle tip location during insertion for use in both assistive systems as well as for automatic control. Most prior studies have focused on the maneuvering of solid flexible needles using external force measurements at the base of the needle holder. However, hollow needles are used in several procedures and measurements of forces in proximity of such needles can eliminate the need for estimating frictional forces that have high variations. These measurements are also significant for endoscopic procedures in which measurement of forces at the needle holder base is difficult. Fiber Bragg grating sensors, due to their small size, inert nature, and multiplexing capability, provide a good option for this purpose. Force measurements have been undertaken during needle insertion into tissue mimicking phantoms made of polydimethylsiloxane as well as chicken tissue using an 18-G needle instrumented with FBG sensors. The results obtained show that it is possible to estimate the different stages of needle penetration including partial rupture, which is significant for procedures in which precise estimation of needle tip position inside the organ or tissue is required.
Article
Due to their small size and flexibility, fiber Bragg grating (FBG) sensors have been integrated into needle-sized continuum robots for shape estimation and force measurement. The challenge in extending previous shape and force sensing technologies to pre-curved continuum robots, such as concentric-tube robots, is that torsion information is essential for accurate shape estimation, and the force-strain relationship is nonlinear. In this letter, a novel helically wrapped FBG sensor design and corresponding force-curvature-strain model are developed to provide simultaneous curvature, torsion, and force measurement. To validate this design and modeling technique, two sensorized Nitinol tubes were fabricated and tested in an experimental setup. The results showed that accurate and sensitive curvature, torsion, and force measurements can be obtained at a 100 Hz sampling rate.
Article
This paper presents calibration and user test results of a 3-D tip-force sensing needle with haptic feedback. The needle is a modified MRI-compatible biopsy needle with embedded fiber Bragg grating (FBG) sensors for strain detection. After calibration, the needle is interrogated at 2 kHz, and dynamic forces are displayed remotely with a voice coil actuator. The needle is tested in a single-axis master/slave system, with the voice coil haptic display at the master, and the needle at the slave end. Tissue phantoms with embedded membranes were used to determine the ability of the tip-force sensors to provide real-time haptic feedback as compared to external sensors at the needle base during needle insertion via the master/slave system. Subjects were able to determine the position of the embedded membranes with significantly better accuracy using FBG tip feedback than with base feedback using a commercial force/torque sensor (p = 0.045) or with no added haptic feedback (p = 0.0024).
Conference Paper
Needle insertion procedures are commonly used for surgical interventions. In this paper, we develop a three-dimensional (3D) closed-loop control algorithm to robotically steer flexible needles with an asymmetric tip towards a target in a soft-tissue phantom. Twelve Fiber Bragg Grating (FBG) sensors are embedded on the needle shaft. FBG sensors measure the strain applied on the needle during insertion. A method is developed to reconstruct the needle shape using the strain data obtained from the FBG sensors. Four experimental cases are conducted to validate the reconstruction method (single-bend, double-bend, 3D double-bend and drilling insertions). In the experiments, the needle is inserted 120 mm into a soft-tissue phantom. Camera images are used as a reference for the reconstruction experiments. The results show that the mean needle tip accuracy of the reconstruction method is 1.8 mm. The reconstructed needle shape is used as feedback for the steering algorithm. The steering algorithm estimates the region that the needle can reach during insertion, and controls the needle to keep the target in this region. Steering experiments are performed for 110 mm insertion, and the mean targeting accuracy is 1.3 mm. The results demonstrate the capability of using FBG sensors to robotically steer needles.