Feasibility check: can audio be a simple
alternative to force-based feedback for needle
Anonymous12[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗], Anonymous2 [∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗],
Anonymous2[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗], and Anonymous1[∗∗∗∗−∗∗∗∗−∗∗∗∗−∗∗∗∗]
Abstract. Accurate needle placement is highly relevant for puncture
of anatomical structures. The clinician’s experience and medical imag-
ing are essential to complete these procedures safely. However, imaging
may come with inaccuracies due to image artifacts. Sensor-based solu-
tions have been proposed for acquiring additional guidance information.
These sensors typically require to be embedded in the instrument tip,
leading to direct tissue contact, sterilization issues, and added device
complexity, risk, and cost. Recently, an audio-based technique has been
proposed for ”listening” to needle tip-tissue interactions by an externally
placed sensor. This technique has shown promising results for diﬀerent
applications. But the relation between the interaction event and the gen-
erated audio excitation is still not fully understood. This work aims to
study this relationship, using a force sensor as a reference, by relating
events and dynamical characteristics occurring in the audio signal with
those occurring in the force signal. We want to show that dynamical in-
formation that a well-known sensor as force can provide could also be
extracted from a low-cost and simple sensor such as audio. In this aim,
the Pearson coeﬃcient was used for signal-to-signal correlation between
extracted audio and force indicators. Also, an event-to-event correlation
between audio and force was performed by computing features from the
indicators. Results show high values of correlation between audio and
force indicators in the range of 0.53 to 0.72. These promising results
demonstrate the usability of audio sensing for tissue-tool interaction and
its potential to improve telemanipulated and robotic surgery in the fu-
Keywords: Audio guidance ·Force feedback ·Needle interventions.
Percutaneous needle insertion is one of the most common minimally invasive pro-
cedures. The experience of the clinician is an important requirement for accurate
placement of needles, given the reduced visual and tactile information transmit-
ted to the clinician via the instruments. Imaging techniques such as magnetic
2 Anonymous et al.
resonance, computed tomography, or ultrasound can support clinicians in this
type of procedure, but the accuracy can still not be fully assured because of
artifacts present in the images [20, 14].
Sensor-based solutions have been proposed for providing haptic feedback dur-
ing the procedure [11, 10, 18, 7, 8, 1, 5, 12, 23, 19, 3]. However, most of these solu-
tions require sophisticated sensors that sometimes need to be embedded in the
instrument tip or shaft, leading to direct contact with human organs, steril-
ization issues, the use of non-standard and quality-reduced tools, added device
complexity, risk and cost. This imposes serious design limitations, and therefore
they have encountered diﬃculties in being adopted for regular clinical use.
Recently, an audio-based technique has been proposed in  for listening to
the needle tip-tissue interaction dynamics using a sensor placed at the proximal
end of the tool. The authors of this work has shown promising preliminary results
for monitoring medical interventional devices such as needles , guide wires
, and laparoscopic tools . However, even if audio has proved to be a tool
with potential for providing guidance information such as tissue-tissue passage,
puncture and perforation events or palpation information, the generated audio
dynamics are still not fully understood.
The aim of this work is to investigate the audio dynamics generated from
needle-tissue interactions during needle insertion to have a better understand-
ing of the generation of the audio excitation using the audio-based guidance
technique. In this purpose, force is used as a reference since it has been widely
employed in the literature to understand interactions between needles and tis-
sue. The main idea is to relate events and dynamical characteristics extracted
from the audio signal with those extracted from the force signal through indica-
tors and event features computed by processing both signals. The audio signal
is processed by extracting its homomorphic envelope. Indicators related to local
event intensity, derivative, and curvature are computed from the force signal.
The Pearson coeﬃcient is used for signal-to-signal correlation between audio
and force indicators. Then, event-to-event correlation between audio and force
events is performed by computing features from the indicators.
Results show values of Pearson coeﬃcient between audio and force indicators
in the range of 0.53 to 0.72, being the highest one the correlation of audio
with force curvature. Additionally, events of high correlated indicators exhibit
a clear relationship that can be important for understanding audio behavior.
Both analyses show that audio, acquired non invasively with a simple and low-
cost sensor, can contain signiﬁcant information that can be used as additional
feedback to clinicians.
Needle insertion and its interaction with soft tissue has been widely studied
using force sensors, being possible to distinguish three phases of interaction [17,
2, 6]. During the ﬁrst phase or pre-puncture phase, the needle tip deforms the
surface in contact with the tissue producing an increase in the force. The second
Title Suppressed Due to Excessive Length 3
phase starts with the puncture event or tissue breakage, characterized by a peak
in the force, followed by a sharp decrease. The third phase corresponds to the
post-puncture phase, where the force can vary due to friction, collision with
interior structures, or due to the puncture of a new tissue boundary. During the
ﬁrst phase, when audio is acquired, no audio excitation occurs since there is no
tissue breakage or structure collision. The puncture during the second phase and
the collisions, friction, and new punctures during the second and third phases
can produce signiﬁcant and complex audio excitation dynamics. However, even
if an audio response is complicated, its dynamics should be related to dynamical
characteristics of the force during the second and third phases. This is what we
want to explore in this work.
Our aim is to extract characteristics or feature indicators from the force that
can be related to dynamical characteristics of the audio excitation. The ﬁrst
indicator that we want to explore is the local intensity of the force or detrended
force, which aims to emphasize the increase of force from a local deﬂection (con-
tact of needle tip with the tissue) passing through its peak (puncture event) and
coming back to a steady stage. We also believe that the cumulative energy stored
during the boundary displacement and the fast drop in force after the puncture
also inﬂuences the audio excitation, and this is why derivative and curvature
indicators are also extracted.
The idea of this work is not to explain mechanical properties and fundamen-
tals of needle insertion in soft tissue, but to demonstrate that characteristics of
audio and force, even resulting from sensors of entirely diﬀerent nature, can be
strongly related through a sort of transfer function between both sensor modal-
ities. Through this relationship, we also want to show the wealth of information
that an audio signal can contain concerning tip-tissue interaction dynamics.
Fig. 1 displays a block diagram with the main steps to relate audio and force
characteristics. First, the audio signal and the force signal are processed in order
to compute the diﬀerent indicators extracted for enhancing the signal features
that want to be compared: one audio indicator (IA), and four force indicators,
related to the local intensity (IFint ), to the curvature IFc), to the derivative
(IFd), and one indicator that integrates curvature and intensity (IFci ). Then,
a signal-to-signal correlation is performed between audio and force indicators in
order to assess similarity between features and also for optimizing the parameters
of the processing algorithms. Finally, an event-to-event correlation is performed
by computing features from the extracted indicators.
2.1 Audio indicator extraction
The audio signal is ﬁrst pre-processed using a 7th-order Butterworth bandpass
ﬁlter (3−6 KHz) in order to enhance the needle tip tissue interaction information,
as shown in .
Fig. 2 displays the ﬁltered audio signal together with the synchronized force
signal during needle insertion into soft tissue, using a recording from the dataset
presented in , which will be introduced in Section 3. It is possible to observe
from the three marked events (denoted by 1, 2 and 3) in Fig. 2(a) and (b) and
4 Anonymous et al.
( . )
Signal to signal
Fig. 1. General block diagram with the methodology to relate force and audio signals.
the zooms in Fig. 2(c), that a main puncture event, identiﬁable in the force as a
signiﬁcant peak, exhibits a complicated succession of events in the audio signal.
This set of events denotes the accumulation of energy through modulation of the
signal amplitude at the time interval just after the rupture of the tissue. To rep-
resent this energy event accumulation, we compute the homomorphic envelope
of the audio signal, which represents the amplitude modulation of the signal and
that it is obtained using homomorphic ﬁltering in the complex domain .
1 2 3 123
Fig. 2. (a) Bandpassed audio signal with labeled main puncture events. (b) force signal
with labeled main puncture events. (c) Audio zoom over the puncture events. (d)
Homomorphic envelope of the audio signal.
Title Suppressed Due to Excessive Length 5
2.2 Force processing for indicators extraction
As explained above, four indicators are extracted from the force signal by en-
hancing information related to characteristics such as intensity, derivative, and
curvature. The diagram of Fig. 1 shows the main steps for the computation of
the force indicators. The ﬁrst step that is common to all the indicators is the
force signal smoothing, consisting of the application of a moving average ﬁlter
in order to reduce the high-frequency ripple dynamics from the force.
The force indicator IFiintends to extract the information concerning the
local intensity of the force. In this aim, a force detrending has to be performed
to attenuate the very low-frequency progression present in the force signal during
needle insertion. For that, the signal baseline is ﬁrst estimated using a two-stage
median ﬁlter  that is then subtracted to the smoothed force signal. Finally,
the positive part of the resulting signal is extracted.
For the computation of the force indicator related to the derivative IFd, a
derivative ﬁlter in series with a smoothing ﬁlter is ﬁrst applied following .
To only keep the most important positive characteristics of the derivative, the
homomorphic envelope is extracted.
The curvature is estimated with the algorithm proposed in  that enhances
curvature and intensity in signals. A 2nd-degree polynomial is used to ﬁt the
force signal inside a sliding window. The 2nd-degree coeﬃcient is related to the
curvature of the signal. The force indicator IFcis ﬁnally computed by applying
a homomorphic envelope to the resulting curvature.
As explained in , an indicator that enhance intensity and curvature jointly
can be extracted by computing the product between the constant polynomial
coeﬃcient and the 2nd-degree coeﬃcient of the polynomial. IFci is computed as
the square root of the joint indicator.
Fig. 3 shows the four extracted indicators, including also the original force
signal and the baseline estimation (in red).
Force and baseline Curvature indicator Derivative indicator
Detrended force Curvature + intensity
Fig. 3. Four indicators extracted from the force signal.
2.3 Force and audio information correlation methodology
Two approaches are applied for putting in relation the indicators extracted from
the force with the one extracted from the audio. A signal-to-signal correlation
using the Pearson coeﬃcient is ﬁrst performed. This step allows assessing the
similarity between both types of indicators and also to optimize the parameters
6 Anonymous et al.
involved in the computation of the indicators. In fact, the extraction of the force
and audio indicators requires to set some parameters:
–The frequency cut-oﬀ of the low-pass ﬁlter to be used in the envelope extrac-
tion for the audio and force signals, denoted as lpfaand lpff, respectively.
–The length of the sliding window used for computing the polynomial ﬁtting
of the force signal, denoted as hwin.
–The ﬁrst and second stage averaging window length of the median ﬁlter
applied for the force detrending, denoted by L1and L2, respectively.
Each parameter is optimized by maximizing the Pearson coeﬃcient between the
force indicator and the audio indicator.
Using the optimized parameters, we perform an event-to-event correlation
analysis. Signiﬁcant puncture events in the force are ﬁrst detected using a stan-
dard peak detector algorithm. Then, a window Wis deﬁned around the detected
peak instant in the audio signal. Finally, for each detected event, we relate the
maximal value of the force indicator inside Wto the energy of the audio indica-
tor, also inside W.
For evaluating the presented approach, the dataset generated in  was used.
This dataset includes 80 audio recordings acquired during the insertion of an 18G
200 mm length biopsy needle into ex-vivo porcine tissue phantom. The insertion
was done automatically using a testing machine (Zwicki, Zwick GmbH & Co.KG,
Ulm) at an insertion velocity of 3 mm/s that also recorded the axial insertion
force. The audio frequency sampling was 44100 Hz, and one force sample was
acquired every 0.03 mm. If we assume the velocity nearly constant, the force
sampling frequency can be estimated at around 100 Hz.
Table 1 shows the optimized results of the correlation between the four force
indicators and the audio indicator. The table also shows the values of the pa-
rameters for the extraction of the indicators. In bold are marked the parameters
that required optimization (lpfa,L1,L2,hwin , and lpff). The span of the MA
smoother used for the computation of the force indicators was set to 10 samples,
equivalent to 0.1 seconds for a sampling frequency of 100 Hz. The last column of
the table shows the average Pearson coeﬃcient value (ρ) over the 80 recordings
of the dataset. It is possible to observe that the best correlation between audio
and force is obtained with the indicator related to the force curvature.
Fig. 4 displays a further analysis concerning the obtained correlation Pearson
coeﬃcients. In Fig. 4(a), the histograms of the Pearson coeﬃcients between the
four force indicators and the audio indicator are displayed. It is possible to verify
that the correlation values range in general between 0.3 and 0.9, but that for
curvature and the joint curvature and intensity indicators, 50% of the record-
ing has a correlation value over 0.6, which is high considering the completely
diﬀerent nature between audio and force sensors. Fig. 4(b), which shows the
accumulative histogram of the Pearson coeﬃcient, conﬁrms the analysis made
Title Suppressed Due to Excessive Length 7
Table 1. Average Pearson coeﬃcients for the correlation between the optimized four
force indicators and the optimized audio indicator for the 80 needle insertion recordings.
Comparison Audio Parameters Force parameter ρ
lpfaMAspan L1L2hwin lpff
IA vs I Fi4 10 100 100 n/a n/a 0.531
IA vs I Fc1 10 n/a n/a 7 1 0.717
IA vs I Fd1 10 n/a n/a n/a 1 0.664
IA vs I F ci 1 10 n/a n/a 13 1 0.672
previously. It is possible to observe that the best correlation between audio and
force is obtained with the curvature indicator IFc, followed by the joint curva-
ture and intensity indicator IFci. The derivative of the force IFdalso provides
high values of correlation, while the local intensity indicator I Fiis the weakest
indicator inﬂuencing the audio.
Fig. 5 shows examples of high correlations between each force indicator and
the audio indicator for diﬀerent recordings belonging to the dataset. It is possible
to see how the main dynamics extracted from the audio indicator can follow the
dynamics obtained from the force indicator, i.e., many of the information involve
in force it is somehow visible in the audio indicator signal.
Pearson coefficent Pearson coefficent
nb. of recordingf
nb. of recordingf
Fig. 4. Standard and cumulative histograms of the Pearson coeﬃcients of the four force
indicators with audio, for the 80 recordings.
Fig. 6 shows the results of the event-to-event correlation of the two best
correlations obtained in the signal-to-signal analysis: curvature IFcindicator
versus the audio indicator IA and the joint curvature and intensity indicator
IFci and IA. We explore two scenarios, the ﬁrst one (Fig. 6(a) and (b)) by taking
into account a large number of puncture events and the second one (Fig. 6(c) and
(d)) by taking into account only the most important puncture events occurring
during the needle insertion process. This is done by only modifying a simple
threshold in the peak detector; higher is the threshold more events will be taken
into account. For both force indicators, a clear event correlation can be observed
using both types of event thresholding. When a large number of events are
8 Anonymous et al.
10 15 20 25 10 15 20 25510 15 20510 15 20 255
Time [s] Time [s]Time [s]Time [s]
Fig. 5. Examples of force and audio indicators where high correlation were obtained.
taken into account, we can see that in the range of force events presenting low
intensities, it is not exactly clear that a high-intensity event will produce a high
audio excitation. However, when the number of events is reduced, it is possible
to observe an evident linear correlation between force and audio events, and this
is even more evident with IFci in Fig. 6(c).
a) b) c) d)
Fig. 6. Event-to-event correlation for (a) IFcvs. I A with a low intensity-event thresh-
old, (b) IFci vs. I A with a low intensity-event threshold, (c) I Fcvs. IA with a high
intensity-event threshold, and (d) IFci vs. IA with a high intensity-event threshold.
In this work, we explored the audio dynamics generated from the tip/tissue in-
teraction during needle insertion into soft tissue using a recently proposed audio
guidance technique. The main idea was to observe the eﬀect of diﬀerent char-
acteristics of force measurement on the audio excitation. We have shown that
information resulting from sensors of entirely diﬀerent nature, such audio and
force, can be strongly related during needle insertion. This suggests that audio
can contain valuable information for monitoring tip/tissue interaction dynamics.
Signal-to-signal and event-to-event correlation analysis showed that the curva-
ture of the force and the integration of force curvature and intensity strongly
aﬀect the behavior of the audio excitation. The obtained results are highly signif-
icant since they show that audio contains considerable information on the needle
insertion process, and that it can be a low-cost and simple complementary tool
for guiding needles.
Title Suppressed Due to Excessive Length 9
1. Abayazid, M., Kemp, M., Misra, S.: 3d ﬂexible needle steering in soft-tissue phan-
toms using ﬁber bragg grating sensors. In: 2013 IEEE International Conference on
Robotics and Automation. pp. 5843–5849. IEEE (2013)
2. Abolhassani, N., Patel, R., Moallem, M.: Needle insertion into soft tissue: A survey.
Medical engineering & physics 29(4), 413–431 (2007)
3. Chadda, R., Wismath, S., Hessinger, M., Sch¨afer, N., Schlaefer, A., Kupnik, M.:
Needle tip force sensor for medical applications. In: 2019 IEEE SENSORS. pp. 1–4.
4. Chen, C., S¨uhn, T., Kalmar, M., Maldonado, I., Wex, C., Croner, R., Boese,
A., Friebe, M., Illanes, A.: Texture diﬀerentiation using audio signal analysis
with robotic interventional instruments. Computers in biology and medicine 112,
5. Elayaperumal, S., Bae, J.H., Daniel, B.L., Cutkosky, M.R.: Detection of membrane
puncture with haptic feedback using a tip-force sensing needle. In: 2014 IEEE/RSJ
International Conference on Intelligent Robots and Systems. pp. 3975–3981. IEEE
6. van Gerwen, D.J., Dankelman, J., van den Dobbelsteen, J.J.: Needle–tissue inter-
action forces–a survey of experimental data. Medical engineering & physics 34(6),
7. Henken, K., Van Gerwen, D., Dankelman, J., Van Den Dobbelsteen, J.: Accuracy
of needle position measurements using ﬁber bragg gratings. Minimally Invasive
Therapy & Allied Technologies 21(6), 408–414 (2012)
8. Ho, S.C.M., Razavi, M., Nazeri, A., Song, G.: Fbg sensor for contact level moni-
toring and prediction of perforation in cardiac ablation. Sensors 12(1), 1002–1013
9. Illanes, A., Boese, A., Maldonado, I., Pashazadeh, A., Schauﬂer, A., Navab, N.,
Friebe, M.: Novel clinical device tracking and tissue event characterization using
proximally placed audio signal acquisition and processing. Scientiﬁc reports 8(1),
10. Iordachita, I., Sun, Z., Balicki, M., Kang, J.U., Phee, S.J., Handa, J., Gehlbach, P.,
Taylor, R.: A sub-millimetric, 0.25 mn resolution fully integrated ﬁber-optic force-
sensing tool for retinal microsurgery. International journal of computer assisted
radiology and surgery 4(4), 383–390 (2009)
11. Kalvøy, H., Frich, L., Grimnes, S., Martinsen, Ø.G., Hol, P.K., Stubhaug, A.:
Impedance-based tissue discrimination for needle guidance. Physiological measure-
ment 30(2), 129 (2009)
12. Kumar, S., Shrikanth, V., Amrutur, B., Asokan, S., Bobji, M.S.: Detecting stages
of needle penetration into tissues through force estimation at needle tip using ﬁber
bragg grating sensors. Journal of biomedical optics 21(12), 127009 (2016)
13. Laguna, P., Thakor, N.V., Caminal, P., Jane, R., Yoon, H.R., de Luna, A.B.,
Marti, V., Guindo, J.: New algorithm for qt interval analysis in 24-hour holter ecg:
performance and applications. Medical and Biological Engineering and Computing
28(1), 67–73 (1990)
14. Lal, H., Neyaz, Z., Nath, A., Borah, S.: Ct-guided percutaneous biopsy of intratho-
racic lesions. Korean journal of radiology 13(2), 210–226 (2012)
15. Mahmoodian, N., Schauﬂer, A., Pashazadeh, A., Boese, A., Friebe, M., Illanes,
A.: Proximal detection of guide wire perforation using feature extraction from
bispectral audio signal analysis combined with machine learning. Computers in
biology and medicine 107, 10–17 (2019)
10 Anonymous et al.
16. Manriquez, A.I., Zhang, Q.: An algorithm for qrs onset and oﬀset detection in single
lead electrocardiogram records. In: 2007 29th Annual International Conference of
the IEEE Engineering in Medicine and Biology Society. pp. 541–544. IEEE (2007)
17. Okamura, A.M., Simone, C., O’leary, M.D.: Force modeling for needle insertion
into soft tissue. IEEE transactions on biomedical engineering 51(10), 1707–1716
18. Park, Y.L., Elayaperumal, S., Daniel, B., Ryu, S.C., Shin, M., Savall, J., Black,
R.J., Moslehi, B., Cutkosky, M.R.: Real-time estimation of 3-d needle shape and
deﬂection for mri-guided interventions. IEEE/ASME Transactions On Mechatron-
ics 15(6), 906–915 (2010)
19. Ravali, G., Manivannan, M.: Haptic feedback in needle insertion modeling and
simulation. IEEE reviews in biomedical engineering 10, 63–77 (2017)
20. Reusz, G., Sarkany, P., Gal, J., Csomos, A.: Needle-related ultrasound artifacts
and their importance in anaesthetic practice. British journal of anaesthesia 112(5),
21. Rezek, I., Roberts, S.J.: Envelope extraction via complex homomorphic ﬁltering.
Technical Report TR-98-9 Technical report (1998)
22. Sameni, R., Shamsollahi, M.B., Jutten, C.: Model-based bayesian ﬁltering of car-
diac contaminants from biomedical recordings. Physiological Measurement 29(5),
23. Xu, R., Yurkewich, A., Patel, R.V.: Curvature, torsion, and force sensing in contin-
uum robots using helically wrapped fbg sensors. IEEE Robotics and Automation
Letters 1(2), 1052–1059 (2016)