ArticlePDF Available

Abstract

Introduction: Laparoscopic surgery often relies on a fixed Remote Center of Motion (RCM) for robot mobility control, which assumes that the patient’s abdominal walls are immobile. However, this assumption is inaccurate, especially in collaborative surgical environments. In this paper, we present a force-based strategy for the mobility of a robotic camera-holder system for laparoscopic surgery based on a pivoting motion. This strategy re-conceptualizes the conventional mobility control paradigm of surgical robotics. Methods: The proposed strategy involves direct control of the Tool Center Point’s (TCP) position and orientation without any constraints associated with the spatial position of the incision. It is based on pivoting motions to minimize contact forces between the abdominal walls and the laparoscope. The control directly relates the measured force and angular velocity of the laparoscope, resulting in the reallocation of the trocar, whose position becomes a consequence of the natural accommodation allowed by this pivoting. Results: The effectiveness and safety of the proposed control were evaluated through a series of experiments. The experiments showed that the control was able to minimize an external force of 9 N to ±0.2 N in 0.7 s and reduce it to 2 N in just 0.3 s. Furthermore, the camera was able to track a region of interest by displacing the TCP as desired, leveraging the strategy’s property that dynamically constrains its orientation. Discussion: The proposed control strategy has proven to be effective minimizing the risk caused by sudden high forces resulting from accidents and maintaining the field of view despite any movements in the surgical environment, such as physiological movements of the patient or undesired movements of other surgical instruments. This control strategy can be implemented for laparoscopic robots without mechanical RCMs, as well as commercial collaborative robots, thereby improving the safety of surgical interventions in collaborative environments.
Force-based control strategy for a
collaborative robotic camera
holder in laparoscopic surgery
using pivoting motion
Carlos Fontúrbel*, Ana Cisnal, Juan Carlos Fraile-Marinero and
Javier Pérez-Turiel
Escuela de Ingenierías Industriales, Medical Robotics Group, Instituto de las Tecnologías Avanzadas de la
Producción (ITAP), Universidad de Valladolid, Valladolid, Spain
Introduction: Laparoscopic surgery often relies on a xed Remote Center of
Motion (RCM) for robot mobility control, which assumes that the patients
abdominal walls are immobile. However, this assumption is inaccurate,
especially in collaborative surgical environments. In this paper, we present a
force-based strategy for the mobility of a robotic camera-holder system for
laparoscopic surgery based on a pivoting motion. This strategy re-
conceptualizes the conventional mobility control paradigm of surgical robotics.
Methods: The proposed strategy involves direct control of the Tool Center Points
(TCP) position and orientation without any constraints associated with the spatial
position of the incision. It is based on pivoting motions to minimize contact forces
between the abdominal walls and the laparoscope. The control directly relates the
measured force and angular velocity of the laparoscope, resulting in the
reallocation of the trocar, whose position becomes a consequence of the
natural accommodation allowed by this pivoting.
Results: The effectiveness and safety of the proposed control were evaluated
through a series of experiments. The experiments showed that the control was
able to minimize an external force of 9 N to ±0.2 N in 0.7 s and reduce it to 2 N in
just 0.3 s. Furthermore, the camera was able to track a region of interest by
displacing the TCP as desired, leveraging the strategys property that dynamically
constrains its orientation.
Discussion: The proposed control strategy has proven to be effective minimizing
the risk caused by sudden high forces resulting from accidents and maintaining
the eld of view despite any movements in the surgical environment, such as
physiological movements of the patient or undesired movements of other surgical
instruments. This control strategy can be implemented for laparoscopic robots
without mechanical RCMs, as well as commercial collaborative robots, thereby
improving the safety of surgical interventions in collaborative environments.
KEYWORDS
robotic surgery, laparoscopy, force control, collaborative robotics, admittance control
OPEN ACCESS
EDITED BY
Anzhu Gao,
Shanghai Jiao Tong University, China
REVIEWED BY
Gianni Borghesan,
Faculty of Engineering Sciences, KU
Leuven, Belgium
Hunter Gilbert,
Louisiana State University, United States
*CORRESPONDENCE
Carlos Fontúrbel,
carlos.fonturbel@uva.es
SPECIALTY SECTION
This article was submitted to Biomedical
Robotics,
a section of the journal
Frontiers in Robotics and AI
RECEIVED 15 January 2023
ACCEPTED 29 March 2023
PUBLISHED 17 April 2023
CITATION
Fontúrbel C, Cisnal A, Fraile-Marinero JC
and Pérez-Turiel J (2023), Force-based
control strategy for a collaborative
robotic camera holder in laparoscopic
surgery using pivoting motion.
Front. Robot. AI 10:1145265.
doi: 10.3389/frobt.2023.1145265
COPYRIGHT
© 2023 Fontúrbel, Cisnal, Fraile-Marinero
and Pérez-Turiel. This is an open-access
article distributed under the terms of the
Creative Commons Attribution License
(CC BY). The use, distribution or
reproduction in other forums is
permitted, provided the original author(s)
and the copyright owner(s) are credited
and that the original publication in this
journal is cited, in accordance with
accepted academic practice. No use,
distribution or reproduction is permitted
which does not comply with these terms.
Frontiers in Robotics and AI frontiersin.org01
TYPE Original Research
PUBLISHED 17 April 2023
DOI 10.3389/frobt.2023.1145265
1 Introduction
Laparoscopic surgery is a widely accepted and extended
technique in the current medical context. In this type of
interventions, small incisions are made in the patients abdominal
area. Through these open orices, laparoscopic forceps, needle
holders, laparoscope, and any other surgical tools are introduced.
This minimally invasive surgical technique is benecial for both
surgeons and patients, leading to a postoperative recovery with fewer
complications (Rau and Hünerbein, 2005).
The da Vinci is the market leader in robotic surgery. This robotic
system is capable of performing complete laparoscopic surgeries
under surgeons control. However, the large investment required in
infrastructure and its high cost make it unaffordable for a large
number of hospitals (Crew, 2020). One cost-effective solution is to
integrate partially automated surgical systems, such as robotic
assistants for performing individual tasks (Voros et al., 2010).
For example, the laparoscope handling could be automated,
mitigating human errors and saving resources, since it is
currently handled by a medical assistant. Robotic solutions for
assistance in laparoscopy surgery, such as laparoscope handling,
have been proposed in the literature since the end of the 20th
century (Hurteau et al., 1994), (Taylor et al., 1995). However, their
integration into such collaborative systems remains a challenge due
to the complexity of the humanrobot interaction (Haddadin, 2014).
In addition to the price, another limitation of the da Vinci is its
absence of force feedback, which makes it difcult to control the
contact forces at the incision.
Nowadays, the most widespread mobility method of robotic
systems in the context of laparoscopy surgery is based on a remote
centre of motion (RCM), coinciding with the entry point of the tool
through the incision. It simplies the development and control of
robotic laparoscopic assistants (Voros et al., 2010;Kuo et al., 2012;
Voros et al., 2010). There are strategies for the RCM implementation
based on the mechanical design, as in the da Vinci (Freschi et al.,
2013)orinAksungur (2015) and Zhou et al. (2018), or based on
control (Sandoval et al., 2021), where a virtual RCM is dened at the
start of the surgery. These robots that include the RCM as a central
feature of their design are based on the immobility of the entry point,
thus dening the mobility constraints of the robot.
The premise of the staticity of the RCM approach omits the actual
movement due to the physiological activity of the patient, such as the
patients abdomen ventilation, which causes movement at the incision
point (Valenza et al., 2010). Additionally, the trocars rest position varies
for each patient. It is mainly affected by the patients abdominal
elasticity coefcient, which depends on fat and other body
parameters. This cannot be taken lightly in collaborative
environments, where accidents can occur, or larger forces can be
exerted by the surgical instruments that displace the patients
abdomen. Control strategies have been proposed to reduce the force
exerted at the contact point between the trocar and the patient by
measuring this force. This approach was tested in the early 21st century
by Krupa et al. (2000), implementing what is now considered a mobile
RCM (Marinho et al., 2016). The results were not fully satisfactory due
to the low frequency of the force sensor, which made a truly effective
control infeasible. This may be the reason why these force-based control
strategies which ensures low contact forces are not widely extended in
real surgical settings nowadays. Due to the improvement in technology,
some hands-on robot approaches have appeared (Kastritsi and
Doulgeri, 2021;Kim et al., 2021). They use a force sensor to move
the robot to the desired position but always using the RCM as a
fundamental mobility restriction.
In contrast to the traditional or mobile RCM-based solutions, this
paper presents a mobility strategy based on pivoting the laparoscope to
minimize the efforts measured by a force/torque sensor located in the
coupling of the tool. The measured force is used to pivot the camera,
minimizing the contact force without losing control of the cameras
TCP. This approach allows to perform an automatic control of the
orientation of the tool, thus releasing the TCP control position. In this
way, a pivoting motion is achieved, similar to the proposal made by
Muñoz et al. (2005), with the advantages of having rigid joints and the
use of a commercial robot. It allows to keep the desired eld of view
(FoV) using modern mobility and camera tracking strategies as
proposed by Sandoval et al. (2021), with a greater degree of security
since the contact forces are minimized in the background. The proposed
control strategy has been tested for the handling of a laparoscope using a
6-DoF Universal Robots UR3e robot and a pelvitrainer. Trainee
surgeons often use pelvitrainersastheyallowthemtopractice
individual aspects of minimally invasive surgical procedures, such as
camera navigation or instrument handling, on a model rather than the
patient (Ackermann et al., 2022). Tests have been carried out with
different contact forces caused by the movement of a pelvitrainer or the
laparoscopes tip, thus simulating the patients physiological movement,
or sudden accidental movements. In general terms, the control is able to
minimize accidents in the operating room through rotations of the tool;
it reduces the exerted forces at the trocar without losing the control of
the laparoscopestip.
FIGURE 1
Four DoFs available using the RCM approach, which is coincident
with the trocar position.
Frontiers in Robotics and AI frontiersin.org02
Fontúrbel et al. 10.3389/frobt.2023.1145265
2 Materials and methods
2.1 Traditional RCM approach
The traditional RCM approach considers the incision as a static
point characterized by four DoFs: a translation along the axis of the
instrument and three rotations around this axis (Figure 1). The
surgical instrument penetrates the patients body through the
incision point. The 4-DoF mobility constraints featured by the
RCM generate a spherical working space for the TCP and can be
mechanically imposed or virtually dened (Dombre et al., 2013).
However, considering the incision as a xed point that coincides
with the RCM has some disadvantages. It requires a very precise
positioning of the RCM with the incision point during surgery to
avoid tissue damage (Dong and Morel, 2016) and also causes the
Cartesian position of the TCP to be controlled indirectly.
Marinho et al. (2016) reported that it was possible to generate
TCP linear movements through the implementation of a RCM by
controlling the 4 available DoFs without an error in the RCM
positioning. The primary goal of this approach is to keep the tool
static at the RCM, preventing the occurrence of contact forces.
Consequently, the TCP positioning is left as a secondary role in
RCM-based implementations (Kim et al., 2021).
The popularity of the RCM approach, which considers the
incision a static point, has limited the development of methods
for minimizing the forces exerted at the trocar due to physiological
movements of the patient or collaborative environments. Some
methods that dynamically modify the position of the RCM by
estimating the trocar position have been developed (Riviere et al.,
2006;Dong and Morel, 2016;Richa et al., 2011). This approach,
known as mobile RCM, causes difculties in the camera
repositioning as a result of the trocars movement during the
intervention. If the exact application point of the force is
unknown, the entire camera must perform a Cartesian motion in
order to reduce the contact force of the trocar (Marinho et al., 2016).
The RCM must coincide with the trocar to ensure that no force is
exerted on the patient, since the position of the TCP is conditioned
by the position of the RCM. To avoid this continuous movement,
relatively high force thresholds must be applied, which limits the
performance of this strategy. Hence, we considered that the mobile
RCM is not an ideal solution once the non-staticity of the trocar is
considered.
If the incision point is not considered static, the use of the
RCM approach is no longer justied. If it is possible to measure
the contact forces between the trocar and the patient, these forces
should be able to be controlled independently of the TCP
position, allowing precise independent control at the trocar
and at the tip of the camera.
2.2 Proposed pivoting motion control
To overcome the disadvantages of the traditional approach
based on a static RCM, a mobility strategy based on pivoting
motion is presented. It consists of varying the orientation of the
camera so as to minimize the contact forces with the abdominal
walls of the patient. It adapts the laparoscopes orientation to the real
location of the incision, or in other words, to the trocar position at
each moment. Since the laparoscope must keep track of the scenario
and should not interfere with other instruments or tissues, we can
infer that no other contact forces should affect the cameras
navigation. Therefore, an admittance control can determine the
angular velocity (pivoting movement) of the camera required to
minimize the contact forces, which are measured by a force/torque
(F/T) sensor located in the coupling of the tool. In this way, the
position of the TCP remains under direct control, while the pivoting
motion minimizes the forces on the trocar, which always tends to a
resting position.
This motion strategy is a restatement of the strategy proposed
by Krupa et al. (2000), which considered that the position of the
trocar varies over time. Although this control strategy was too
complex to implement at the time, it is a valid approach today due
to the improvements in sensorization in the last years. In this
article, the absolute tool velocity was determined at the height of
the trocar, displacing this pointhorizontallyasaresultofthe
force at the trocar. Krupas strategy was directly related to those
that implement a mobile RCM, which have not been particularly
fruitful in the literature. We consider that this approach,
although it could not be widely extended due to technological
deciencies, as mentioned in the article itself, can be simplied by
displacing the controlled point to the tip.
This force-based strategy allows to control the pivoting
motion without the need of the passive joint. Multiple
advantages associated with pivoting motion control have been
previously reported in the literature (Muñoz et al., 2005). These
advantages include, above all, repositioning the tool, in this case
the camera, without losing control of its tip. Thus, it is possible to
track the image even during the pivoting that may occur due to
contact between the camera and patient. This allows lowering the
threshold of admitted force before starting the reorientation, as
well as solving the problem with a less aggressive movement,
avoiding displacement of the entire camera and also through low
magnitude turns.
FIGURE 2
Pivoting movement as a consequence of forces exerted in the
direction of the Y-axis of {B}.
Frontiers in Robotics and AI frontiersin.org03
Fontúrbel et al. 10.3389/frobt.2023.1145265
In the following section, the admittance control will be
introduced. It generates the pivoting motion of the tool to
minimize the forces produced at the trocar. Second, the
feedforward control is described, which avoids the occurrence of
exerting forces due to the movement of the laparoscope. Third, the
mobility constraints associated with the proposed control are
addressed.
2.2.1 Admittance control
The mechanical admittance (Y) of a system is dened as the ratio
of its velocity (v) to its force (F), as expressed in Equation 1. This
denition is related to the second-order differential equation which
describes a mass, spring, and dumping serial system and hence, it is
characterized by a mass (m), a stiffness (k), and a viscous damping
(b) (Mihelj and Podobnik, 2012).
Ys
()v
F1
ms +b+k
s
.(1)
Theobjectiveofanadmittancecontrolistoshapethe
mechanical admittance of a device such that it possesses
desired characteristics. In this proposal, it is used to determine
the angular velocity that minimizes the interaction forces.
Figure 2 shows that the rotational velocity of the TCP that
characterizes the pivoting motion is related to the forces
exerted at the trocar. The exerted force FB
ywould be avoided
by a negative rotation in the X-axis of {B}. Analogously, if the
force is exerted in the X-axis, FB
x, it can be avoided by a rotation
around the Y-axis of {B}.
The mechanical admittance can be simplied so that it consists
of only one damping. Eq. 2describes this simplied admittance,
which provides good results in terms of robustness and control
accuracy despite its simplicity.
Ys
()v
F1
b.(2)
By this model, we can determine a relationship between force
and angular velocity that constrains the working space of the camera
around the X- and Y-axes, dened as ωB
Ax and ωB
Ax at Eqs 3,4,
respectively, where the sub-index A stands for admittance. The gains
applied to relate force and angular velocity, kAx and kAy , are
experimentally calculated and are dependent on the
environmental stiffness (Zeng and Hemami, 1997).
ωB
Ax kAxFB
y,(3)
ωB
Ay kAyFB
x.(4)
For its implementation, a F/T sensor placed at the laparoscopes
coupling is used to determine the applied force at the trocar, taking
{S} as a reference system (Figure 2). Note that to relate this force to
the velocity using the admittance control, both variables must refer
to the same system. The absolute reference system, which
corresponds to the base of the robot {B} (Figure 2), is used.
Hence, it is necessary to transform the force measurements to
the absolute reference system {B}.
The aforementioned angular velocities allow the
parameterization of the twist vector, (sB
A), which represents the
linear and angular velocity of the TCP as shown in Eq. 5. This
vector characterizes the pivoting motion of the tool that minimizes
the interaction forces at the trocar, constraining the tool orientation
and thus freeing the Cartesian control of the TCP.
sB
AvA,ωA
()
B

T0,0,0,ωB
Ax,ωB
Ay,0

T.(5)
Forces are used to calculate the pivoting motion of the camera.
Forces close to zero causes low pivoting speed, while higher order
forces provoke higher response speeds. This strategy favours that the
camera always tends to a resting position without aggressive
movements, and consequently the forces exerted on the patient
are minimized.
2.2.2 Feedforward orientation control
The vector calculated at the TCP by the admittance control (sB
A)
allows to minimize the forces exerted when the Cartesian velocity at
the tools tip is zero. However, it is also convenient to maintain low
contact forces once the camera is in motion to change the eld of
view. When there is a movement of the camera, the exclusive use of
the admittance control would require a too elevated velocity or a
high gain, causing oscillations and abrupt changes of direction, and
therefore high forces on the trocar. This problem has been
previously reported by Calanca et al. (2016) and may be solved
with the addition of a position control loop that prevents large forces
from being exerted, so a lower gain can be applied to the admittance
control, avoiding an oscillatory behaviour.
This position control loop can be implemented as a feedforward
(ff). Figure 3 shows that it is possible to geometrically determine
which rotation the tool should perform against the future Cartesian
velocity of the TCP. The objective of this control is to limit the
Cartesian velocity of the camera when Cartesian movement occur in
the immediate future. The ff predictive action complements the
admittance control, which minimizes the forces exerted in the
present by pivoting, displacing the tool to a rest position as a
consequence of the forces measured. Hence, this control ensures
FIGURE 3
An adequate rotation related to the Cartesian velocity vB
c
prevents exerting forces when the TCP of the laparoscope, or point
pB
TCP, is displaced.
Frontiers in Robotics and AI frontiersin.org04
Fontúrbel et al. 10.3389/frobt.2023.1145265
that the forces exerted are minimized without giving up the direct
control of the TCP position.
For the calculation of the angular velocity, the ff controller
needs to know the position of the camera point whose Cartesian
velocity would be reduced due to this control. This point will be
referred to hereinafter as the fulcrum position (pB
f), and it is the
projection of the contact point between the laparoscope and the
trocar at the laparoscopes longitudinal axis. Eqs 6,7allow the
calculation of the tools angular velocity at pB
TCP that keeps the
camera immobile at the point pB
f.dB
tisthedistancetravelled
between pB
TCP and pB
finthetimeperiodΔt.Intheexampleof
Figure 3,whendisplacingpB
TCP at a speed vB
y,theangleαof
rotation that prevents forces around the X-axisisthedifference
between the angle formed by the vector dB
tintheYZplaneatthe
current and future instances. Thiscalculationisperformedinthe
YZ and XZ planes to determine the rotation using the X-and
Y-axes, respectively, allowing to create the velocity vector of the ff
control sB
f,asshowninEq.8.
ωB
fx α
ΔtdB
ty +vB
y·Δt
dB
tz +vB
z·ΔtdB
ty
dB
tz

·1
Δt,(6)
ωB
fy − dB
tx +vB
x·Δt
dB
tz +vB
z·ΔtdB
tx
dB
tz

·1
Δt,(7)
sB
fvf,ωf

B

T
0,0,0,ωB
fx,ωB
fy,0

T.(8)
While the admittance control pivots the camera in response to
the contact force, the ff control calculates the angle that the
laparoscope must pivot in advance to maintain the fulcrum point
at a low speed. In this way, the ff controller reduces the contact forces
and ensures that the admittance control maintains its corrective
functionality. The angular velocity vector calculated by the ff is
added to the provided by the admittance control, thus allowing both
preventive and reactive orientation controls even if the TCP is
moving.
2.2.3 Movement in the released degrees of
freedom
The proposed force-based pivoting motion control liberates the
TCP movement, characterized by velocity vB
ix,v
B
iy,v
B
iz. Movement is
also released around the longitudinal axis of the laparoscope es
z
characterized by the rotation velocity wz, since this will not cause
any contact forces with the patient. This allows to transform the four
DoFs available in the RCM approach (Figure 1) into 3 DoFs of
Cartesian motion and 1 DoF of rotation (Figure 4).
The twist velocity vector sB
i, which denes the four DoFs released
at the TCP referred to the robot base{B}, is described in Eq. 7a.sB
iis
referred to the system {B}, and consequently, the Cartesian velocities
and the angular velocity must be also referred to{B}. Hence, the
angular velocity wzof the tool through its longitudinal axis eS
zis
transformed to the system {B} using the rotation matrix RB
S.
sB
ivi,ωi
()
B

TvB
ix,v
B
iy,v
B
iz,w
zeS
zRB
S

T. (7a)
The reference velocity, considering the four DoFs of the TCP,
can be dened without worrying about the exerted forces since they
would be minimized by the control.
2.3 Complete force-based pivoting motion
control strategy
The complete force-based pivoting motion control is shown in
Figure 5. This control strategy is the result of the sum of the
admittance and ff controls described previously. A PI controller
is also included, although it could be omitted in case the exact
position of the fulcrum point pB
fis known. The PI and the ff
controllers, shown with dashed lines in Figure 5, should only act
when the camera is in motion.
PI complements the predictive action of the ff when the position of
the fulcrum point pB
fis not accurate, since the ff control would not
completely eliminate the forces exerted on the trocar during the
movement. This would cause a stationary error during Cartesian
pB
TCP movement that the admittance control could not correct.
Therefore, the inclusion of the PI integral action compensates the
imperfection of the ff controller associated with a poor fulcrum point
estimation. Due to the dynamic nature of the system, where continuous
trocar displacements may occur due to physiological and camera
movements, we proposed an operation mode for this PI control
since an over-action would affect the quality of the control strategy.
The PI control is only active when the camera is in motion, and
therefore, it is congured to reset after each change of direction or stop
of pB
TCP. In addition, the PI control must be adjusted with a low gain and
an appropriate control frequency in such a way that past states do not
affect the present control, or in other words, so that it can be determined
whether the stationary error is still present and to prevent past forces
already compensated to affect the control. In the experimental results,
control parameters and its performance are analysed.
To sum up, interaction forces are reduced due to the pivoting
motion provided by the admittance control. Additionally, the ff
control anticipates the laparoscopes motion to prevent the
appearance of forces due to this motion, and the PI control
eliminates errors that have not been corrected in the immediate
past. For this purpose, the forces exerted at the trocar point, once
FIGURE 4
Four DoFs released at the TCP by the proposed force-based
pivoting motion control.
Frontiers in Robotics and AI frontiersin.org05
Fontúrbel et al. 10.3389/frobt.2023.1145265
gravity has been compensated, are inputted to the admittance
control and the PI, while the ff control requires the current and
desired (future) positions of the TCP (pB
TCP) and the current position
of the fulcrum (pB
f). Additionally, not only the force measurements
and orientation of the tool but also the weight of the tool is needed to
calculate the gravity-compensated force. As previously mentioned,
all control variables are expressed in the same reference system
associated with the robot base {B}. Hence, the gravity can be directly
compensated (Figure 6).
2.4 Setup platform
The setup platform used to test the performance of the presented
force-based control strategy is shown in Figure 7. The experimental
platform mainly consists of a 6-DoF UR3e robot (Universal Robots,
Denmark), a Hex-H F/T sensor (OnRobot A/S, Denmark), and a
SZABO-BERCI-SACKIER pelvitrainer (KARL STORZ SE & Co.
KG, Germany). The reference systems which are required by the
theoretical proposal {B} and {S} are located at the base of the UR3e
Robot and the F/T sensor, respectively. A new reference system {P} is
attached to the pelvitrainer (Figure 7).
The pelvitrainer recreates the mobility restrictions in
laparoscopic surgeries, due to the rigidity of its orices, which
FIGURE 5
Diagram of the force-based angular velocity control for the laparoscope. The dashed lines connect elements that are only activated by moving the
laparoscopes tip.
FIGURE 6
Forces measured on the trocar and their reference systems.
FIGURE 7
Experimental setup mainly consisting of the robot, F/T sensor,
and pelvitrainer.
Frontiers in Robotics and AI frontiersin.org06
Fontúrbel et al. 10.3389/frobt.2023.1145265
mimic those of a real abdomen. To test the safety of the proposed
control, a 3D-printed PLA rails were arranged for the pelvitrainer to
move along ^
pB
xaxes. In this way, it is possible to test the control
response to small movements that mimic the patients physiological
movement, as well as larger movements, which simulates accidents
that might happen in collaborative environments.
The F/T sensor, used to measure the interaction forces at the trocar,
has an accuracy of 0.2 N in the XY plane of {S}, and its maximum
frequency sample is 500 Hz. The surgical instrument is introduced
inside the abdomen through 10-mm and 12-mm ENDOPATH XCEL
®
trocars. A Storz Telecam One-Chip Camera Head 20212030 is used
which is integrated in a HOPKINS telescope 0°with 10 mm diameter.
This cylindrical device is inserted into the trocar and in turn into the
interior of the pelvitrainer. It has been coupled to the robot by means of
a rigid resin coupling tool of our own manufacture, in order to avoid
unwanted plastic deformation and inaccuracy in the positioning of
the TCP.
2.5 Control system implementation
The proposed control architecture is implemented using ROS
(Robot Operating System) and is illustrated in Figure 8. The green
blocks correspond to ROS nodes, being three main blocks: the
navigationcontrol,theUR3erobot,andtheHEXF/Tsensor.The
navigation control is subdivided into three specic tasks. The Cartesian
velocity control and the fulcrum point estimation (white blocks in
Figure 8) can be freely implemented and are not detailed in this
manuscript. For the experimental tests, the reference Cartesian
velocity has been dened according to the experimental goals.
Additionally, experimental results showed that it is not necessary to
provide the precise fulcrum location for the control strategy to work
correctly. In fact, a 5 cm height error provides an appropriate response,
which improves if the point is determined accurately.
The angular velocity control (orange block in Figure 8)istheone
responsible of controlling the TCPsangularvelocityunderthe
principles presented in this paper. A 4-point average lter is
applied to the force measurements to reduce the noise, resulting in
altered force with a frequency of 125 Hz. This ltered force
combined with the reference Cartesian velocity of the TCP,
fulcrum position, and tool orientation is used to calculate the
reference angular velocity by the angular velocity control.
The communication between the ROS nodes is carried out by
means of ROS topics, through the subscriberpublisher relationship
that they maintain. The SpeedL command, provided by Universal
Robots on 0F
1
URScript, is sent to the UR3e robot with the desired
velocity vector of the pB
TCP point, generated on the navigation
control block, as shown in Figure 8. This command requests the
robot every 8 ms to control the absolute velocity of a point in real
time in a uent way.
3 Results
The proposed force-based pivoting motion control has been
tested using the experimental setup shown in Figure 7. First, the
pivoting motion generated by the admittance control has been
evaluated as a method to control the forces exerted on the trocar.
Second, the effectiveness of the ff and PI together with the
admittance control has also been tested to avoid exerting
forces on the patient due to the Cartesian motion of the TCP.
Third, the performance of the proposed control has been tested
against errors in the estimation of the fulcrum position, to
validate the robustness of the control in real environments.
The detection and minimization of risks in the event of
accidents have been tested. Finally, the ability of the control to
maintain the FoV of the camera at a xed point during
pelvitrainer movements has also been evaluated.
FIGURE 8
ROS implementation diagram of the robot control.
Frontiers in Robotics and AI frontiersin.org07
Fontúrbel et al. 10.3389/frobt.2023.1145265
3.1 Admittance control
The rst experiment aimed to determinate the gain of the
admittance control by evaluating its response to the contact
forces produced between the laparoscope and the patient. The
patients abdomen was simulated by a pelvitrainer characterized
by a radial stiffness of 3 N/mm, which is a representative value of the
human tissue. In this experiment, the pelvitrainer was moved in the
^
pB
xdirection (Figure 7), resulting in a movement of 3 mm in the X-
and Y-axes of {B}. This displacement caused a constant force of 9 N
in each axis. The response of the system to this external force is
shown in Figure 9. The force on the X-axis is shown as a solid line,
while the force on the Y-axis is represented with a dashed line. The
gure shows that once the admittance control is activated, the
contact forces of 9 N are minimized by the pivoting motion of
the TCP.
The gains kAx and kAy of the admittance control are
experimentally determined. Both gains are equal since the system
should respond in the same manner on both axes. Therefore, three
experimental tests were carried out with the following gain values:
0.03, 0.06 and 0.075 radN
-1
s
-1
.
There is also a clear need to avoid high or low gains since the
system loses efciency in both cases. A high gain (blue line in
Figure 9) provokes undesired and unsafe oscillations, while a low
gain (green line in Figure 9) causes an overdamped response, which
increases the time response of the system. In the case of a gain of
0.03 radN
-1
s
-1
(orange line in Figure 9), the system stabilizes quickly,
not exceeding ±0.2 N again at 0.2 s after the over-peak and 0.7 s after
the start of the pivoting. It is noteworthy that, with this gain, the
system takes 0.32 s to reduce the exerted forces of 2 N, a threshold at
which the risk can be considered avoided.
It has been proved that the system provides an adequate
response to high forces and has therefore been validated to be
FIGURE 9
Response of the admittance control to a contact force of 9 N between the laparoscope and the pelvitrainer, characterized by a radial stiffness of
3 N/mm.
FIGURE 10
Linear movement performed to test the ff pivoting control for the
Cartesian movement of the TCP.
Frontiers in Robotics and AI frontiersin.org08
Fontúrbel et al. 10.3389/frobt.2023.1145265
effective in minimizing the contact forces produced by any
movement of the patient, including slight physiological
movements. The following experimental tests are carried out
with an admittance control determined in this experiment at
0.03 radN
-1
s
-
1. Note that the mechanical characteristics of the
pelvitrainer, such as the stiffness, may slightly differ from a
patients abdominal wall, so the gain might require some ne-
tuning.
3.2 Force-based pivoting motion control
The performance of the complete force-based pivoting
control to minimize the forces exerted by the camera during
its movement has also been evaluated. The admittance control in
combination with the action of the PI and ff controls is tested
against repetitive linear movements of the TCP between pB
TCP1
and pB
TCP2, producing the motion shown in Figure 10.The
displacement of the TCP is 3 cm in the X-andY-axes of {B},
without height variation, introducing the laparoscope to a depth
of 15 cm inside the pelvitrainer. In the following experiments, the
reference Cartesian velocity is 8mm/sinbothaxes,decreasing
linearly as the TCP approaches the target zone, to observe if there
were any changes in the behaviour at lower speeds. Figure 11
represents the reference Cartesian velocity in Y (black dashed
line) to show the relationship between the TCP velocity in Y-axis
andtheforceappearinginthisaxisinthedifferenttests.TheX
Cartesian velocity of the TCP has been omitted to simplify the
graph, as it is opposite to the Y velocity, producing an X force also
opposite to the Y force.
The rst test (green line in Figure 11) aimed to evaluate the
performance of the admittance control during these linear
movements of the TCP. Although the admittance controller
operates alone (neither ff nor PI has been implemented), the
system response is closed to be considered safe, narrowly
exceeding ±1 N as the maximum force. The contact force
decreases during deceleration of the TCP displacement,
indicating that once the ff is implemented, it will effectively
reduce the force exerted at the contact point, even if its position
is not accurately estimated.
The second test (orange line in Figure 11) evaluated the response
of the system to the TCP displacement, when the system is
composed of the admittance control and the PI control. The PI,
which requires current and past force measurements, is
characterized by a low integral gain of 0.00015 rad/(s·N) and a
frequency of 100 Hz. Note that the PI control has been activated only
during the TCP point. A low value of this gain allows to reduce part
of the stationary error during the Cartesian motion of the TCP,
while a high value produces oscillations. The inclusion of the PI
controller improves the response of the system without
implementing yet any kind of position control. Figure 11 (orange
line) shows that the forces exerted during the TCP displacement
never surpass ±0.9 N, and therefore, the control can be already
considered safe.
The last test (red line in Figure 11) was performed with the
complete angular velocity control, which consisted of the
FIGURE 11
Force at the trocar during Cartesian movement of the TCP by the combination of the different control algorithms. Green, admittance control;
orange, admittance control and PI; red, complete control strategy: admittance control, PI, and ff. Continuous lines represent forces at the X-axis, while
dashed lines show forces at the Y-axis. The dashed black line represents the TCP Cartesian speed at the Y-axis.
Frontiers in Robotics and AI frontiersin.org09
Fontúrbel et al. 10.3389/frobt.2023.1145265
combination of the admittance, PI, and ff controllers. The ff needs
the fulcrum position and the reference Cartesian velocity to provide
a predictive position control. In this experiment, the depth of the
fulcrum point was manually measured. During 95% of the test time,
the forces in the X- and Y-axes exerted by the camera during its
displacement were in the range [-0.13 N 0.13 N]. Therefore, the
predictive action of the ff reduces signicantly the force exerted
compared to the previous tests.
3.3 Force-based pivoting motion control
with uncertainty at the trocar
The performance of the complete angular velocity control has
been previously evaluated against forces exerted by the camera
motion. However, the fulcrum position, which is required by the
ff controller, may be uncertain or vary during surgery due to
physiological movements of the patient, such as breathing.
Hence, in this section, we aimed to repeat the previous test but
including a depth error in the fulcrum position.
Figure 12 shows the response of the system to repetitive linear
movement of the TCP (dashed black line) with a fulcrum height
error of 3 cm (orange line) and 5 cm (green line). The
effectiveness of the ff decreases compared to the analogue
results without the height error (Figure 11, red line). However,
the complete angular velocity control outperformed the
admittance and PI controls (Figure 11, orange line) even with
an erroneous estimation of the fulcrum position.
The predictive action of the angular velocity ff control limits the
speed of the laparoscope at the contact point, improving the effectiveness
of the admittance control in reducing the contact force caused by the
movement of the tool. In fact, the contact forces in the X-andY-axes are
reduced to a range [-0.6 N 0.6 N] even with a 5 cm height error
(Figure 12, green line). With a height error of 3 cm (Figure 12,
orange line), the exerted forces are within the [-0.4 N 0.4 N] range. If
the entry point for the laparoscope is taken as a pB
fz approximation, the
error would always be less than 3 cm which can be considered the most
unfavourable case.
Additionally, to test the performance of the control with less
contact information, a trocar with a clearance of 2 mm is used. The
system reacts more slowly than the previous tests (Figure 12, blue
line), as the trocar tolerance generates a less restrictive scenario. We
can therefore assure the performance of the control in various ranges
of possible situations and forces applied.
3.4 Test against accidents
IftheTCPofthecameraisimmobile,theadmittancecontrol
is the only control that is operative, and it will produce a pivoting
motion to minimize any contact force. If the TCP is moving, the ff
is activated so its predictive action reduces the movement at the
contact point, ensuring a good response of the admittance
control. It has been tested that the proposed force-based
pivoting motion control efciently reduces the contact forces
at any time.
FIGURE 12
System response to repetitive linear movement of the TCP in different uncertainty scenarios. Green, with a fulcrum height error of 5 cm; orange,
with a fulcrum height error of 3 cm; blue, with a trocar clearance of 2 mm. The dashed black line represents the TCP Cartesian speed.
Frontiers in Robotics and AI frontiersin.org10
Fontúrbel et al. 10.3389/frobt.2023.1145265
Additionally, a safety mode has been implemented to detect
force-related abnormal conditions and hence drastically reduce any
risk situation to the patient. If the force exceeds a predetermined
threshold, the safety mode immobilises the laparoscopes tip. The
threshold is set to 2 N in the horizontal plane, which is considered
the safe force limit. The reference Cartesian velocity of the tip is
automatically set to zero, while the admittance control would
reposition the camera by pivoting, thus minimizing the
interaction force.
Although for this implementation the safety mode immobilizes
the laparoscopes tip, another alternative would be to provide a
reference Cartesian velocity such that it counteracts the excessive
force. This alternative, although it requires a more complex
implementation of the Cartesian velocity control, would have a
faster response time since the admittance control is complemented
by this reference motion.
The following experimental test has been performed with the
safe mode that sets the reference speed to zero when the force
threshold is exceeded since it is the simplest and most
unfavourable case. The response of the system to a dangerous
situation is shown in Figure 13. At the beginning, the laparoscope
performed the linear movement of the previous tests at a depth of
15 cm. Then, an accident was recreated by making the UR3e
robot to displace the pelvitrainer in the ^
pB
xdirection of 1 cm at a
speed of 10 m/s in each X-andY-axis of {B}. Therefore, a peak
force of about 30 N should have appeared with the displacement
and the elastic constant of the pelvitrainer (3 N/mm). However,
the pivoting damped the impact force. Considering the stiffness
of the pelvitrainer, which is 3 N/mm, and the displacement in this
test, which is 10 mm, the pivoting movement helps to cushion the
initial impact. The peak force is reduced from an expected value
of 30 N to 10.34 N. After the initial peak, the force returns to a
safe value of 2N within 0.24 s. The over-peak occurred at 0.51 s
from the peak with a value of 0.7 N, a force that can be considered
completely safe. After 0.06 s, the force decreased to the
0.2 N range, where forces can be considered negligible. When
detecting excessive forces, the safe mode set the reference
Cartesian velocities to zero. After a considerable time from the
accident detection, the reference Cartesian velocity of the TCP
was reactivated.
3.5 Capability to maintain the eld of view
The aim of the proposed control strategy is to allow keeping the
FoV of the camera using the four DoFs while minimizing the exerted
forces. The laparoscope was inserted into the trocar up to a depth of
10 cm, and a target point (pB
r) was set for the camera to track during
the experiment. pB
rwas dened as the intersection between the initial
longitudinal direction of the laparoscope and the plane Z
B
=0.1 m
(Figure 14). The pelvitrainer was moved 5 cm at 3 mm/s in the X-
and Y-axes of {B}, which coincides with the ^
pB
xdirection (Figure 7).
The reference Cartesian velocity was set so that the laparoscopes
TCP moves to keep track of pB
r.Figure 14 shows how the camera
keeps track of the target point and hence maintaining the
desired FoV.
FIGURE 13
System response to an accident, simulated by a fast movement of the pelvitrainer. The dashed black line represents the TCP Cartesian speed, and the
green lines represent the contact forces.
Frontiers in Robotics and AI frontiersin.org11
Fontúrbel et al. 10.3389/frobt.2023.1145265
The force exerted at the trocar and the actual Cartesian
velocity of the TCP is displayed in Figure 15.Theforces
exerted were at a low risk level, not exceeding 1.1 N. During
the movement of the pelvitrainer, the camera also moved to keep
pointing to pB
r, while the force-based control generated the
pivoting motion to minimize the contact forces. Once the
movement of the pelvitrainer stopped (16.68 s), the
laparoscope kept moving to maintain the FoV. At this point,
forces decreased rapidly even while the laparoscope was still
displacing, since the PI and ff prevented exerting forces caused by
the laparoscopes tip movement.
4 Discussion
The proposed control strategy, which considers the trocar as a
non-static element, is closely related to the proposal by Krupa
et al. (2000) since both control approaches have been designed
under the same theoretical considerations. The experimental
results presented by Krupa showed that the control they
proposed reduced the contact forces at the trocar to ±2 N.
However, that force is relatively high since it is equal to the
maximum force limit that we have considered safe. These results
maybecausedbythelowrefresh rate of the F/T sensor used,
which was 50 Hz. Despite the later widespread use of the RCM
approach, we consider that the approach proposed by Krupa with
proper sensing technology has a longer run.
The experimental results presented in this article veried
that the developed strategy based on the theoretical proposal
using a current F/T sensor minimizes the contact forces and
hence makes the laparoscopy surgery safer. Among these
improvements, the positioning of the robot prior to the
intervention is facilitated, as less precision is required, and no
stationary forces are exerted on the patient. This is the reason
why we would like to rethink the use of force-based controls and
to highlight Krupaswork.
In addition to the technological gap, there is also a difference
in the mobility principles in both proposals. While Krupas
proposal and other mobile RCM-based solutions control the
position of the RCM, our proposal controls the velocity of the
tip of the camera, with the inherent advantage that there is no
need to know the precise position of the fulcrum. By the nature of
the pivoting motion, once the force is known, it is possible to
rotate the tip of the camera until the force disappears without
knowing the exact fulcrum position. Therefore, the pivoting
motion for repositioning the laparoscope is an appropriate
strategy to control the forces at the trocar.
The effectiveness of the proposed force-based pivoting
motion control has been veried when the tip of the camera,
or pivot point, is static. In the presence of an external force of 9 N,
the control takes 0.7 s to minimize the force to ±0.2 N and only
needs 0.32 s to reduce the force to 2 N. However, when the pivot
point is moved, forces are exerted on the patient. Therefore, a
predictive ff controller is activated when there is motion in order
to anticipate it and rotate the camera around its tip. In this way,
theforcesexertedbythetoolarelimitedinadvancefortheforce-
based control to function as effectively as it does when the tip
remains static.
Although the mobile RCM and our proposed pivoting motion
control strategies have a similar goal, the latter has the advantages
that there is no need either to determine the exact fulcrum
position or to displace the complete laparoscope to minimize
the contact forces (Marinho et al., 2016). A mobile RCM that is
designed to maintain control of the laparoscope tip would
respond with a pivoting motion. This is equivalent to the
control system proposed in this manuscript. Our approach
decouples the camera tip position control from the orientation
control that minimizes the contact forces in the background. This
makes it possible to not lose sight of the tracked area even when
physiological movements occur. This implies the possibility of
maintaining the FoV, which was also tested.
This specic mobility strategy for force-feedback robots is
ideal for camera handling, due to the precise control of the TCP
position. It allows to control where the camera is pointing at any
given moment without a precise method of determining the
trocar position. We foresee that the proposed mobility strategy
can be applicable to the robotic handling of other types of
laparoscopic tools. For the camera, only contact forces at the
trocar are exerted, but during the handling of other tools, tissue
FIGURE 14
Camera keeping track of the target point pB
rwhile the pelvitrainer
is in motion.
Frontiers in Robotics and AI frontiersin.org12
Fontúrbel et al. 10.3389/frobt.2023.1145265
contact forces also occur. Therefore, it will be necessary to
differentiate the contact forces on the trocar from those
applied on the tool. This has been discussed in the literature
as an inherent problem in this environment, forcing the
implementation of the mobility constraints of the RCM to
directly minimize the forces on the trocar. However, we
believe that if contact forces are distinguished, this mobility
strategy would maintain its characteristics and benets for tool
handling while allowing haptic feedback for the surgeon.
5 Conclusion
This paper presents a force-based pivoting motion control
strategy for a collaborative robotic system for laparoscopic
surgery. Its performance was evaluated in a set of experiments
that showed a drastic reduction of the contact force that the
instrument exerts to the abdominal walls, even in the presence of
motion. It can be therefore concluded that this approach ensures
safety during the laparoscopy surgery. Additionally, this force-based
control strategy has some advantages over the mobile RCM
approach: there is no need to know the exact position of the
trocar and liberates the motion of the camera tip. Therefore, the
camera can point to any location of interest during surgery since the
real environment constrains dynamically the camera orientation.
Itisimportanttopointoutthattheperformedexperiments
have been conceived as a proof of concept. They were designed to
evaluate the performance of the force-based control strategy for
minimizing forces both against patient movements and camera
movements. This allowed to dene the scope of the method, and
future experiments should be carried out based on scenarios
closer to real surgery conditions (i.e., replacing the pelvitrainer
with a dead or living animals and human body donors).
Data availability statement
The raw data supporting the conclusion of this article will be
made available by the authors, without undue reservation.
Author contributions
In this work, CF was responsible for the theoretical
development and experimentationofthestudy,aswellasfor
co-writing the manuscript. AC supervised and supported the
theoretical development as well as the experiments and co-wrote
the manuscript. JF provided intellectual guidance throughout the
process, performed the data analysis, co-wrote the manuscript
and approved the theoretical aspects dening the manuscript. JP-
T reviewed the paper to ensure its relevance and validity by
conducting literature research and validating its assumptions. All
authors read and corrected the intermediate manuscripts, and all
authors approved the nal manuscript.
FIGURE 15
Forces exerted at the trocar in blue and Cartesian velocity of the TCP in green and orange.
Frontiers in Robotics and AI frontiersin.org13
Fontúrbel et al. 10.3389/frobt.2023.1145265
Funding
This research was funded by the Spanish Ministry of Science
and Innovation through research project PID2019-111023RB-
C33 and the Regional Ministry of Education for the pre-doctoral
recruitment of research staff co-nanced by the European Social
Fund (ESF).
Acknowledgments
The authors would like to thank all participants that dedicated
their time and effort to contribute to this research, especially to Eva
de la Fuente and Alberto Jesús Fraile.
Conict of interest
The authors declare that the research was conducted in the
absence of any commercial or nancial relationships that could be
construed as a potential conict of interest.
Publishers note
All claims expressed in this article are solely those of the authors and
do not necessarily represent those of their afliated organizations, or
those of the publisher, the editors, and the reviewers. Any product that
may be evaluated in this article, or claim that may be made by its
manufacturer, is not guaranteed or endorsed by the publisher.
References
Ackermann, J., Pahls, J., Baumann, J., Spuntrup, C., Holthaus, B., Noe, G/., et al.
(2022). The pelvitrainer for training in laparoscopic surgery - a prospective, multicenter,
interdisciplinary study: Experimental research. Int. J. Surg. 101, 106604. doi:10.1016/J.
IJSU.2022.106604
Aksungur, S. (2015). Remote center of motion (RCM) mechanisms for surgical
operations. Int. J. Appl. Math. Electron. Comput. 3 (2), 119. doi:10.18100/ijamec.
84097
Calanca, A., Muradore, R., and Fiorini, P. (2016). A review of algorithms for
compliant control of stiff and xed-compliance robots. IEEE/ASME Trans.
Mechatronics 21 (2), 613624. doi:10.1109/TMECH.2015.2465849
Crew, B. (2020). Worth the cost? A closer look at the da Vinci robotsimpact
on prostate cancer surgery. Nature 580, S5S7. doi:10.1038/d41586-020-
01037-w
Dombre, E., Poignet, P., and Pierrot, F. (2013). Design of medical robots,in
Medical robotics (Netherlands: Wiley), 141176. doi:10.1002/
9781118562147.CH5
Dong, L., and Morel, G. (2016). Robust trocar detection and localization during robot-
assisted endoscopic surgery. Proc. IEEE Int. Conf. Robot. Autom. 2016, 41094114.
doi:10.1109/ICRA.2016.7487602
Freschi, C., Ferrari, V., Mel, F., Ferrari, M., Mosca, F., and Cuschieri, A. (2013).
Technical review of the da Vinci surgical telemanipulator. Int. J. Med. Robotics Comput.
Assisted Surg. 9 (4), 396406. doi:10.1002/rcs.1468
Haddadin, S. (2014). Towards the robotic Co-worker. Springer Tracts Adv. Robotics
90, 195215. doi:10.1007/978-3-642-40308-8_8
Hurteau, R., Desantis, S., Begin, E., and Gagner, M. (1994). Laparoscopic
surgery assisted by a robotic cameraman: Concept and experimental results.
IEEE Int. Conf. Robotics Automation 3, 22862289. doi:10.1109/ROBOT.1994.
350945
Kastritsi, T., and Doulgeri, Z. (2021). A controller to impose a RCM for hands-on
robotic-assisted minimally invasive surgery. IEEE Trans. Med. Robot. Bionics 3 (2),
392401. doi:10.1109/TMRB.2021.3077319
Kim, M., Zhang, Y., and Jin, S. (2021). Control strategy for direct teaching of non-
mechanical remote center motion of surgical assistant robot with force/torque sensor.
Appl. Sci. 11, 4279. doi:10.3390/APP11094279
Krupa, A., Doignon, C., Gangloff, J., de Mathelin, M., Solert, L., and Morel, G. (2000).
Towards semi-autonomy in laparoscopic surgery through vision and force feedback
control,in Experimental Robotics VII, Waikiki, Hawaii, USA, 11-13 December 2000
(Berlin: Springer), 189198. doi:10.1007/3-540-45118-8_20
Kuo, C. H., Dai, J. S., and Dasgupta, P. (2012). Kinematic design considerations for
minimally invasive surgical robots: An overview. Int. J. Med. Robotics Comput. Assisted
Surg. 8 (2), 127145. doi:10.1002/rcs.453
Marinho, M. M., Bernardes, M. C., and Bo, A. P. L. (2016). Using general-purpose
serial-link manipulators for laparoscopic surgery with moving remote center of motion.
J. Med. Robot. Res. 1 (4), 1650007. doi:10.1142/S2424905X16500070
Mihelj, M., and Podobnik, J. (2012). Haptics for virtual reality and teleoperation.
Dordrecht: Springer Netherlands. doi:10.1007/978-94-007-5718-9
Muñoz, V. F., Gómez-de-Gabriel, J. M., García-Morales, I., Fernández-Lozano, J., and
Morales, J. (2005). Pivoting motion control for a laparoscopic assistant robot and
human clinical trials. Adv. Robot. 19 (6), 694712. doi:10.1163/1568553054255664
Rau, B., and Hünerbein, M. (2005). Diagnostic laparoscopy: Indications and benets.
Langenbecks Arch. Surg. 390 (3), 187196. doi:10.1007/s00423-004-0483-x
Richa, R., Bó, A. P. L., and Poignet, P. (2011). Towards robust 3D visual tracking for
motion compensation in beating heart surgery. Med. Image Anal. 15 (3), 302315.
doi:10.1016/J.MEDIA.2010.12.002
Riviere, C. N., Gangloff, J., and De Mathelin, M. (2006). Robotic compensation of
biological motion to enhance surgical accuracy. Proc. IEEE 94 (9), 17051716. doi:10.
1109/JPROC.2006.880722
Sandoval, J., Laribi, M. A., Faure, J. P., Breque, C., Richer, J. P., and Zeghloul, S. (2021).
Towards an autonomous robot-assistant for laparoscopy using exteroceptive sensors:
Feasibility study and implementation. IEEE Robot. Autom. Lett. 6 (4), 64736480.
doi:10.1109/LRA.2021.3094644
Taylor, R. H., Funda, J., Eldridge, B., Gomory, S., Gruben, K., LaRose, D., et al. (1995).
A telerobotic assistant for laparoscopic surgery. IEEE Eng. Med. Biol. Mag. 14 (3),
279288. doi:10.1109/51.391776
Valenza, F., Chevallard, G., Fossali, T., Salice, V., Pizzocri, M., and Gattinoni, L.
(2010). Management of mechanical ventilation during laparoscopic surgery. Best Pract.
Res. Clin. Anaesthesiol. 24 (2), 227241. doi:10.1016/j.bpa.2010.02.002
Voros, S., Haber, G. P., Menudet, J. F., Long, J. A., and Cinquin, P. (2010). ViKY
robotic scope holder: Initial clinical experience and preliminary results using instrument
tracking. IEEE/ASME Trans. Mechatronics 15 (6), 879886. doi:10.1109/TMECH.2010.
2080683
Zeng, G., and Hemami, A. (1997). An overview of robot force control. Cambridge,
United Kingdom: Cambridge University Press.
Zhou, X., Zhang, H., Feng, M., Zhao, J., and Fu, Y. (2018). New remote centre of
motion mechanism for robot-assisted minimally invasive surgery. Biomed. Eng. Online
17 (1), 170. doi:10.1186/s12938-018-0601-6
Frontiers in Robotics and AI frontiersin.org14
Fontúrbel et al. 10.3389/frobt.2023.1145265
... This article presents a vision model for real-time tool localization, employing two Hourglass modules in series. Building on our previous work [24], which introduced a novel force-based control strategy utilizing pivoting motions instead of the regular remote center of the tool (RCM) for the camera holder, this paper aims to advance this research. Our ultimate goal is to develop a fully autonomous robotic camera holder that integrates this mobility approach with the proposed vision model. ...
... Sensors 2024,24, 4191 ...
Article
Full-text available
Partially automated robotic systems, such as camera holders, represent a pivotal step towards enhancing efficiency and precision in surgical procedures. Therefore, this paper introduces an approach for real-time tool localization in laparoscopy surgery using convolutional neural networks. The proposed model, based on two Hourglass modules in series, can localize up to two surgical tools simultaneously. This study utilized three datasets: the ITAP dataset, alongside two publicly available datasets, namely Atlas Dione and EndoVis Challenge. Three variations of the Hourglass-based models were proposed, with the best model achieving high accuracy (92.86%) and frame rates (27.64 FPS), suitable for integration into robotic systems. An evaluation on an independent test set yielded slightly lower accuracy, indicating limited generalizability. The model was further analyzed using the Grad-CAM technique to gain insights into its functionality. Overall, this work presents a promising solution for automating aspects of laparoscopic surgery, potentially enhancing surgical efficiency by reducing the need for manual endoscope manipulation.
... The proposed strategy uses both constraint-based and guideline-based potential fields on the basis of Guidance Virtual Fixtures and Forbidden-Region Virtual Fixtures [13]. This approach enables the integration of various real-world elements, making it compatible with scenarios involving surgical human-robot collaboration, an area that is largely unaddressed and still an open field [14]. The tests to be conducted are intended to verify the feasibility of performing an autonomous suture in a simplified and highly structured surgical scenario, in order to demonstrate the viability of using Potential Fields in this context. ...
Article
Full-text available
This paper presents a control strategy that secures both precision and manipulation sensitivity of remote center motion with direct teaching for a surgical assistant robot. Remote center motion is an essential function of conventional laparoscopic surgery, and the most intuitive way a surgeon manipulates a robot is through direct teaching. The surgical assistant robot must maintain the position of the insertion port in three-dimensional space during the four-degree-of-freedom motions such as pan, tilt, spin, and forward/backward. In addition, the robot should move smoothly when controlling it with the hands during the surgery. In this study, a six-degree-of-freedom collaborative robot performs the cone-shaped trajectory with pan and tilt motion of an end-effector keeping the position of the remote center. Instead of the bulky mechanically constrained remote center motion mechanism, a conventional collaborative robot is used to mimic the wrist movement of a scrub nurse. A force/torque sensor that is attached between the robot and end-effector estimates the surgeon’s intention. A direct teaching control strategy based on position control is applied to guarantee precise remote center position maintenance performance. A motion generation algorithm is designed to generate motion by utilizing a force/torque sensor value. The parameters of the motion generation algorithm are optimized so that the robot can be operated with uniform sensitivity in all directions. The precision of remote center motion and the torque required for direct teaching are analyzed through pan and tilt motion experiments.
Article
Full-text available
Background Robot-assisted minimally invasive surgery (RMIS) is promising for improving surgical accuracy and dexterity. As the end effector of the robotic arm, the remote centre of motion mechanism is one of the requisite terms for guaranteeing patient safety. The existing remote centre of motion mechanisms are complex and large in volume, as well as high assembly requirement and unsatisfactory precise. This paper aimed to present a new remote centre of motion mechanism for solving these problems. Methods A new mechanism based on the RMIS requirements is proposed for holding the laparoscope and generating a remote centre of motion for the laparoscope. The mechanism kinematics is then analysed from the perspective of the structural function, and its inverse kinematics is determined with a small number of calculations. Finally, the position deviation of the laparoscope rotational point is chosen as the index to evaluate the mechanism performance. The experiments are performed to test the deviation. Results The position deviations of the laparoscope rotational point do not exceed 2 mm, which is lower than that of the existing remote centre of motion mechanism. The 2 mm positioning error of the laparoscope won’t affect surgeon observation of the surgical field, and the pressure caused by the positioning error was acceptable for the skin elasticity. The proposed mechanism meets the RMIS requirement. Conclusions The proposed mechanism can achieve the remote centre of motion for the laparoscope. Its simple and compact structure is beneficial to avoid the collision of robotic arms, and it can be applied on other robots for providing the instrument necessary motion in minimally invasive surgery.
Article
Full-text available
Minimally invasive surgical systems are being widely used to aid operating rooms across the globe. Although arguably successful in laparoscopic surgery, the da Vinci robotic system has limitations mostly regarding cost and lack of patient physiological motion compensation. To obtain a more cost-effective alternative, earlier works used general-purpose fully actuated serial-link robots to control instruments in laparoscopic research using constrained Jacobian techniques. In contrast with those works, we present a new technique to solve the laparoscopic constraints for the serial-link manipulator by using a constrained trajectory. This novel technique allows complex 3D remote center-of-motion trajectories to be taken into account. Moreover, it does not have problems related to drifting, and is less prone to singularity related issues as it can be used with redundant manipulators. The proof-of-concept experiments are done by performing artificial trajectories with static and moving trocar points using a physical robot manipulator. Furthermore, the system is tested using user input of 13 medically untrained personnel in an endoscope navigation task. The experiments show that the system can be operated reliably under arbitrary and unpredictable user inputs.
Article
Full-text available
This survey presents the state of the art of basic compliant control algorithms in a unified view of past and present literature. Compliant control is fundamental when dealing with unstructured environments, as in the case of human-robot interaction. This is because it implicitly controls the energy transfer to the environment, providing a safe interaction. In this review, we analyze solutions from traditional robotics, usually involving stiff joints, and recent literature to find common control concepts and differences. To this aim, we bring back every schemas and relative mathematics formulation to a common and simplified scenario. Then, for each schema, we explain its intuitive meaning and report issues raised in the literature. We also propose an expansion of taxonomy to account for recent research.
Article
Background Minimally invasive surgical procedures have a flat learning curve, especially in the initial period of a surgeon's training. Pelvitrainers enable the prospective surgeon to drill the surgical technique, including camera navigation, instrument manipulation, and the individual steps of the operation, on a model rather than a patient. Integrating the pelvitrainer into standardized surgical training programs is challenging, but would be essential to achieve optimum effects of long duration in surgical education. Material and methods The pelvitrainer Realsimulator 2.0 (Endodevelop) was evaluated in 16 standardized training courses (at three training centers) of the German Working Group for Gynecological Endoscopy (AGE e.V.) from 2017 to 2020, The training concept was implemented and evaluated in a prospective, multicenter, interdisciplinary, multinational setting. Results One hundred and eighty of 206 physicians (response rate 87.4%) completed the multi-stage training concept successfully. A significant (p < 0.001) objective improvement (positive learning curve) was observed for all exercises on the pelvitrainer. The trainer's subjective evaluation revealed positive ratings for design (median 82%, IQR 71–91%), camera navigation (87%, IQR 76–95%), and instrument manipulation (median 87%, IQR 77–94%). A follow-up survey performed six months after the course confirmed its sustainable and high benefits in clinical routine (median 82%, IQR 70–97%). Conclusion The present investigation proved the high educational value of pelvitrainers, which can be enhanced by using a structured training concept. The benefit of training courses for health care justifies their incorporation into a standardized training curriculum. The completion of such training courses should be regarded as a prerequisite for a doctor to qualify as an independent surgeon.
Article
A new autonomous camera-holder robotic system for Laparoscopic Surgery (LS) is presented in this paper. In the proposed system, a 7-DoF collaborative robot holds and moves the laparoscope, or surgical camera, so that it permanently points towards the instruments tips. A motion capture (MoCap) system, tracking online the surgical instruments, provides the robots controller with instruments motion data. Unlike the existing systems, this paper proposes an autonomous robot-assistant, freeing up surgeons from the laparoscope motion control, fully responding to the surgical community requirements. Moreover, a fully compliance control law is implemented. In task-space, it is suitable for smoothing robot movements and reducing the contact forces generated at the trocar. Null-space is also exploited through a compliance control law restricting the elbows robot motion into a desired range, defined to avoid undesired collisions with the rest of the medical equipment. Robot Operating System (ROS) framework has been used to establish the communication between the robot and MoCap system, using the UDP protocol for data exchange. Training tasks, such as suturing training, have been performed by surgeons to validate the usefulness of the proposed platform, proving that a collaborative robot can autonomously comply with camera-holder assistant tasks.
Article
In Robotic-Assisted Minimally Invasive Surgery a long and thin instrument attached to the robotic arm enters the human body through a tiny incision. To ensure that no injury occurs when the surgeon is manipulating the instrument, the incision point must be a remote center of motion (RCM) for the instrument. For this purpose, a novel target admittance model is designed in the joint space for hands-on procedures that can be applied in all commercially available general-purpose manipulators with six or more degrees of freedom. It is proved that the joint reference trajectories generated by the proposed target admittance model under the exertion of a human force are stable and satisfy the RCM constraint. The measurements of the human force and the robot’s forward kinematic model are only required. Its use spans all hands-on surgical procedures. The proposed model can be easily extended to achieve additional objectives. Simulation results validate the theoretical findings and experimental results utilizing a KUKA LWR4+ demonstrate that trocar displacements are less than 1mm.
Article
The da Vinci robotic system has become the ubiquitous method for prostate removal, but its cost and long-term outcomes for patients are raising questions. The da Vinci robotic system has become the ubiquitous method for prostate removal, but its cost and long-term outcomes for patients are raising questions.
Article
Various human-friendly motion control methods were presented and analyzed. These are independently useful tools for numerous applications as they open up entirely new robot behaviors. However, due to their complex interrelationship in this chapter it is discussed how to integrate the presented methods into a more general hybrid state-based control architecture. Even though the focus is on robotic co-workers, the elaborated schemes are also applicable to service robots. The implementation of such a sensor-based robotic co-worker that brings robots closer to humans in industrial settings and achieve close cooperation is currently a challenging goal in robotics. Pioneering examples of intimate collaboration between human and robot, whose origin can be found in [10], are Intelligent Assist Devices (IADs), as the skill assist described in [18]. In 1983 a method was proposed at DLR for allowing immediate “programming by touch” of a robot through a force/torquesensor- ball [8], see Fig. 8.1 (left). Despite being a common vision in robotics the robotic co-worker has not become reality yet, as there are various open questions still to be answered. Apart from the control and safety aspects, the architectural level also poses significant challenges.