Conference PaperPDF Available

Evolutionary Motion Control Optimization in Physical Human-Robot Interaction

  • Halodi Robotics

Abstract and Figures

Given that the success of an interaction task depends on the capability of the robot system to handle physical contact with its environment, pure motion control is often insufficient. This is especially true in the context of medical freehand ultrasound where the human body is a deformable surface and an unstructured environment, representing both a safety concern and a challenge for trajectory planning and control. The systematic tuning of practical high degree-of-freedom physical human-robot interaction (pHRI) tasks is not trivial and there are many parameters to be tuned. While traditional tuning is generally performed ad hoc and requires knowledge of the robot and environment dynamics, we propose a simple and effective online tuning framework using differential evolution (DE) to optimize the motion parameters for parallel force/impedance control in a pHRI and medical ultrasound motion application. Through real-world experiments with a KUKA LBR iiwa 7 R800 collaborative robot, the DE framework tuned motion control for optimal and safe trajectories along a human leg phantom. The optimization process was able to successfully reduce the mean absolute error of the motion contact force to 0.537N through the evolution of eight motion control parameters.
Content may be subject to copyright.
Evolutionary Motion Control Optimization in Physical Human-Robot
Nicholas A. Nadeau1,2and Ilian A. Bonev1
Abstract Given that the success of an interaction task
depends on the capability of the robot system to handle physical
contact with its environment, pure motion control is often insuf-
ficient. This is especially true in the context of medical freehand
ultrasound where the human body is a deformable surface and
an unstructured environment, representing both a safety con-
cern and a challenge for trajectory planning and control. The
systematic tuning of practical high degree-of-freedom physical
human-robot interaction (pHRI) tasks is not trivial and there
are many parameters to be tuned. While traditional tuning is
generally performed ad hoc and requires knowledge of the robot
and environment dynamics, we propose a simple and effective
online tuning framework using differential evolution (DE) to
optimize the motion parameters for parallel force/impedance
control in a pHRI and medical ultrasound motion application.
Through real-world experiments with a KUKA LBR iiwa 7
R800 collaborative robot, the DE framework tuned motion
control for optimal and safe trajectories along a human leg
phantom. The optimization process was able to successfully
reduce the mean absolute error of the motion contact force
to 0.537 N through the evolution of eight motion control
Collaborative industrial robots are increasingly being
used outside of traditional manufacturing where physical
human-robot interaction (pHRI) plays an important role in
the application. Given that the success of an interaction task
depends on the capability of the robot system to handle phys-
ical contact with its environment, pure motion control is often
insufficient. This is especially true in the context of medical
freehand ultrasound where the human body is a deformable
surface and an unstructured environment, representing both
a safety concern and a challenge for trajectory planning and
Within the research domain of robot-assisted freehand
ultrasound, previous studies have focused on ergonomic ben-
efits [1], image quality optimization [2] and reconstruction
[3], pHRI safety [4], and application design [5]. Regardless
of the objective, force control techniques are necessary
to achieve effective and adaptable behaviour of a robotic
system in the unstructured ultrasound environment while also
ensuring safe pHRI. However, while force control does not
require explicit knowledge of the environment, to achieve an
acceptable dynamic behaviour, the control parameters (e.g.,
gain, stiffness, damping) must be tuned.
The performance of a force control strategy is greatly
affected by the tuned parameters. Past studies have explored
This work was funded by the Canada Research Chairs Program.
Ecole de technologie sup´
erieure, Montr´
eal, Qu´
ebec, Canada
2Corresponding author,
Fig. 1: The motion task coordinate system. The primary goal
was to maintain a constant normal force (fz) with the probe
while performing a linear motion along a human leg phantom
(xaxis direction).
effective pHRI control and tuning including, iterative tuning
[6], adaptive control [7]–[9], fuzzy learning [10], impact
collision modelling [11], and passivity-based control [12].
Regardless of the approach, the tuning is generally performed
on an ad hoc basis [13] or requires knowledge of the robot
and environment dynamics [14].
As noted in [6], the systematic tuning of practical high
degree-of-freedom (DOF) robotic tasks is not trivial; there
are many parameters to be tuned simultaneously and heuris-
tically, in addition to the coupling dynamics that must be
considered. Moreover, the authors of [15] point out that
the choice of control parameters depends on the application
and involves a compromise between interaction forces and
position accuracy. Within the context of medical ultrasound,
different human body locations have different stiffnesses [16]
and thus will require different tunings.
Given these challenges, we present an online optimiza-
tion framework for parallel force/impedance control using
differential evolution (DE) [17] in the context of pHRI and
medical ultrasound. While previous research has explored the
integration of DE in robot control, such as simulated control
tuning of a PUMA-560 [18] or PID tuning in a five-bar
mechanism [19], there is a lack of knowledge with regards to
the potential use of evolutionary algorithms in pHRI motion
control. The current study takes inspiration from [20]–[22]
and leverages task repetition to tune motion control for
optimal and safe trajectories along a human leg phantom. In
our approach, DE can enable the robot system to explore the
large parameter domain for candidate tuning solutions using
metaheuristics without any assumptions about the problem
space or environment. Further, a primary advantage of the
framework is the preservation of a simple control structure,
as no additional complexities have been added (i.e., fuzzy
logic, neural networks).
The rest of the paper is organized as follows. First,
Section II explains the problem of controlled motions along
a deformable surface. Second, Section III describes the
proposed optimization method and implementation of DE.
Third, Section IV outlines the experiment and the validation
of the framework. Finally, Section V concludes the paper
and discusses potential future work.
The context of medical freehand ultrasound on a human
leg is the basis of this study with the primary goal of
maintaining a constant normal force with the probe, as
compression effects would distort the resulting images [23].
In the ideal application, robot-assisted ultrasound would
allow for a sonographer to collaborate with the robot
through hand-guidance while the robot performs the repet-
itive force-based motion tasks. Fundamentally, this appli-
cation is a surface following operation on an unstructured
and deformable body, a classical exercise in robotics and
force control (in-depth surveys may be seen in [24] and
[25]). Furthermore, given the pHRI context of this study,
impedance control is an essential component to ensure safe
motion and to compensate for the unstructured environment
In this paper we combine force and impedance control in
a parallel control model. As shown in [27], parallel control
strategies allow for control in the full-dimensional space
without selection matrices. Parallel control loops override
one another given a prioritization policy. For this task, the
impedance control loop allows for safety and environment
compensation, giving compliance to the tool frame in all
Cartesian axes. At a higher priority level, task-orientated
force control allows for constant force-tracking along the
Thus, given an initial contact with the surface, the task
can be achieved by assigning the following constraints [15]:
a non-zero linear velocity along the xaxis
zero linear velocity along the yaxis
zero angular velocity about the x,y, and zaxes
a non-zero force along the zaxis
Fig. 2: The high-level steps of differential evolution (DE):
(1) A population of candidate solution vectors is initialized;
(2) Mutations are generated for each candidate; (3) The
mutations are crossed with the original candidates; (4) The
fitness of the crossed candidates is scored; (5) The population
is updated based on the scored fitness. Steps 2-5 are repeated
until a convergence criterion is met.
where the above motion task constraints and coordinate
system are illustrated in Fig. 1.
DE, first presented in [17] and reviewed in [28], is a simple
yet powerful evolutionary algorithm for global optimization,
being able to minimize nonlinear and non-differentiable
continuous space functions. Typical direct search optimiza-
tion methods (e.g., Nelder–Mead) use a greedy criterion
to select newly generated parameters. Under this criterion,
the new parameters are only kept if it reduces the fitness
function value, risking convergence to local minima. In
contrast, DE maintains a population of candidate solutions
and creates new solutions through the strategic and stochastic
combination of existing ones, allowing the full (but bounded)
solution space to be continuously explored. This is especially
important with regards to robotics, as the solution space
fitness of a high DOF motion task is assumed to be noisy and
is not guaranteed to be differentiable. An illustrated overview
of DE may be seen in Fig. 2 with the specific implementation
shown in Algorithm 1.
The DE solver for the present study was implemented
using Python v3.6.1 and SciPy v1.0.0 on an external PC
as a client to the robot controller over a local network. As
optimization of the solver itself was outside the scope of
the current study, the solver hyperparameters were set using
the recommendations of both [17] and the SciPy package.
Notably, the population size was set to 5×D, where Dis the
dimension of the input vector. The mutation constant was set
in the range of [0.5,1], whereby dithering randomly changes
this constant every generation, helping to speed convergence.
The recombination constant (crossover probability) was set
to 0.7. The population was initialized using Latin hypercube
sampling (LHS) to maximize coverage of the available
parameter space. Finally, the best1bin evolution strategy
was selected, where for each mutation, two candidates of
the population are randomly chosen, and their difference is
used to mutate the best candidate into a mutant vector. The
crossover process then combines the given candidate with
the mutant, based on a binomial distribution. The resulting
trial vector is to be evaluated by a robot motion session,
generating a fitness score. A detailed overview of this process
is seen in Algorithm 1.
Algorithm 1 The Differential Evolution (DE) solver process.
1: // initialize solver
2: CR 0.7// crossover probability
3: DgetN umP arameters() // number of parameters
4: populationSiz e 5×D
5: population initializeP opul ation(populationSize)
7: // each loop represents a single generation
8: loop
9: // get a random mutation within the given range
10: Frandom(0.5,1)
11: for all candidate population do
12: // ‘best1bin‘ mutation
13: r0population.getRandomC andidate()
14: r1population.getRandomC andidate()
15: r0,1r0r1
16: bc population.getBestCandidate()
17: mutantCandidate bc +F×r0,1
19: // ‘best1bin‘ crossover
20: trialC andidate candidate
21: for ito Ddo
22: cr random(0,1)
23: if cr < CR then
24: trialC andidate[i]mutantCandidate[i]
25: end if
26: end for
27: // one random parameter is always from the mutant
28: irandomInt(0, D)
29: trialC andidate[i]mutantCandidate[i]
31: // perform a session with the trial candidate
32: // this blocks until the motion session is complete
33: result perf ormSession(trialCandidate)
35: // evalute the fitness and update the population
36: fitnesstrial evaluateF itness(result)
37: if fitnesstrial < f itnesscandidate then
38: candidate trialC andidate
39: end if
40: end for
41: end loop
The proposed optimization framework was evaluated with
a 7-DOF KUKA LBR iiwa 7 R800 (LBR) collaborative
robot and the experiment setup may be seen in Fig. 3.
The LBR robot was equipped with a rounded probe tool
that was designed to approximate the tool referenced in
Annex A of [16]. In addition, the robot is equipped with
torque sensors at every joint, allowing for calculations of
Cartesian forces acting on the tool. The mass data of the
tool assembly (0.948 kg) was calibrated using the built-in
Fig. 3: The experiment setup. A KUKA LBR iiwa 7 R800
(LBR) collaborative robot (A) was equipped with a rounded
probe tool (B). A mannequin leg with an internal skeletal
structure (C) was used to simulate the ultrasound subject and
water-based lubricant was used on the leg to fully simulate
real ultrasound examination and reduce friction.
calibration procedure of the robot to remove the tool load
from the force measurements.
A mannequin leg with an internal skeletal structure was
used to simulate the ultrasound subject and water-based
lubricant was used on the leg to fully simulate real ultrasound
examination and reduce friction. The desired contact force
for the motion was derived from a review of previous
robot-assisted ultrasound studies. Forces between 15 N,
3.74.6 N, and 520 N were reported in [4], [2], and [29],
respectively. As such, a setpoint force of 5 N was selected.
As discussed in Section II, parallel force/impedance con-
trol was selected as the control strategy and was implemented
as two layers. First, the control architecture exploited the
built-in real-time impedance control of the LBR robot as the
base control strategy (KUKA Sunrise.OS v1.10). The basic
model is a virtual spring-damper system with configurable
values for stiffness and damping, allowing the LBR to
be highly sensitive and compliant (Fig. 4). Second, the
high-priority task-oriented force control was implemented
through the Sunrise.Connectivity DirectServo control loop
interface. This allows for non-deterministic soft real-time
control and fast corrections to the robot path.
Each motion session begins with the DE solver process
providing a candidate Session State (Protocol Buffers v3.5.1)
to the robot controller through a remote procedure call
interface (gRPC v1.9.0). The Session State input vector
defines the parameters that form the optimization problem
Fig. 4: The built-in real-time impedance control of the
KUKA LBR iiwa robot (LBR). The model is a virtual
spring-damper system with configurable values for stiffness
(K) and damping (ζ), allowing the LBR to be highly
sensitive and compliant. The resulting interaction forces are
calculated from the displacement (X) between the desired
position (Pd) and the actual position (Pa) in each Cartesian
and is given by:
SessionState =
where kx,y,z and ζx,y,z are the Cartesian stiffnesses
(05000 N/m) and Lehr’s damping ratios (0.11 1), respec-
tively, for the LBR impedance control. dxdefines the xaxis
position displacement per servo iteration (0.110 mm) and
gzdefines the gain of the force-tracking (0.110 1). The
bounds of the impedance control parameters are hard limits
set by the robot while the remaining bounds were selected ex-
perimentally. Furthermore, these parameters remain constant
for the duration of a single session. Finally, the stiffness
and damping of the rotation axes were set to a constant
300 N m/rad and 0.7, respectively, as they were not part
of the optimization process.
Following session initialization, the robot performed a
linear downwards motion until the force condition was
triggered at which point the servo loop was initiated for the
ultrasound motion. A motion session is judged as a success
after 100 mm of travel in the xaxis direction from the initial
contact point (visualized in Fig. 1). A timeout of 3000 servo
iterations was also implemented to avoid endless loops
(successful sessions took an average of 1129 iterations).
Throughout the session, the zaxis force (fz) is logged at
1000 Hz.
Following the completion of a motion session, the robot
controller updated the external client with a Session Result,
consisting of the recorded fzdata and the session success
status. The DE solver used the Mean Absolute Error (MAE)
of the fzdata with respect to the setpoint force as a fitness
score. The first second of data was discarded to avoid edge
effects with respect to the initial contact event (highlighted
in Fig. 6). However, if a timeout occurred, the session
was penalized with a score of infinity. Iteratively, motion
sessions were performed, evaluating candidate control pa-
rameter solutions as generated by the DE solver with the
parameters evolving over time. A detailed description of the
motion optimization process and framework is specified in
Algorithm 2.
Algorithm 2 The motion control session.
1: // receive candidate vector from DE solver
2: LBR SessionState
4: // initialize session
5: numServoIterations 0
6: maxServoIterations 3000
7: fz,d ← −5// Newtons, desired force
8: xd100 // millimeters, desired travel distance
9: ¯
P0getC urrentC artesianP osition()
11: // initialize internal impedance control layer
12: LBRkx,ky,kzSessionStatekx,ky,kz
13: LBRkrx,r y,rz 300 // Nm/rad
14: LBRζxyzSessionStateζxyz
15: LBRζrx,r y,rz 0.7// Lehr’s damping ratio
17: isServoRunning T rue
18: while isServoRunning do
19: numServoIterations numServoIterations + 1
20: if numServoIterations > maxS ervoI terations or
21: isServoRunning F alse
22: else
23: // force control layer
24: ¯
FigetF orceV ector()
25: fzfz,d ¯
26: zfz
27: zz×SessionS tategz
29: // calculate next servo position
30: ¯
PigetC urrentC artesianP osition()
31: ¯
Px,i+1 ¯
Px,i +SessionS tatedx
32: ¯
Py,i+1 ¯
33: ¯
Pz,i+1 ¯
Pz,i + ∆z
34: ¯
Prx,ry,r z,i+1 ¯
Prx,ry,r z,0
35: setDestination(¯
36: end if
37: end while
A. Results
A total of 795 sessions were performed, representing
nearly 20 generations of candidate solutions, with 709 suc-
cessful sessions (89.2 %). The optimization process was
terminated due to the fitness score leveling off, as clearly
seen in Fig. 5. The mean servo loop period was 3.12 ms
(320.49 Hz), on par with the responsive ultrasound force
control reported in [2].
Comparisons of motion behaviour throughout the process
is shown in Figs. 6 and 7. The poor performing sessions
noticeably suffer from strong oscillations or completely miss
the desired contact force, as compared to stable and accurate
optimized motion. It should be noted that the optimization
of the initial contact event was outside the scope of this
study, as this transient event would require its own set
of motion control parameters, thus resulting in the force
overshoots seen outside the shaded area in Fig. 6. In an actual
robot-assisted application, the initial contact would most
likely be achieved through the sonographer hand-guiding the
Over the evolution of the candidate parameters, corre-
lations were found between three key parameters (ζz,gz,
and kz), the fitness score, and the session index. The raw
parameter evolution may be seen in Fig. 8a and Fig. 8b.
These correlations give insight into the evolutionary process
of the DE solver as it generates candidate solutions. It is also
interesting to note how even at later indexes the DE solver
still explores the full parameter domain and does not simply
converge to local parameter minima.
Finally, the evolution of the best candidates is visualized
in Fig. 9 and the optimal parameters are summarized in
Table I. While hundreds of motion sessions were performed,
the MAE fitness reached 0.607 N after only 28 sessions
and the optimal value of 0.537 N after 689 sessions. Close
examination of the evolution in Fig. 9 shows relatively
wide parameter exploration in sessions prior to session
28 and relative stabilization afterwards. The trend towards
high z-direction rigidity (Fig. 8) suggests the application
favours stronger position control, but the results in Table I
demonstrate that the optimal parameters were not simply
at maximum. Moreover, Fig. 7 clearly demonstrates that a
simple fixed z-depth motion would not be a valid approach
to this application.
The systematic tuning of practical high DOF robotic tasks
is not trivial and there are many parameters to be tuned, espe-
cially in the context of pHRI force control. While traditional
tuning is generally performed ad hoc and requires knowledge
of the robot and environment dynamics, we propose a simple
and effective online framework using DE to optimize the
motion parameters for parallel force/impedance control in a
medical ultrasound motion application.
Through real-world experiments with a KUKA LBR iiwa
7 R800 collaborative robot, probe tool, and a mannequin
leg with an internal skeletal structure, the DE framework
was able to successfully reduce the MAE of the motion
0 100 200 300 400 500 600 700
Session Index
Fitness Score
Session Index
Best Fitness Score
Fig. 5: Raw fitness scores and linear regression (top) and
best fitness scores (bottom) vs. session index.
Force [N]
0 1 2 3 4 5
Time [s]
Force [N]
0 1 2 3 4 5
Time [s]
(a) First few sessions.
0.0 0.5 1.0 1.5 2.0 2.5 3.0
Time [s]
Force [N]
(b) Best session.
Fig. 6: Comparisons of the probe contact force, fzthrough-
out the differential evolution optimization process. The
shaded region highlights the motion data used to score the
candidate fitness.
0.0 0.5 1.0 1.5 2.0 2.5 3.0
Time [s]
Z Position [mm]
Best Session
Worst Session
Fig. 7: Probe z-position of the best and worst sessions of the
differential evolution optimization process.
Lehr'S Damping
Zeta Z
0 100 200 300 400 500 600 700
Session Index
Stiffness [N/m]
Lehr'S Damping
Zeta Z
Fitness Score
Stiffness [N/m]
Fig. 8: Evolution of key parameters with the optimization
process (a) and the fitness score (b). Linear regressions are
represented by the red lines.
1 2 6 13 28 181 392 587 622 625 689
Session Index
Zeta X
Zeta Y
Zeta Z
Normalized Value
Fig. 9: Evolution of the motion control parameters in the
best candidates. The columns represent the parameters of
best candidate up to a given session. Each parameter row is
normalized over its evolution.
TABLE I: Parameters and fitness of the best differential
evolution (DE) candidate. The fitness is calculated as the
mean absolute error (MAE) between the desired contact force
and the recorded force through a motion session.
Parameter Value
dx8.865 mm
kx3703.233 N/m
ky2777.930 N/m
kz4885.149 N/m
MAE 0.537 N
task to 0.537 N through the optimization and evolution of
eight motion control parameters. Although real-world fitness
function evaluations are a relatively expensive process, the
total optimization time was just 55 min for 20 generations,
well under the 50 generation benchmark set in [18].
This simplicity of the framework has three key benefits.
First, inferring the motion parameters from the application
is not trivial, especially since the human leg has varying
stiffness [16]. Second, it can easily be used to extend and
improve existing control schemes through a simple com-
munication network between the DE solver and the robot.
Third, it allows for the optimization process to be well
understood by the user, unlike black-box methods such as
neural networks. As this approach is model-based, the motion
parameters may be easily updated with respect to changing
environments (e.g., change in leg orientation), whereas a
black-box model would have difficulty adapting.
Consequently, while the mannequin leg may not perfectly
represent a human subject, this framework can be adopted as
a starting point for more advanced control paradigms, such as
reinforcement learning. Future motion control architectures
for pHRI and intelligent systems will likely rely on forms
of continuous learning, highlighting the importance of an
optimization framework that allows the robot system to
explore, evolve, and evaluate the solution space for more
effective motion behaviour.
As stability analysis was outside the scope of the current
study, it provides a foundation for future work. Other po-
tential extensions of this study include the optimization of
the DE solver, the exploration of new motion parameters
and control architectures, and the development of smooth
transitions between initial contact and surface following
motion control.
[1] S. E. Salcudean, G. Bell, S. Bachmann, W. H. Zhu, P. Abolmaesumi,
and P. D. Lawrence, “Robot-assisted diagnostic ultrasound – design
and feasibility experiments,” in Medical Image Computing and
Computer-Assisted Intervention – MICCAI’99. Springer Berlin
Heidelberg, 1999, pp. 1062–1071. [Online]. Available: https:
// 115
[2] P. Chatelain, A. Krupa, and N. Navab, “Confidence-driven control
of an ultrasound probe,” IEEE Transactions on Robotics, vol. 33,
no. 6, pp. 1410–1424, dec 2017. [Online]. Available: https:
[3] S. Merouche, L. Allard, E. Montagnon, G. Soulez, P. Bigras, and
G. Cloutier, “A robotic ultrasound scanner for automatic vessel
tracking and three-dimensional reconstruction of b-mode images,”
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency
Control, vol. 63, no. 1, pp. 35–46, jan 2016. [Online]. Available:
[4] F. Pierrot, E. Dombre, E. D´
egoulange, L. Urbain, P. Caron,
S. Boudet, J. Gari´
epy, and J.-L. M´
egnien, “Hippocrate: a safe robot
arm for medical applications with force feedback,” Medical Image
Analysis, vol. 3, no. 3, pp. 285–300, sep 1999. [Online]. Available:
[5] K. Mathiassen, J. E. Fjellin, K. Glette, P. K. Hol, and O. J. Elle,
“An ultrasound robotic system using the commercial robot UR5,
Frontiers in Robotics and AI, vol. 3, feb 2016. [Online]. Available:
[6] W. Yu, X. Li, and R. Carmona, “A novel PID tuning method
for robot control,” Industrial Robot: An International Journal,
vol. 40, no. 6, pp. 574–582, oct 2013. [Online]. Available: 406
[7] B. Alqaudi, H. Modares, I. Ranatunga, S. M. Tousif, F. L.
Lewis, and D. O. Popa, “Model reference adaptive impedance
control for physical human-robot interaction,” Control Theory and
Technology, vol. 14, no. 1, pp. 68–82, feb 2016. [Online]. Available:
[8] S. Jung, T. C. Hsia, and R. G. Bonitz, “Force tracking impedance
control of robot manipulators under unknown environment,IEEE
Transactions on Control Systems Technology, vol. 12, no. 3, pp.
474–483, May 2004.
[9] E. Gribovskaya, A. Kheddar, and A. Billard, “Motion learning and
adaptive impedance for robot control during physical interaction
with humans,” in 2011 IEEE International Conference on Robotics
and Automation. IEEE, may 2011. [Online]. Available: https:
[10] Z. Du, W. Wang, Z. Yan, W. Dong, and W. Wang, “Variable
admittance control based on fuzzy reinforcement learning for
minimally invasive surgery manipulator,Sensors, vol. 17, no. 12, p.
844, apr 2017. [Online]. Available:
[11] L. Roveda, N. Iannacci, F. Vicentini, N. Pedrocchi, F. Braghin, and
L. M. Tosatti, “Optimal impedance force-tracking control design
with impact formulation for interaction tasks,” IEEE Robotics and
Automation Letters, vol. 1, no. 1, pp. 130–136, jan 2016. [Online].
[12] A. Albu-Sch¨
affer, C. Ott, and G. Hirzinger, “A unified passivity-based
control framework for position, torque and impedance control
of flexible joint robots,” The International Journal of Robotics
Research, vol. 26, no. 1, pp. 23–39, jan 2007. [Online]. Available:
[13] H. G. Sage, M. F. D. Mathelin, and E. Ostertag, “Robust control
of robot manipulators: A survey,International Journal of Control,
vol. 72, no. 16, pp. 1498–1522, jan 1999. [Online]. Available:
[14] R. Kelly, “A tuning procedure for stable PID control of robot manip-
ulators,” Robotica, vol. 13, no. 2, p. 141–148, 1995.
[15] L. Villani and J. De Schutter, Force Control. Berlin, Heidelberg:
Springer Berlin Heidelberg, 2008, pp. 161–185. [Online]. Available: 8
[16] ISO/TS 15066:2016, “Robots and robotic devices – Collaborative
robots,” International Organization for Standardization, Tech. Rep.,
[17] R. Storn and K. Price, “Differential evolution – a simple and efficient
heuristic for global optimization over continuous spaces,Journal
of Global Optimization, vol. 11, no. 4, pp. 341–359, Dec 1997.
[Online]. Available:
[18] Q. Zheng, D. Simon, H. Richter, and Z. Gao, “Differential particle
swarm evolution for robot control tuning,” in 2014 American Control
Conference, June 2014, pp. 5276–5281.
[19] M. G. Villarreal-Cervantes and J. Alvarez-Gallegos, “Off-line PID
control tuning for a planar parallel robot using DE variants,
Expert Systems with Applications, vol. 64, pp. 444 – 454, 2016.
[Online]. Available:
[20] J. Buchli, F. Stulp, E. Theodorou, and S. Schaal, “Learning
variable impedance control,The International Journal of Robotics
Research, vol. 30, no. 7, pp. 820–833, apr 2011. [Online]. Available:
[21] T. Inoue, G. D. Magistris, A. Munawar, T. Yokoya, and R. Tachibana,
“Deep reinforcement learning for high precision assembly tasks,” in
2017 IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS), Sept 2017, pp. 819–825.
[22] D. Jeon and M. Tomizuka, “Learning hybrid force and position
control of robot manipulators,” IEEE Transactions on Robotics and
Automation, vol. 9, no. 4, pp. 423–431, 1993. [Online]. Available:
[23] M. W. Gilbertson and B. W. Anthony, “Force and position control
system for freehand ultrasound,” IEEE Transactions on Robotics,
vol. 31, no. 4, pp. 835–849, aug 2015. [Online]. Available:
[24] D. E. Whitney, “Historical perspective and state of the art in robot
force control,” The International Journal of Robotics Research, vol. 6,
no. 1, pp. 3–14, 1987.
[25] B. Siciliano and L. Villani, Robot force control. Springer Science &
Business Media, 2012, vol. 540.
[26] F. Ficuciello, L. Villani, and B. Siciliano, “Variable impedance
control of redundant manipulators for intuitive human–robot physical
interaction,” IEEE Transactions on Robotics, vol. 31, no. 4, pp.
850–863, aug 2015. [Online]. Available:
[27] S. Chiaverini and L. Sciavicco, “The parallel approach to
force/position control of robotic manipulators,” IEEE Transactions on
Robotics and Automation, vol. 9, no. 4, pp. 361–373, 1993. [Online].
[28] S. Das and P. N. Suganthan, “Differential evolution: A survey of the
state-of-the-art,” IEEE Transactions on Evolutionary Computation,
vol. 15, no. 1, pp. 4–31, feb 2011. [Online]. Available: https:
[29] F. Courreges, P. Vieyres, and G. Poisson, “Robotized tele-echography,
in Teleradiology. Springer Berlin Heidelberg, 2008, pp. 139–154.
[Online]. Available: 13
... The main challenge of evolutionary optimization is the slow convergence due to the many trials the system must run, especially when working with hardware-in-the-loop optimization [16], [17]. Furthermore, an unconstrained optimization may employ parameters that could lead to instability, especially when dealing with humans. ...
... Lahr et al. [13] optimized an admittance controller for industrial robots along one DOF in an experimental setup, optimizing in an experimental design taking into account several nonlinearities. Nadeau and Bonev [16] used an evolutionary algorithm to optimize the impedance controller during a human-machine task with multiple DOFs using an industrial robot. In contrast, Lahr et al. [17] optimized an admittance controller for an industrial robot with three-DOF for an assembly task using dimensionality reduction for faster convergence. ...
Full-text available
The field of physical human-robot interaction has dramatically evolved in the last decades. As a result, the robotic system's requirements have become more challenging, including personalized behavior for different tasks and users. Various machine learning techniques have been proposed to give the robot such adaptability features. This paper proposes a model-based evolutionary optimization algorithm to tune the apparent impedance of a wrist rehabilitation device. We used passivity to define boundaries for the possible controller outcomes, limiting the shared autonomy of the robot and ensuring the coupled system stability. The experiment consists of a hardware-in-the-loop optimization and a one-degree-of-freedom robot used for wrist rehabilitation. Experimental results showed that the proposed technique could generate customized passive impedance controllers for three subjects. Furthermore, when compared with a constant impedance controller, the method suggested decreased in 20\% the root mean square of interaction torques while maintaining stability during optimization.
... This results in the capability to easily calibrate a robot model and forms the foundation of the research presented in Nadeau, Bonev, & Joubair (2019). Furthermore, real-time robot optimization applications, an example of which is presented in Nadeau & Bonev (2018), can be augmented with machine learning through the vectorial interface of Pybotics and the Scikit-learn framework (Pedregosa et al., 2011). ...
Full-text available
Conference Paper
High precision assembly of mechanical parts requires accuracy exceeding the robot precision. Conventional part mating methods used in the current manufacturing requires tedious tuning of numerous parameters before deployment. We show how the robot can successfully perform a tight clearance peg-in-hole task through training a recurrent neural network with reinforcement learning. In addition to saving the manual effort, the proposed technique also shows robustness against position and angle errors for the peg-in-hole task. The neural network learns to take the optimal action by observing the robot sensors to estimate the system state. The advantages of our proposed method is validated experimentally on a 7-axis articulated robot arm.
Full-text available
In order to get natural and intuitive physical interaction in the pose adjustment of the minimally invasive surgery manipulator, a hybrid variable admittance model based on Fuzzy Sarsa(λ)-learning is proposed in this paper. The proposed model provides continuous variable virtual damping to the admittance controller to respond to human intentions, and it effectively enhances the comfort level during the task execution by modifying the generated virtual damping dynamically. A fuzzy partition defined over the state space is used to capture the characteristics of the operator in physical human-robot interaction. For the purpose of maximizing the performance index in the long run, according to the identification of the current state input, the virtual damping compensations are determined by a trained strategy which can be learned through the experience generated from interaction with humans, and the influence caused by humans and the changing dynamics in the robot are also considered in the learning process. To evaluate the performance of the proposed model, some comparative experiments in joint space are conducted on our experimental minimally invasive surgical manipulator.
Full-text available
Optimization methods have shown to be a very important approach for control engineers. They emulate the decision-making ability of a human expert to tune the control gains for a process or system with the formulation and solution of a mathematical optimization problem. In such formulation, evolutionary algorithms (EAs) have been widely used to obtained the control gains. Nevertheless a bad selection of the control gains through the optimization process can result in instability of the closed-loop control system such that the convergence and diversity in the EAs can be compromised. In this paper the PID control tuning for a planar parallel robot with a five-bar mechanism to follow a highly nonlinear trajectory is stated as an off-line nonlinear dynamic optimization problem (OLNLDOP). In order to promote individuals with a stable behavior in the closed-loop control system, a dynamic constraint and a method to handle such constraint is proposed into the OLNLDOP and into eight different variants of the differential evolution algorithm, respectively. Comparative analysis shows that the proposal finds suitable solutions for the OLNLDOP with a better convergence time. Laboratory testing with the optimum PID control gains on a real prototype validates the tuning optimization method.
Full-text available
The use of robots in health care has increased dramatically over the last decade. One area of research has been to use robots to conduct ultrasound examinations, either controlled by a physician or autonomously. This paper examines the possibility of using the commercial robot UR5 from Universal Robots to make a tele-operated robotic ultrasound system. Physicians diagnosing patients using ultrasound probes are prone to repetitive strain injuries, as they are required to hold the probe in uncomfortable positions and exert significant static force. The main application for the system is to relieve the physician of this strain by letting the them control a robot that holds the probe. A set of requirements for the system is derived from the state-of-the-art systems found in the research literature. The system is developed through a low-level interface for the robot, effectively building a new software framework for controlling it. Compliance force control and forward flow haptic control of the robot was implemented. Experiments are conducted to quantify the performance of the two control schemes. The force control is estimated to have a bandwidth of 16.6 Hz, while the haptic control is estimated to have a bandwidth of 65.4 Hz for the position control of the slave and 13.4 Hz for the force control of the master. Overall, the system meets the derived requirements and the main conclusion is that it is feasible to use the UR5 robot for robotic ultrasound applications.
Full-text available
The letter presents a force-tracking impedance controller granting a free-overshoots contact force (mandatory performance for many critical interaction tasks such as polishing) for partially unknown interacting environments (such as leather or hard-fragile materials). As in many applications, the robot has to gently approach the target environment (whose position is usually not well-known), then execute the interaction task. Therefore, the algorithm has been designed to deal with both the free space approaching motion (phase a.) and the succeeding contact task (phase b.) without switching from different control logics. Control gains have to be properly calculated for each phase in order to achieve the target force tracking performance (i.e., free-overshoots contact force). In detail, phase a. control gains are optimized based on the impact collision model to minimize the force error during the following contact task, while phase b. control gains are analytically calculated based on the solution of the LQR optimal control problem. The analytical solution grants the continuous adaptation of the control gains during the contact phase on the estimated value of the environment stiffness (obtained through an on-line extended Kalman filter). A probing task has been carried out to validate the performance of the control with partially unknown contact environment properties. Results show the avoidance of force overshoots and instabilities.
We present a control framework for optimizing image quality during robotic ultrasound acquisitions. The quality of the ultrasound signal across the field of view is represented by a confidence map that is computed online from the B-mode frames, following a model of sound propagation. Moments extracted from this confidence map are used to design a control law for optimizing imaging quality, based on the task function approach. The proposed confidence control is combined with force and position control to build two illustrative applications. First, we use force control and confidence control in order to maintain a correct pressure and a good orientation of the probe during teleoperation. Thus, control is shared between a human operator and the robot. Then, we add an automatic positioning task, so that the quality is optimized while maintaining a target in the image center. We show experimentally that confidence-driven control can effectively optimize the acoustic window in real time. In addition, we show that it can improve the tracking robustness, by preventing the target from being shadowed. Finally, we present the results of experiments performed on a human volunteer.
A fundamental requirement for the success of a manipulation task is the capability to handle the physical contact between a robot and the environment. Pure motion control turns out to be inadequate because the unavoidable modeling errors and uncertainties may cause a rise of the contact force, ultimately leading to an unstable behavior during the interaction, especially in the presence of rigid environments. Force feedback and force control becomes mandatory to achieve a robust and versatile behavior of a robotic system in poorly structured environments as well as safe and dependable operation in the presence of humans. This chapter starts from the analysis of indirect force control strategies, conceived to keep the contact forces limited by ensuring a suitable compliant behavior to the end effector, without requiring an accurate model of the environment. Then the problem of interaction tasks modeling is analyzed, considering both the case of a rigid environment and the case of a compliant environment. For the specification of an interaction task, natural constraints set by the task geometry and artificial constraints set by the control strategy are established, with respect to suitable task frames. This formulation is the essential premise to the synthesis of hybrid force/motion control schemes.
This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.
Locating and evaluating the length and severity of a stenosis is very important for planning adequate treatment of peripheral arterial disease (PAD). Conventional ultrasound (US) examination cannot provide maps of entire lower limb arteries in 3D. We propose a prototype 3D-US robotic system with B-mode images, which is non-ionizing, non-invasive, and is able to track and reconstruct a continuous segment of the lower limb arterial tree between the groin and the knee. From an initialized cross-sectional view of the vessel, automatic tracking was conducted followed by 3D-US reconstructions evaluated using Hausdorff distance, cross-sectional area and stenosis severity in comparison with 3D reconstructions with computed tomography angiography (CTA). A mean Hausdorff distance of 0.97 ± 0.46 mm was found in vitro for 3D-US compared with 3D-CTA vessel representations. To evaluate the stenosis severity in vitro, 3D-US reconstructions gave errors of 3%-6% when compared with designed dimensions of the phantom, which are comparable to 3D-CTA reconstructions, with 4-13% errors. The in vivo system's feasibility to reconstruct a normal femoral artery segment of a volunteer was also investigated. These results encourage further ergonomic developments to increase the robot's capacity to represent lower limb vessels in the clinical context.
A hand-held force-controlled ultrasound probe has been developed for medical imaging applications. The probe–patient contact force can be held constant to improve image stability, swept through a range, or cycled. The mechanical portion of the device consists of a ball screw linear actuator driven by a servo motor, along with a load cell, accelerometer, and limit switches. The performance of the system was assessed in terms of the frequency response to simulated sonographer hand motion and in hand-held image feature tracking during simulated patient motion. The system was found to attenuate contact force variation by 97% at 0.1 Hz, 83% at 1 Hz, and 33% 10 Hz, a range that spans the typical human hand tremor frequency spectrum. In studies with 15 human operators, the device applied the target contact force with ten times less variation than in conventional ultrasound imaging. An ergonomic, human-in-the-loop, imaging-workflow enhancing control scheme, which combines both force- and position-control, permits smooth making and breaking of probe–patient contact, and helps the operator keep the probe centered within its range of motion. By controlling ultrasound probe contact force and consequently the amount of tissue deformation, the system enhances the repeatability, usability, and diagnostic capabilities of ultrasound imaging.