ArticlePDF Available

Model-mediated teleoperation with improved stability

Authors:

Abstract and Figures

Model-mediated teleoperation has been developed to improve both transparency and stability in teleoperation. It uses local model of remote environment to provide non-delayed force feedback rather than using delayed force signals from slave side and thus is robust to arbitrary time delay. However, updating parameters in the local model may cause sudden force change during the operation. Meanwhile, the undesirable deep penetration or overlarge contact force may occur on the slave side due to the modeling error. Both of them will jeopardize the system stability. In this article, we propose a novel force-based model updating algorithm, which restrains the abrupt force caused by parameter updating. The update efficiency has been greatly improved by comparing with the existing solution; meanwhile, it ensures a stable human–machine interaction at the same time. Then, a new adaptive impedance controller that restricts both overlarge force and penetration is introduced. The obtained results on a one-degree of freedom contact experiment with a delay of 5 s demonstrate the superiority of proposed approaches in comparison with state-of-the-art methods.
Content may be subject to copyright.
Research Article
Model-mediated teleoperation
with improved stability
Jingzhou Song
1
, Yukun Ding
1
, Zhihao Shang
1
and Ji Liang
2
Abstract
Model-mediated teleoperation has been developed to improve both transparency and stability in teleoperation. It uses
local model of remote environment to provide non-delayed force feedback rather than using delayed force signals from
slave side and thus is robust to arbitrary time delay. However, updating parameters in the local model may cause sudden
force change during the operation. Meanwhile, the undesirable deep penetration or overlarge contact force may occur on
the slave side due to the modeling error. Both of them will jeopardize the system stability. In this article, we propose a
novel force-based model updating algorithm, which restrains the abrupt force caused by parameter updating. The update
efficiency has been greatly improved by comparing with the existing solution; meanwhile, it ensures a stable human–
machine interaction at the same time. Then, a new adaptive impedance controller that restricts both overlarge force and
penetration is introduced. The obtained results on a one-degree of freedom contact experiment with a delay of 5 s
demonstrate the superiority of proposed approaches in comparison with state-of-the-art methods.
Keywords
Teleoperation, model-mediated teleoperation, impedance control
Date received: 24 June 2017; accepted: 20 January 2018
Topic: Robot Manipulation and Control
Topic Editor: Andrey V Savkin
Associate Editor: Alexander Pogromsky
Introduction
Robotic teleoperation system enables operators get a pres-
ence perception of interaction with remote environment
where human presence is hazardous or costly. To this end,
one critical step is the force feedback that gives the oper-
ator the perception of the force exerted by the slave
manipulator.
1
However, the force feedback could be out
of phase compared to operators command due to the delay
in telecommunication system and it has been recognized
for many years that even a small time delay in the control
loop may jeopardize the stability and performance.
Although the stability can be achieved by more conserva-
tive control strategy, for example, passivity-based
schemes, the transparency will be significantly
decreased.
2
Considerable effort has been made to deal
with the trade-off between stability and transparency. The
mainstream techniques adopted include four-channel
architecture,
3,4
wave variable method,
5,6
and sliding mode
control.
7,8
Despite these new improvements, there are still
many tasks, which require a high degree of transparency
under long time delay. But it is hardly to be realized by
bidirectional teleoperation.
9
Furthermore, while many state-of-the-art researches
assume a constant time delay, the time-varying delay and
packet loss problems are particular challenges for both
1
School of Automation, BeijingUniversity of Posts and Telecommunications,
Beijing, China
2
Technology and Engineering Center for Space Utilization, Chinese
Academy of Sciences, Beijing, China
Corresponding author:
Jingzhou Song, School of Automation, Beijing University of Posts and
Telecommunications, No. 10 Xitucheng Road, Beijing 100876, China.
Email: sjz2008@bupt.edu.cn
International Journal of Ad vanced
Robotic Systems
March-April 2018: 1–19
ªThe Author(s) 2018
DOI: 10.1177/1729881418761136
journals.sagepub.com/home/arx
Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License
(http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without
further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/
open-access-at-sage).
stability and transparency.
10
Therefore, alternative
approaches that are less dependent on a high-performance
communication network are becoming more and more
desirable.
Model-mediated teleoperation (MMT) was proposed to
improve both stability and transparency of teleoperation
system.
11
The typical architecture of MMT is illustrated
in Figure 1. On the master side, a model representing both
geometric properties and contact dynamic properties of
remote environment is built to be as accurate as possible.
The user hand motion information is send to remote robot
via a force-feedback device. Force feedback is provided
based on the local virtual model of remote environment
under negligible delay. On the slave side, the robot tracks
the incoming commands while simultaneously collects sen-
sor data (e.g. force, position, and image) for online estimat-
ing parameters of the model of the environment that it
contacts. Rather than sending slave sensory data to the
master side, estimated model parameters are transmitted
back to master side under communication delay. Then, the
parameters of the model on master side were updated
according to the received parameters.
The steady state in MMT is defined as the state that the
estimated model parameters converge and the model mis-
match error between the estimated model and real environ-
ment is neglectable. A transition state in MMT refers to a
period that a model mismatch between local model and
actual environment is existing.
12
Using estimated a model of remote environment, which
include both geometric properties and contact dynamics,
the operator could interact with local model rather than
remote environment and thus could percept non-delayed
force feedback. Theoretically, the teleoperation system can
obtain ideal transparency in the presence of arbitrary time
delay if the local model is identical with real remote envi-
ronment. Different from conventional bilateral teleopera-
tion algorithms, the MMT leads to the two subsystems and
its stability condition is easy to meet. The study by Xu
et al.
12
provides more details about MMT.
The application and extension of MMT have attracted
much attention and its most problems are at least partly solved.
MMT has been adopted in multi-operator multi-robot sys-
tems by the coordination of the master devices using one
centralized variable position-based admittance controller.
13
Based on the traditional position and force sensors, vision
sensor is also used to strengthen the prediction ability of
MMT systems.
14
Interaction with moving object that is
much more complicated than fixed object was realized,
while the modeling of general moving object is still diffi-
cult.
15
For better overall performance, other teleoperation
techniques such as virtual fixture were actively incorporated
in MMT systems.
16
Some promising variants of MMT were
also developed.
17,18
Two key components of MMT are the estimation and
obtaining the model of the remote environment and design-
ing the slave side controller.
19
Multiple degrees of freedom
mass-spring models and finite element models are highly
accurate for simulation of deformable object. But they are
computationally prohibitive for teleoperation system.
20
The main environment models used in robotics research
are spring model, linear Kelvin–Voigt model, and non-
linear Hunt–Crossley model, which is the result of a bal-
ance between complexity and accuracy. All of them enable
user to percept the virtual object’s mechanical impedance
to some extent, which is necessary for the haptic rendering
on master side.
21
Meanwhile, it is usually difficult to model the remote
environment accurately before operation tasks or even
after. In order to obtain better approximation or keep pace
with the changing environments, the parameters describing
the local model are continuously estimated on slave side
and transferred back to the master, which is referred to
online parameter estimation. Different techniques in envi-
ronment modeling and online parameter estimation had
been extensively studied and compared.
22–24
Then, the local model on master side is updated based on
the estimated parameters, which is known as model updat-
ing. Special attention must be paid in this process as impro-
per update schemes may lead to undesired motion and
instability. When the data received from slave side trigger
a sharp change of the model parameters, such as the stiff-
ness and environment location, the force feedback may be
changed accordingly. It is difficult for users to stabilize the
master controller if the force changed suddenly. Thus, the
abrupt changing force may disrupt the user’s operation and
cause unsafety issue on slave side because the abnormal
command also is transmitted to slave. The negative impact
of the abrupt change of model parameter is referred to as
“model jump effect.”
25
Different schemes were adopted to
deal with this problem, but these approaches delayed the
updating of model parameters or changed them in a limited
rate in order to maintain the stability or passivity of the
system.
26,27
There is an alternative approach, the user is
informed by a small force impulse when model updating
is needed. Then, the user must move the master end to a
safe area to enable the update of the model parameters. This
approach will interrupt the operation frequently.
The transition state of model updating can be defined as
the period when the force rendered by the master device is
Figure 1. Structure of the MMT. MMT: model-mediated
teleoperation.
2International Journal of Advanced Robotic Systems
different from the original one that is computed based on
the received parameters directly. To provide force feedback
accurately, the transition period should be as short as pos-
sible. However, in order to obtain a stable haptic rendering,
a trade-off of between smooth force rendering and quick
transition is preferred.
Another challenge of MMT is the slave controller.
While tracking the delayed command from the master side,
the command is often built on inaccurate information due
to the error between local model and real environment.
Two state-of-the-art control approaches for slave control
in MMT are the switching position control/force control
method and the relative tracking method.
12
In switching position/force control approach, the
slave is position controlled in free space and force
controlled while in contact with the environment. Thus,
the slave is controlled to maintain same contact force
as applied on the master side. However, while the stiff-
ness of the remote environment is smaller than that of
the local model, the slave end-effector will penetrate
into the environment more than that on the master side.
The overlarge penetration may cause damages to the
environment.
The relative tracking method proposed in the study by
Winck and Okamura
19
is a modified PD controller. Instead
of tracking the absolute position on master side, the relative
tracking method takes the mismatch position into account.
Environment model information is also sent to the slave
side. The aim of relative tracking is to make the slave to
track the master position relative to the model and thus
maintains the same penetration for the slave side and mas-
ter side. Even though the relative tracking avoids the unde-
sired penetration on the slave side, it is not capable to deal
with the stiffness mismatch between the local model and
remote environment. Once the stiffness mismatch exsits,
the contact force is deviating and the overlarge force will
jeopardize the operation safety.
In this article, we propose a simplified MMT system
and address the stability problem of MMT by introdu-
cing a new model updating approach and an adaptive
impedance controller. The proposed model updating
approach limited the ‘abnormal’ force changing and
ensured stable human–machine interaction on master
side. The proposed controller is kind of a supervisory
controller. In addition to position commands, the corre-
sponding local model information is also sent over com-
munication link to the slave side. The slave controller
interprets the operator’s intention on the basis of corre-
sponding master state. After the comparison between the
expected model and the real environment, it carries out
command from master in a conservative way to prevent
from both overlarge force and penetration. The proposed
impedance control approach enables the controller to
adjust its behavior smoothly without switching among
different states and maintains a compliant contact with
the environment. Eventually, the proposed controller can
limit overlarge force and penetration in the presence of
both position uncertainty and stiffness uncertainty in the
transition state. Meanwhile, accurate tracking of both
position and force is achieved in the steady state, that
is, the system tracking capability is not compromised
evidently. Note that the accurate tracking of both posi-
tion and force in steady state does not mean the
controller is able to control the force and position simul-
taneously, it is the visualized description of the result
that both force and position on slave side are following
that of the master side.
Furthermore, we made some discussion on the limitation
of existing position-dependent environment model
28
and
proposed to use interpolation mechanism to estimate envi-
ronment impedance in an unmeasured position prior to the
contact happened. By incorporating the prior knowledge
and available vision information, the method is expected
to shorten the transition state in MMT and accelerate the
parameter estimation.
The article is organized as follows. The environment
modeling and identificationarepresentedinsection
“Environment modeling and identification.” Section
“Environment model updating algorithm” presents the pro-
posed model updating strategy. The adaptive impedance
controller is presented in section “Adaptive impedance
controller.” Experimental setup and results are described
in “Experiments” section. Section “Discussion and con-
clusions” provides a discussion about the extension of the
environment model in MMT and concludes the article with
a summary and outlook.
Environment modeling and identification
Environment modeling
We first have a quick review and discussion on several
mainstream contact dynamic model in MMT. Geometry
parameters of environment are out of the scope as they are
normally not included in the environment model.
12
The
identification of geometric profile can be seen in the liter-
ature.
14,29–31
The spring model is the simplest contact force model.
Though different models had shown better consistency
with real environment, it is still one of the dominate
approach in related research.
29,32,33
It denotes a proportion
relationship between the penetration and contact force. The
only parameter is the stiffness K
Fe¼Kdxdx>0
0dx<0
ð1Þ
where dxrepresents the penetration depth into the environ-
ment and Kdenotes the environment stiffness.
Another common environment contact dynamic
model for teleoperation is the Kelvin–Voigt model,
which incorporates the dynamics of a linear damper-
spring system
Song et al. 3
Fe¼KdxþB_
dxdx>0
0dx<0
(ð2Þ
where Bdenotes the environment damping. While the
inertia term may be included in popular second-order
model, it is often neglected as the environment is usually
stationary or quasi-static for most applications.
Nonlinear model has been shown to have better agree-
ment with the real dynamic behavior of physical environ-
ment. The Hunt–Crossley model is the most popular
nonlinear model for contact dynamics in teleoperation
Fe¼KdxnþBdxn_
dxdx>0
0dx<0
(ð3Þ
where nis a constant typically lies between 1 and 2. The
complexity and accuracy of the above-mentioned three
models increases in turn and the selection of modeling
approach requires a appropriate trade-off in each context.
Although the Hunt–Crossley model is nonlinear, a
single-stage method is proposed to linearized it by taking
the natural logarithm of both sides of the model for
dx>0.
34
Consequently, the Hunt–Crossley model can be
identified using common methods. But even so, the Hunt–
Crossley model is still have difficulties for practical appli-
cation. The applicability of estimation method is limited as
its validity and consistency requirement, for example, low
damped environment and low operation speed. More
importantly, experiments in the studies by Haddadi and
Hashtrudi-Zaad
24
and Achhammer et al.
35
show much
slower converge speed comparing to Kelvin–Voigt model,
which probably caused by additional exponential para-
meter n.
For the Kelvin–Voigt model, experiment results in
the studies by Achhammer et al.
35
and Yamamoto
et al.
36
show that the Bhardly converges and it may
fluctuate significantly while the Khas good conver-
gence. While the Kelvin–Voigt model obtains slight
smaller force error at the cost of complicity comparing
to spring model, the identified damping character is
useless for most controllers. Moreover, as slight differ-
ence in haptic feedback signals is not perceivable for
human operator, it is hard for the operator to find this
minor improvement. The threshold of human percep-
tual discrimination for haptic signals is referred to as
just noticeable difference (JND). As reported in the
study by Hirche and Buss
37
, the JND of force is
approximately 10%. In contrast, the various adaptive
robot controllers of usually accords with the environ-
ment stiffness and have acceptable accuracy. The sim-
plicity of spring model also facilitates the estimation
process and makes it a more practical choice. This
partly explains the popularity of simple spring model
that is also used in our experiments.
Model identification
A number of model identification method were investi-
gated for estimating the model parameters.
23,38
The self-
disturbing recursive least squares (SPRLS) is a common
method used in recent studies as it can be immune to noise
and track variable environment at the same time.
23,24,35
The
update equations of SPRLS can be written as follows
^
qk¼^
qk1þKkðykfT
k^
qk1Þ
Kk¼Pk1fkð1þfT
kPk1fkÞ1
Pk¼ðIKkfT
kÞPk1þbNINTðg^
e2
k1ÞIð4Þ
where Pkis the covariance matrix at time instant; k,^
q, and y
are the vector of estimated dynamic parameters and system
output, respectively; fconsisting of input variables; band
gare designed constants that determine the sensitivity and
gain; Iis the identity matrix of the same size as the matrix
P; and ^
eis the estimation error calculated by ^
e¼y^
y. The
NINT function is a round off operator
NINTðg^
e2
k1Þ¼ g^
e2
k1g^
e2
k10:5
0g^
e2
k1<0:5
(ð5Þ
When the error is lower than the maximum error bound
determined by g, the self-disturbing term is equal to zero
and it is identical to the regular exponentially weighted
recursive least squares algorithm that have good conver-
gence character. Otherwise the Pkis increased according to
the error and the sensitivity gain b, which means the latest
data are endowed with larger weight in the estimation pro-
cess. Thus, the SPRLS can be immune to noise and have
great ability of tracking variable parameters as well. One
limitation of SPRLS is that its performance heavily
depends on appropriate value of the designed parameters
gand b. Although the SPRLS had been used in many
studies, guide or discussion on the selection of design para-
meters is missing.
In practice, prediction error is usually caused by the
mismatch between the real contact dynamics and the model
characteristic instead of the force measurement noise. Such
a mismatch causes inevitable error between the calculated
force and measured force even with the optimal parameters.
gdetermines a minimum error level is to be considered as
signal of changed environment dynamics. Thus, the thresh-
old determined by gis tuned to be slight larger than the
maximum error caused by the mismatch and bis tuned
according to the scale of parameters in our experiment.
Environment model updating algorithm
Stable force rendering on the master side is vital for MMT.
However, the model updating during the operation task
may induce unstable force rendering. For example, when
the salve end just made contact with a object, which is
4International Journal of Advanced Robotic Systems
stiffer than that in local model, the stiffness in local model
should be updated as soon as the real stiffness is received so
that it could reflect the real environment property correctly.
However, the force feedback is rendered continuous and
relies on the stiffness. The update of stiffness will cause
larger force increment in short time and such discontinuity
of the force feedback would hazard the system stability.
Parameter-based model updating
Gradual-update scheme has been adopted in MMT. With
the gradual-update strategy, the model parameters were
changed in a fixed change rate or in a fixed time period
in some cases. The update law for position updating can be
written as below while the stiffness updating is similar
xm
e;tþ1¼
xm
e;tþvm
eDTx
e;tþ1>xm
e;tþvm
eDT
xe;tþ1else
xm
e;tvm
eDTx
e;tþ1<xm
e;tvm
eDT
8
>
<
>
:
ð6Þ
where xeis the received environment position, xm
eis the
environment position in local model, and vm
eis the change
rate of xm
e. Though it can mitigate the model jump effec-
tively, stability is not ensured theoretically. More impor-
tantly, a fixed maximum parameter change rate does not
confine the bound of the force variation. In the contacting
stage, the force feedback on master side is computed as
fm¼kmðxmxm
eÞð7Þ
where fm,xm, and kmrepresent the force feedback, position,
and stiffness on master side, respectively. The force varia-
tion induced by parameter vfcan be describe as
vf¼_
km
@fm
@kþ_
xm
e
@fm
@xm
e
¼vm
kðxmxm
eÞvm
ekmð8Þ
where vm
kis the change rate of km. As shown in equation (8),
the force variation rate caused by parameter change, which
affects users’ experience and system stability directly, is
also affected by current contact state. As a result, the fixed
parameter change rates have to be low to avoid large force
variation rate in some cases. Thus, this approach is not
efficient enough.
There are two passivity-based model update strategies
that ensure system passivity. The one in the study by
Mitra et al.
26
was a basic approach that delayed the
parameter changes until the update introduces no energy
increment.TheneweroneproposedinthestudybyXu
et al.
27
used an adaptive virtual damper to dissipate the
energy generate of the model parameters, and thus it
was more efficient than the former one. For spring
model in one-degree of freedom (1-DOF), the passivity
condition for separate stiffness updating and position
updating was given as
_
ktþ1¼Dk
DT2aþbde ktDT
2

_
xt
xtþ1

2
ð9Þ
2DT_
xt
kt
½ktðxm;txm
e;tþðaþbde Þ_
xtþðxm;txm
e;tÞ2
ðxm;tþ1xm
e;tDxm
e;tþ1Þ
ð10Þ
where Dxm
e;tþ1¼xm
e;tþ1xm
e;t,bde is the device damping, a
is the given adaptive damping, kdenotes the stiffness of
local model, and the subscript of mis omitted for clarity. In
practice, the adaptive damping often needs to be large or
even infinite. To avoid a damping value exceeding the
capability of device, awas set with a upper bound.
While the passivity-based methods guarantee the system
passivity, the impedance of human arm has not been con-
sidered. Human arm contributes to system stability by
adjusting itself. Small energy generation is most likely to
be dissipated by human arm and the whole system includ-
ing the human operator is still stable. Therefore, a update
strategy that strictly complies with the passivity condition
is too conservative in application. However, this is still an
open issue as it is so difficult to take human arm damping
into design of passivity-based methods (the damping prop-
erty of human arm is too complicated to be modeled).
Another important defect of passivity-based methods is that
it cannot deal with the condition that the system energy is
decreasing. These weaknesses of gradual-update scheme
and passivity-based methods motivate us to find a better
approach and the proposed force-based model updating
method is introduced in next section.
Force-based model updating
To the best of our knowledge, all existing approaches aim-
ing to solve the effect of model jump, control the variation
of the force feedback in an indirect way, that is, they con-
trol the change of model parameters, and thus limit the
abrupt change of force. However, we do not have to control
the model parameter in the transition state, as the output
that user can perceived the force feedback is the force
rendered by the force-feedback device. In other words, the
key novel idea is that we can deal with the model jump
effect by directly controlling the rendered force instead of
updating the parameters step by step. In the model updating
process, the goal is to make the force feedback from the
master device to be identical with that obtained the com-
putation result with latest parameters as soon as possible.
The only constraint is to avoid abrupt force change and
make it easy for human to handle. Both the goal and con-
straint are about the force rather than parameters. Thus,
implementing model updating in the level of force instead
of model parameters is a more reasonable solution, which
can be referred as force-based scheme. By modifying the
force instead of model parameters on which the force is
based, we are able to avoid the detour and redundancy and
the model updating could be much more efficient.
The difference of parameter-based model updating and
force-based model updating is shown in Figure 2.
Song et al. 5
For now, the challenge becomes how to develop such a
model updating algorithm at the level of force rather than
model parameters. An intuitional approach is to limit the
change rate of the force feedback to avoid the strike of
“model jump effect.” We can constrain the force change
rate to a safe range with smaller value than a threshold to
ensure stable interaction. However, according to our
experiment, a smaller force change rate can bring signif-
icant negative effect on user’s operation while a larger
force change rate does not. An intuitive example can make
it more clear. It is easy for the operator to decrease the
contact force toward a virtual wall from 5 N to 1 N in
approximately 0.2 s with a neglectable movement. But if
theuserisholdingstillwithaforceat5N,andthenletthe
force decreases to 1 N in 0.2 s without any warning in
advance, the user’s hand will move forward suddenly with
a feeling of “missed step.” Note that the force change rate
in the two cases is the same. We can conclude that an
appropriate force change rate depends on the specific con-
ditions and thus is difficult to set. Instead of force change
rate, we need a new criterion that helps us define a safe
range for stable interaction.
Once human is involved on the master side, whenever
the user executes a motion or hold still, a force is expect
roughly and unconsciously due to human’s physical intui-
tion. It is assumed that when a movement is executed, both
remaining unchanged or changing according the displace-
ment and the stiffness of the environment are acceptable
and easy to handle for users. Meanwhile, the force should
not be larger or smaller than these two. Such an assumption
has been validated through experiments and further expla-
nation can be found in last section. The range between
these two values will be used as a safe range in the design
of force-based methods.
The aim of model updating is to make the force rendered
by the force-feedback device fmequal to the predicted force
fp, which calculated by the latest model parameters. Thus, it
is preferred that fmchanges in the direction that reduce the
error between fmand fp, the opposite direction is undesired.
Thus, a value within the aforementioned range can be set as
the expected force fex. So the force fpcalculated with the
latest parameters transmitted from the slave side can be
divided into two parts, the expected force fex and the abrupt
force fab. By limiting fab , the force feedback is close to the
range that is easily to be handled by operators and thus the
stable interaction is ensured.
Based on the foregoing discussion, the update scheme of
the force feedback fmon master side is defined as follows.
First, the predicted force fpis calculated
fp;t¼km
e;tðxm
m;txm
e;tÞxm
m;txm
e;t>0
0xm
m;txm
e;t0
(ð11Þ
Then, the maximum and minimum expected force fmax
ex
and fmin
ex are defined based on the fm,km
e, and the displace-
ment in the last sampling period
fmin
ex ¼minðfm;t1;fm;t1þdxtkm
e;t1Þð12Þ
fmax
ex ¼maxðfm;t1;fm;t1þdxtkm
e;t1Þð13Þ
Then, the selected expected force is obtained as
fex;t¼
fmax
ex fp;t>fmax
ex
fp;tfmin
ex fp;tfmax
ex
fmin
ex fp;t<fmin
ex
8
>
<
>
:
ð14Þ
The force to be rendered by the force-feedback device
can be obtained
fm;t¼fex;tþSATðfab;tÞð15Þ
where rendered force fab;t¼fp;tfex;tand SATðÞ is a
saturation function defined as
SATðfabÞ¼
fc
mfab >fc
m
fab fc
m<fab <fc
m
fc
mfab <fc
m
8
>
<
>
:
ð16Þ
where fc
mis the value of allowed abrupt force in a sampling
period.
The rendered force can be divided into two parts: one is
the reasonable force expected by the user fex and the other is
the abnormal part. Note that in the steady state where no
model updating is needed, fex is exactly equal to fpand thus
we obtain fm¼fp, which means the force render is accu-
rate. In the transition state when fmis not equal to fp,thefm
is close to fex. Stable interaction is ensured since fex is an
expected force that can be handled effortlessly. Moreover,
via the selection of fex,thefmis approaching to the fpas
soon as possible. Once the fmis equal to fp, the modeling
updating is finished and normal force rendering is also
achieved with the given force updating scheme.
Adaptive impedance controller
Slave controller in MMT
Transition states occur irregularly during teleoperation due
to inaccurate environment estimation or environment
Figure 2. Parameter-based and force-based model update
schemes.
6International Journal of Advanced Robotic Systems
changes. In this case, the command made on master side is
based on inaccurate state information, which may result in
undesirable and risky deep penetration or large contact
force on slave side. The main challenge of slave controller
in MMT is to carry out the master command under an
uncertain or unknown environment meanwhile maintains
the stability of the teleoperation system.
Figure 3 shows model mismatch issue during the transi-
tion state. Both position mismatch and stiffness mismatch
may cause overlarge contact force using conventional posi-
tion control approach. Fortunately, the model mismatch
problem is only partly solved, either by switching posi-
tion/force control or by relative tracking method.
Switching position/force control is the most frequently
used slave controller in MMT. The slave switches to force
control mode when the measured contact force exceeds a
preset threshold. In the force control mode, the contact
force is effectively controlled and thus the overlarge force
is avoided in both cases of position mismatch and stiffness
mismatch. However, if the stiffness of remote environment
is smaller than that of the local model, in order to apply the
same contact force, the slave will penetrate to the contact
object large than the master side even though the larger
penetration is probably not the user’s intention, that is,.
once the stiffness mismatch exist, potential overlarge pene-
tration may occurred and damage the contact object.
Relative tracking had the same framework of PD control
but modified it based on the relative positions and veloci-
ties. The original PD controller for position tracking is
FIm ¼kpsðxmxsÞþkds ðvmvsÞð17Þ
where FIm is the force applied to the slave and kps and kds
are the PD gains. Relative tracking method modified it to
FIm ¼kpsððxmxm
eÞðxsxeÞÞ
þkdsððvmvm
eÞðvsveÞÞ ð18Þ
where xm
eand xeare the position of local model and the real
environment of master and slave, respectively. vm
eand ve
denote the velocities.
With such a modification, relative tracking method
greatly improved the position tracking performance in the
case of position mismatch. However, it cannot solve the
stiffness mismatch issue. Considering the stiffness mis-
match exists, for example, when the slave is to contact an
object, which is stiffer than what supposed on master side.
There would be no position mismatch, for example,
xm
e¼xeand vm
e¼ve. Then, the relative tracking degener-
ates to standard PD control that tracks the position on mas-
ter side. In the transition state, the position controlled slave
follows the master position and the resulting contact force
is larger than it on the master side due to larger stiffness.
Such a larger force may also damage the object.
Adaptive impedance controller
As discussed earlier, either one of the two slave controller
cannot address the potential stability issues in transition
state. A possible solution suggest in the study by Xu
et al.
12
is using a hybrid control scheme and the slave do
not execute any motion commands whenever the contact
force or the penetration has reached the values as them on
the master side. However, it has not been verified that the
frequently interruption and switching operation would
degrade of execution efficiency significantly.
Inspired by the wide used impedance controller in com-
pliant control and its advantages, we figure out an adaptive
impedance controller for the slave control in MMT. The
original idea of impedance control is to control force and
position simultaneously and it indirectly controls a
dynamic relationship between these two, that is, using ref-
erence inputs of both desired force and position is allowed.
But the application of impedance controller usually
involves only one input, while the other is neglected or
adjusted adaptively.
39,40
One possible reason is using two
reference input requires the command conforming to envi-
ronment dynamics, which is not possible in most cases.
However, the local model in MMT enables the operator
to give both force and position input at the same time. More
importantly, they are reproducible on slave side and useful
for slave controller to understand the operator’s intention.
In the steady state, the rendered force on master side is
calculated by the accurate estimating of remote environ-
ment dynamics. This means both the rendered force and its
corresponding position conform to real environment and
can be reproduced on slave side simultaneously. Although
minor error is inevitable due to the deviation of local
model, the residual error is no more than the modeling
error. As far as we know, this is the first work that leverages
the inherent interaction between impedance controller and
MMT. In comparison with switching position/force con-
trol, our approach can realize compliant contact while
avoiding the chattering problem. In addition, the adaptive
impedance controller’s behavior is more similar to the
human arm that is identified as an impedance module in
some teleoperation frameworks.
9,41
Figure 3. Model mismatch in MMT. MMT: model-mediated
teleoperation
Song et al. 7
Consider the low inertia and frictionless nature of com-
mon haptic devices on master side, for example, Geomagic
Touch, together with the limited velocity and acceleration
in teleoperation, the rendered force output by haptic device
has very little difference with the operator applied force.
Therefore, it is feasible to use the rendered force on master
side as the desired force on slave side.
The overview of proposed controller is shown in
Figure 4. In the proposed controller, both master position
and rendered force are transmitted to slave side as the
reference inputs of the slave impedance controller. The
design parameters in controller are adaptively adjusted.
For the impedance controller, the desired impedance
model (in the case of 1-DOF) is specified as
Md
xþBdð_
x_
xdÞþKdðxxdÞ¼efð19Þ
x¼M1
d½efBdð_
x_
xdÞKdðxxdÞ ð20Þ
where Md;Bd, and Kdare the desired inertia, damping, and
stiffness, respectively; ef¼fefd; and xdis the desired
position, fdis the desired force, xand feare the actual
position and actual force; _
xand
xdenote the actual velocity
and acceleration.
When the slave is moving in free space, both feand fd
are zero, and the impedance controller is equivalent to a
position controller. With a fixed high desired stiffness Kf,
that is, high proportional gain, accurate position tracking
could be achieved.
In the case that the slave is contacting with environ-
ment, we first assumed a static environment for the sim-
plicity. It is also not considered the direction motion
tangent to the environment surface, which means no slid-
ing. The method can be easily extended to these cases,
with a modification of parameter when estimating the
stiffness and penetration at the same time.
42
Let xsand
xmdenote the penetration on slave and master sides,
respectively. For an environment represented by spring
model, fecan be approximate by
fe¼Ksxsð21Þ
where Ksdenotes the stiffness of environment. With
SPRLS and spring model, Kscan be obtained shortly after
the contact occurred and as the estimation of Ksis on slave
side and thus is immune to communication delay. fdis the
rendered force on master side
fd¼Kmxmð22Þ
At the equilibrium point, we obtain
KdðxmxsÞ¼fefdð23Þ
Plugging equations (21) and (22) to equation (23), we
obtain
xs¼KmþKd
KsþKd
xmð24Þ
For both position mismatch and stiffness mismatch, we
expect the following relative bounded of feand xsfor safe
operation
fe
fd
aFð25Þ
xs
xm
aPð26Þ
where aFand aPare constant larger than 1, respectively,
denoting the acceptable ratios of actual contact force to
desired force and actual penetration to desired penetration.
For the restriction of force, using equations (21) to (23)
and (25), the following condition can be obtained
KdðaF1ÞKsKm
KsaFKm
KsaFKm
KdðaF1ÞKsKm
KsaFKm
Ks<aFKm
8
>
>
>
>
<
>
>
>
>
:
ð27Þ
Similarly, for the restriction of penetration, the follow-
ing condition can be obtained
KdKmaPKs
aP1ð28Þ
Note that 8Kd>0 meet equation (27) when
Ks<aFKmand 8Kd>0 meet equation (28) when
Ks>Km=aP. In addition to restriction of overlarge force
and penetration, the impedance controller’s performance
can be improved by adjusting the parameters based on
environment stiffness.
43
To this end, defines Kd¼FðKsÞ
where FðKsÞis set toward best controller’s performance
without the consideration of potential overlarge force or
penetration issue. Generally, the rule of Kdin order to meet
equations (27) and (28) can be determined as following
Kd¼
maxFðKsÞ;ðaF1ÞKsKm
KsaFKmKsKm=aP
FðKsÞelse
minFðKsÞ;KmaPKs
aP1KsaFKm
8
>
>
>
>
>
>
<
>
>
>
>
>
>
:ð29Þ
Figure 4. Structure of the controller.
8International Journal of Advanced Robotic Systems
With regard to an intuitional explanation, larger Kdcan
be seen as more weight on position tracking. Emphasis on
position tracking is a conservative approach when the envi-
ronment is softer than estimated (as an extreme case, the
master is contacting while the slave is not). Similarly,
smaller Kdmeans force tracking is preferred when the
environment is stiffer than estimated (such as the slave end
is contacting while the master end is not).
Once the Kdis determined, Mdand Bdcan be selected on
the basis of both Kdand Ksto shape the closed-loop tran-
sient behavior. The damping ratio xdis given by
xd¼Bd
2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
MdðKdþKsÞ
pð30Þ
To overcome the contact instability problem, critical
damping or over-damped is preferred.
The modelingerror can be denoted as em¼KmxmKsxm.
Then, eqaution (23) can be transformed as following by
substituting fe¼Ksxsand fd¼Ksxmþeminto it
KdðxmxsÞ¼KsxsðKsxmþemÞð31Þ
Consequently, we obtain the only equilibrium point
xsxm¼em
KsþKd
ð32Þ
Since jemjis lower bounded, it can be easily obtained
that jxsxmjis upper bounded as long as KsþKdis lower
bounded. Kdis set to a high value when Ksis small and jemj
is relatively small (usually less than 0:1fe). It is safe to
assume that the tracking error jxsxmjis acceptable for
most teleoperation tasks. It is worth noting that the fdand xd
are the input of the impedance controller, the tracking abil-
ity of system does not depend on the adaptive parameter
Kd. In fact, when the modeling error is neglectable, xswill
converge to xdno matter how Kdchanges by the estimated
environment stiffness.
Note that the position tracking error are based on the
equilibrium point of contact, the penetration or contact
force may exceed the bounds due to abrupt change of envi-
ronment, modeling error, dynamic error, and so on. How-
ever, as shown in the “Experiments” section, the overall
restriction is effective.
Experiments
Comparison of model updating
Stiffness updating only. A simple experiment setup is used for
comparison of model updating approach. Operator contacts
with a virtual floor with Geomagic Touch as the force-
feedback device on master side and only vertical motion
is considered. A spring model is used as local model and
thus the force feedback is computed as equation (1), while
dxis derived by the endpoint position of Geomagic Touch
and the position of virtual floor. The stiffness and position
of virtual floor was updated regularly. The stiffness of
virtual floor was set as 1100 N/m initially and switches
between 100 N/m and 1100 N/m every 10 s. The operator
was told to make contact with the virtual floor in a static or
fluctuate manner. No model updating scheme is used to
serve as a benchmark. As can be noted in Figure 5, the
operator was unable to keep the handle still when the stiff-
ness changes. The operator pushed the handle deeper acci-
dentally (seeing position increase in the figure) when the
stiffness decreases abruptly at time t¼10 s. The handle
was pushed up (seeing position decrease in the figure)
accidentally and had an overshoot when the force rendered
on device increases suddenly at time t¼20 s. The maxi-
mum value of most force rendered is below 4 N. Even so,
the abrupt change of model parameters imposed a signifi-
cant negative effect to the normal operation.
Three model updating methods were tested in the
same condition. The stiffness change rate in gradual-
update scheme and fc
min force-based method were set to
1000 N/ms and 0.002 N, respectively, by manual tuning so
that they can just be handled the operator. In passivity-
based method,
27
bde was set to zero as the device damping
is small and unfixed when awas 5 Ns/m.
Figure 6 demonstrates the results of passivity-based
method, the force changed very smooth at time t¼20 s.
However, at time t¼10 s, the force diminished suddenly
and caused the operator push down the handle in about 0.1 s
accidentally because the decreased stiffness lead to
decreased energy and the algorithm does nothing through
the variation. This indicates that passivity-based methods
should be used combining with other methods to deal with
time (s)
0
2
4
6
8
10
12
14
16
force (N)
fp
fm
0 5 10 15 20 25
0 5 10 15 20 25
time (s)
–10
–5
0
5
10
15
20
position (mm)
xm
Figure 5. Force and position without model updating.
Song et al. 9
the system energy decrease. Figure 7 shows the change of
stiffness during the operation. The stiffness changes from
1100 N/m to 100 N/m at time t¼10 s directly and varies
from 100 N/m to 1100 N/m adaptively according to the
state of the contact. When the penetration decreases to
smaller value after time t¼25 s, the bigger stiffness
changes rate is allowed and the update process still takes
more than 6 s. The result of gradual-update scheme is
shown in Figure 8, the force fmchanged smooth at time
t¼10 s and t¼20 s. The master handle is pushed down
gently at time t¼10 s due to the decrease of force and then
be handled quickly as the operator noticed the decrease of
force. No position overshoot happened when increased
force increase at time t¼20 s.
The result of force-based updating is shown in Figure 9,
which is similar to gradual-update method and the position
fluctuation is even smaller. The negative effect of model
jump is effectively avoided.
Both position and stiffness updating. To evaluate the force-
based method thoroughly, another experiment with simul-
taneous position and stiffness updating is conducted.
time (s)
0
5
10
15
20
25
force (N)
fp
fm
time (s)
–10
–5
0
5
10
15
20
25
position (mm)
xm
0 5 10 15 20 25
0 5 10 15 20 25
Figure 6. Force and position with passivity-based model
updating.
0 5 10 15 20 25
time (s)
0
200
400
600
800
1000
1200
stiffness (N/m)
Ke
Km
e
Figure 7. Stiffness change in passivity-based model updating.
0 5 10 15 20 25
0 5 10 15 20 25
time (s)
0
2
4
6
8
10
12
force (N)
fp
fm
time (s)
–5
0
5
10
15
20
position (mm)
xm
Figure 8. Force and position with gradual-update scheme.
0 5 10 15 20 25
0 5 10 15 20 25
time (s)
0
2
4
6
8
10
12
force (N)
fp
fm
time (s)
–5
0
5
10
15
20
position (mm)
xm
Figure 9. Force and position with force-based model updating.
10 International Journal of Advanced Robotic Systems
Passivity-based method is not compared as no existed solu-
tion with both location and position updating there is. The
stiffness switches between 1100 N/m and 100 N/m every
10 s, the position of virtual floor switches between 0 mm
and 20 mm every 5 s. The initial value for stiffness and
position is 1100 N/m and 20 mm, respectively. The force
and position profiles are shown in Figures 10 and 11. As
can be noted in the enlarged view, after the parameter
updating at time t¼5s,fmchanged toward to fpdirectly
in force-based method, while it fluctuated in graduate-
update method. Note that in the transition state started from
t¼5s,fpis always bigger than fmwhich means the fluc-
tuation of fmis unnecessary and inefficient. This is also how
the force-based method be more efficient than the
parameter-based methods. As a result, the force-based
method takes only 2.52 s for model updating while the
gradual-update scheme takes 5 s.
The abrupt force during the operation of two methods is
compared and shown in Figures 12 and 13. The maximum
and average values of fab for gradual-updating scheme are
0.062 N and 0.009 N, respectively. In contrast, the maxi-
mum and average values of fab for force-based methods are
both only 0.002 N. Smaller fab indicates that force-based
method may provides more comfortable interaction.
The results of gradual-update scheme and force-based
approach are summarized in Table 1. It shows clearly that
the proposed approach provides a much better model
updating performance with high efficiency and smooth user
experience.
0 5 10 15 20 25
0 5 10 15 20 25
time (s)
0
5
10
15
20
25
30
35
force (N)
fp
fm
2
4
6
time (s)
–20
–10
0
10
20
30
position (mm)
xm
5 5.5 6
Figure 10. Force and position with gradual-update scheme, while
both location and stiffness are updated.
0 5 10 15 20 25
0 5 10 15 20 25
time (s)
0
5
10
15
20
25
force (N)
fp
fm
5 5.1 5.2 5.3
2.2
2.4
2.6
2.8
3
3.2
time (s)
–30
–20
–10
0
10
20
position (mm)
xm
Figure 11. Force and position with force-based model updating,
while both location and stiffness are updated.
0 5 10 15 20 25
time (s)
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
force (N)
fab
Figure 12. Abrupt force profile with gradual-update scheme.
0 5 10 15 20 25
time (s)
0
0.5
1
1.5
2
2.5
force (N)
× 10–3
fab
Figure 13. Abrupt force profile with force-based model updating.
Song et al. 11
Comparison of slave controller
Setups. To evaluate the proposed controller, we used a
MMT system as shown in Figure 14 with a Geomagic
Touch as the master haptic device and a 3-DOF translation
parallel manipulator as the slave. An ATI mini 40 force
sensor (ATI Industrial Automation, Inc., NC, USA) was
mounted at the end-effector of the slave robot to measure
the contact force. Currently, for simple, only 1-DOF
motion in vertical direction was allowed.
For online parameter estimation, spring model was used
as the local model and SPRLS is utilized. The online esti-
mation process was implemented at the slave side by the
force sensor data. Specifically, the estimation result is
updated by equations (4) and (5) at position measurement
frequency 170 Hz. The threshold for detection of contact
was set as 0.05 N. The place where a contact happened was
considered as the position of environment, therefore dxcan
be easily obtained. The estimated environment stiffness
and position were sent back to the master side and the force
feedback is computed using equation. (1). It should be
noted that the environment position and stiffness used for
force computation came from the slave side under time
delay. When accurate position and stiffness is obtained, a
non-delayed accurate force feedback will be provided to
users. One practical trick used is that, when the slave
reaches the environment according to the environment and
no contact is detected, the slaves current position is sent
back to replace the location of environment. In this way, the
known information that the environment location is lower
than the current location is utilized, and the modeling error
is minimized.
For the proposed controller, position-based impedance
control is used to carry out the command of impedance
model. Mdis set to 2 Kg and Bdis set according to equation
(30), while xdis set to 1 for better transient behavior. FðKsÞ
is defined as
FðKsÞ¼ l Ks
2
Ks
0Ks>
8
>
<
>
:
ð33Þ
where and lare constant setting before the operation
tasks. Specifically, ,l, and Kfwere set to 8000, 5000,
and 5000 N/m, respectively. Both aPand aFset to 1.3,
which means 30%larger force and penetration are
expected. The other parameter settings are invariant for all
the following experiments except the time delay. To vali-
date the robustness of the proposed approach, a delay of 2 s
was given in the comparison experiments and 5 s time
delay was used in the continuous contact experiments.
As shown in Figure 15, different contact materials
(environments) were used for the experiment. They
included a rubber pad, a foam, and a metal cover board.
The stiffnesses of the three materials are approximately
400, 6000, and 19 000 N/m, respectively.
The proposed controller was first validated in compari-
son with the switching position/force control method
12
and
the relative tracking method.
19
Then, it was evaluated in a
continuous contact experiment with changing environment
(both stiffness error and position error exist). For the con-
venience of observation, all curves plotted in the figure are
translated by 2 s along the time axis to counteract the for-
ward time delay.
Potential of existing approaches. The experiments were
designed for detecting the potential of existing control
approaches. Both forward and backward communication
time delay were set to be 2 s and the feedback from the
slave side was blocked to maintain a transition state. The
Table 1. Comparison of model updating algorithms.
Algorithm
Time used
fab(N)
(s) Max Mean
Gradual-update 5.0 0.062 0.009
Force-based 2.52 0.002 0.002
Figure 14. Setup of the teleoperation system.
Figure 15. Contact materials in experiments.
12 International Journal of Advanced Robotic Systems
initial stiffness in local model was set as 2000 N/m in every
trial. Comparing with the environment model on master
side, foam is a lower stiffness environment, while rubber
pad and metal cover board have higher stiffness.
In the first set of experiments, the slave was in contact
with a rubber pad using relative tracking, which is a harder
environment than expected. As shown in Figure 16, while
the position tracking is accurate, the slave contact force is
much larger than master due to unexpected larger stiffness.
The overlarge force is unpredictable for operator and may
damage the remote environment and the stability of the
system. The results using the proposed controller in the
same condition are shown in Figure 17. It is verified that
the controller effectively restricted the unexpected over
large force when it encountered an unexpected high envi-
ronment stiffness.
Another experiment is to contact with foam, which is a
softer environment than expected, using switching posi-
tion/force controller. The results are shown in Figure 18.
The force controller drives the slave to a position higher
than master obviously which will cause large and danger-
ous penetration in order to obtain desired contact force. In
contrast, as shown in Figure 19, the overlarge penetration
may be avoided using our impedance controller. The com-
parison results are summarized in Table 2. As could be
expected, during the transition state when the local model
on master side exists error relative to the real environment
on slave side, the proposed impedance controller can
ensure task be carrying out by a conservative way. The
slave contact force and penetration were restricted to a safe
range when the stiffness does not match, which is impos-
sible for other approaches.
Continuous contact experiment. To validate the proposed
controller thoroughly, we conducted the experiments on
0 5 10 15 20 25 30 35
0 5 10 15 20 25 30 35
time (s)
–2
0
2
4
6
8
10
12
14
force (N)
fm
fs
time (s)
1
2
3
4
5
6
7
position (mm)
xm
xs
Figure 16. Force and position with relative tracking (hard
environment).
0 102030405060
0 102030405060
time (s)
–1
0
1
2
3
4
5
force (N)
fm
fs
time (s)
0.5
1
1.5
2
2.5
3
3.5
4
position (mm)
xm
xs
Figure 17. Force and position with adaptive impedance con-
troller (hard environment).
0 2 4 6 8 101214161820
02468101214161820
time (s)
0
0.5
1
1.5
2
2.5
3
3.5
force (N)
fm
fs
time (s)
1
2
3
4
5
6
7
8
9
position (mm)
xm
xs
Figure 18. Force and position with switching position/force
controller (soft environment).
Song et al. 13
changing environment in which both position and stiffness
of the object are varying and unknown by the operator
in advance. The initial stiffness in local model is still
2000 N/m. Different from former experiment sets, the mas-
ter received the estimated parameters from slave side and
updated the local model for accurate approximation. Thus,
the operation will entry to the steady state from transition
state during the operation. To mediate the model jump
effect while updating parameters, the force-based model
updating is utilized and the fc
mis set to 0.002 N.
The operator was allowed to contact with different envi-
ronment. The order of contact material is rubber pad, foam,
double rubber pads, and metal cover board. The full process
of the experiment can be seen in the Online Supplementary
Video. The result of online parameter estimation is shown
in Figure 20. The initial stiffness value was 2000 N/m and
converges to corresponding value once the slave is in con-
tacting with a new environment. Four steady stages at 7000,
400, 5000, and 19,000 N/m represent four different mate-
rial stiffness, respectively. The converge of stiffness esti-
mating takes only a few seconds and no undesired
overshoot happened. Besides, the estimation algorithm can
distinguish the difference between the rubber pad and dou-
ble rubber pad apparently, which further suggests the accu-
racy of the parameter estimation algorithm. Overall, the
online stiffness estimation employed in the system
worked effectively. On the other hand, the quick varia-
tion of stiffness at such a magnitude challenges the
model updating on the master side. It will be shown
soon that the proposed model updating algorithm’s per-
formance is satisfactory as expected.
The position tracking and force-feedback results are
shown in Figure 21 and the enlarged views of four stages
are shown in Figures 22 to 25.
0 5 10 15 20 25 30 35 40 45 50 55
0 5 10 15 20 25 30 35 40 45 50 55
time (s)
0
0.5
1
1.5
2
2.5
3
3.5
4
force (N)
fm
fs
time (s)
2
2.5
3
3.5
4
4.5
5
5.5
6
position (mm)
xm
xs
Figure 19. Force and position with an adaptive impedance con-
troller (soft environment).
Table 2. Comparison of slave controller.
Algorithm Force overshoot Position overshoot
Adaptive impedance 0.7 N 0.4 mm
Force/position N/A 5.7 mm
Relative tracking 9.5 N N/A
0 20 40 60 80 100 120 140 160 180 200 220
time (s)
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
stiffness (N/m)
× 104
Ke
Figure 20. Result of online parameter estimation.
0 20 40 60 80 100 120 140 160 180 200 220
0 20 40 60 80 100 120 140 160 180 200 220
time (s)
–1
0
1
2
3
4
force (N)
fm
fs
time (s)
–80
–60
–40
–20
0
20
40
60
80
position (mm)
xm
xs
Figure 21. Continuous experiment of adaptive controller.
14 International Journal of Advanced Robotic Systems
Since position and stiffness of all four different environ-
ments are unknown to user, there are position error and
stiffness error at the initial phase of the each contact.
As a result, both accurate position tracking and force
feedback may not be achievable and proper action must
be done to avoid overlarge force or penetration. For exam-
ple, as shown in Figure 22, the slave tracking the master
before time t¼8 s. Because the position of environment
model on master side is lower than the real one, the master
time (s)
force (N)
fm
fs
time (s)
position (mm)
xm
xs
5 10152025303540
5 10152025303540
0
1
2
3
4
20
25
30
35
40
45
50
55
60
Figure 22. Continuous experiment of adaptive controller: con-
tact stage with rubber pad.
55 60 65 70 75 80 85 90 95
50 55 60 65 70 75 80 85 90 95
time (s)
force (N)
time (s)
position (mm)
xm
xs
fm
fs
50
–0.5
0
0.5
1
1.5
2
2.5
3
–10
–5
0
5
10
15
20
25
30
Figure 23. Continuous experiment of adaptive controller: con-
tact stage with foam.
time (s)
–0.5
0
0.5
1
1.5
2
2.5
3
3.5
force (N)
fm
fs
time (s)
5
10
15
20
25
30
35
position (mm)
xm
xs
105 110 115 120 125 130 135 140 145 150
105 110 115 120 125 130 135 140 145 150
Figure 24. Continuous experiment of adaptive controller: con-
tact stage with double rubber pads.
150 160 170 180 190 200 210 220
150 160 170 180 190 200 210 220
time (s)
–0.5
0
0.5
1
1.5
2
2.5
3
3.5
force (N)
fm
fs
time (s)
58
59
60
61
62
63
64
position (mm)
xm
xs
Figure 25. Continuous experiment of adaptive controller: con-
tact stage with metal cover board.
Song et al. 15
kept moving downward without force feedback. While the
slave was in contact with the environment but the master
did not, the impedance parameters of the controller were
adjusted in order to avoid overlarge contact force so the
slave did not go on tracking the master. Meanwhile, the
new environment position was transmitted back to the mas-
ter. After the command based on corrected environment
model is sent to the slave, both position tracking on slave
side and force prediction on master side are achieved. The
impedance control scheme adopted in the controller pro-
vides a high degree of robustness to robot unmodeled
dynamics and the unknown environments. As a result,
which also demonstrated in the plots, the controller not
only effectively avoided overlarge contact force and pene-
tration but also guaranteed the smooth trajectories.
It is worth noting that the master position changed
smoothly when the model parameter abruptly at time
t¼10, 55, 112, and 175 s. The operator also can feel
smooth force changing during the whole experiments,
which validated the effectiveness of the model updating
strategy.
As shown in Figure 25, when the position of local model
on master side is higher than the real one, the operator feels
like the slave is in contact but it is actually not, then the
controller will drive the slave to slightly surpass the posi-
tion of master. It can help the slave “found” that the real
environment position is farther than expected. As a result,
the slave is able to keep moving until it makes contact with
the environment and the environment position in local
model becomes correct.
The point is that both overlarge force and penetration are
effectively avoided under unknown environment and large
time delay, which is impossible for conventional bilateral
teleoperation approaches. The residual tracking error
comes from the dynamic error and modeling error. The
tracking error, when contacting with metal cover board,
is minimum in the four cases due to high stiffness and small
modeling error.
Discussion and conclusions
In this section, we offer some discussion about the pro-
posed methods and position-dependent environment
dynamics model, which is necessary for the application
of MMT at a higher level.
Discussion on proposed methods
For the force-based model updating algorithm, it is
believed to be more effective than existing parameter-
based approaches. Actually, the goal of the parameter
updating in MMT is to maintain the stable human–machine
interaction and make the rendered force equal to the force
calculated via latest model parameters as soon as possible
rather than update the model parameters step by step. While
updating the parameters step by step is surely feasible and
useful, which also have been used wildly, the force-based
method can make full use of the feasible space to accom-
plish its task. Thus, model force updating is a more reason-
able and more effective solution in comparison with the
parameter-based methods. Note that the same allowed
abrupt force fc
mmay mean different parameter updating rate
under different conditions. For example, when the penetra-
tion is small even lager stiffness variation causes limited
force change, thus the model updating will be finished
quickly. Meanwhile, small stiffness change may cause
large force change and the model updating will be finished
slowly. In other words, while fc
mis fixed to ensure stable
interaction, the update rate is adaptive.
For the slave controller, the primary limitation is that
either position or force cannot be controlled precisely. It is
also worth to point out that, for the proposed approach,
obtaining ideal tracking in steady state requires environ-
ment can be represented by spring model. As shown in
Figures 22 and 25, the strong nonlinear dynamics of foam
lead to apparent larger tracking error. Modification
toward nonlinear contact model needs to be investigated,
thought it may lead to a complicated boundary condition.
Besides, as can be noticed in Figure 22, the slave position
tends to be lower than the master side. The reason of this
trendpossiblyisthattheslaveisdesignedtobecontrolled
by a position controller based on a simple PID controller
without gravity compensation.
Another possible concern is the stiffness of the robot. It
is well-known that stiffness is important to any robotic
system’s performance.
44
In a MMT system with the pro-
posed controller, the geometry change of the robot means
that the estimated environment stiffness is coupled with
the stiffness of the environment and slave manipulator, so
the modeling accuracy may degrades. In addition, the
stiffness of end-effector force sensors also contributes to
the error. Fortunately, even though the stiffness properties
of the slave manipulator cannot be neglected. We can
easily found that since the environment is modeled to
include the stiffness of both environment and slave
manipulator, the force tracking does not degrade like the
position tracking, which is comforting for most force-
feedback teleoperation tasks.
The stiffness analysis of a robotic system is a general
issue in robot control rather than a teleopration problem.
However, a complete analysis that takes the slave manip-
ulator’s stiffness performance into consideration should be
done in future work due to their importance for a successful
robotic systems.
Position-dependent environment dynamics model
Different from typical simplified task in the experiment
that has only one contact point, in real complex tasks, envi-
ronment model must be position-dependent so as to match
the position-dependent nature of environment impedance.
Position-dependent model was proposed and has shown its
16 International Journal of Advanced Robotic Systems
advantages comparing with time-based approach.
28
The
workspace is quantized into discrete components and each
of them represents a small area in the slave robot’s work-
space. The estimated parameters are stored in the data node
corresponding to the current tip position of the robot. Thus,
prestored estimated parameters can provide prior knowl-
edge when the robot back to the same position to shorten
transition state.
However, it is still an open issue. In current position-
dependent model, only parameters in measured position
are updated while others remain the initial value. Consid-
ering the large amount of data nodes, only a few part of
information in the model can be updated using the direct
measurement. The model works only when the new con-
tact point is the exactly one of the previous contact points.
While the operator on the master side may be able to
estimate the impedance in an unmeasured position empiri-
cally and manual adjustment by operator leads to signif-
icant decrease in operation efficiency. Meanwhile,
making use of more available information to estimate the
parameters before the real contact happened plays an
important role as it determines the difference between
local model and remote environment at current position.
If the estimation is accurate, there will be no transition
state in current contact point. In addition, unnecessary
dramatic changes between updated points and the others
will be introduced if the estimated parameters differ
greatly from the initial value.
To solve this problem, a mechanism to estimate the
environment model parameters in unmeasured position
is needed. Meanwhile, its computation complexity must
be limited so that it can keep compatible with high
refresh rate.
Considering the physical characteristic of common
environment surface, the parameters are continuous in
most areas even in the presence of dramatic changes at
the boundaries, which can be either visible or invisible.
The environment impedance in one position is usually
similar or even identical to the impedance in near posi-
tion when the material in these positions is same. In
addition, teleoperation system usually has a camera and
the vision information is valuable for estimating para-
meters of contact dynamics model because the images
show which part surface of environment are in the same
appearances. Same appearances mean possible same
material and thus similar dynamic characteristic. The
perceivable boundary may mean different object or
material surface, which has different or unrelated impe-
dance characteristics. Note that estimating environment
impedance by vision and measured sample is in accord
with human common sense. It is reasonable to assume
that the near area has the similar impedance.
In order to estimating environment parameters before
contact, we propose a method that consists by two steps.
First, get the image of the environment surface and extract
the contour. Then divides the position-dependent model
into many regions using the extracted contour. For the
accuracy, the division can be done by operator manually.
The expense on this process is acceptable because it is only
a one-time effort. Second, every time a new estimated
result is given, updating the data node corresponding to the
current position. Then using nearest neighbor interpolation
to obtain the value of all unmeasured data nodes.
Although the interpolation does not guarantee the accu-
racy as there is no any direct information, it is difficult to
prove its effectiveness theoretically as we cannot obtain
mathematical formulation of the unknown real environ-
ment. It is believed that this method is effective in most
cases based on the discussion before. An improved estima-
tion of environment impedance not only shortens the tran-
sition state but also accelerates the parameter estimation
process and thus improves the impedance controller’s per-
formance.
28
In addition, the improved model can be used to
display a continuous impedance map of the environment to
give the operator a visible description of environment
dynamics. This is particular valuable for palpation or dam-
age detection tasks. A similar application was investigated
in the study by Yamamoto et al.
22
Conclusions
As an alternative to bilateral force-feedback teleoperation,
MMT is developed to enable efficient operation especially
under large time delay. In this article, we put forward a
force-based model updating algorithm and an adaptive
impedance controller to improve the stability of MMT.
We firstly proposed a new force-based approach to
solve the model jump effect in MMT. The force calculated
by the latest model parameters is set as the target, and it
makes the rendered force to approach the target as soon as
possible. By dividing the rendered force into normal part
and abnormal part, then the abnormal part is limited to
ensure stable interaction and thenormalpartisretainedto
respond the normal operation. By accomplishing the
model updating more effectively, the force-based methods
areabletosignificantlyimprove the updating efficiency
compared to the existing solutions while ensuring a stable
human-machine interaction.
On the slave side, motivated by the potential risk of
current slave controllers, an adaptive impedance controller
was introduced. We leveraged the specialty of MMT that
both force and position can be acquired for slave controller,
which give two reference input to the controller. Mean-
while, by comparing the real environment stiffness with
the one in the prediction model on the master side, the
impedance parameters were adjusted automatically. As a
result shown in both theory and experiment, when there are
errors between the prediction model and real environment,
the slave executes the task in a conservative way that
emphasis on force or position accordingly to avoid unde-
sired large penetration or contact force. Together with the
switching-free characteristic, the proposed controller is
Song et al. 17
deemed to provide better stability and safety for MMT. We
implemented the controllers in a 1-DOF experiment sys-
tem. Experiments results on different environment material
verified the superiority of the proposed methods.
Although the results in the experiment is encouraging, it
is worth noting that obtaining ideal tracking and force feed-
back under large time delay requires more accurate envi-
ronment model. In addition to the environment dynamics
model, the geometry model of environment is also needed
and vital for the application of MMT.
Future work consists in the extension of proposed con-
troller to a multiple degree of freedom tasks and the experi-
ments with improved position-dependent environment
model. Furthermore, a comprehensive and theoretical sta-
bility analysis is needed.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) disclosed receipt of the following financial support
for the research, authorship, and/or publication of this article: The
project was supported by the Open Research Fund of Key Labora-
tory of Space Utilization, Chinese Academy of Science (no. LSU-
2016-05-2).
Supplemental material
Supplementary material for this article is available online.
References
1. Becquet MC. Teleoperation: numerical simulation and
experimental validation, vol. 4. Dordrecht: Springer Science
& Business Media, 2012.
2. Lawrence DA. Stability and transparency in bilateral teleo-
peration. IEEE Trans Robot Autom 1993; 9(5): 624–637.
3. Rebelo J and Schiele A. Time domain passivity controller for
4-channel time-delay bilateral teleoperation. IEEE Trans
Haptics 2015; 8(1): 79–89.
4. Sun D, Naghdy F, and Du H. Wave-variable-based passivity
control of four-channel nonlinear bilateral teleoperation
system under time delays. IEEE/ASME Trans Mechatron
2016; 21(1): 238–253.
5. Niemeyer G and Slotine JJ. Stable adaptive teleoperation.
IEEE J Ocean Eng 1991; 16(1): 152–162.
6. Li H and Kawashima K. Achieving stable tracking in wave-
variable-based teleoperation. IEEE/ASME Trans Mechatron
2014; 19(5): 1574–1582.
7. Hace A, Franc M, and Jezernik K. Sliding mode control for
scaled bilateral teleoperation. In: IECON 2011-37th annual
conference on IEEE industrial electronics society,Mel-
bourne, Australia, 7–10 November 2011, pp. 3430–3435.
IEEE.
8. Gonzalez N, Salas-Pen
˜a O, DeLeon-Morales J, et al.
Observer-based integral sliding mode approach for bilateral
teleoperation with unknown time delay. Automatika J Con-
trol Meas Electron Comput Commun 2016; 57(3): 749–760.
9. Ajoudani A, Tsagarakis NG, and Bicchi A. Tele-impedance:
teleoperation with impedance regulation using a body-
machine interface. Int J Rob Res 2012; 31(13): 1642–1656.
DOI: 0278364912464668.
10. Ye Y, Pan YJ, and Hilliard T. Bilateral teleoperation with
time-varying delay: a communication channel passification
approach. IEEE/ASME Trans Mechatron 2013; 18(4):
1431–1434.
11. Mitra P and Niemeyer G. Model-mediated telemanipulation.
Int J Rob Res 2008; 27(2): 253–262.
12. Xu X, Cizmeci B, Schuwerk C, et al. Model-mediated
teleoperation: toward stable and transparent teleoperation
systems. IEEE Access 2016; 4: 425–449.
13. Passenberg C, Peer A, and Buss M. Model-mediated teleo-
peration for multi-operator multi-robot systems. In: 2010
IEEE/RSJ international conference on intelligent robots and
systems (IROS), Taipei, Taiwan, 18–22 October 2010, pp.
4263–4268. IEEE.
14. Willaert B, Bohg J, Van Brussel H, et al. Towards multi-dof
model mediated teleoperation: using vision to augment feed-
back. In: 2012 IEEE international workshop on haptic audio
visual environments and games (HAVE), Munich, Germany,
8–9 October 2012, pp. 25–31. IEEE.
15. Winck RC, Sketch SM, Hawkes EW, et al. Time-delayed
teleoperation for interaction with moving objects in space.
In: 2014 IEEE international conference on robotics and auto-
mation (ICRA), Hongkong, China, 31 May–7 June 2014, pp.
5952–5958. IEEE.
16. Xia T, L´eonard S, Kandaswamy I, et al. Model-based tele-
robotic control with virtual fixtures for satellite servicing
tasks. In: 2013 IEEE international conference on robotics
and automation (ICRA), Karlsruhe, Germany, 6–10 May
2013, pp. 1479–1484. IEEE.
17. Smisek J, van Paassen RM, and Schiele A. Naturally-
transitioning rate-to-force controller robust to time delay by
model-mediated teleoperation. In: 2015 IEEE international
conference on systems, man, and cybernetics (SMC), Hong
Kong, China, 9–12 October 2015, pp. 3066–3071. IEEE.
18. Feth D, Peer A, and Buss M. Incorporating human haptic
interaction models into teleoperation systems. In: 2010
IEEE/RSJ international conference on intelligent robots and
systems (IROS), Taipei, Taiwan, 18–22 October 2010, pp.
4257–4262. IEEE.
19. Winck RC and Okamura AM. Model-mediated teleopera-
tion with predictive models and relative tracking. In: ASME
2013 dynamic systems and control conference, Palo Alto,
United States, 21–23 October 2013, pp. V002T26A003–
V002T26A003. American Society of Mechanical Engineers.
20. Lloyd B, Sz´ekely G, and Harders M. Identification of spring
parameters for deformable object simulation. IEEE Trans Vis
Comput Graphics 2007; 13(5): 1081–1094.
21. Mihelj M and Podobnik J. Haptics for virtual reality and
teleoperation, vol. 67. Dordrecht: Springer Science & Busi-
ness Media, 2012.
18 International Journal of Advanced Robotic Systems
22. Yamamoto T, Vagvolgyi B, Balaji K, et al. Tissue property
estimation and graphical display for teleoperated robot-
assisted surgery. In: IEEE international conference on robotics
and automation, 2009. ICRA’09, Kobe, Japan, 12–17 May
2009, pp. 4239–4245. IEEE.
23. Haddadi A and Hashtrudi-Zaad K. Online contact impedance
identification for robotic systems. In: 2008 IEEE/RSJ inter-
national conference on intelligent robots and systems, Nice,
France, 22–26 September 2008, pp. 974–980. IEEE/RSJ.
24. Haddadi A and Hashtrudi-Zaad K. Real-time identification of
hunt–crossley dynamic models of contact environments.
IEEE Trans Robot 2012; 28(3): 555–566.
25. Willaert B, Van Brussel H, and Niemeyer G. Stability of
model-mediated teleoperation: discussion and experiments.
In: International conference on human haptic sensing and
touch enabled computer applications,vol.1.Springer:
Springer Berlin Hedidelberg, 2012, pp. 625–636.
26. Mitra P, Gentry D, and Niemeyer G. User perception and
preference in model mediated telemanipulation. In: Second
joint eurohaptics conference and symposium on haptic inter-
faces for virtual environment and teleoperator systems
(WHC’07), Tsukaba, Japan, 22–24 March 2007, pp.
268–273. IEEE.
27. Xu X, Schuwerk C, and Steinbach E. Passivity-based model
updating for model-mediated teleoperation. In: 2015 IEEE
international conference on multimedia & expo workshops
(ICMEW), Turin, Italy, 29 June–3 July 2015, pp. 1–6. IEEE.
28. Love LJ and Book WJ. Force reflecting teleoperation with
adaptive impedance control. IEEE Trans Syst Man Cybern
Syst 2004; 34(1): 159–165.
29. Xu X, Cizmeci B, Al-Nuaimi A, et al. Point cloud-based
model-mediated teleoperation with dynamic and
perception-based model updating. IEEE Trans Instrum Meas
2014; 63(11): 2558–2569.
30. Mobasser F and Hashtrudi-Zaad K. Predictive teleoperation
using laser rangefinder. In: 2006 Canadian conference on
electrical and computer engineering, Ottawa, Canada, 7–10
May 2006, pp. 1279–1282. IEEE.
31. Huijun L and Aiguo S. Virtual-environment modeling and
correction for force-reflecting teleoperation with time delay.
IEEE Trans Ind Electron 2007; 54(2): 1227–1233.
32. Xu X, Kammerl J, Chaudhari R, et al. Hybrid signal-based and
geometry-based prediction for haptic data reduction. In: 2011
IEEE international workshop on haptic audio visual envi-
ronments and games (HAVE), Qinhuangdao, China, 14–17
October 2011, pp. 68–73. IEEE.
33. Verscheure D, Swevers J, Bruyninckx H, et al. On-line iden-
tification of contact dynamics in the presence of geometric
uncertainties. In: IEEE international conference on robotics
and automation, Pasadena, United States, 19–23 May 2008,
pp. 851–856. ICRA 2008 IEEE.
34. Haddadi A and Hashtrudi-Zaad K. A new method for online
parameter estimation of hunt-crossley environment dynamic
models. In: 2008 IEEE/RSJ international conference on intel-
ligent robots and systems, Nice, France, 22–26 September
2008, pp. 981–986. IEEE.
35. Achhammer A, Weber C, Peer A, et al. Improvement of
model-mediated teleoperation using a new hybrid environ-
ment estimation technique. In: 2010 IEEE international
conference on robotics and automation (ICRA),Ancho-
rage, United States, 3–8 May 2010, pp. 5358–5363. IEEE.
36. Yamamoto T, Bernhardt M, Peer A, et al. Techniques for
environment parameter estimation during telemanipula-
tion. In: 2008 2nd IEEE RAS & EMBS international con-
ference on biomedical robotics and biomechatronics,
Scottsdale, United states, 19–22 October 2008, pp.
217–223. IEEE.
37. Hirche S and Buss M. Human-oriented control for haptic
teleoperation. Proc IEEE 2012; 100(3): 623–647.
38. Erickson D, Weber M, and Sharf I. Contact stiffness and
damping estimation for robotic systems. Int J Rob Res
2003; 22(1): 41–57.
39. Jung S, Hsia TC, and Bonitz RG. Force tracking impedance
control of robot manipulators under unknown environment.
IEEE Trans Control Syst Technol 2004; 12(3): 474–483.
40. Stanczyk B, Peer A, and Buss M. Development of a high-
performance haptic telemanipulation system with dissimilar
kinematics. Adv Rob 2006; 20(11): 1303–1320.
41. Hogan N. Controlling impedance at the man/machine inter-
face. In: 1989 IEEE international conference on robotics
and automation, 1989. Proceedings, Scottsdale, United
States, 14–19 May 1989, pp. 1626–1631. IEEE.
42. Tzafestas C, Velanas S, and Fakiridis G. Adaptive impedance
control in haptic teleoperation to improve transparency under
time-delay. In: IEEE international conference on robotics
and automation, 2008, Pasadena, United States, 19–23 May
2008, pp. 212–219. ICRA 2008, IEEE.
43. Love LJ and Book WJ. Environment estimation for enhanced
impedance control. In: 1995 IEEE international conference
on robotics and automation, 1995. Proceedings, vol. 2.
Nagoya, Japan, 21–27 May 1995, pp. 1854–1859. IEEE.
44. Carbone G. Stiffness analysis and experimental validation of
robotic systems. Front Mech Eng 2011; 6(2): 182–196.
Song et al. 19
... Velocity of the proxy is limited to make the master force change gradually without abrupt jump. Song et al. propose a method limiting the force magnitude that can change during one sample time [37]. These methods, however, have differences between the forces rendered to the operator and the applied to objects in the environment, which can degrade task performance in control of force and discrimination of stiffness. ...
... Figs. 10 and 11 are the results of a previous MMT with the force-based model update algorithm (FMU) [37]. The algorithm limits the change rate of the force rendered to the master. ...
... The algorithm limits the change rate of the force rendered to the master. The force-limiting gain is set to the same value used in [37]. Figs. 10 and 11 show the results without and with 50 ms time delay, respectively. ...
Article
Full-text available
Model-mediated teleoperation (MMT) employs an environment model at the master side to compute feedback output to the master at a faster rate. This approach improves system stability in the presence of time delay. MMT, however, does not generally perform well if the employed model is not accurate. Model mismatch is unavoidable when the environment is unknown in advance or varies. This paper proposes MMT employing an adaptive model. The proposed method adaptively moves the reference point of the employed model, whereas the previous MMTs used reference points fixed to the surface of objects in the environment. This can make system stability independent of the time delay. Experiments show that the proposed method improves stability compared to the previous MMTs when there are model mismatches. User studies are conducted to compare the operator’s performance in two tasks, control of force exerted to objects in the environment, and discrimination of object stiffness. The result shows that the error in the forces applied to objects in the environment significantly decreases in the proposed method. Errors in forces rendered to the master are also improved by at least 20.2 %. The experiment result also shows that subjects can discriminate up to 40.9 % smaller differences in the stiffness than the previous MMT under the same time delay.
... In [183], the authors proposed a control design to minimize model jumping based on an adaptive impedance controller and novel model updating algorithm. The adaptive impedance controller was deployed on the slave-side to estimate physical environment parameters with the ability to adjust its behavior without state transition; and a force-based updating local controller was employed on the master-side to update stiffness. ...
Article
Full-text available
The Tactile Internet has become a revolution for Internet technology, greatly improving the transmission of skill sets (audio, video, text, and haptics) over communication channels compared with traditional triple-play data (audio, video, text). It is a strong candidate to support next-generation delay-sensitive and loss-intolerant smart applications. However, stringent requirements for the Tactile Internet, including ultra-low latency, ultra-high reliability, high availability, and ultra-security, present critical challenges to ensure Quality of Service (QoS). Consequently, several approaches have been proposed to meet these QoS requirements. This article reviews QoS provisioning approaches for the Tactile Internet. First, we present key concepts for the fifth-generation and beyond technologies, Tactile Internet, and haptic communication. Second, we discuss the Tactile Internet use cases along with strict QoS requirements. Third, we classify existing solutions, including haptic codecs, control system designs, hybrid schemes, and intelligent prediction models; provide in-depth discussion regarding these approaches to improve QoS for the Tactile Internet applications; and investigate strengths and weaknesses for each proposed solution. Finally, we present open research challenges and discuss potential future research avenues to realize the Tactile Internet services.
... In [80], authors developed a model mediated system based on force updating algorithm and adaptive impedance controller. The force updating algorithm is introduced to overcome the force disturbance problem associated with the model updating, and thus achieves a higher level of system stability. ...
Article
Full-text available
Enabling haptic communication as well as voice and data over the future 5G cellular system has become a demand. Tactile Internet is one of the main use cases of the 5G system that will allow the transfer of haptic communications in real time. Latency, availability, reliability, and security are the main design challenges of the tactile Internet system and haptic based bilateral teleoperation systems. An end-to-end latency of 1 ms remains the main challenge toward tactile Internet system realization, not only for the processing and coding delays but mainly for the limitations of light. In this work, we analyze the key solutions to overcome the light limitations and enable the tactile Internet over any distances with the required latency. Building a virtual model or model mediated for the remote environment at the edge cloud unit near to the end user is the main solution. By means of AI, the virtual model can predict the behavior of the remote environment and thus, the end user can interact with the virtual environment with a high system experience. This literature review covers the existing work of the model mediated bilateral teleoperated systems and discusses its availability for the tactile Internet system. Furthermore, the security issues of tactile Internet system and the effect of model mediated system on the required security level of tactile Internet applications are discussed. Finally, a structure for the tactile Internet system, with the deployment of model mediation, is suggested.
... In [28], authors developed a model mediated system based on force updating algorithm and adaptive impedance controller. The force updating algorithm is introduced to overcome the force disturbance problem associated with the model updating, and thus achieves a higher level of the system stability. ...
Conference Paper
Enabling haptic communication as well as voice and data over the future 5G cellular system become a demand. Tactile Internet is one of the main use cases of the 5G system that will allow the transfer of haptic communications in real time. The end-to-end latency of 1ms remain the main challenge toward the Tactile Internet system realization, not only for the processing and coding delays but mainly for the limitations of light. In this work, we analyze the key solutions to overcome the light limitations and enable the Tactile Internet over any distances with the required latency. Building a virtual model or model mediated for the remote environment at the edge cloud unit near to the end user is the main solution. By means of AI the virtual model can predict the behavior of the remote environment and thus the end user can interact with the virtual environment with a high system experience. We review the exiting work for the model mediated bilateral systems and discuss its availability for the Tactile Internet system. Finally, we suggest a structure for the Tactile Internet system with the deployment of model mediated.
Article
Full-text available
In the haptic human-robot interaction systems, stability and transparency factors are critical but conflicting with one another. Ensuring safety and accuracy of bilateral haptic teleoperation systems is always an important tradeoff to be carefully balanced. Attaining a reasonable operating point for less painful compensation between stability and transparency has been the main concern when designing haptic human-robot systems. Some important works discussed in the paper include model-based control approaches like wave-variable transformation and scattering, time domain passivity/wave prediction with energy regulation, model mediated teleoperation approaches, along with model-free control approaches like neural networks and fuzzy control approaches. The objective to obtain a better agreement between stability and transparency criteria, along with a comprehensive review of these methods and a newly proposed technique attracted most interests of the paper. Earlier solutions tried to achieve a decent tradeoff but were limited to varying time delay and data loss encountered during transmission in the communication channels. The comparison of these methods demonstrates their performance, by illustrating their respective outlines, viability, and limitations, which can aid in the identification of compensation among the state-of-the-art methods and inspire novel ideas. The hardware platforms developed in literature are also summarized here to show the physical implementation of such systems. The paper concludes by suggesting the need of a hybrid method inclusive of an Active Disturbance Rejection Controller (ADRC), towards an even better operating point for the tradeoff between the transparency and stability of haptic human-robot systems.
Article
Full-text available
This paper deals with force-reflecting control design for teleoperation of bilateral robots under unknown constant time delay. The proposed impedance teleoperator control is based on integral sliding mode approaches, avoiding undesirable chattering effect. With the aim of implementing the proposed controller and taking into account that only position measurement is available, a Nonlinear Observer based on Super Twisting Algorithm is proposed to estimate velocity and acceleration in the slave side of the teleoperation system. Furthermore, owing to the finite-time convergence properties of the observer, the proposed control scheme guarantees robust tracking under unknown constant time delay. Experimental results illustrate the effectiveness of the proposed scheme.
Conference Paper
Full-text available
In this paper a new model-mediated teleoperation framework based on rate-control input is presented. The system facilitates force feedback teleoperation and is inherently robust to severe time delays yet still provides high level of rate/force tracking fidelity. Previously published methods on model-mediated teleoperation are here extended to accept rate commands and use the slave robot's programmed impedance as a contact model for force feedback generation. The system is designed to ensure decoupling of the control loops in case of environment changes, which also allows to cope with severe and possibly variable time-delay in the communication channel. Stability and performance is analyzed and the system is shown to remain stable for finite delays. Tuning guidelines with design trade-offs are provided. The presented method is demonstrated on an experimental 1-DOF master/slave setup in a teleoperation scenario, showing stability and reference rate & force tracking performance over 2 seconds of round-trip communication delay in bilateral control.
Article
Bilateral teleoperation systems with haptic feedback allow human users to interact with objects or perform complex tasks in remote or inaccessible environments. Communication delays in teleoperation systems jeopardize system stability and transparency, leading to degraded system performance and poor user experience. In this paper, we provide a survey of the model-mediated teleoperation (MMT) approach, which has been developed to guarantee both system stability and transparency in the presence of arbitrary communication delays. This survey focuses on two major parts: 1) the historical development of the MMT approach from the late 1980s to the present and 2) the main challenges facing the design of a reliable MMT system. Along with the discussion of the MMT challenges and the proposed solutions, a series of experiments has been conducted to compare the performance between the existing techniques and to supply data that were missing in the previous studies on the MMT approach.
Conference Paper
Telerobotics has The potential To facilitate The repair of satellites in geosynchronous orbit by allowing human operators To interact naturally with remote objects. Time delays on The order of seconds make it difficult To provide immersive feedback To The operator, motivating The use of predictive visual and haptic displays of The robot and environment. A Teleoperation framework developed for This scenario invokes a Two-part environment model That predicts motion of objects in The environment, both in free space and during contact with The robot. When objects in The environment are in free space, a propagated model using delayed data provides predictive feedback To The operator. However, when The robot interacts with The environment, a local environment model That does not propagate delayed data is used. This reduces computational load and ensures stability during robot-environment interactions. Two experiments were carried out To Test The Teleoperation system. Results demonstrate The ability of The prediction algorithm To provide reliable feedback and improve operator performance before, during, and after robot-environment interactions.
Article
Tools for analyzing teleoperation system performance and stability when communication delays are present are presented. A general multivariable system architecture is utilized which includes all four types of data transmission between master and slave: force and position in both directions. It is shown that the proper use of all four channels is of critical importance in achieving high-performance telepresence in the sense of accurate transmission of task impedances to the operator. It is also shown that transparency and robust stability (passivity) are conflicting design goals in teleoperation systems. Achieved transparency and stability properties of two common architectures, as well as a 'passivated' approach and a novel 'transparency optimized' architecture, are quantitatively compared on one-degree-of-freedom examples
Article
An innovative wave-based Time Domain Passivity Approach (TDPA) applied to a four-channel (4-CH) nonlinear teleoperation system is proposed. The primary objective of this approach is to enhance system transparency while maintaining stability in the presence of random time delays. The system stability for different scenarios of human and environment condition is analyzed. The method is validated through experimental work based on a 3-DOF bilateral teleoperation system. The experimental results show that the proposed control algorithm can robustly guarantee the system stability and simultaneously provide better performance than methods developed in previous work.
Article
In this paper, we extend the concept of model-mediated teleoperation for complex environments and six degrees of freedom interaction using point cloud surface models. In our system, a time-of-flight camera is used to capture a high-resolution point cloud model of the object surface. The point cloud model and the physical properties of the object (stiffness and surface friction coefficient) are estimated at the slave side in real-time and transmitted to the master side using the modeling and updating algorithm proposed in this paper. The proposed algorithm adaptively controls the updating of the point cloud model and the object properties according to the slave movements and by exploiting known limitations of human haptic perception. As a result, perceptually irrelevant transmissions are avoided, and thus the packet rate in the communication channel is substantially reduced. In addition, a simple point cloud-based haptic rendering algorithm is adopted to generate the force feedback signals directly from the point cloud model without first converting it into a 3-D mesh. In the experimental evaluation, the system stability and transparency are verified in the presence of a round-trip communication delay of up to 1000 ms. Furthermore, by exploiting the limits of human haptic perception, the presented system allows for a significant haptic data reduction of about 90% for teleoperation systems with time delay.