Conference PaperPDF Available

Online Computation of Safety-Relevant Regions for Human Robot Interaction

Authors:
  • ORIGITECH GmbH

Abstract and Figures

For scenarios of robotic manipulators interacting with humans in a shared workspace, it is of paramount importance to avoid collision and to guarantee safe interaction. Thus, safety-relevant regions, i.e., regions of the workspace which are possibly occupied by a human, need to be reliably predicted and excluded from the workspace in which robot trajectories are planned. In our previous work, we have proposed using the probabilistic framework of Hidden Markov Models (HMMs) to account for the non-determinism in human motion. Such models often lead to sufficiently accurate predictions of safety-relevant regions (as opposed to single, most likely human positions) if the human motion follows one of a finite set of known motion patterns. However, predictions may be inaccurate to the extent of being useless if an unknown motion pattern is encountered, which may degrade the robots performance and may also cause safety problems. In this paper, the latter problem is avoided by combining HMM-based prediction with reachability analysis. This analysis is based on the present observation of the human's position, on dynamic limits of human motion such as velocity and acceleration limits, and on bounds of measurement errors. Based on the proposed method, a prediction of the unsafe region of the workspace can be computed online. This way, one arrives at motion plans for the robot, which are safe in the sense that collision between human and robot is avoided with a probability not less than a predefined threshold. The practicability of our method is successfully demonstrated by online predicting the motion of human hands in two scenarios with different human workers, multiple motion patterns, and multiple goals. The method is embedded into a scheme for online optimization of trajectories for an industrial robot which interacts with a human worker.
Content may be subject to copyright.
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
Online Computation of Safety-Relevant Regions
for Human Robot Interaction
Hao Ding
ABB
Corporate Research
Germany
hao.ding@de.abb.com
Kurniawan Wijaya
Institute of Control and
System Theory, University
of Kassel, Germany
Gunther Reißig
Chair of Control Engineering,
University of the Federal
Armed Forces Munich,
Germany
http://www.reiszig.de/gunther/
Olaf Stursberg
Institute of Control and
System Theory, University
of Kassel, Germany
stursberg@uni-kassel.de
AbstractFor the purpose of robot motion planning in
human robot interaction, the efficient and accurate prediction
of safety-relevant regions that are possibly occupied by the
human within the planning horizon is a necessity. The proba-
bilistic framework of Hidden Markov Models (HMMs) has been
proposed to account for the nondeterminism in human motion.
Such models often lead to sufficiently accurate predictions if
the human motion follows one of a finite set of known motion
patterns. But the HMMs need to be continuously updated
during use in order to cope with new patterns. Existing update
methods require any newly observed motion pattern to be
completed before it can be used for prediction. In this paper, the
latter problem is avoided by employing a combination of HMM
based prediction with information obtained from reachability
analysis based on dynamic limits of human motion. The method
has been successfully implemented to predict the motion of a
human hand online.
Keywords: Human motion modeling and prediction, Robot
safety, Reachability analysis, Hidden Markov Models, Human
robot interaction
I. INTRODUCTION
For scenarios of robotic manipulators interacting with
human operators in a shared workspace, it is of paramount
importance to avoid collision and to guarantee safe in-
teraction. Thus, safety-relevant regions, i.e., regions of
the workspace which are possibly occupied by a human,
need to be reliably predicted and excluded from the free
workspace in which robot trajectories are planned. It should
be stressed that it is in general not sufficient in specific
scenarios to only compute the most likely or expected human
positions, namely prediction in the form of single points, but
the nondeterminism in human behavior must be accounted
for. Moreover, since the robot and human share the same
workspace and prediction of human motion at the initial time
is not sufficient, methods for online adaptation of models of
human motion and for predicting human motion within a
planning horizon are important in this context.
Classical estimation techniques based on kinematic or
dynamic models of human motion [1], [2], [11] and other
deterministic methods [4], [5], [9] may produce accurate
short-term prediction. However, their performance in longer-
term prediction is usually not satisfactory since they do
not take into account aspects of human motion such as
intention. Thus, most current research focuses on implicit
modeling methods. Based on the assumption that humans
tend to follow motion patterns, the probabilistic framework
of Hidden Markov Models (HMMs) has been proposed [3],
[10]. In general, this approach consists of two phases,
namely, the identification phase and the prediction phase. An
online variant of HMMs, named Growing HMMs (GHMMs),
which incrementally improves the HMM based on newly
observed motion, has been presented in [16]. Based on the
GHMMs, we have proposed a method for computing safety-
relevant regions of the workspace [6]. These regions should
serve as safety constraints when robot motion is planned or
optimized. This way, one arrives at motion plans that are
safe in the sense that collision between human and robot
is avoided with a probability not less than a predefined
threshold.
However, GHMMs can only be updated once a trajectory
sequence is completed. That is, when the human motion
follows an unknown pattern, the prediction obtained from the
current HMM is usually useless until the motion has been
completed such that the HMM can be updated – this may
lead to performance degradation and safety issues. In this
paper, the latter problem is avoided by combining HMM-
based prediction with reachability analysis of the dynami-
cally feasible behavior of the human operator. This analysis
is based on the dynamic limits of human motion such
as velocity and acceleration limits including measurement
errors. Based on the proposed method, the predicted safety-
relevant region of the workspace can be computed online by
combining the probabilistic model and reachability analysis
with a given probability threshold.
The paper is organized as follows: Section II gives the
formulation of an optimal control problem for robotic manip-
ulators safely interacting with human operators. This leads to
the necessity of generating the safety-relevant regions. Fol-
lowing the background of modeling and prediction based on
HMMs, a schematic example is shown there for illustrating
the enhancement of the prediction accuracy by combining
reachable sets in Section III. We then provide the method for
computing reachable sets and the improved safety-relevant
region. Finally, we demonstrate our method in two scenarios
with different human operators for accurately predicting
the human hand motion with multiple motion patterns and
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
multiple goals in Section IV.
II. PROBLEM STATEMENT
The following notation is used throughout the paper:
Workspace /Cartesian space:Z=R3, in which
the robot and human work; points [z1, z 2, z3]TZ
represent positions.
Configuration space:C=Rn;[q1, q2,...,qn]TC
represent joint angles of the robotic manipulator with n
degrees of freedom.
State space: X=R2n;x(t) = [q(t),˙q(t)]TX
represent the state of the robotic manipulator.
zjZdenotes the position of joint jincluding the
position of the end effector in the workspace with j
{1,...,n+ 1}. For the sake of notational simplicity, zj
is used as a shortcut for z(qj).
Observation space:O=Z×Z;O(t) =
[z1(t), z2(t), z3(t),˙z1(t),˙z2(t),˙z3(t)]TOrepresent
data of human motion measured or derived; specifically,
one observation consists of position and velocity in the
workspace.
The motion planning problem of robotic manipulators can
be formulated as an optimization problem as follows:
min
u(t)
g
X
τ=0
Jτ(x(τ), u(τ)) (1a)
s.t. x(t+ 1) = f(x(t), u(t)),t∈ {0,1,...,g1}
(1b)
x(t0) = x0, zn+1(g)G(1c)
x(t)L, u(t)U(1d)
Ω(x(t)) F(t) = ,t∈ {0,1,...,g},(1e)
where
Jτis a cost function which represents a performance
measure of the robot at each sampling time τ;
(1b) is derived from the actual differential equations
modeling the robot in continuous time, and describes
the system dynamics in discrete time, where x(t)X
is the system state, u(t)Uis an admissible control
(torques, etc.), and t∈ {0,1, . . . , g}denotes the current
time, and 0,1,...,g are the sampling instances;
x0is the initial state of the robot, and GRn, the
target set for the end effector zn+1(g);
LXrepresents admissible states x(t), excluding e.g.
infeasible configurations of the robot;
Ω(x(t)) Zis the set of points in the workspace
occupied by the robot;
F(t)Zdenotes the set of points in the workspace
that are possibly occupied by a human.
Since positions measurements are obtained with a defined
frequency (e.g. by a camera system) and since the control of
the robotic manipulator has to be realized in discrete time,
the optimization problem (1) is also formulated in a discrete
time setting. In order to still guarantee safety everywhere
in between the sampling instants, dynamic properties of the
human and the robot (such as velocity bounds) can be used
to tighten the constraints (1e) and (1d), respectively [8] [12,
Section 5.3.4].
Online generation of the set Fat future time instances,
namely, F(t+r)with r∈ {1,...,H}, where Hdenotes
the prediction or planning horizon, is also necessary in the
course of solving more general motion planning problems,
e.g. [12]. It is precisely the problem of determining F(t+r)
which is investigated in the paper.
III. HUMAN MOTION MODELING AND PREDICTION
Our method of modeling and predicting human motion,
and particularly, of determining the safety-relevant regions
F(t+r), consists of the following steps:
(i) Based on a set of updated experimental data, a time-
evolving Hidden Markov Model is built.
(ii) According to the current observation O(t)of the po-
sition of the human, a conditional probability density
function for reaching a position at time instants t+
1,...,t+H, and the reachable set of the human position
is determined using dynamic limits of the motion.
(iii) The safety-relevant regions in the workspace, which
contain the positions possibly occupied by the human
operator in future times, are determined by combining
the probabilistic model and the reachable set.
A. Background
1) Probabilistic model: HMM:
III.1 Definition. The H M M = (S, A, S(0), O, P (O|S))
used in the paper consists of (according to [14]):
S(t)is the stochastic state variable S(t)
{S1, S2,...,SN} ⊂ Nat time t. The value of S(t)
refers to a point in the observation space O. In general,
this hidden state can also include the acceleration.
ARN×Nis a transition matrix whose entry aij in
row iand column jis given by aij =P([S(t+ 1) =
Si]|[S(t) = Sj]).
Initial state probability:
P(S(t= 0)) = [P(S(t= 0) = S1), P (S(t= 0) = S2),
. . . , P (S(t= 0) = SN)],(2)
where each element contains the initial probability for
the corresponding discrete state at time t= 0.
The observation variable O(t)O.
The observation probability density function:
P(O(t)|[S(t) = Si]) = G(Ot;µi,Σ) (3)
is represented by a Gaussian distribution for every
discrete state Si.µiis the mean value of each dis-
tribution, namely µi=Si. All discrete states have the
same covariance matrix Σ. The set of all Gaussian
parameters are denoted by b={Σ, µ1,...,µN}.
The full set of parameters for the HMM is λ={P(S(t=
0)), A, b}, which is computed based on the sequences of
observations O1:Tgwith Tgduration of the human motion
from the start to the goal. Details of the identification of the
HMM as well as the time-varying adaptation of the HMM
can be found in [6], [16].
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
2) Model-based prediction:
III.2 Definition. Given a new observation O(t)at time t,
the prediction of the belief state for the object is estimated
using:
P(S(t+r) = Si|O(t))
=
N
X
j=1
P(S(t+r) = Si|S(t+r1) = Sj)
·P(S(t+r1) = Sj|O(t))
=
N
X
j=1
aij ·P(S(t+r1) = Sj|O(t)),(4)
where iand jare state indices.
The conditional probability with a given point Onin the
observation space at time t+r,r∈ {1,...,H}, denoted by
P(O(t+r) = On|O(t)), is defined as:
P(O(t+r) = On|O(t)) = y(O(t+r) = On|O(t))
RRny(τ|O(t)),(5)
where
y(O(t+r) = On|O(t)) =
N
X
i=1
P(S(t+r) = Si|O(t))
·P(O(t+r) = On|S(t+r) = Si),
and P(O(t+r) = On|S(t+r) = Si)is defined by (3) with
the state Si∈ {S1,...,SN}.
The derivation of the equations can be found in [6].
3) Generation of safety-relevant regions using a given
probability threshold: For any given point Onin the observa-
tion space, the conditional probability density P(O(t+r) =
On|O(t)) of reaching the point Onat future time t+rwith
respect to the current observation O(t)is obtained from (5),
where only the position information in OnOis used.
In order to efficiently compute F(t+r), we need to avoid
working with an infinite number of points in the observation
space. Therefore, the observation space is divided into D
cells Qi,i∈ {1,...,D}. The centroid Odof each cell
is selected for computing the probability of landing in the
corresponding cell. For the sake of simplicity, it is assumed
that the probability density is constant in each cell, and (5)
is approximated by
P(O(t+r) = Od|O(t))
y(O(t+r) = Od|O(t))
PD
d=1 y(O(t+r) = Od|O(t)).(6)
The indices {1,...,D}are arranged in an array wsuch
that
P(O(t+r) = Owi|O(t)) P(O(t+r) = Owj|O(t)) (7)
whenever j > i, where widenotes the entry at position i
of w. Finally, let a threshold δ[0,1] be given and define
F(t+r)by
F(t+r) =
ϕ
[
i=1
Qwi,(8)
where
ϕ= min
κ∈{1,...,D} κ
X
l=1
P(O(t+r) = Owl|O(t)) δ!.(9)
This means, we are neglecting cells Qiof low probability.
With this choice of F(t+r), the probability pof the future
observation O(t+r)being contained in F(t+r)is lower
bounded by δ:
p=
ϕ
X
l=1
P(O(t+r) = Owl|O(t)) δ. (10)
B. Improving Prediction Quality using Reachability Analysis
1) Principle: As mentioned in the introduction, the online
scheme for updating the HMMs can only change the model
when a motion pattern is completed. This leads to an inaccu-
racy of the model, if a new motion pattern is encountered. A
schematic example is shown in Fig. 1 for further illustration.
Motion patterns
New behavior
Prediction based on HMM
Reachable set ψ(t+r)
Current observation
Start
Goal
Fig. 1. Schematic example.
Two motion patterns for
human motion are marked by
solid lines from the start to
the goal. In a new motion,
marked by the dashed line,
the human moves along a dif-
ferent path. The prediction
from an intermediate point on
this path can only be based on
the current observation and
the HMMs, which results in
the safety-relevant region represented by the dark gray area.
Clearly, this ‘safety-relevant region’ is actually not safe. This
motivates the introduction of the reachable set ψ(t+r)to
enhance the prediction. ψ(t+r)is computed based on the
current measurement (including the measurement errors for
positions and velocities) and uses lower and upper bounds
for the velocity and acceleration of human motion to con-
servatively approximate the attainable part of the workspace
for the prediction horizon.
2) Online computation of the reachable set: In order to
increase the prediction accuracy, the computation of the
reachable set ψ(t+r)at time t+ris carried out based
on the knowledge of O(t) = (z(t),˙z(t)) and the dynamic
limits of human motion as follows.
The method we propose is based on the following reacha-
bility result, in which a function γrepresents one component
of the position vector zR3, and for the sake of simplicity
of presentation, the position has been shifted in time in order
to arrive at an the interval [0, T ]of definition for γ.
III.3 Lemma. Let T > 0and assume γ: [0, T ]Ris twice
differentiable. Assume further that γfulfills the following
conditions:
(i)γ(0) [γ, γ+]for some γγ+; (measurement
error);
(ii) ˙γ(0) [ ˙γ,˙γ+]for some ˙γ˙γ+; (measure-
ment/estimation error);
(iii)t[0,T ]˙γ(t)[v, v+]for some v,v+with v
˙γ˙γ+v+, where vand v+denote lower and
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
upper bounds, respectively, on the speed of the human
motion;
(iv)t[0,T ]¨γ(t)[a, a+]for some a,a+with a
a+, where aand a+denote lower and upper bounds,
respectively, on the acceleration of the human motion.
Then γfulfills the following upper bound for all t[0, T ]:
γ(t)γ(t)upper =
γ++
t·˙γ++t2·a+
2,
if a+= 0 or tt1=v+˙γ+
a+;
v+˙γ+
a+·˙γ++(v+˙γ+)2
2a++v+·(tv+˙γ+
a+),
otherwise, (11)
as well as the lower bound obtained from (11) by substituting
+with of γ,aand v, and with .
Therefore, the reachable set ψ(t+r)at time t+rin the
future can be computed as below:
ψ(t+r) := [¯γ(r)lower ,¯γ(r)upper ],(12)
where the interval is a subset of R3and the vector-valued
bounds are ¯γlower and ¯γupper, obtained for the three com-
ponents of the position zR3.
3) Combining the probabilistic model and reachability
analysis: The crucial point about the combination of HMMs
with reachability analysis for the purpose of improving the
quality of prediction is the following observation: Ordering
the cell indices i={1,...,D}according to (7), i.e., starting
from the highest probability cell first, prior to the computa-
tion of the safety-relevant region (8) is not a prerequisite
for the methods of Sec. III-A.3 to work. In fact, the order
(7) has been introduced for the sole purpose of keeping the
safety-relevant region (8) small, and the method is justified
for any order of the cell indices as long as (9) is fulfilled,
which implies (10). Consequently, this degree of freedom
may be used to improve the quality of prediction if additional
information is available that cannot be obtained from the
data from which the HMM was built. In the present paper,
we propose to take advantage of such additional information
by replacing the order (7) by the following one: Before
ordering, the cells that intersect the reachable set ψ(t+r)are
selected first. The set of those cells is Sβ
i=1 Qiwith βbeing
the number of cells that intersect the reachable set ψ(t+r).
Then the indices {1,...,β}are arranged in an array u
according to the probabilities determined before by (6)
P(O(t+r) = Oui|O(t)) P(O(t+r) = Ouj|O(t)) (13)
with βj > i 1, where uidenotes the entry at position
iof u.
The indices of the remaining cells {β+ 1,...,D}are
arranged in an array vsuch that their order matches the
relation:
P(O(t+r) = Ovi|O(t)) P(O(t+r) = Ovj|O(t)) (14)
for Dj > i β+ 1, where videnotes the entry at
position iof v.
Finally, with a specified δ[0,1], the enhanced safety-
relevant region F(t+r)is defined by
F(t+r) =
ψ
[
i=1
(Qui+Qvi),(15)
where
ψ=
argminκ∈{1,...,β}(Pκ
l=1 P(O(t+r) = Oul|O(t)) δ),
if Pβ
l=1 P(O(t+r) = Oul|O(t)) δ;
argminκ∈{β+1,...,D}(Pκ
l=1 P(O(t+r) = Ovl|O(t))
δPβ
l=1 P(O(t+r) = Oul|O(t)),
if Pβ
l=1 P(O(t+r) = Oul|O(t)) < δ;(16)
This new order implies that in the course of assembling
the enhanced safety-relevant region (15), cells intersecting
the reachable set ψ(t+r)are used first, followed by some
of the remaining cells if necessary to satisfy (16). Hence, if
the prediction that could be obtained from the HMM alone is
of high quality, the safety-relevant region should be a subset
of ψ(t+r)and the effect of the new ordering is negligible.
However, if an unexpected motion pattern occurs, then the
prediction that could be obtained from the HMM alone is
of a low quality. In this case, the probability assigned to
the cells in the reachable set will be very small, hence the
reachable set will be a subset of the computed safety-relevant
region, and hence, the latter will contain the next observation
with certainty.
We finally remark that for the purpose of reducing the
complexity of solving the optimal control problem in (1a),
the safety-relevant region F(t+r)may also be over-
approximated by sets that have a simpler structure, e.g. [15].
IV. APPLICATION RESULTS
The proposed method is realized in the following human
assembly task scenarios. The safety-relevant regions can be
online generated.
A. Scenario 1
The first human assembly task scenario is shown in Fig. 2.
A human operator moves his hand from Start to Goal1
for picking up a workpiece to then accomplish an assembly
task at Start. He has also to pick up another workpiece at
Goal2. A red marker is attached to his hand and detected by
a stereo camera system which is operated at 10 frames per
second. The workspace of the human operator is divided
into 252 cells, which are translated copies of the cuboid
[0,12.27] ×[0,12.75] ×[0,13.5]cm3that are arranged in a
12 ×7×3-grid. For better illustration, here and in what
follows, the 3D workspace is represented as a combination
of three planes, which correspond to cross-sections parallel
to the (z1z2)-plane.
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
z1
z1
z1
z1
z1
z1
z1z1
z2
z2
z2
z2
z2
z2
z2z2
z3
z3z3
Start Start
Start
Start
Goal1
Goal1
Goal1
Goal1
Goal2
Goal2
Goal2
Goal2
Workspace
Cell
Fig. 2. Illustration of scenario 1.
1) Online generation of the safety-relevant regions using
a given probability threshold: The HMM identifying the
motion of the human hand is adapted online after capturing
the motion of the human operator with multiple motion
patterns from the start to the goal. The prediction of the
mean value of the state positions, which indicates the trend
of the motion is marked by the green stripe. It is computed
according to Eq. (13) in [6].
Based on the description in Sec. III-A.3, the safety-
relevant region F(t+r)with a given threshold δ= 0.9996
for the motion with the existing motion patterns can be
generated online, using the current observation O(t). Fig. 3
shows the predictions F(t+r)for r∈ {1,...,8}. Different
colors according to the colorbar in Fig. 3 indicate the
corresponding probabilities in the cells computed by (6).
.
patterns
t+ 1 2 3 45 6 7 8
0
0.01
0.2
1
O(t)
Start
Goal1
Motion patterns Mean value prediction
Fig. 3. Predicted safety-relevant regions based on existing motion patterns.
2) Online generation of the safety-relevant regions using
a combination of a probabilistic model and reachability
analysis: As shown in Fig. 4, the prediction based only
on the HMM is not accurate enough with a new trajectory
that does not follow an existing motion pattern. The human
operator moves his arm towards Goal2, yet the mean value
of the state prediction (the green dot shown in Fig. 4(a))
follows the existing motion pattern towards Goal1.
We now additionally consider the reachability analysis
to enhance the quality of the prediction. The prediction
of the reachable set (uniform gray color) based on the
current observation O(t)is shown in Fig. 4(c). Only
the reachable set inside the workspace is considered here.
Dynamic limits such as bounds on velocity and acceleration
of human motion, and particularly, of the motion of the hand,
are taken from literature [9]. Here, we use the parameters
v±=±60cm/s and a±=±40cm/s2. In addition, we
use the bounds γ±=zi(t)±1cm and ˙γ±= ˙zi(t)±
3cm/s as required in the application of Lemma III.3, where
zi(t)and ˙zi(t)are the actual measurement and estimation,
respectively.
Goal1
Goal2
(a) Prediction.
(b) Prediction based on HMM solely.
t+ 1 2 3 45 6 7 8
(c) Computation of the reachable set.
Fig. 4. Prediction of a new trajectory towards Goal2, which does not
belong to existing motion patterns towards Goal1.
Fig. 5(a) shows the probability Paof the real trajectories
O(t+r)lying in the corresponding F(t+r)for t
{0,...,g}and r= 8, depending on the probability threshold
δ. For the case that an unforeseen motion pattern occurs, the
prediction quality is unsatisfactory if only the HMM is used.
The value of Pacorresponding to predictions solely based on
HMMs (red dashed line in Fig. 5(a)) is relatively small. Pa
drastically improves for the combination of the probabilistic
model with reachability analysis, as represented by the blue
solid line. Furthermore, the real trajectories lie always inside
the predicted reachable sets. Of course, the sets may be
rather large when the velocity and acceleration of the human
are large.
The graphical representation of the updated motion pat-
terns is finally shown in Fig. 5(b), which is obtained only
after the unforeseen motion patterns have been taken into
account.
0
δ
Pa
0.2
0.4
0.6
0.8
0.8
0.9
0.9
1
1
(a) Probability of trajectories for un-
expected motion patterns at t+ 8
lying in the enhanced F(t+ 8).
Start
Goal1
Goal2
(b) Graphical represen-
tation of the updated
motion patterns.
Fig. 5. Evaluation of the proposed method in scenario 1.
B. Scenario 2
In the second scenario, a different human operator is going
to accomplish the assembly task similar as in scenario 1.
Here he needs to pick up a workpiece only at Goal1 but
with two more motion patterns. The workspace is divided
Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,
Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.
into 180 cells of the same size as before and arranged in a
12 ×5×3-grid.
1) Online generation of F(t+r)using a specified δ:The
HMM modeling the motion of the human hand contains 70
states and is adapted online while capturing his motion of the
human operator. F(t+r)with a given threshold δ= 0.9996
is generated online, using the current observation O(t). Fig.
6(c) shows the probability Paof the real trajectories O(t+
r)inside the corresponding F(t+r)with r= 8 for the
specified δ. As illustrated in Fig. 6(a), the prediction quality
is satisfactory; Pais larger than 0.8whenever δ > 0.83.
The parameters of the HMM are computed using only one
iteration of the Baum-Welch algorithm [13] in order to meet
the time-constraints of online motion planning. Pacan be
further improved by using more iterations and more training
data [6].
Pattern1
Pattern2
(a) Motion patterns.
O(t)
Start
Goal1
(b) Prediction.
δ
Pa
0.4
0.6
0.8
0.8
0.9
0.9
1
1
(c) Evaluation.
Fig. 6. Motion prediction based on existing motion patterns.
2) Online generation of F(t+r)using a combination
of a probabilistic model and reachability analysis: With
a new trajectory that does not follow any existing motion
pattern shown in Fig. 7(a), the prediction based only on the
HMM is not accurate enough. The human operator moves
his arm along the boundary of the table, yet the mean value
of the states prediction (green stripe) indicating the trend
of the motion follows the existing motion pattern. We now
additionally consider the reachability analysis to enhance the
quality of the prediction. Here, the dynamic limits of this
human is different from the one in the previous scenario,
which are v±=±100cm/s and a±=±80cm/s2, As
illustrated in Fig. 7(a), the prediction quality is unsatisfactory
due to an unforeseen motion pattern, shown as the red dotted
line in Fig. 7(b). Pacorresponding to predictions based on
HMMs solely (red dashed line in 7(b)) is relatively small. It
drastically improves for the combination of the probabilistic
model with the reachable set (blue solid line).
z1
z2z3
StartStart
GoalGoal
Prediction
O(t)
(a) Prediction.
δ
Pa
0.2
0.4
0.8
0.8
0.9
0.9
1
1
(b) Evaluation.
Fig. 7. Prediction of an unforeseen trajectory that does not belong to the
existing motion patterns. Probability Paof the real trajectories at O(t+ 8)
lying inside F(t+ 8) for specified δ.
V. CONCLUSIONS
The selected probabilistic model HMM is appropriate for
human motion modeling and prediction, especially when
the human intention is considered. In order to enhance the
quality of prediction accuracy, we have proposed a method
for combining the probabilistic model with reachability
analysis considering dynamic limits of human motion and
measurement error. The practicability of our method was
successfully demonstrated by online predicting the motion of
human hands in two scenarios with different human workers,
multiple motion patterns, and multiple goals. The method
has been embedded into a scheme for online optimization
of trajectories for an industrial robot which interacts with a
human worker [7].
REFERENCES
[1] Marjan A. Admiraal, Martijn J.M.A.M. Kusters, and Stan C.A.M.
Gielen. Modeling kinematics and dynamics of human arm movements.
Motor Control, 8:312–338, 2004.
[2] N. Badler and D. Tolani. Real time inverse kinematics on human arm.
Presence, 5(4):393–401, 1996.
[3] M. Bennewitz, W. Burgard, G. Cielniak, and S. Thrun. Learning
motion patterns of people for compliant robot motion. Int. J. of Robot.
Research, 24(1):31–48, 2005.
[4] A. Biess, D. Liebermann, and T. Flash. A computational model for
redundant human three-dimensional pointing movements: Integration
of independent spatial and temporal motor plans simplifies movement
dynamics. J. of Neuroscience, 27(48):13045–13064, 2007.
[5] A. Biess, M. Nagurka, and T. Flash. Simulating discrete and rhyth-
mic multi-joint human arm movements by optimization of nonlinear
performance indices. Biol. Cybern., 95(1):31–53, 2006.
[6] H. Ding, G. Reißig, K. Wijaya, D. Bortot, K. Bengler, and O. Sturs-
berg. Human arm motion modeling and long-term prediction for safe
and efficient human-robot-interaction. In Proc. of the IEEE Int. Conf.
Robot. and Autom., pages 5875–5880, 2011.
[7] H. Ding, K. Wijaya, G. Reißig, and O. Stursberg. Optimizing motion
of robotic manipulators in interaction with human operators. In
S. Jeschke et al., editor, Int. Conf. on Intell. Robot. and Appl., LNCS,
pages 520–531. Springer, 2011.
[8] G. Fainekos, A. Girard, H. Kress-Gazit, and G. Pappas. Temporal
logic motion planning for dynamic robots. Automatica, 45(2):343–
352, 2009.
[9] T. Flash and N. Hogan. The coordination of arm movements: an
experimentally confirmed mathematical model. J. of Neuroscience,
5(7):1688–1703, 1985.
[10] W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank. A system for
learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach.
Intell., 28(9):1450–1464, 2006.
[11] Jr. J. Laviola. Double exponential smoothing: an alternative to kalman
filter-based predictive tracking. In Proc. of the Immersive Project
Technol. and Virtual Environments, pages 199–206, 2003.
[12] S.M. LaValle. Planning Algorithms. Cambridge University Press,
2006.
[13] R.M. Neal and G.E. Neal. A new view of the em algorithm that
justifies incremental, sparse and other variants. In M.I. Jordan, editor,
Learning in Graphical Models. Dordrecht, Kluwer, 1998.
[14] L.R. Rabiner. A tutorial on hidden markov models and selected
applications in speech recognition. Proc. of the IEEE, 77(2):257–286,
1989.
[15] O. Stursberg and B.H. Krogh. Efficient representation and computation
of reachable sets for hybrid systems. In Hybrid Systems: Comput. and
Control, volume 2623 of LNCS, pages 482–497. Springer, 2003.
[16] D. Vasquez, T. Fraichard, and C. Laugier. Growing hidden markov
models: An incremental tool for learning and predicting human and
vehicle motion. Int. J. of Robot. Research, 28(11-12):1486–1506,
2009.
... Offline simulation reduces time-expensive online calculation, especially when using complex models of obstacles. Modelling human arm motion in relation to human-robot interaction is performed using Hidden Markov Models by Ding et al. in [10], [11]. However, simulation and statistical analysis have the inherent drawback that they cannot account for all possible eventualities in finite time; indeed, reachable sets are proposed by the authors of [11] to account for unknown motions. ...
... Modelling human arm motion in relation to human-robot interaction is performed using Hidden Markov Models by Ding et al. in [10], [11]. However, simulation and statistical analysis have the inherent drawback that they cannot account for all possible eventualities in finite time; indeed, reachable sets are proposed by the authors of [11] to account for unknown motions. ...
... Having proven that reachable sets can be computed for a robot under Computed Torque Control, it remains to compute the reachable set of the human obstacle to check for intersection. This expands upon the idea in [11] that the reachable sets of human motion be used to supplement a Hidden Markov Model where unexpected motion is detected. Our method, however, involves computing the reachable set of all possible motion with no probabilistic element in the algorithm. ...
Article
Full-text available
A failsafe control strategy is presented for online safety certification of robot movements in a collaborative workspace with humans. This approach plans, predicts and uses formal guarantees on reachable sets of a robot arm and a human obstacle to verify the safety and feasibility of a trajectory in real time. The robots considered are serial link robots under Computed Torque schemes of control. We drastically reduce the computation time of our novel verification procedure through precomputation of non-linear terms and use of interval arithmetic, as well as representation of reachable sets by zonotopes, which scale easily to high dimensions and are easy to convert between joint space and Cartesian space. The approach is implemented in a simulation, to show that real time is computationally within reach.
... Markov chains are used to model human dynamics in [24], and probabilistic reachability analysis, based on the dynamics obtained, is used in a collision avoidance algorithm. Ding et al. [25] present another probabilistic prediction of the human occupancy using a hidden Markov model (HMM). Mainprice and Berenson [26] compute a voxel map of occupancy probability, which is used in a cost function to guide a path-planning algorithm. ...
... This may happen during a reflex movement when touching something sharp or grabbing a toppling object. The authors in [25] point out that unforeseen or unusual movements would not be accounted for by their approach, and propose formal techniques, specifically reachability analysis, as a complementary technique to account for such movements. There exists no previous work that can rigorously predict the set of future occupancies given an uncertain dynamic model of human motion. ...
... In order to predict human motion, one requires a model of the human. The modeling task can be integrated into the prediction approach, as it is often done in machine learning, e.g., the aforementioned HMMs [25], or one obtains the model Fig. 2. Top: Offline, we capture archetypal movements of subjects and fit to a kinematic model; from the resulting joint positions, velocities, and accelerations, we obtain a dynamic model. Bottom: Online, we fit sensor data (not necessarily motion capture) to the kinematic model to obtain the state in joint space, and calculate the reachable set until a future time t f with the dynamics obtained offline. ...
Article
Predicting the occupancy of a human in real time is of great interest in human-robot coexistence for obtaining regions that a robot should avoid in safe motion planning. The human body is composed of joints and links, suiting approximation by a kinematic chain, but the control strategy of the human is completely unknown, meaning the potential occupancy grows very fast and it is difficult to compute tightly in real time. As such, most previous work considers only specific, known, or probable movements, and usually does not account for a range of human dimensions. Focusing on the human arm, we analyze archetypal movements performed by test subjects to create a dynamic model. Motion-capture data of subjects are fitted, for modeling purposes, to two abstractions: a 4-degree of freedom (DOF) model and a 3-DOF model, to obtain dynamic parameters. We validate our approach on movements from a publicly available database. The prediction is shown to be computationally fast, and reachable sets of the abstraction are shown to enclose all possible future occupancies of the arm for different subjects, tightly but overapproximatively. The 3-DOF model has advantages over the 4-DOF in terms of speed, though the 4-DOF model is tighter at smaller time horizons. Such an overapproximative representation is intended for certifiable safety-guaranteed collision avoidance algorithms for robots.
... Since human intention cannot be predicted reliably, probabilistic methods have been proposed. Among others, Hidden Markov Models [6] and Markov chains [7] can be used to find probable movements and then calculate the most likely occupancy accordingly [8]. Though these methods work well for most human behaviour, unusual movements like reflex movements, tripping, or grabbing falling objects, may not occur in the training data and may not be accounted for. ...
... From these markers, we identify the shoulder x S , elbow x E and wrist x W , and a base coordinate system at the shoulder as in [11]. 6. Markers and local base coordinate system. ...
... Since the human moves fast and with uncertain intention, most approaches which predict future human movement to avoid collisions are probabilistic. Ding et al. present a Hidden Markov Model [1] to predict hand occupancy in reaching motions. Mainprice and Berenson use a Gaussian Mixture Model to estimate the probability of space around the robot being occupied by a human [2]; this is used in a trajectory planner. ...
... Consider first (3a). Since the global trajectory is known in advance, we can calculate θ, the maximum of d 2 ξ ds 2 over the global trajectory, and substitute this into (3a) 1 . Sinceṡ ∈ [0, 1], the numerator of the righthand side attains a minimum at a m −θ. ...
... from touching hot or sharp objects, catching a falling object, or swatting a fly. Ding et al. [5] address this limitation: they propose a hybrid stochastic/formal prediction using a Hidden Markov Model to predict occupancy of familiar motions and reachable sets in case of unknown movements. Ragaglia et Al. ...
... Mainprice and Berenson [12] predict future motion using a Gaussian Mixture Model and compute a voxel map of occupancy probabilities, which is used in a human-avoidance cost function to plan trajectories. Ding et al. predict human occupancy with a Hidden Markov Model (HMM) [13]. The latter authors note that unusual or previously unobserved movements would not be accounted for by an HMM and propose that this probabilistic method be supplemented by reachability analysis. ...
Conference Paper
Human motion is fast and hard to predict. To implement a provably safe collision-avoidance strategy for robots in collaborative spaces with humans, an overapproximative prediction of the occupancy of the human is required, which needs to be calculated faster than real time. We present a method for computing volumes containing the entire possible future occupancy of the human, given its state, faster than real time. The dynamic model of the human is built from analysing a set of archetypal movements performed by test subjects. The occupancy prediction is tested on a publicly available database of motion capture data, and shown to be overapproximative for all movements relating to everyday activities, sport and dance. Our novel algorithm is useful to guarantee safety in human-robot collaboration scenarios.
Article
Traditionally safety in industrial manufacturing environments has been addressed by rigid separation between robots and human operators. With the advent of new technologies and the transition of production to industry 4.0, a more flexible approach to manufacturing is pursued to achieve higher productivity, where robots and human operators are allowed to collaborate and interact. This transformation leads to overcoming traditional safety procedures and the development of new safety-assuring technologies for the minimization of risks connected with human robot collaboration. In this work we focus on the prediction of movements of operators’ upper torso and arms by developing a method which combines data driven methodologies with formal methods. The approach is based on a predictive model of human motion compared against the planned robot trajectory and online monitoring of satisfaction of safety requirements with formal methods. We provide an early assessment of the robustness of the proposed method with results from simulations in a virtual environment.
Article
Full-text available
Modeling and predicting human and vehicle motion is an active research domain. Owing to the difficulty in modeling the various factors that determine motion (e.g. internal state, perception) this is often tackled by applying machine learning techniques to build a statistical model, using as input a collection of trajectories gathered through a sensor (e.g. camera, laser scanner), and then using that model to predict further motion. Unfortunately, most current techniques use offline learning algorithms, meaning that they are not able to learn new motion patterns once the learning stage has finished. In this paper, we present an approach where motion patterns can be learned incrementally, and in parallel with prediction. Our work is based on a novel extension to hidden Markov models, called growing hidden Markov models, which gives us the ability to learn incrementally both the parameters and the structure of the model. The proposed approach has been evaluated using synthetic and real trajectory data. In our experiments our approach consistently learned motion models that were more compact and accurate than those produced by two other state-of-the-art techniques.
Conference Paper
Full-text available
Recently, the problem of how to manipulate industrial robots that interact with human operators attracts a lot of attention in robotics research. This interest stems from the insight that the integration of human operators into robot based manufacturing systems may increase productivity by combining the abilities of machines with those of humans. In such a Human-Robot-Interaction (HRI) setting, the challenge is to manipulate the robots both safely and efficiently. This paper proposes an online motion planning approach for robotic manipulators with HRI based on model predictive control (MPC) with embedded mixed-integer programming. Safety-relevant regions, which are potentially occupied by the human operators, are generated online using camera data and a knowledge-base of typical human motion patterns. These regions serve as constraints of the optimization problem solved online to generate control trajectories for the robot. As described in the last part of the paper, the proposed method is realized for a HRI scenario.
Conference Paper
Full-text available
Modeling and predicting human behavior is indispensable when industrial robots interacting with human operators are to be manipulated safely and efficiently. One challenge is that human operators tend to follow different motion patterns, depending on their intention and the structure of the environment. This precludes the use of classical estimation techniques based on kinematic or dynamic models, especially for the purpose of long-term prediction. In this paper, we propose a method based on Hidden Markov Models to predict the region of the workspace that is possibly occupied by the human within a prediction horizon. In contrast to predictions in the form of single points such as most likely human positions as obtained from previous approaches, the regions obtained here may serve as safety constraints when the robot motion is planned or optimized. This way one avoids collisions with a probability not less than a predefined threshold. The practicability of our method is demonstrated by successfully and accurately predicting the motion of a human arm in two scenarios involving multiple motion patterns.
Article
Full-text available
Whenever people move through their environments they do not move randomly. Instead, they usually follow specific trajectories or motion patterns corresponding to their intentions. Knowledge about such patterns enables a mobile robot to robustly keep track of persons in its environment and to improve its behavior. In this paper we propose a technique for learning collections of trajectories that characterize typical motion patterns of persons. Data recorded with laser-range finders are clustered using the expectation maximization algorithm. Based on the result of the clustering process, we derive a hidden Markov model that is applied to estimate the current and future positions of persons based on sensory input. We also describe how to incorporate the probabilistic belief about the potential trajectories of persons into the path planning process of a mobile robot. We present several experiments carried out in different environments with a mobile robot equipped with a laser-range scanner and a camera system. The results demonstrate that our approach can reliably learn motion patterns of persons, can robustly estimate and predict positions of persons, and can be used to improve the navigation behavior of a mobile robot.
Article
We present novel algorithms for predictive tracking of user position and orientation based on double exponential smoothing. These algorithms, when compared against Kalman and extended Kalman filter-based predictors with derivative free measurement models, run approximately 135 times faster with equivalent prediction performance and simpler implementations. This paper describes these algorithms in detail along with the Kalman and extended Kalman Filter predictors tested against. In addition, we describe the details of a predictor experiment and present empirical results supporting the validity of our claims that these predictors are faster, easier to implement, and perform equivalently to the Kalman and extended Kalman filtering predictors.
Article
In this paper, we address the temporal logic motion planning problem for mobile robots that are modeled by second order dynamics. Temporal logic specifications can capture the usual control specifications such as reachability and invariance as well as more complex specifications like sequencing and obstacle avoidance. Our approach consists of three basic steps. First, we design a control law that enables the dynamic model to track a simpler kinematic model with a globally bounded error. Second, we built a robust temporal logic specification that takes into account the tracking errors of the first step. Finally, we solve the new robust temporal logic path planning problem for the kinematic model using automata theory and simple local vector fields. The resulting continuous time trajectory is provably guaranteed to satisfy the initial user specification.
Conference Paper
Computing reachable sets is an essential step in most anal- ysis and synthesis techniques for hybrid systems. The representation of these sets has a deciding impact on the computational complexity and thus the applicability of these techniques. This paper presents a new ap- proach for approximating reachable sets using oriented rectangular hulls (ORHs), the orientations of which are determined by singular value de- compositions of sample covariance matrices for sets of reachable states. The orientations keep the over-approximation of the reachable sets small in most cases with a complexity of low polynomial order with respect to the dimension of the continuous state space. We show how the use of ORHs can improve the eciency of reachable set computation signif- icantly for hybrid systems with nonlinear continuous dynamics.
Article
This paper presents studies of the coordination of voluntary human arm movements. A mathematical model is formulated which is shown to predict both the qualitative features and the quantitative details observed experimentally in planar, multijoint arm movements. Coordination is modeled mathematically by defining an objective function, a measure of performance for any possible movement. The unique trajectory which yields the best performance is determined using dynamic optimization theory. In the work presented here, the objective function is the square of the magnitude of jerk (rate of change of acceleration) of the hand integrated over the entire movement. This is equivalent to assuming that a major goal of motor coordination is the production of the smoothest possible movement of the hand. Experimental observations of human subjects performing voluntary unconstrained movements in a horizontal plane are presented. They confirm the following predictions of the mathematical model: unconstrained point-to-point motions are approximately straight with bell-shaped tangential velocity profiles; curved motions (through an intermediate point or around an obstacle) have portions of low curvature joined by portions of high curvature; at points of high curvature, the tangential velocity is reduced; the durations of the low-curvature portions are approximately equal. The theoretical analysis is based solely on the kinematics of movement independent of the dynamics of the musculoskeletal system and is successful only when formulated in terms of the motion of the hand in extracorporal space. The implications with respect to movement organization are discussed.