Content uploaded by Gunther Reissig

Author content

All content in this area was uploaded by Gunther Reissig on Jan 06, 2014

Content may be subject to copyright.

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

Online Computation of Safety-Relevant Regions

for Human Robot Interaction

Hao Ding

ABB

Corporate Research

Germany

hao.ding@de.abb.com

Kurniawan Wijaya

Institute of Control and

System Theory, University

of Kassel, Germany

Gunther Reißig

Chair of Control Engineering,

University of the Federal

Armed Forces Munich,

Germany

http://www.reiszig.de/gunther/

Olaf Stursberg

Institute of Control and

System Theory, University

of Kassel, Germany

stursberg@uni-kassel.de

Abstract—For the purpose of robot motion planning in

human robot interaction, the efﬁcient and accurate prediction

of safety-relevant regions that are possibly occupied by the

human within the planning horizon is a necessity. The proba-

bilistic framework of Hidden Markov Models (HMMs) has been

proposed to account for the nondeterminism in human motion.

Such models often lead to sufﬁciently accurate predictions if

the human motion follows one of a ﬁnite set of known motion

patterns. But the HMMs need to be continuously updated

during use in order to cope with new patterns. Existing update

methods require any newly observed motion pattern to be

completed before it can be used for prediction. In this paper, the

latter problem is avoided by employing a combination of HMM

based prediction with information obtained from reachability

analysis based on dynamic limits of human motion. The method

has been successfully implemented to predict the motion of a

human hand online.

Keywords: Human motion modeling and prediction, Robot

safety, Reachability analysis, Hidden Markov Models, Human

robot interaction

I. INTRODUCTION

For scenarios of robotic manipulators interacting with

human operators in a shared workspace, it is of paramount

importance to avoid collision and to guarantee safe in-

teraction. Thus, safety-relevant regions, i.e., regions of

the workspace which are possibly occupied by a human,

need to be reliably predicted and excluded from the free

workspace in which robot trajectories are planned. It should

be stressed that it is in general not sufﬁcient in speciﬁc

scenarios to only compute the most likely or expected human

positions, namely prediction in the form of single points, but

the nondeterminism in human behavior must be accounted

for. Moreover, since the robot and human share the same

workspace and prediction of human motion at the initial time

is not sufﬁcient, methods for online adaptation of models of

human motion and for predicting human motion within a

planning horizon are important in this context.

Classical estimation techniques based on kinematic or

dynamic models of human motion [1], [2], [11] and other

deterministic methods [4], [5], [9] may produce accurate

short-term prediction. However, their performance in longer-

term prediction is usually not satisfactory since they do

not take into account aspects of human motion such as

intention. Thus, most current research focuses on implicit

modeling methods. Based on the assumption that humans

tend to follow motion patterns, the probabilistic framework

of Hidden Markov Models (HMMs) has been proposed [3],

[10]. In general, this approach consists of two phases,

namely, the identiﬁcation phase and the prediction phase. An

online variant of HMMs, named Growing HMMs (GHMMs),

which incrementally improves the HMM based on newly

observed motion, has been presented in [16]. Based on the

GHMMs, we have proposed a method for computing safety-

relevant regions of the workspace [6]. These regions should

serve as safety constraints when robot motion is planned or

optimized. This way, one arrives at motion plans that are

safe in the sense that collision between human and robot

is avoided with a probability not less than a predeﬁned

threshold.

However, GHMMs can only be updated once a trajectory

sequence is completed. That is, when the human motion

follows an unknown pattern, the prediction obtained from the

current HMM is usually useless until the motion has been

completed such that the HMM can be updated – this may

lead to performance degradation and safety issues. In this

paper, the latter problem is avoided by combining HMM-

based prediction with reachability analysis of the dynami-

cally feasible behavior of the human operator. This analysis

is based on the dynamic limits of human motion such

as velocity and acceleration limits including measurement

errors. Based on the proposed method, the predicted safety-

relevant region of the workspace can be computed online by

combining the probabilistic model and reachability analysis

with a given probability threshold.

The paper is organized as follows: Section II gives the

formulation of an optimal control problem for robotic manip-

ulators safely interacting with human operators. This leads to

the necessity of generating the safety-relevant regions. Fol-

lowing the background of modeling and prediction based on

HMMs, a schematic example is shown there for illustrating

the enhancement of the prediction accuracy by combining

reachable sets in Section III. We then provide the method for

computing reachable sets and the improved safety-relevant

region. Finally, we demonstrate our method in two scenarios

with different human operators for accurately predicting

the human hand motion with multiple motion patterns and

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

multiple goals in Section IV.

II. PROBLEM STATEMENT

The following notation is used throughout the paper:

•Workspace /Cartesian space:Z=R3, in which

the robot and human work; points [z1, z 2, z3]T∈Z

represent positions.

•Conﬁguration space:C=Rn;[q1, q2,...,qn]T∈C

represent joint angles of the robotic manipulator with n

degrees of freedom.

•State space: X=R2n;x(t) = [q(t),˙q(t)]T∈X

represent the state of the robotic manipulator.

•zj∈Zdenotes the position of joint jincluding the

position of the end effector in the workspace with j∈

{1,...,n+ 1}. For the sake of notational simplicity, zj

is used as a shortcut for z(qj).

•Observation space:O=Z×Z;O(t) =

[z1(t), z2(t), z3(t),˙z1(t),˙z2(t),˙z3(t)]T∈Orepresent

data of human motion measured or derived; speciﬁcally,

one observation consists of position and velocity in the

workspace.

The motion planning problem of robotic manipulators can

be formulated as an optimization problem as follows:

min

u(t)

g

X

τ=0

Jτ(x(τ), u(τ)) (1a)

s.t. x(t+ 1) = f(x(t), u(t)),∀t∈ {0,1,...,g−1}

(1b)

x(t0) = x0, zn+1(g)∈G(1c)

x(t)∈L, u(t)∈U(1d)

Ω(x(t)) ∩F(t) = ∅,∀t∈ {0,1,...,g},(1e)

where

•Jτis a cost function which represents a performance

measure of the robot at each sampling time τ;

•(1b) is derived from the actual differential equations

modeling the robot in continuous time, and describes

the system dynamics in discrete time, where x(t)∈X

is the system state, u(t)∈Uis an admissible control

(torques, etc.), and t∈ {0,1, . . . , g}denotes the current

time, and 0,1,...,g are the sampling instances;

•x0is the initial state of the robot, and G⊆Rn, the

target set for the end effector zn+1(g);

•L⊆Xrepresents admissible states x(t), excluding e.g.

infeasible conﬁgurations of the robot;

•Ω(x(t)) ⊆Zis the set of points in the workspace

occupied by the robot;

•F(t)⊆Zdenotes the set of points in the workspace

that are possibly occupied by a human.

Since positions measurements are obtained with a deﬁned

frequency (e.g. by a camera system) and since the control of

the robotic manipulator has to be realized in discrete time,

the optimization problem (1) is also formulated in a discrete

time setting. In order to still guarantee safety everywhere

in between the sampling instants, dynamic properties of the

human and the robot (such as velocity bounds) can be used

to tighten the constraints (1e) and (1d), respectively [8] [12,

Section 5.3.4].

Online generation of the set Fat future time instances,

namely, F(t+r)with r∈ {1,...,H}, where Hdenotes

the prediction or planning horizon, is also necessary in the

course of solving more general motion planning problems,

e.g. [12]. It is precisely the problem of determining F(t+r)

which is investigated in the paper.

III. HUMAN MOTION MODELING AND PREDICTION

Our method of modeling and predicting human motion,

and particularly, of determining the safety-relevant regions

F(t+r), consists of the following steps:

(i) Based on a set of updated experimental data, a time-

evolving Hidden Markov Model is built.

(ii) According to the current observation O(t)of the po-

sition of the human, a conditional probability density

function for reaching a position at time instants t+

1,...,t+H, and the reachable set of the human position

is determined using dynamic limits of the motion.

(iii) The safety-relevant regions in the workspace, which

contain the positions possibly occupied by the human

operator in future times, are determined by combining

the probabilistic model and the reachable set.

A. Background

1) Probabilistic model: HMM:

III.1 Deﬁnition. The H M M = (S, A, S(0), O, P (O|S))

used in the paper consists of (according to [14]):

•S(t)is the stochastic state variable S(t)∈

{S1, S2,...,SN} ⊂ Nat time t. The value of S(t)

refers to a point in the observation space O. In general,

this hidden state can also include the acceleration.

•A∈RN×Nis a transition matrix whose entry aij in

row iand column jis given by aij =P([S(t+ 1) =

Si]|[S(t) = Sj]).

•Initial state probability:

P(S(t= 0)) = [P(S(t= 0) = S1), P (S(t= 0) = S2),

. . . , P (S(t= 0) = SN)],(2)

where each element contains the initial probability for

the corresponding discrete state at time t= 0.

•The observation variable O(t)∈O.

•The observation probability density function:

P(O(t)|[S(t) = Si]) = G(Ot;µi,Σ) (3)

is represented by a Gaussian distribution for every

discrete state Si.µiis the mean value of each dis-

tribution, namely µi=Si. All discrete states have the

same covariance matrix Σ. The set of all Gaussian

parameters are denoted by b={Σ, µ1,...,µN}.

The full set of parameters for the HMM is λ={P(S(t=

0)), A, b}, which is computed based on the sequences of

observations O1:Tgwith Tgduration of the human motion

from the start to the goal. Details of the identiﬁcation of the

HMM as well as the time-varying adaptation of the HMM

can be found in [6], [16].

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

2) Model-based prediction:

III.2 Deﬁnition. Given a new observation O(t)at time t,

the prediction of the belief state for the object is estimated

using:

P(S(t+r) = Si|O(t))

=

N

X

j=1

P(S(t+r) = Si|S(t+r−1) = Sj)

·P(S(t+r−1) = Sj|O(t))

=

N

X

j=1

aij ·P(S(t+r−1) = Sj|O(t)),(4)

where iand jare state indices.

The conditional probability with a given point Onin the

observation space at time t+r,r∈ {1,...,H}, denoted by

P(O(t+r) = On|O(t)), is deﬁned as:

P(O(t+r) = On|O(t)) = y(O(t+r) = On|O(t))

RRny(τ|O(t))dτ ,(5)

where

y(O(t+r) = On|O(t)) =

N

X

i=1

P(S(t+r) = Si|O(t))

·P(O(t+r) = On|S(t+r) = Si),

and P(O(t+r) = On|S(t+r) = Si)is deﬁned by (3) with

the state Si∈ {S1,...,SN}.

The derivation of the equations can be found in [6].

3) Generation of safety-relevant regions using a given

probability threshold: For any given point Onin the observa-

tion space, the conditional probability density P(O(t+r) =

On|O(t)) of reaching the point Onat future time t+rwith

respect to the current observation O(t)is obtained from (5),

where only the position information in On∈Ois used.

In order to efﬁciently compute F(t+r), we need to avoid

working with an inﬁnite number of points in the observation

space. Therefore, the observation space is divided into D

cells Qi,i∈ {1,...,D}. The centroid Odof each cell

is selected for computing the probability of landing in the

corresponding cell. For the sake of simplicity, it is assumed

that the probability density is constant in each cell, and (5)

is approximated by

P(O(t+r) = Od|O(t)) ≈

y(O(t+r) = Od|O(t))

PD

d=1 y(O(t+r) = Od|O(t)).(6)

The indices {1,...,D}are arranged in an array wsuch

that

P(O(t+r) = Owi|O(t)) ≥P(O(t+r) = Owj|O(t)) (7)

whenever j > i, where widenotes the entry at position i

of w. Finally, let a threshold δ∈[0,1] be given and deﬁne

F(t+r)by

F(t+r) =

ϕ

[

i=1

Qwi,(8)

where

ϕ= min

κ∈{1,...,D} κ

X

l=1

P(O(t+r) = Owl|O(t)) ≥δ!.(9)

This means, we are neglecting cells Qiof low probability.

With this choice of F(t+r), the probability pof the future

observation O(t+r)being contained in F(t+r)is lower

bounded by δ:

p=

ϕ

X

l=1

P(O(t+r) = Owl|O(t)) ≥δ. (10)

B. Improving Prediction Quality using Reachability Analysis

1) Principle: As mentioned in the introduction, the online

scheme for updating the HMMs can only change the model

when a motion pattern is completed. This leads to an inaccu-

racy of the model, if a new motion pattern is encountered. A

schematic example is shown in Fig. 1 for further illustration.

Motion patterns

New behavior

Prediction based on HMM

Reachable set ψ(t+r)

Current observation

Start

Goal

Fig. 1. Schematic example.

Two motion patterns for

human motion are marked by

solid lines from the start to

the goal. In a new motion,

marked by the dashed line,

the human moves along a dif-

ferent path. The prediction

from an intermediate point on

this path can only be based on

the current observation and

the HMMs, which results in

the safety-relevant region represented by the dark gray area.

Clearly, this ‘safety-relevant region’ is actually not safe. This

motivates the introduction of the reachable set ψ(t+r)to

enhance the prediction. ψ(t+r)is computed based on the

current measurement (including the measurement errors for

positions and velocities) and uses lower and upper bounds

for the velocity and acceleration of human motion to con-

servatively approximate the attainable part of the workspace

for the prediction horizon.

2) Online computation of the reachable set: In order to

increase the prediction accuracy, the computation of the

reachable set ψ(t+r)at time t+ris carried out based

on the knowledge of O(t) = (z(t),˙z(t)) and the dynamic

limits of human motion as follows.

The method we propose is based on the following reacha-

bility result, in which a function γrepresents one component

of the position vector z∈R3, and for the sake of simplicity

of presentation, the position has been shifted in time in order

to arrive at an the interval [0, T ]of deﬁnition for γ.

III.3 Lemma. Let T > 0and assume γ: [0, T ]→Ris twice

differentiable. Assume further that γfulﬁlls the following

conditions:

(i)γ(0) ∈[γ−, γ+]for some γ−≤γ+; (measurement

error);

(ii) ˙γ(0) ∈[ ˙γ−,˙γ+]for some ˙γ−≤˙γ+; (measure-

ment/estimation error);

(iii)∀t∈[0,T ]˙γ(t)∈[v−, v+]for some v−,v+with v−≤

˙γ−≤˙γ+≤v+, where v−and v+denote lower and

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

upper bounds, respectively, on the speed of the human

motion;

(iv)∀t∈[0,T ]¨γ(t)∈[a−, a+]for some a−,a+with a−≤

a+, where a−and a+denote lower and upper bounds,

respectively, on the acceleration of the human motion.

Then γfulﬁlls the following upper bound for all t∈[0, T ]:

γ(t)≤γ(t)upper =

γ++

t·˙γ++t2·a+

2,

if a+= 0 or t≤t1=v+−˙γ+

a+;

v+−˙γ+

a+·˙γ++(v+−˙γ+)2

2a++v+·(t−v+−˙γ+

a+),

otherwise, (11)

as well as the lower bound obtained from (11) by substituting

′+′with ′−′of γ,aand v, and ′≤′with ′≥′.

Therefore, the reachable set ψ(t+r)at time t+rin the

future can be computed as below:

ψ(t+r) := [¯γ(r)lower ,¯γ(r)upper ],(12)

where the interval is a subset of R3and the vector-valued

bounds are ¯γlower and ¯γupper, obtained for the three com-

ponents of the position z∈R3.

3) Combining the probabilistic model and reachability

analysis: The crucial point about the combination of HMMs

with reachability analysis for the purpose of improving the

quality of prediction is the following observation: Ordering

the cell indices i={1,...,D}according to (7), i.e., starting

from the highest probability cell ﬁrst, prior to the computa-

tion of the safety-relevant region (8) is not a prerequisite

for the methods of Sec. III-A.3 to work. In fact, the order

(7) has been introduced for the sole purpose of keeping the

safety-relevant region (8) small, and the method is justiﬁed

for any order of the cell indices as long as (9) is fulﬁlled,

which implies (10). Consequently, this degree of freedom

may be used to improve the quality of prediction if additional

information is available that cannot be obtained from the

data from which the HMM was built. In the present paper,

we propose to take advantage of such additional information

by replacing the order (7) by the following one: Before

ordering, the cells that intersect the reachable set ψ(t+r)are

selected ﬁrst. The set of those cells is Sβ

i=1 Qiwith βbeing

the number of cells that intersect the reachable set ψ(t+r).

Then the indices {1,...,β}are arranged in an array u

according to the probabilities determined before by (6)

P(O(t+r) = Oui|O(t)) ≥P(O(t+r) = Ouj|O(t)) (13)

with β≥j > i ≥1, where uidenotes the entry at position

iof u.

The indices of the remaining cells {β+ 1,...,D}are

arranged in an array vsuch that their order matches the

relation:

P(O(t+r) = Ovi|O(t)) ≥P(O(t+r) = Ovj|O(t)) (14)

for D≥j > i ≥β+ 1, where videnotes the entry at

position iof v.

Finally, with a speciﬁed δ∈[0,1], the enhanced safety-

relevant region F(t+r)is deﬁned by

F(t+r) =

ψ

[

i=1

(Qui+Qvi),(15)

where

ψ=

argminκ∈{1,...,β}(Pκ

l=1 P(O(t+r) = Oul|O(t)) ≥δ),

if Pβ

l=1 P(O(t+r) = Oul|O(t)) ≥δ;

argminκ∈{β+1,...,D}(Pκ

l=1 P(O(t+r) = Ovl|O(t)) ≥

δ−Pβ

l=1 P(O(t+r) = Oul|O(t)),

if Pβ

l=1 P(O(t+r) = Oul|O(t)) < δ;(16)

This new order implies that in the course of assembling

the enhanced safety-relevant region (15), cells intersecting

the reachable set ψ(t+r)are used ﬁrst, followed by some

of the remaining cells if necessary to satisfy (16). Hence, if

the prediction that could be obtained from the HMM alone is

of high quality, the safety-relevant region should be a subset

of ψ(t+r)and the effect of the new ordering is negligible.

However, if an unexpected motion pattern occurs, then the

prediction that could be obtained from the HMM alone is

of a low quality. In this case, the probability assigned to

the cells in the reachable set will be very small, hence the

reachable set will be a subset of the computed safety-relevant

region, and hence, the latter will contain the next observation

with certainty.

We ﬁnally remark that for the purpose of reducing the

complexity of solving the optimal control problem in (1a),

the safety-relevant region F(t+r)may also be over-

approximated by sets that have a simpler structure, e.g. [15].

IV. APPLICATION RESULTS

The proposed method is realized in the following human

assembly task scenarios. The safety-relevant regions can be

online generated.

A. Scenario 1

The ﬁrst human assembly task scenario is shown in Fig. 2.

A human operator moves his hand from Start to Goal1

for picking up a workpiece to then accomplish an assembly

task at Start. He has also to pick up another workpiece at

Goal2. A red marker is attached to his hand and detected by

a stereo camera system which is operated at 10 frames per

second. The workspace of the human operator is divided

into 252 cells, which are translated copies of the cuboid

[0,12.27] ×[0,12.75] ×[0,13.5]cm3that are arranged in a

12 ×7×3-grid. For better illustration, here and in what

follows, the 3D workspace is represented as a combination

of three planes, which correspond to cross-sections parallel

to the (z1−z2)-plane.

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

z1

z1

z1

z1

z1

z1

z1z1

z2

z2

z2

z2

z2

z2

z2z2

z3

z3z3

Start Start

Start

Start

Goal1

Goal1

Goal1

Goal1

Goal2

Goal2

Goal2

Goal2

Workspace

Cell

Fig. 2. Illustration of scenario 1.

1) Online generation of the safety-relevant regions using

a given probability threshold: The HMM identifying the

motion of the human hand is adapted online after capturing

the motion of the human operator with multiple motion

patterns from the start to the goal. The prediction of the

mean value of the state positions, which indicates the trend

of the motion is marked by the green stripe. It is computed

according to Eq. (13) in [6].

Based on the description in Sec. III-A.3, the safety-

relevant region F(t+r)with a given threshold δ= 0.9996

for the motion with the existing motion patterns can be

generated online, using the current observation O(t). Fig. 3

shows the predictions F(t+r)for r∈ {1,...,8}. Different

colors according to the colorbar in Fig. 3 indicate the

corresponding probabilities in the cells computed by (6).

.

patterns

t+ 1 2 3 45 6 7 8

0

0.01

0.2

1

O(t)

Start

Goal1

Motion patterns Mean value prediction

Fig. 3. Predicted safety-relevant regions based on existing motion patterns.

2) Online generation of the safety-relevant regions using

a combination of a probabilistic model and reachability

analysis: As shown in Fig. 4, the prediction based only

on the HMM is not accurate enough with a new trajectory

that does not follow an existing motion pattern. The human

operator moves his arm towards Goal2, yet the mean value

of the state prediction (the green dot shown in Fig. 4(a))

follows the existing motion pattern towards Goal1.

We now additionally consider the reachability analysis

to enhance the quality of the prediction. The prediction

of the reachable set (uniform gray color) based on the

current observation O(t)is shown in Fig. 4(c). Only

the reachable set inside the workspace is considered here.

Dynamic limits such as bounds on velocity and acceleration

of human motion, and particularly, of the motion of the hand,

are taken from literature [9]. Here, we use the parameters

v±=±60cm/s and a±=±40cm/s2. In addition, we

use the bounds γ±=zi(t)±1cm and ˙γ±= ˙zi(t)±

3cm/s as required in the application of Lemma III.3, where

zi(t)and ˙zi(t)are the actual measurement and estimation,

respectively.

Goal1

Goal2

(a) Prediction.

(b) Prediction based on HMM solely.

t+ 1 2 3 45 6 7 8

(c) Computation of the reachable set.

Fig. 4. Prediction of a new trajectory towards Goal2, which does not

belong to existing motion patterns towards Goal1.

Fig. 5(a) shows the probability Paof the real trajectories

O(t+r)lying in the corresponding F(t+r)for t∈

{0,...,g}and r= 8, depending on the probability threshold

δ. For the case that an unforeseen motion pattern occurs, the

prediction quality is unsatisfactory if only the HMM is used.

The value of Pacorresponding to predictions solely based on

HMMs (red dashed line in Fig. 5(a)) is relatively small. Pa

drastically improves for the combination of the probabilistic

model with reachability analysis, as represented by the blue

solid line. Furthermore, the real trajectories lie always inside

the predicted reachable sets. Of course, the sets may be

rather large when the velocity and acceleration of the human

are large.

The graphical representation of the updated motion pat-

terns is ﬁnally shown in Fig. 5(b), which is obtained only

after the unforeseen motion patterns have been taken into

account.

0

δ

Pa

0.2

0.4

0.6

0.8

0.8

0.9

0.9

1

1

(a) Probability of trajectories for un-

expected motion patterns at t+ 8

lying in the enhanced F(t+ 8).

Start

Goal1

Goal2

(b) Graphical represen-

tation of the updated

motion patterns.

Fig. 5. Evaluation of the proposed method in scenario 1.

B. Scenario 2

In the second scenario, a different human operator is going

to accomplish the assembly task similar as in scenario 1.

Here he needs to pick up a workpiece only at Goal1 but

with two more motion patterns. The workspace is divided

Authors’ Version of a paper presented at the 43rd Intl. Symp. Robotics (ISR), Taipei, Taiwan,

Aug 29-31, 2012. See http://www.reiszig.de/gunther/pubs/i12rHMM.html for a BibTeX entry.

into 180 cells of the same size as before and arranged in a

12 ×5×3-grid.

1) Online generation of F(t+r)using a speciﬁed δ:The

HMM modeling the motion of the human hand contains 70

states and is adapted online while capturing his motion of the

human operator. F(t+r)with a given threshold δ= 0.9996

is generated online, using the current observation O(t). Fig.

6(c) shows the probability Paof the real trajectories O(t+

r)inside the corresponding F(t+r)with r= 8 for the

speciﬁed δ. As illustrated in Fig. 6(a), the prediction quality

is satisfactory; Pais larger than 0.8whenever δ > 0.83.

The parameters of the HMM are computed using only one

iteration of the Baum-Welch algorithm [13] in order to meet

the time-constraints of online motion planning. Pacan be

further improved by using more iterations and more training

data [6].

Pattern1

Pattern2

(a) Motion patterns.

O(t)

Start

Goal1

(b) Prediction.

δ

Pa

0.4

0.6

0.8

0.8

0.9

0.9

1

1

(c) Evaluation.

Fig. 6. Motion prediction based on existing motion patterns.

2) Online generation of F(t+r)using a combination

of a probabilistic model and reachability analysis: With

a new trajectory that does not follow any existing motion

pattern shown in Fig. 7(a), the prediction based only on the

HMM is not accurate enough. The human operator moves

his arm along the boundary of the table, yet the mean value

of the states prediction (green stripe) indicating the trend

of the motion follows the existing motion pattern. We now

additionally consider the reachability analysis to enhance the

quality of the prediction. Here, the dynamic limits of this

human is different from the one in the previous scenario,

which are v±=±100cm/s and a±=±80cm/s2, As

illustrated in Fig. 7(a), the prediction quality is unsatisfactory

due to an unforeseen motion pattern, shown as the red dotted

line in Fig. 7(b). Pacorresponding to predictions based on

HMMs solely (red dashed line in 7(b)) is relatively small. It

drastically improves for the combination of the probabilistic

model with the reachable set (blue solid line).

z1

z2z3

StartStart

GoalGoal

Prediction

O(t)

(a) Prediction.

δ

Pa

0.2

0.4

0.8

0.8

0.9

0.9

1

1

(b) Evaluation.

Fig. 7. Prediction of an unforeseen trajectory that does not belong to the

existing motion patterns. Probability Paof the real trajectories at O(t+ 8)

lying inside F(t+ 8) for speciﬁed δ.

V. CONCLUSIONS

The selected probabilistic model HMM is appropriate for

human motion modeling and prediction, especially when

the human intention is considered. In order to enhance the

quality of prediction accuracy, we have proposed a method

for combining the probabilistic model with reachability

analysis considering dynamic limits of human motion and

measurement error. The practicability of our method was

successfully demonstrated by online predicting the motion of

human hands in two scenarios with different human workers,

multiple motion patterns, and multiple goals. The method

has been embedded into a scheme for online optimization

of trajectories for an industrial robot which interacts with a

human worker [7].

REFERENCES

[1] Marjan A. Admiraal, Martijn J.M.A.M. Kusters, and Stan C.A.M.

Gielen. Modeling kinematics and dynamics of human arm movements.

Motor Control, 8:312–338, 2004.

[2] N. Badler and D. Tolani. Real time inverse kinematics on human arm.

Presence, 5(4):393–401, 1996.

[3] M. Bennewitz, W. Burgard, G. Cielniak, and S. Thrun. Learning

motion patterns of people for compliant robot motion. Int. J. of Robot.

Research, 24(1):31–48, 2005.

[4] A. Biess, D. Liebermann, and T. Flash. A computational model for

redundant human three-dimensional pointing movements: Integration

of independent spatial and temporal motor plans simpliﬁes movement

dynamics. J. of Neuroscience, 27(48):13045–13064, 2007.

[5] A. Biess, M. Nagurka, and T. Flash. Simulating discrete and rhyth-

mic multi-joint human arm movements by optimization of nonlinear

performance indices. Biol. Cybern., 95(1):31–53, 2006.

[6] H. Ding, G. Reißig, K. Wijaya, D. Bortot, K. Bengler, and O. Sturs-

berg. Human arm motion modeling and long-term prediction for safe

and efﬁcient human-robot-interaction. In Proc. of the IEEE Int. Conf.

Robot. and Autom., pages 5875–5880, 2011.

[7] H. Ding, K. Wijaya, G. Reißig, and O. Stursberg. Optimizing motion

of robotic manipulators in interaction with human operators. In

S. Jeschke et al., editor, Int. Conf. on Intell. Robot. and Appl., LNCS,

pages 520–531. Springer, 2011.

[8] G. Fainekos, A. Girard, H. Kress-Gazit, and G. Pappas. Temporal

logic motion planning for dynamic robots. Automatica, 45(2):343–

352, 2009.

[9] T. Flash and N. Hogan. The coordination of arm movements: an

experimentally conﬁrmed mathematical model. J. of Neuroscience,

5(7):1688–1703, 1985.

[10] W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank. A system for

learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach.

Intell., 28(9):1450–1464, 2006.

[11] Jr. J. Laviola. Double exponential smoothing: an alternative to kalman

ﬁlter-based predictive tracking. In Proc. of the Immersive Project

Technol. and Virtual Environments, pages 199–206, 2003.

[12] S.M. LaValle. Planning Algorithms. Cambridge University Press,

2006.

[13] R.M. Neal and G.E. Neal. A new view of the em algorithm that

justiﬁes incremental, sparse and other variants. In M.I. Jordan, editor,

Learning in Graphical Models. Dordrecht, Kluwer, 1998.

[14] L.R. Rabiner. A tutorial on hidden markov models and selected

applications in speech recognition. Proc. of the IEEE, 77(2):257–286,

1989.

[15] O. Stursberg and B.H. Krogh. Efﬁcient representation and computation

of reachable sets for hybrid systems. In Hybrid Systems: Comput. and

Control, volume 2623 of LNCS, pages 482–497. Springer, 2003.

[16] D. Vasquez, T. Fraichard, and C. Laugier. Growing hidden markov

models: An incremental tool for learning and predicting human and

vehicle motion. Int. J. of Robot. Research, 28(11-12):1486–1506,

2009.