Content uploaded by He Wang

Author content

All content in this area was uploaded by He Wang

Content may be subject to copyright.

Energy-based Pose Unfolding and Interpolation

for 3D Articulated Characters

He Wang and Taku Komura

1Edinburgh University, 10 Crichton Street, EH8 9AB, United Kingdom

Abstract. In this paper, we show results of controlling a 3D articulated

human body model by using a repulsive energy function. The idea is based

on the energy-based unfolding and interpolation, which are guaranteed

to produce intersection-free movements for closed 2D linkages. Here, we

apply those approaches for articulated characters in 3D space. We present

the results of two experiments. In the initial experiment, starting from

a posture that the body limbs are tangled with each other, the body is

controlled to unfold tangles and straighten the limbs by moving the body

in the gradient direction of an energy function based on the distance

between two arbitrary linkages. In the second experiment, two diﬀerent

postures of limbs being tangled are interpolated by guiding the body using

the energy function. We show that intersection free movements can be

synthesized even when starting from complex postures that the limbs are

intertwined with each other. At the end of the paper, we discuss about

the limitations of the method and future possibilities of this approach.

Keywords: Character animation, Motion planning, Pose interpolation

1 Introduction

Controlling characters that are closely interacting with its own body, other char-

acters and the environment is a diﬃcult problem. As the joint angle representa-

tion, which is the most prevalent way to express the posture of characters, does

not consider the spatial relationship of the body limbs, the resultant movements

suﬀer from collisions and penetrations.

In this paper, we present the results of controlling a character by using a

repulsive energy function that has been successfully applied to control 2D link-

ages for unfolding [3] and interpolating [12] diﬀerent conﬁgurations. We deﬁne

a similar energy function used in [3, 12] that is based on the Euclidian distance

of the linkages of character skeletons in 3 dimensional space.

We show the results of two experiments, which are the 3D and open linkage

versions of what are done in [3] and [12]. In the ﬁrst experiment, starting from a

posture in which the limbs of the body are tangled with each other, the character

is controlled in the gradient direction of the energy function and the limbs are

successfully unfolded (see Figure 3). Next, we interpolate two diﬀerent postures

in which the limbs are tangled with each other. For several pair of postures, we

can successfully interpolate them without any collisions or penetrations of the

2 Energy-based Pose Unfolding and Interpolation

limbs (see Figure 4). These preliminary results hint the possibility to use the

repulsive energy function for path planning and motion synthesis.

The rest of the paper is composed as follows. In section 2, we review some

related work of motion synthesis. In section 3, we describe about the energy

function that we deﬁne to control the character. In section 4, we show the ex-

perimental results of unfolding postures in which the limbs are closely interacting

with one another. In section 5, we describe the algorithm and experimental re-

sults of interpolating diﬀerent postures by using this energy function. In section

6, we discuss about the problems of the method and the possible approaches for

coping with the problem, and conclude the paper.

2 Related Works

Simulating the close interactions of multiple characters is an interest of many

researchers in character animation. We ﬁrst review some work in the ﬁeld. We

then review path-planning methods for synthesizing movements of close inter-

actions. Finally, we give a brief review of some work in computational geometry

that handles problems of unfolding and interpolation.

2.1 Character interaction

The simulation of interactions between multiple characters has many applica-

tions such as computer games, virtual environments and ﬁlms. Liu et al.[16]

simulate the close dense interactions of two characters by repetitively updating

the motion of each character by spacetime constraints. Lee and Lee [15] sim-

ulate the boxing match by using reinforcement learning. Treuille et al [20] use

reinforcement learning to simulate the pedestrians avoiding each other. Shum

et al [17, 19] use min-max search to ﬁnd the optimal action in a competitive

environment. They also propose a real-time approach based on an automatically

produced ﬁnite state machine [18]. These researches do not handle very close

interactions such as holding or wrestling. Ho and Komura [9] propose to evalu-

ate the similarity of character postures based on the topological relationships.

When equivalent postures are found, the postures are linearly interpolated at the

level of generalized coordinates. In [8], Ho and Komura deﬁne a new coordinate

system called Topology Coordinates to synthesize and interpolate keyframes in

which the characters tangle their limbs with each other. The method is applied

for a prototype of a wrestling game in [10]. The postures are restricted to those

involve tangling and it is not very suitable when there is no tangle between the

characters.

2.2 Path-planning movements of close interactions

Up to now, if we want to generate scenes of close contacts such as one character

holds, carries or wrestles with another, it is necessary to plan the motions by

Energy-based Pose Unfolding and Interpolation 3

Fig. 1. An example of a locked 5 link chain.

global path planning / collision avoidance approaches, such as Rapidly-exploring

Random Trees (RRT) [14] or Probabilistic Road Maps (PRM) [13]. Yamane et

al. [21] simulate motions to move luggage from one place to another by combining

IK and RRT. Hirano et al [6] and Berenson et al. [1] propose to use RRT to

search the state space for grasping objects. Ho and Komura [7] use RRT to plan

motions such as holding and piggybacking. The problems of global planning

methods are (1) they require a huge amount of computation to ﬁnd collision free

paths, (2) as the paths are randomly searched, they are not consistent, and (3)

the resulting motion requires further reﬁnement such as smoothing due to its

jaggyness. Therefore, these approaches are not practical from the viewpoint of

interactive character motion synthesis and editing.

2.3 Linkage unfolding in computational geometry

In computational geometry, a number of researchers have been motivated by the

question about locked chains. The main question is which types of linkages al-

ways have connected conﬁguration spaces [5]. If the conﬁguration is connected,

it is called unlocked, and any two conﬁgurations can be interpolated. On the

contrary, if there are disconnected conﬁgurations, it is called locked. In 3D, a

locked open chain of ﬁve links was found by Cantarella and Johnson [2] (see

Figure 1). In [4], Connelly et al. prove that there are no locked 2D chains. Upon

this discovery, Cantarella et al. [3] propose an energy-driven approach to unfold

linkages, and Iben et al. [12] propose an extended approach to interpolate arbi-

trary conﬁgurations. We apply these approaches for unfolding and interpolating

postures of 3D articulated characters. Although it is proven that there can be

locked conﬁgurations for 3D articulated characters, we show that this approach

works well for various complex postures which we observe in daily life.

3 Unfolding the Body by Repulsive Energy

In this section, we ﬁrst explain about the repulsive energy function that we use

for the experiments in this paper. Next we show experimental results of unfolding

a folded body posture by moving the body in the gradient direction of this energy

function.

3.1 Repulsive Energy

The repulsive energy move each body segment away from each other. We design

the energy such that it converges to inﬁnity when the distance between two

4 Energy-based Pose Unfolding and Interpolation

segments decrease to zero:

G=X

i,j,i6=j

E(qi, qj) = 1

D(qi, qj)(1)

where qi, qjare the conﬁgurations of the two body segments, and D(qi, qj) is

the shortest distance between the two body segments. We use this function to

guide the character to either unfold from a folded posture and interpolate two

diﬀerent postures. The energy becomes larger when the body is folded and will

be smaller when all the limbs are stretched out, as shown in Figure 2.

Fig. 2. A folded posture (left) and an unfolded posture (right), and their repulsive

energies shown at the bottom of each posture.

3.2 Unfolding Folded Postures

We use the energy function deﬁned in Eq.1 to unfold the postures. Starting from

various folded postures in which the limbs are tangled with one another, the

Jacobian of the repulsive energy is computed, and the body posture is updated

by moving the joint angles in the gradient direction. The updates of the body

joint angles are computed by the following equation:

∆q = (−I+JJT)J

kJk(2)

where qis the vector of joint angles and Jis the Jacobian of the repulsive

energy G:J=∂G

∂q . After solving ∆q, it can be normalized and used to update

q:q:= q+∆q. However, we also need to consider other constraints, such as

positional constraints and joint limits. Here we linearize the nonlinear constraints

and turn inequality constraints into equality constraints by imposing them only

Energy-based Pose Unfolding and Interpolation 5

when the original inequality constraints are violated [11]. Let us represent all

the linear constraints by

K∆q =C. (3)

Then, we project the original ∆q onto the null space of these constraints by:

∆q′=∆q −KTl(4)

where

l= (KK T)−1(K ∆q +αǫ),(5)

αis constant whose value is set to 0.01 in our experiments and ǫ=K∆q −C.

Sometimes, after projection, ∆q′might be facing the gradient direction, i.e.,

J·∆q′>0. In such a case, we need to update ∆q′by a further projection:

∆q′′ =∆q′−DTl′(6)

where

l′= (DDT)−1(D∆q′+αγ) (7)

D=K

G, γ =ǫ

σ/α ,(8)

and σis a small negative value which is set to −0.01 in our experiments.

Starting from various conﬁgurations shown in the left-most column in Figure

3, the body can be unfolded into postures shown in the right-most column. It can

be observed that the body can succesfully unfold the limbs in these examples.

Although the unfolding is known to always converge for 2D linkages [3], this

is not always the case in 3D. This is simply because simply moving the body

in the gradient of a repulsive energy can fall into local minima especially when

the limbs are making knots with each other. However, we have not faced such

situations in our experiments, even for a very complex yoga pose shown in the

bottom of Figure 3.

4 Interpolating Postures by the Repulsive Energy

In this section, we present the method and the experimental results of interpo-

lating diﬀerent postures using the repulsive energy deﬁned in section 3.1. We are

applying the algorithm of interpolating arbitrary conﬁgurations by 2D linkages

[12] to 3 dimensional tree structures.

4.1 Methodology

Assume we are interpolating two diﬀerent postures represented by q1and q2. The

algorithm proceeds by iteratively updating the two postures such that they ap-

proach towards each other. Again we assume there are linear constraints imposed

6 Energy-based Pose Unfolding and Interpolation

Fig. 3. Examples of unfolding movements starting from postures in the left

Energy-based Pose Unfolding and Interpolation 7

to the system represented by Eq.3. In each iteration, we ﬁrst examine the energy

of two postures, then move the posture with the higher energy, say qh, towards

the lower one, say ql. We ﬁrst calculate the update vector: ∆q =ql−qh. Since

moving along this direction could violate some linear constraints, we project the

vector to the null space of the constraints by Eq. 4. Because moving in this

direction might increase the repulsive energy, we apply another projection by

Eq. 6. If such a direction cannot be found (if DDTin Eq.7 is not invertible), we

project the gradient-descent direction vector of the energy to the null space of

the linear constraints. Once the update vector is calculated, we move qhalong

this direction by a small amount.

Below is the pseudo code of the interpolation algorithm:

1. Initialization: q1←q0

1and q2←q0

2.

2. Compute the energy G1and G2at q1and q2by Eq.1

3. ∆q ←sign(G2−G1)(q1−q2)

4. if(G1> G2)q←q1else q←q2

5. Compute the Jacobian of Gby partial diﬀerence : J=∂G

∂q (q)

6. Project ∆q onto the null space of Kby Eq. 4

7. if (∆q ·J > 0)

if (DDTin Eq.7 is not invertible) update ∆q by Eq. 6

else ∆q ← −J, project ∆q to the null space of Kby Eq. 4

8. q←q+β∆q

k∆qkwhere βis a small constant.

9. if (G1> G2)q1←qelse q2←q

10. If q1and q2are close enough, terminate.

11. Go to step 2.

Once the algorithm is converged, a collision free path can be synthesized by

connecting the trajectories of q1and q2.

4.2 Experimental Results

Here we show results of interpolating two diﬀerent postures explained in the

previous subsection. The initial and ﬁnal postures are shown in the left-most and

right-most columns in Figure 4, respectively. The snapshots of the interpolated

motions are shown in the middle columns. It can be observed that the postures

can be interpolated without any collisions or penetrations. In our experiments,

all diﬀerent combinations of interpolations succeeded without getting stuck at

local minima. It is to be noted that the interpolation of these postures using

joint angles or joint positions can easily result in penetrations of the limbs in

these examples.

5 Discussions and Conclusion

In this paper, we have shown experimental results of unfolding and interpolating

tangled body postures using a repulsive energy function. Although intersection-

free movements are assured for linkages in 2D, this is not the case for 3D skeletons

8 Energy-based Pose Unfolding and Interpolation

Fig. 4. Examples of interpolating two diﬀerent poses.

Energy-based Pose Unfolding and Interpolation 9

due to locks and possible knotted postures. Despite the concern, valid movements

can be synthesized in most of our experiments. This can be because the body

limbs are only composed of three to four short linkages, which is not enough to

compose complex knots. In order to unfold or interpolate conﬁgurations with

complex knots, it will be necessary to analyze the postures and ﬁnd out the

knots, and control the limbs to desolve them.

One problem of the energy-based approach is that even the postures can be

unfolded / interpolated in most of the cases, the motion synthesized is usually

very dynamic and not realistic in terms of human movements. Producing nat-

ural smooth motions using the repulsive energy function can be an interesting

research direction to follow.

References

[1] Berenso, D., Diankovage, R., Nishiwaki, K., agami, S.K., Kuﬀner, J.: Grasp plan-

ning in complex scenes. IEEE/RAS Humanoids (2007)

[2] Cantarella, J., Johnson, H.: Nontrivial embeddings of polygonal intervals and

unknots in 3-space. Journal of Knot Theory and Ramiﬁcations 7(8), 1027–1039

(1998)

[3] Cantarella, J.H., Demaine, E.D., Iben, H.N., O’Brien, J.F.: An energy-driven

approach to linkage unfolding. In: Proceedings of the 20th Annual Symposium

on Computational Geometry (Jun 2004), http://graphics.cs.berkeley.edu/

papers/Cantarella-AED-2004-06/

[4] Connelly, R., Demaine, E.D., Rote, G.: Straightening polygonal arcs and convex-

ifying polygonal cycles. In: Discrete and Computational Geometry. pp. 432–442

(2000)

[5] Demaine, E.D., O’Rourke, J.: Geometric Folding Algorithms: Linkages, Origami,

Polyhedra. Cambridge University Press, New York, NY, USA, reprint edn. (2008)

[6] Hirano, Y., Kitahama, K., Yoshizawa, S.: Image-based object recognition and

dexterous hand/arm motion planning u sing rrts for grasping in cluttered scene.

IEEE/RSJ International Conference on Intelligent Robots and Systems (2005)

[7] Ho, E.S.L., Komura, T.: Planning tangling motions for humanoids. In: Proceedings

of Humanoids 2007 (2007)

[8] Ho, E.S.L., Komura, T.: Character motion synthesis by topology coordinates.

Computer Graphics Forum (Proceedings of Eurographics 2009) 28(2) (2009)

[9] Ho, E.S.L., Komura, T.: Indexing and retrieving motions of characters in close

contact. IEEE Transactions on Visualization and Computer Graphics 15(3), 481–

492 (2009)

[10] Ho, E.S.L., Komura, T.: A ﬁnite state machine based on topology coordinates for

wrestling games. Computer Animation and Virtual Worlds pp. n/a–n/a (2011),

http://dx.doi.org/10.1002/cav.376

[11] Ho, E.S.L., Komura, T., Tai, C.L.: Spatial relationship preserving character mo-

tion adaptation. ACM Trans. Graph. 29(4) (2010)

[12] Iben, H.N., O’Brien, J.F., Demaine, E.D.: Refolding planar polygons. Discrete

and Computational Geometry 41(3), 444–460 (Apr 2009), http://graphics.cs.

berkeley.edu/papers/Iben-RPP-2009-04/

[13] Kavraki, L., Svestka, P., Latombe, J., Overmars, M.: Probabilistic roadmaps for

path planning in high-dimensional conﬁguration spaces. Tech. rep., Stanford, CA,

USA (1994)

10 Energy-based Pose Unfolding and Interpolation

[14] LaValle, S., Kuﬀner, J.: Rapidly-exploring random trees: Progress and prospects.

Robotics: The Algorithmic Perspective. 4th Int’l Workshop on the Algorithmic

Foundations of Robotics (2001)

[15] Lee, J., Lee, K.H.: Precomputing avatar behavior from human motion data. Pro-

ceedings of 2004 ACM SIGGRAPH/Eurographics Symposium on Computer An-

imation pp. 79–87 (2004)

[16] Liu, C.K., Hertzmann, A., Popovic, Z.: Composition of complex optimal multi-

character motions. ACM SIGGRAPH / Eurographics Symposium on Computer

Animation pp. 215–222 (2006)

[17] Shum, H.P.H., Komura, T., Yamazaki, S.: Simulating competitive interactions us-

ing singly captured motions. Proceedings of ACM Virtual Reality Software Tech-

nology 2007 pp. 65–72 (2007)

[18] Shum, H.P.H., Komura, T., Yamazaki, S.: Simulating interactions of avatars in

high dimensional state space. ACM SIGGRAPH Symposium on Interactive 3D

Graphics (i3D) 2008 pp. 131–138 (2008)

[19] Shum, H.P., Komura, T., Yamazaki, S.: Simulating multiple character interactions

with collaborative and adversarial goals. IEEE Transactions on Visualization and

Graphics (2011)

[20] Treuille, A., Lee, Y., Popovic’, Z.: Near-optimal character animation with contin-

uous control. ACM Transactions on Graphics 26(3), 7:1–7:7 (2007)

[21] Yamane, K., Kuﬀner, J., Hodgins, J.: Synthesizing animations of human manip-

ulation tasks. ACM Transactions on Graphics 23(3), 532–539 (2004)