Energy-Based Pose Unfolding and Interpolation for 3D Articulated Characters

Abstract

In this paper, we show results of controlling a 3D articulated human body model by using a repulsive energy function. The idea is based on the energy-based unfolding and interpolation, which are guaranteed to produce intersection-free movements for closed 2D linkages. Here, we apply those approaches for articulated characters in 3D space. We present the results of two experiments. In the initial experiment, starting from a posture that the body limbs are tangled with each other, the body is controlled to unfold tangles and straighten the limbs by moving the body in the gradient direction of an energy function based on the distance between two arbitrary linkages. In the second experiment, two different postures of limbs being tangled are interpolated by guiding the body using the energy function. We show that intersection free movements can be synthesized even when starting from complex postures that the limbs are intertwined with each other. At the end of the paper, we discuss about the limitations of the method and future possibilities of this approach.

Full text

Energy-based Pose Unfolding and Interpolation
for 3D Articulated Characters
He Wang and Taku Komura
1Edinburgh University, 10 Crichton Street, EH8 9AB, United Kingdom
Abstract. In this paper, we show results of controlling a 3D articulated
human body model by using a repulsive energy function. The idea is based
on the energy-based unfolding and interpolation, which are guaranteed
to produce intersection-free movements for closed 2D linkages. Here, we
apply those approaches for articulated characters in 3D space. We present
the results of two experiments. In the initial experiment, starting from
a posture that the body limbs are tangled with each other, the body is
controlled to unfold tangles and straighten the limbs by moving the body
in the gradient direction of an energy function based on the distance
between two arbitrary linkages. In the second experiment, two different
postures of limbs being tangled are interpolated by guiding the body using
the energy function. We show that intersection free movements can be
synthesized even when starting from complex postures that the limbs are
intertwined with each other. At the end of the paper, we discuss about
the limitations of the method and future possibilities of this approach.
Keywords: Character animation, Motion planning, Pose interpolation
1 Introduction
Controlling characters that are closely interacting with its own body, other char-
acters and the environment is a difficult problem. As the joint angle representa-
tion, which is the most prevalent way to express the posture of characters, does
not consider the spatial relationship of the body limbs, the resultant movements
suffer from collisions and penetrations.
In this paper, we present the results of controlling a character by using a
repulsive energy function that has been successfully applied to control 2D link-
ages for unfolding [3] and interpolating [12] different configurations. We define
a similar energy function used in [3, 12] that is based on the Euclidian distance
of the linkages of character skeletons in 3 dimensional space.
We show the results of two experiments, which are the 3D and open linkage
versions of what are done in [3] and [12]. In the first experiment, starting from a
posture in which the limbs of the body are tangled with each other, the character
is controlled in the gradient direction of the energy function and the limbs are
successfully unfolded (see Figure 3). Next, we interpolate two different postures
in which the limbs are tangled with each other. For several pair of postures, we
can successfully interpolate them without any collisions or penetrations of the

2 Energy-based Pose Unfolding and Interpolation
limbs (see Figure 4). These preliminary results hint the possibility to use the
repulsive energy function for path planning and motion synthesis.
The rest of the paper is composed as follows. In section 2, we review some
related work of motion synthesis. In section 3, we describe about the energy
function that we define to control the character. In section 4, we show the ex-
perimental results of unfolding postures in which the limbs are closely interacting
with one another. In section 5, we describe the algorithm and experimental re-
sults of interpolating different postures by using this energy function. In section
6, we discuss about the problems of the method and the possible approaches for
coping with the problem, and conclude the paper.
2 Related Works
Simulating the close interactions of multiple characters is an interest of many
researchers in character animation. We first review some work in the field. We
then review path-planning methods for synthesizing movements of close inter-
actions. Finally, we give a brief review of some work in computational geometry
that handles problems of unfolding and interpolation.
2.1 Character interaction
The simulation of interactions between multiple characters has many applica-
tions such as computer games, virtual environments and films. Liu et al.[16]
simulate the close dense interactions of two characters by repetitively updating
the motion of each character by spacetime constraints. Lee and Lee [15] sim-
ulate the boxing match by using reinforcement learning. Treuille et al [20] use
reinforcement learning to simulate the pedestrians avoiding each other. Shum
et al [17, 19] use min-max search to find the optimal action in a competitive
environment. They also propose a real-time approach based on an automatically
produced finite state machine [18]. These researches do not handle very close
interactions such as holding or wrestling. Ho and Komura [9] propose to evalu-
ate the similarity of character postures based on the topological relationships.
When equivalent postures are found, the postures are linearly interpolated at the
level of generalized coordinates. In [8], Ho and Komura define a new coordinate
system called Topology Coordinates to synthesize and interpolate keyframes in
which the characters tangle their limbs with each other. The method is applied
for a prototype of a wrestling game in [10]. The postures are restricted to those
involve tangling and it is not very suitable when there is no tangle between the
characters.
2.2 Path-planning movements of close interactions
Up to now, if we want to generate scenes of close contacts such as one character
holds, carries or wrestles with another, it is necessary to plan the motions by

Energy-based Pose Unfolding and Interpolation 3
Fig. 1. An example of a locked 5 link chain.
global path planning / collision avoidance approaches, such as Rapidly-exploring
Random Trees (RRT) [14] or Probabilistic Road Maps (PRM) [13]. Yamane et
al. [21] simulate motions to move luggage from one place to another by combining
IK and RRT. Hirano et al [6] and Berenson et al. [1] propose to use RRT to
search the state space for grasping objects. Ho and Komura [7] use RRT to plan
motions such as holding and piggybacking. The problems of global planning
methods are (1) they require a huge amount of computation to find collision free
paths, (2) as the paths are randomly searched, they are not consistent, and (3)
the resulting motion requires further refinement such as smoothing due to its
jaggyness. Therefore, these approaches are not practical from the viewpoint of
interactive character motion synthesis and editing.
2.3 Linkage unfolding in computational geometry
In computational geometry, a number of researchers have been motivated by the
question about locked chains. The main question is which types of linkages al-
ways have connected configuration spaces [5]. If the configuration is connected,
it is called unlocked, and any two configurations can be interpolated. On the
contrary, if there are disconnected configurations, it is called locked. In 3D, a
locked open chain of five links was found by Cantarella and Johnson [2] (see
Figure 1). In [4], Connelly et al. prove that there are no locked 2D chains. Upon
this discovery, Cantarella et al. [3] propose an energy-driven approach to unfold
linkages, and Iben et al. [12] propose an extended approach to interpolate arbi-
trary configurations. We apply these approaches for unfolding and interpolating
postures of 3D articulated characters. Although it is proven that there can be
locked configurations for 3D articulated characters, we show that this approach
works well for various complex postures which we observe in daily life.
3 Unfolding the Body by Repulsive Energy
In this section, we first explain about the repulsive energy function that we use
for the experiments in this paper. Next we show experimental results of unfolding
a folded body posture by moving the body in the gradient direction of this energy
function.
3.1 Repulsive Energy
The repulsive energy move each body segment away from each other. We design
the energy such that it converges to infinity when the distance between two

4 Energy-based Pose Unfolding and Interpolation
segments decrease to zero:
G=X
i,j,i6=j
E(qi, qj) = 1
D(qi, qj)(1)
where qi, qjare the configurations of the two body segments, and D(qi, qj) is
the shortest distance between the two body segments. We use this function to
guide the character to either unfold from a folded posture and interpolate two
different postures. The energy becomes larger when the body is folded and will
be smaller when all the limbs are stretched out, as shown in Figure 2.
Fig. 2. A folded posture (left) and an unfolded posture (right), and their repulsive
energies shown at the bottom of each posture.
3.2 Unfolding Folded Postures
We use the energy function defined in Eq.1 to unfold the postures. Starting from
various folded postures in which the limbs are tangled with one another, the
Jacobian of the repulsive energy is computed, and the body posture is updated
by moving the joint angles in the gradient direction. The updates of the body
joint angles are computed by the following equation:
∆q = (−I+JJT)J
kJk(2)
where qis the vector of joint angles and Jis the Jacobian of the repulsive
energy G:J=∂G
∂q . After solving ∆q, it can be normalized and used to update
q:q:= q+∆q. However, we also need to consider other constraints, such as
positional constraints and joint limits. Here we linearize the nonlinear constraints
and turn inequality constraints into equality constraints by imposing them only

Energy-based Pose Unfolding and Interpolation 5
when the original inequality constraints are violated [11]. Let us represent all
the linear constraints by
K∆q =C. (3)
Then, we project the original ∆q onto the null space of these constraints by:
∆q′=∆q −KTl(4)
where
l= (KK T)−1(K ∆q +αǫ),(5)
αis constant whose value is set to 0.01 in our experiments and ǫ=K∆q −C.
Sometimes, after projection, ∆q′might be facing the gradient direction, i.e.,
J·∆q′>0. In such a case, we need to update ∆q′by a further projection:
∆q′′ =∆q′−DTl′(6)
where
l′= (DDT)−1(D∆q′+αγ) (7)
D=K
G, γ =ǫ
σ/α ,(8)
and σis a small negative value which is set to −0.01 in our experiments.
Starting from various configurations shown in the left-most column in Figure
3, the body can be unfolded into postures shown in the right-most column. It can
be observed that the body can succesfully unfold the limbs in these examples.
Although the unfolding is known to always converge for 2D linkages [3], this
is not always the case in 3D. This is simply because simply moving the body
in the gradient of a repulsive energy can fall into local minima especially when
the limbs are making knots with each other. However, we have not faced such
situations in our experiments, even for a very complex yoga pose shown in the
bottom of Figure 3.
4 Interpolating Postures by the Repulsive Energy
In this section, we present the method and the experimental results of interpo-
lating different postures using the repulsive energy defined in section 3.1. We are
applying the algorithm of interpolating arbitrary configurations by 2D linkages
[12] to 3 dimensional tree structures.
4.1 Methodology
Assume we are interpolating two different postures represented by q1and q2. The
algorithm proceeds by iteratively updating the two postures such that they ap-
proach towards each other. Again we assume there are linear constraints imposed

6 Energy-based Pose Unfolding and Interpolation
Fig. 3. Examples of unfolding movements starting from postures in the left

Energy-based Pose Unfolding and Interpolation 7
to the system represented by Eq.3. In each iteration, we first examine the energy
of two postures, then move the posture with the higher energy, say qh, towards
the lower one, say ql. We first calculate the update vector: ∆q =ql−qh. Since
moving along this direction could violate some linear constraints, we project the
vector to the null space of the constraints by Eq. 4. Because moving in this
direction might increase the repulsive energy, we apply another projection by
Eq. 6. If such a direction cannot be found (if DDTin Eq.7 is not invertible), we
project the gradient-descent direction vector of the energy to the null space of
the linear constraints. Once the update vector is calculated, we move qhalong
this direction by a small amount.
Below is the pseudo code of the interpolation algorithm:
1. Initialization: q1←q0
1and q2←q0
2.
2. Compute the energy G1and G2at q1and q2by Eq.1
3. ∆q ←sign(G2−G1)(q1−q2)
4. if(G1> G2)q←q1else q←q2
5. Compute the Jacobian of Gby partial difference : J=∂G
∂q (q)
6. Project ∆q onto the null space of Kby Eq. 4
7. if (∆q ·J > 0)
if (DDTin Eq.7 is not invertible) update ∆q by Eq. 6
else ∆q ← −J, project ∆q to the null space of Kby Eq. 4
8. q←q+β∆q
k∆qkwhere βis a small constant.
9. if (G1> G2)q1←qelse q2←q
10. If q1and q2are close enough, terminate.
11. Go to step 2.
Once the algorithm is converged, a collision free path can be synthesized by
connecting the trajectories of q1and q2.
4.2 Experimental Results
Here we show results of interpolating two different postures explained in the
previous subsection. The initial and final postures are shown in the left-most and
right-most columns in Figure 4, respectively. The snapshots of the interpolated
motions are shown in the middle columns. It can be observed that the postures
can be interpolated without any collisions or penetrations. In our experiments,
all different combinations of interpolations succeeded without getting stuck at
local minima. It is to be noted that the interpolation of these postures using
joint angles or joint positions can easily result in penetrations of the limbs in
these examples.
5 Discussions and Conclusion
In this paper, we have shown experimental results of unfolding and interpolating
tangled body postures using a repulsive energy function. Although intersection-
free movements are assured for linkages in 2D, this is not the case for 3D skeletons

8 Energy-based Pose Unfolding and Interpolation
Fig. 4. Examples of interpolating two different poses.

Energy-based Pose Unfolding and Interpolation 9
due to locks and possible knotted postures. Despite the concern, valid movements
can be synthesized in most of our experiments. This can be because the body
limbs are only composed of three to four short linkages, which is not enough to
compose complex knots. In order to unfold or interpolate configurations with
complex knots, it will be necessary to analyze the postures and find out the
knots, and control the limbs to desolve them.
One problem of the energy-based approach is that even the postures can be
unfolded / interpolated in most of the cases, the motion synthesized is usually
very dynamic and not realistic in terms of human movements. Producing nat-
ural smooth motions using the repulsive energy function can be an interesting
research direction to follow.
References
[1] Berenso, D., Diankovage, R., Nishiwaki, K., agami, S.K., Kuffner, J.: Grasp plan-
ning in complex scenes. IEEE/RAS Humanoids (2007)
[2] Cantarella, J., Johnson, H.: Nontrivial embeddings of polygonal intervals and
unknots in 3-space. Journal of Knot Theory and Ramifications 7(8), 1027–1039
(1998)
[3] Cantarella, J.H., Demaine, E.D., Iben, H.N., O’Brien, J.F.: An energy-driven
approach to linkage unfolding. In: Proceedings of the 20th Annual Symposium
on Computational Geometry (Jun 2004), http://graphics.cs.berkeley.edu/
papers/Cantarella-AED-2004-06/
[4] Connelly, R., Demaine, E.D., Rote, G.: Straightening polygonal arcs and convex-
ifying polygonal cycles. In: Discrete and Computational Geometry. pp. 432–442
(2000)
[5] Demaine, E.D., O’Rourke, J.: Geometric Folding Algorithms: Linkages, Origami,
Polyhedra. Cambridge University Press, New York, NY, USA, reprint edn. (2008)
[6] Hirano, Y., Kitahama, K., Yoshizawa, S.: Image-based object recognition and
dexterous hand/arm motion planning u sing rrts for grasping in cluttered scene.
IEEE/RSJ International Conference on Intelligent Robots and Systems (2005)
[7] Ho, E.S.L., Komura, T.: Planning tangling motions for humanoids. In: Proceedings
of Humanoids 2007 (2007)
[8] Ho, E.S.L., Komura, T.: Character motion synthesis by topology coordinates.
Computer Graphics Forum (Proceedings of Eurographics 2009) 28(2) (2009)
[9] Ho, E.S.L., Komura, T.: Indexing and retrieving motions of characters in close
contact. IEEE Transactions on Visualization and Computer Graphics 15(3), 481–
492 (2009)
[10] Ho, E.S.L., Komura, T.: A finite state machine based on topology coordinates for
wrestling games. Computer Animation and Virtual Worlds pp. n/a–n/a (2011),
http://dx.doi.org/10.1002/cav.376
[11] Ho, E.S.L., Komura, T., Tai, C.L.: Spatial relationship preserving character mo-
tion adaptation. ACM Trans. Graph. 29(4) (2010)
[12] Iben, H.N., O’Brien, J.F., Demaine, E.D.: Refolding planar polygons. Discrete
and Computational Geometry 41(3), 444–460 (Apr 2009), http://graphics.cs.
berkeley.edu/papers/Iben-RPP-2009-04/
[13] Kavraki, L., Svestka, P., Latombe, J., Overmars, M.: Probabilistic roadmaps for
path planning in high-dimensional configuration spaces. Tech. rep., Stanford, CA,
USA (1994)

10 Energy-based Pose Unfolding and Interpolation
[14] LaValle, S., Kuffner, J.: Rapidly-exploring random trees: Progress and prospects.
Robotics: The Algorithmic Perspective. 4th Int’l Workshop on the Algorithmic
Foundations of Robotics (2001)
[15] Lee, J., Lee, K.H.: Precomputing avatar behavior from human motion data. Pro-
ceedings of 2004 ACM SIGGRAPH/Eurographics Symposium on Computer An-
imation pp. 79–87 (2004)
[16] Liu, C.K., Hertzmann, A., Popovic, Z.: Composition of complex optimal multi-
character motions. ACM SIGGRAPH / Eurographics Symposium on Computer
Animation pp. 215–222 (2006)
[17] Shum, H.P.H., Komura, T., Yamazaki, S.: Simulating competitive interactions us-
ing singly captured motions. Proceedings of ACM Virtual Reality Software Tech-
nology 2007 pp. 65–72 (2007)
[18] Shum, H.P.H., Komura, T., Yamazaki, S.: Simulating interactions of avatars in
high dimensional state space. ACM SIGGRAPH Symposium on Interactive 3D
Graphics (i3D) 2008 pp. 131–138 (2008)
[19] Shum, H.P., Komura, T., Yamazaki, S.: Simulating multiple character interactions
with collaborative and adversarial goals. IEEE Transactions on Visualization and
Graphics (2011)
[20] Treuille, A., Lee, Y., Popovic’, Z.: Near-optimal character animation with contin-
uous control. ACM Transactions on Graphics 26(3), 7:1–7:7 (2007)
[21] Yamane, K., Kuffner, J., Hodgins, J.: Synthesizing animations of human manip-
ulation tasks. ACM Transactions on Graphics 23(3), 532–539 (2004)