Content uploaded by Brent Gillespie
Author content
All content in this area was uploaded by Brent Gillespie on Aug 19, 2014
Content may be subject to copyright.
2004-01-2178
Modeling the Coordinated Movements of the Head and Hand
Using Differential Inverse Kinematics
K. Han Kim1, R. Brent Gillespie2 and Bernard J. Martin3
1,3HUMOSIM Laboratory, The University of Michigan
2Department of Mechanical Engineering, The University of Michigan
Copyright © 2004 SAE International
ABSTRACT
Hand reach movements for manual work, vehicle
operation, and manipulation of controls are planned and
guided by visual images actively captured through eye
and head movements. It is hypothesized that reach
movements are based on the coordination of multiple
subsystems that pursue the individual goals of visual
gaze and manual reach. In the present study, shared
control coordination was simulated in reach movements
modeled using differential inverse kinematics. An 8-DOF
model represented the torso-neck-head link (visual
subsystem), and a 9-DOF model represented the torso-
upper limb link (manual subsystem), respectively. Joint
angles were predicted in the velocity domain via a
pseudo-inverse Jacobian that weighted each link for its
contribution to the movement. A secondary objective
function was introduced to enable both subsystems to
achieve the corresponding movement goals in a
coordinated manner by manipulating redundant degrees
of freedom. Simulated motions were compared to motion
recordings from ten subjects performing right-hand
reaches in a seated posture. Joint angles were predicted
with and without the contribution of the coordination
function, and model accuracy was determined using the
RMS error and differences in end posture angles. The
results indicated that prediction accuracy was generally
better when the coordination function was included. This
improvement was more pronounced for low and
eccentric targets, as they required greater contribution of
the joints shared by both visual and manual subsystems.
INTRODUCTION
Motion prediction models of lifting, reaching, or pointing
tasks have been commonly based on a multiple body
link system with a single end-effector [1, 2, 3, 4].
However, even simple reaching movements may include
multiple task components, other than moving the hand
toward the goal target. For example, reaching involves
the movement of the head and the eyes to capture
images of the environment and build an internal
representation of the space in which hand movements
are planed and guided. It has been shown that head
and/or eye movements are modulated by the movement
of the whole body and the hand [5, 6, 7]. In addition,
whole body and/or hand movements are also adjusted
as a function of visual perception of the environment [8,
9, 10]. Hence it may be suggested that the central
nervous system (CNS), while planning and executing a
movement, simultaneously coordinates multiple
subsystems that pursue individual goals (guiding the
hand, displacing the gaze, etc) in order to achieve the
general aim of the task (reaching for the target).
Seated reach movements include the movements of the
visual sub-system (eye – head – neck – torso), and the
manual subsystem (finger – hand – forearm – upper arm
– clavicle – torso). Depending on target location and
initial posture, both systems may move synergistically (in
a same direction) or antagonistically (in different
directions) with respect to each other. Since the visual
and manual subsystems share a common link (torso), it
is hypothesized that the two subsystems negotiate the
control of this common link involved in the motion of their
respective end-effectors. The CNS should plan and
coordinate the movement of each subsystem in order to
allocate the use of the common link in such a manner
that both subsystems achieve their individual goals. The
coordination of multiple subsystems is made possible by
manipulating the redundant degrees of freedom of each
subsystem.
The aims of the present study are to 1) construct a
multibody link system representing the visual and
manual subsystems; 2) develop optimization-based
inverse kinematics models to simulate the movements of
the subsystems separately and integratively; and 3)
quantify the benefit of the integration of multiple
subsystems by comparing simulated and actual
movements.
METHODS
DIFFERENTIAL INVERSE KINEMATICS
The manual subsystem (finger-hand-forearm-upper arm-
clavicle-torso, Figure 1A) and the visual sub-system
(eye-head-neck-torso links, Figure 1B) were modeled to
represent a human subject performing a seated reach
task. The manual and visual subsystems are composed
of nine and eight revolute joints, respectively (Table 1).
Table 1. Joint composition of manual and visual subsystem
Manual subsystem joints Visual subsystem joints
qm1 Torso extension (+) qv1 Torso extension (+)
qm2 Torso lateral bending (+ left) qv2 Torso lateral bending (+ left)
qm3 Torso axial rotation (+ ccw) qv3 Torso axial rotation (+ccw)
qm4 Clavicle horizontal (+ forward) qv4 Neck vertical (+ up)
qm5 Clavicle vertical (+ up) qv5 Neck horizontal (+ left)
qm6 Shoulder flexion (+) qv6 Head extension (+)
qm7 Shoulder abduction (+) qv7 Head tilt (+ left)
qm8 Upper arm axial rotation (+ccw) qv8 Head axial rotation (+ccw)
qm9 Elbow flexion (+)
The joint angles of each subsystem are combined and
represented by a vector as follows:
⎥
⎥
⎥
⎦
⎤
⎢
⎢
⎢
⎣
⎡
=
v
m
c
q
q
q
q
where
[]
T
321 ,, qqq
c=q (torso joint angles- common in both
manual and visual subsystems)
[]
T
987654 ,,,,, mmmmmmm qqqqqq=q (clavicle-shoulder-
elbow angles: manual subsystem)
[]
T
87654 ,,,, vvvvvv qqqqq=q (neck-head angles: visual
subsystem)
A B
Figure 1. Multi-link composition of 9-dof manual subsystem (A) and 8-
dof visual subsystem (B). The arrow extending from each joint indicates
the positive direction of joint rotation in a right-hand sense.
The position of the manual subsystem end-effector is
represented by Cartesian coordinates as follows:
[
]
T
mfff )(),(),( 321 qqqp = (Eq. 1)
where f denotes a function of direct kinematics of the
movement. In contrast, the end-effector of the visual
subsystem is defined using a two-dimensional image
coordinate system [11].
⎥
⎦
⎤
⎢
⎣
⎡
=
⎥
⎦
⎤
⎢
⎣
⎡
=
y
x
z
vp
p
p
k
f
f
)(
)(
5
4
q
q
p (Eq. 2)
where the k a spatial scaling factor, and px, py, and pz
represent x-, y-, and z-coordinates of the target in a
head-centered reference frame.
In general, the velocity of the end-effector p can be
obtained by:
qqJq
q
q
p&&& )(
)( =
∂
∂
=i
f (Eq. 3)
where J is a Jacobian matrix. We can use Eq. 3 for both
the manual and visual subsystem separately:
qJp && mm
=
qJp && vv
=
(Eq. 4)
where Jm and Jv
represent the Jacobian matrices for the
manual and visual subsystem, respectively.
Generally, we wish to obtain q
& as a function of p
&, but
due to the redundant degrees of freedom of the multi-
link systems, the ordinary inverse of J cannot be
obtained. Alternatively, a weighted pseudo-inverse of J
(denoted as J†) may be used.
pJpJJWJWq &&& †111 )( == −−− TT (Eq. 5)
where W is a weighting matrix that characterizes the
instantaneous contribution of each joint. The weighting
matrices for the manual subsystem were adapted from
an earlier study [2]. For the visual subsystem, regression
models of peak velocities of the joint angles, whose
measurements are described below, were used to
estimate the weighting matrices. Hence, Eq. 5 attempts
to satisfy the primary objectives of 1) obtaining joint
angles that place the end-effector at the desired
positions at a given time; 2) manipulating joint angles in
a way that the squared sum of all joint velocity is
minimized; and 3) setting the relative contribution of
each joint as determined by the weighting matrix.
Since we have redundant degrees of freedom, we can
introduce a secondary objective that determines/
reconfigures the joint angles of the linkage system
without changing the end-effector position by using a
matrix (I – J†J) which projects an arbitrary vector 0
q
&into
the null space of J [12], where I is an identity matrix.
Hence for the manual subsystem:
0
†† )( qJJIpJq &&& mmmm −+= (Eq. 6)
Multiplying Jv for on both sides and solving for 0
q
&:
0
†† )( qJJIJpJJqJ &&& mmvmmvv −+=
)()]([ †††
0mmvvmmv pJJqJJJIJq &&& −−= (Eq. 7)
By plugging Eq.7 into Eq.6:
)()]()[( †††††
mmvvmmvmmmm pJJqJJJIJJJIpJq &&&& −−−+=
(Eq. 8)
which is simplified to Eq.9 with a gain term (
α
) scaling
the secondary objective function [13].
)()]([ ††††
mmvvmmvmm pJJpJJIJpJq &&&& −−+=
α
(Eq. 9)
In the present study,
α
was set to unity. Then q is
obtained from the numerical integration of q
&, and the
drift of the end-effector position was stabilized by a
feedback control algorithm [14].
MOTION MEASUREMENTS
Subjects Ten healthy subjects, 23±3 (mean ± sd) years
old with normal vision participated in the experiments.
Equipment A circular array containing visual targets was
placed on the right frontal hemisphere of the subject.
The target area extended from 0 (mid-sagittal plane) to
90° (rightward) at the distance of 60 and 95cm from the
hip-point (Figure 2A). The height of the target array was
either at eye level or 50cm below eye level (Figure 2B).
Visual targets are distributed over the array at 15°
intervals. Each target was a seven-segment digital
display (visual angle < 1°).
Procedure In a seated posture, the subjects were
asked initially to look at the home target located in the
mid-sagittal plane while both hands were resting on the
lap (Figure 2A). Then the subject reached with the right
hand for a target illuminated at a random location and
touched it with the index finger (Figure 2B). Movements
of the body links, identified by electromagnetic markers
placed on body landmarks, were recorded by a motion
capture system (Flock of BirdsTM).
RESULTS
Movements were simulated using a model of the manual
subsystem alone (Eq. 5), the visual subsystem alone
(Eq. 5), or the coordinated manual and visual
subsystems (Eq. 9). Examples of superimposed stick
figures generated by each model and the corresponding
motion measurements are presented in Figure 3. It was
observed that all models are capable of predicting the
motion trajectories similar to actual motion recordings;
however, noticeable differences between predicted and
actual motions were found for shoulder and upper arm,
and forearm links.
For torso angles, the model of the manual subsystem
alone (Figure 4B) made an accurate prediction for qm2
(lateral bending), while the model of the visual
subsystem was better at predicting qm1 (extension) and
qm3 (axial rotation). This observation is in agreement with
a direct kinematics model (Figure 1) indicating that
lateral bending of the torso may not contribute
significantly to head rotation in the direction of a target
located in the right hemisphere while it may be of
primary importance for moving the torso and the hand
toward the target. Hence it is suggested that the
coordinated model, which benefits from both individual
models (manual and visual) provides a better
“combined” accuracy for all three torso angles.
In general, the accuracy for eight of the fourteen joint
angles of the combined visuo-manual linkage system
was significantly improved by the use of a coordinated
model (Table 2). Even though qm1 (torso
flexion/extension angle) did not show a significant
difference as a main effect when contrasted by the non-
coordinated or coordinated models, the coordination ×
target height interaction effect indicated that the
coordinated model made significantly greater accurate
predictions for the low target positions, where downward
flexion movements are required for visual gaze and
manual reach. Coordination × target eccentricity
interaction effects indicated that prediction accuracy for
qm2 (torso lateral bending) increases with target
eccentricity in the coordinated model. These results
indicate that coordination improves model prediction
accuracy when the body segments are effectively
involved in a motion.
A
B
Figure 2. Configuration of targets in a top view (A) and rear view (B).
Figure 4. Model simulation result for torso angles; 1: flexion (up +), 2:
lateral bending (left+), 3: axial rotation (CCW+). Actual angles from
motion measurements (A), and predictions by a model of the manual
subsystem (B), the visual subsystem (C), and coordinated manual and
visual subsystems (D). The target was located at 90° from the mid-
sagittal plane, 50cm below eye level (distance = 60cm).
Table 2. RMS error of the joint angles predicted by models for all target
locations. Shaded rows correspond to significant improvements in
prediction accuracy by the coordinated model.
Joint Non-Coordinated
(mean ± se°)
Coordinated
(mean ± se°)
Significance
qm1 3.5±0.3 4.2±0.4 p < 0.05
qm2 2.3±0.2 3.0±0.3 Non-significant
qm3 7.7±0.9 5.1±0.5 p < 0.05
qm4 6.2±0.7 8.3±1.0 p < 0.05
qm5 8.1±1.1 7.1±1.0 p < 0.05
qm6 25.4±2.8 19.5±2.3 p < 0.05
qm7 14.2±1.6 11.8±1.1 p < 0.05
qm8 39.1±4.2 35.2±3.7 p < 0.05
qm9 14.7±0.9 11.8±1.0 p < 0.05
qv4 8.9±1.0 5.8±1.0 p < 0.05
qv5 4.0±0.5 5.1±0.6 p < 0.05
qv6 6.5±0.6 5.3±0.5 p < 0.05
qv7 7.9±0.3 7.6±0.4 Non-significant
qv8 4.1±0.5 4.0±0.4 Non-significant
A
B
C
⎯⎯→ time ▬▬▬▬ measured ▬▬▬▬ predicted
Figure 3. Stick figures generated by actual motion measurements and superimposed model predictions. The target was located in the mid-sagittal plane
at the eye level (distance = 60cm). A) manual subsystem; B) visual subsystem; C) manual + visual coordinated. The dotted lines that extend from the
cross hairs represent the head orientation vector [15].
Prediction of end-posture angles was also generally
improved by the coordinated model (Table 3). More
pronouncedly it was observed that the prediction error
for torso axial rotation (qm3) was reduced by 85% when
the coordinated model was used. Also the prediction
accuracy for qm1 (torso flexion/extension) and qm2 (torso
lateral bending) increases for targets at low height and
far eccentricity, respectively, which is consistent with the
observations about the RMS error statistics above.
Table 3. Error of the joint angles of the end posture predicted by
models for all target locations. Shaded rows correspond to significant
improvements in prediction accuracy by the coordinated model.
Joint Non-Coordinated
(mean ± se°)
Coordinated
(mean ± se°)
Significance
qm1 0.8±1.0 2.4±0.9 p < 0.05
qm2 0.7±0.9 -2.4±1.0 p < 0.05
qm3 11.5±1.5 1.7±1.2 p < 0.05
qm4 7.9±2.0 13.0±2.1 p < 0.05
qm5 13.8±1.8 12.2±1.8 p < 0.05
qm6 -41.0±4.4 -31.1±3.7 p < 0.05
qm7 24.7±3.4 14.3±3.6 p < 0.05
qm8 53.3±10.7 46.3±10.7 p < 0.05
qm9 25.8±2.0 19.7±2.4 p < 0.05
qv4 -13.6±1.9 -7.9±1.8 p < 0.05
qv5 2.0±1.2 6.2±1.1 p < 0.05
qv6 -8.2±1.4 -3.6±1.3 p < 0.05
qv7 8.2±1.2 5.7±1.4 p < 0.05
qv8 3.4±1.0 -2.0±0.9 p < 0.05
CONCLUSIONS
In visually guided reach motions, the visual and manual
subsystems act to locate the target and move the hand
to the target, respectively [16, 17]. The present model
proposes a method of incorporating multiple subsystems
with individual end-effectors using an optimization-based
inverse kinematics algorithm. In this process of
incorporation, coordination is required in order to share
common resources and subsystems that are dedicated
to manipulate their respective end-effector. The common
links may be controlled by either one of the subsystems
exclusively, while the other subsystem’s control is
restrained. Movement accuracy can be viewed as the
result of this coordination. Dominance of one subsystem
over another may be a function of tasks requirements in
a specific context.
It was found that in general the accuracy of the predicted
joint angle trajectories was better when the coordination
is introduced as a secondary cost function in the
differential inverse kinematics. Accordingly the results of
this model suggest that 1) the central controller takes
into account the constraints of each subsystem to find
an optimal set of joint angles, hence coordination can be
viewed as the compromise of shared control between
identified subsystems involved in a movement; 2) the
advantage of the coordination model is more prominent
for reach movements to low and eccentric targets. This
latter effect shows that the accuracy of the model
increases with the effective contribution of a joint to a
visually guided reach movement. Hence, accuracy of the
coordinated model may be better than it appears when
considering only the average RMS errors including all
target locations. The statistically significant interaction
effects mentioned above support this hypothesis.
ACKNOWLEDGMENTS
We would like to acknowledge and thank the partners of
the Human Motion Simulation project for their support of
the present study (DaimlerChrysler Company, EDS -
PLM Solutions, Ford Motor Company, General Motors
Corporation, International Truck and Engine
Corporation, Lockheed Martin Corporation / Sandia
National Laboratories, University of Michigan Automotive
Research Center, US Army – Tank-Automotive and
Armaments Command, and US Postal Service).
REFERENCES
1. Dysart, M. J., and Woldstad, J. C. 1996, Posture
prediction for static sagittal-plane lifting, Journal of
Biomechanics, 29(10), 1393-1397.
2. Zhang, X., Kuo, A. D., and Chaffin, D. B. 1999,
Optimization-based differential kinematic modeling
exhibits a velocity-control strategy for dynamic
posture determination in seated reaching
movements, Journal of Biomechanics, 31, 1035-
1042.
3. Wang, X., 1999, Behavior-based inverse kinematics
algorithm to predict arm prehension postures for
computer-aided ergonomic evaluation, Journal of
Biomechanics, 32(5), 453-460.
4. Abdel-Malek, K., Yu, W., Jaber, J., and Duncan, J.
2001, Realistic posture prediction for maximum
dexterity, SAE Technical Papers Series, 2001-01-
2110 (Warrendale, PA: Society of Automotive
Engineers).
5. Kim, K., Martin, B. J., and Park, W. 2000, Head
orientation in visually guided tasks, SAE Technical
Papers Series, 2000-01-2174 (Warrendale, PA:
Society of Automotive Engineers).
6. Delleman, N. J., Huysmans, M. A., and Kujit-Evers,
L. F. M. 2001, Postural behaviour in static gazing
sidewards, SAE Technical Papers Series, 2001-01-
2093 (Warrendale, PA: Society of Automotive
Engineers).
7. Tipper, S. P., Howard, L. A., and Paul, M. A. 2001,
Reaching affects saccade trajectories. Experimental
Brain Research, 136(2), 241-249.
8. Peterka, R. J., and Benolken, M. S. 1995, Role of
somatosensory and vestibular cues in attenuating
visually induced human postural sway, Experimental
Brain Research, 105(1), 101-110.
9. Cohn, J. V., DiZio, P., and Lackner J. R. 2000,
Reaching during virtual rotation: context specific
compensations for expected coriolis forces, Journal
of Neurophysiology. 83(6), 3230-3240.
10. van der Kooij, H., Jacobs, R., Koopman, B., van der
Helm, F. 2001, An adaptive model of sensory
integration in a dynamic environment applied to
human stance control, Biological Cybernetics. 84(2),
103-115.
11. Hashimoto, K. 1999, Observer-based Visual
Servoing, In Robotics and Automation. Eds: B.
Ghosh, N. Xi and T. J. Tarn, Academic Press,
London.
12. Greville, T. N. E. 1959, The pseudoinverse of a
rectangular or singular matrix and its application to
the solution of systems of linear equations. SIAM
Review, 1(1), 38-43.
13. Maciejewski, A. A., and Klein, C. A. 1985, Obstacle
avoidance for kinematically redundant manipulators
in dynamically varying environments, International
Journal of Robotics Research, 4(3), 109-117.
14. Chiacchio, P., Chiaverini, S., Sciavicco, L., and
Siciliano, B. 1991, Closed-loop inverse kinematics
schemes for constrained redundant manipulators
with task space augmentation and task priority
strategy. International Journal of Robotics Research,
10, 410–425.
15. Jampel, R. S., and Shi, D. X. 1992, The primary
position of the eyes, the resetting saccade, and the
transverse visual head plane. Investigative
Ophthalmology and Visual Science. 33(8), 2501-
2510.
16. Kim, K. H. and Martin, B. J. 2002, Coordinated
Movements of the Head in Hand Reaching Tasks.
Proceedings of the 46th Human Factors and
Ergonomics Society Conference, Baltimore,
Maryland.
17. Vercher, J. L., Magenes, G., Prablanc, C., and
Gauthier, G. M. 1994, Eye-head-hand coordination
in pointing at visual targets: Spatial and temporal
analysis. Experimental Brain Research, 99, 507-523.
CONTACT
K. Han Kim
Email: kyunghan@umich.edu
Phone: +1-734-647-3241
Address: 1205 Beal Avenue,
Ann Arbor, MI 48109-2117
USA