Content uploaded by Sean Carver
Author content
All content in this area was uploaded by Sean Carver
Content may be subject to copyright.
Biol. Cybern. 92, 147–163 (2005)
DOI 10.1007/s00422-004-0535-x
© Springer-Verlag 2005
Comparing internal models of the dynamics
of the visual environment
Sean Carver
1
, Tim Kiemel
1,2
, Herman van der Kooij
3
, John J. Jeka
1,4
1
Department of Kinesiology, University of Maryland, College Park, MD 20742, USA
2
Department of Biology, University of Maryland, College Park, MD, USA
3
Institute for Biomedical Technology, University of Twente, Enschede, The Netherlands
4
Program in Neuroscience and Cognitive Science, University of Maryland, College Park, MD, USA
Received: 16 June 2004 / Accepted: 17 November 2004 / Published online: 9 February 2005
Abstract. It is well known that the human postural con-
trol system responds to motion of the visual scene, but
the implicit assumptions it makes about the visual envi-
ronment and what quantities, if any, it estimates about the
visual environment are unknown. This study compares the
behavior of four models of the human postural control sys-
tem to experimental data. Three include internal models
that estimate the state of the visual environment, implic-
itly assuming its dynamics to be that of a linear stochastic
process (respectively, a random walk, a general first-order
process, and a general second-order process). In each case,
all of the coefficients that describe the process are esti-
mated by an adaptive scheme based on maximum likeli-
hood. The fourth model does not estimate the state of the
visual environment. It adjusts sensory weights to minimize
the mean square of the control signal without making any
specific assumptions about the dynamic properties of the
environmental motion.
We find that both having an internal model of the visual
environment and its type make a significant difference in
how the postural system responds to motion of the visual
scene. Notably, the second-order process model outper-
forms the human postural system in its response to sinu-
soidal stimulation. Specifically, the second-order process
model can correctly identify the frequency of the stimu-
lus and completely compensate so that the motion of the
visual scene has no effect on sway. In this case the postural
control system extracts the same information from the
visual modality as it does when the visual scene is station-
ary. The fourth model that does not simulate the motion of
the visual environment is the only one that reproduces the
experimentally observed result that, across different fre-
quencies of sinusoidal stimulation, the gain with respect
to the stimulus drops as the amplitude of the stimulus
increases but the phase remains roughly constant. Our re-
sults suggest that the human postural control system does
not estimate the state of the visual environment to respond
to sinusoidal stimuli.
Correspondence to: S. Carver
(e-mail: sc350@umail.umd.edu,
Tel.: +1-301–4058272, Fax: +1-301–4055578)
1 Introduction
Significant progress in understanding the human postural
control system over the last 10–20 years has emphasized
the need to understand its inherently adaptive nature.
It is now a generally held view that visual, vestibular,
and somatosensory inputs are dynamically reweighted
to maintain upright stance as environmental or ner-
vous system conditions change (Horak and Macpherson
1996; Kiemel et al. 2002; Oie et al. 2002; Peterka 2002;
Shumway-Cook and Woollacott 2001) Environmental
changes such as moving from a light to a dark environment
or from a fixed to a moving support surface (e.g., onto a
moving walkway at an airport) require an updating of sen-
sory weights to current conditions so that muscular com-
mands are based on the most precise and reliable sensory
information available (Teasdale et al. 1991; Wolfson et al.
1985; Woollacott et al. 1986). The emphasis on adaptive
processes has highlighted the need for nonlinear models
to account for multisensory reweighting. One approach
to this problem has involved the implementation of an
internal model (Van der Kooij et al. 2001).
An internal model simulates within the nervous system
some natural process, be it internal or external to the body
(Wolpert et al. 1995). An example of how such simulations
can facilitate motor control is given by a Kalman filter.
Mathematical models of posture that include state esti-
mation typically implement some variation of this optimal
estimator (Van der Kooij et al. 1999; Kiemel et al. 2002). A
Kalman filter simulates the dynamics of the body and its
sensors to predict its afferent sensory signals. The Kalman
filter then compares its predictions to the actual signals to
derive optimal state estimates given the statistics of the
noise in the system. Thus an internal model is typically
assumed to underlie the nervous system’s state estimation
needed to control posture.
State estimation for posture traditionally involves
simulating the body and sensor dynamics. Less prom-
inent is the idea that the nervous system simulates
the dynamics of the environment, although it has
been indirectly discussed in the posture literature for
some time. Early studies (Brandt et al. 1973, 1975)
148
proposed that the perceptual system underlying pos-
tural control distinguishes objects in the environment
(foreground) from the overall environmental motion
(background) based upon evidence of visually induced
self-motion perception (i.e., vection). For instance, vec-
tion is dependent upon the number of moving contrasts
(stripes or dots) within a rotating visual display (Brandt
et al. 1975). With very few moving contrasts in the visual
display, vection is not reported, indicating that self-motion
is distinguishable from environmental motion. However,
vection is experienced more reliably as the number of mov-
ing contrasts increases. Similar effects have been observed
by controlling the area (i.e., small vs. large) of the moving
contrasts (Brandt et al. 1973). More recent experimental
results from Ravaioli et al. (2004) also suggest that the
nervous system decomposes relative motion between the
visual scene and body sway into environmental and self-
motion. Standing subjects were presented with visual dis-
plays whose motion consisted of both an oscillation and
a constant velocity translation that varied between condi-
tions. Gain relative to the oscillation did not initially de-
crease as the translation velocity increased but decreased
significantly at higher translation speeds. These results
suggest that the postural control system can selectively
compensate for translation at low speeds. The authors
proposed that an internal model, having an estimate of
the visual scene translation velocity, simulates the trans-
latory motion of the environment. Their proposed neural
controller subtracts the simulated motion from the visual
measurement to give an estimate of body motion.
Van der Kooij et al. (2001) was the first to propose an
internal model of the environment as part of the postural
control system by expanding the state estimation problem
to include estimation of properties of the environment.
Their internal model derives its prediction of the sensory
signals not only by simulating the dynamics of the body
and its sensors, but also by simulating the dynamics of
the environment. Specifically, the internal model of their
postural control system simulates the motion of the visual
scene, the translation and rotation of the support base,
and the dynamics of an externally applied force.
To implement a simulation of the environment, the ner-
vous system must have a model of how it changes. The
modeled nervous system of Van der Kooij et al. (2001)
implicitly assumes the dynamics of the position of the
visual scene (in our notation x
v
) to be a random walk.
This gives assumed (but possibly incorrect) state equa-
tions for x
v
that can be included with the Kalman filter’s
equations for the dynamics of the body and sensors. Thus
the nervous system keeps track of, utilizes, and updates
an estimate of x
v
, called ˆx
v
. With this estimate the internal
model predicts the sensory measurements. Based on the
properties of a random walk, the internal model consid-
ers motion of x
v
in either direction to be equally likely.
Therefore, it changes ˆx
v
based on discrepancies between
the actual sensory measurements and its predictions.
The update to ˆx
v
will depend not only upon the dis-
crepancies, but also upon assumed noise statistics of the
system, including an estimate of the noise level associ-
ated with the visual scene random walk dynamics (in our
notation d
1
). This estimate is a second quantity associ-
ated with the visual environment that the modeled ner-
vous system maintains and updates. Van der Kooij et al.
(2001) chose the adaptive scheme of Myers and Tapley
(1976) to estimate d
1
. It should be noted that if a random
walk correctly models the visual dynamics, then the ner-
vous system’s estimate of d
1
will approximate its true value.
Once this happens, the internal model of the environment
will match reality and the state estimates upon which the
control is based will be optimal in the sense made precise
by optimal control theory (Bryson and Ho 1975).
On the other hand, if the environmental dynamics do
not match a random walk, the nervous system’s estimate
of d
1
will depend upon the (possibly changing) properties
of the visual stimulus. Thus the nervous system’s behavior
will adapt to changing environmental conditions. But does
the model adapt in the same way as the human postural
control system?
In many respects the model of Van der Kooij et al.
(2001) succeeded in reproducing the behavior of the
human postural control system. Notably, the authors
found good quantitative agreement with the experimen-
tal data of Peterka and Benolken (1995) with regard to
gain in response to a sinusoidal stimulus across different
frequencies and amplitudes of the stimulus. Qualitatively
the model predicts that gain drops as a function of increas-
ing stimulus amplitude across different stimulus frequen-
cies – correctly reproducing this well-known experimental
result (Peterka and Benolken 1995). Van der Kooij et al.
(2001) found their model agreed in other respects with the
data of Peterka and Benolken (1995). Among other things
they found that without the vestibular sensors, their model
reproduced reasonably well gain data from vestibular loss
patients.
This paper tests more rigorously the hypothesis that an
internal model of the environment underlies the adapta-
tion of the human postural control system to changing
environmental conditions. We use observations not con-
sidered by Van der Kooij et al. (2001) to further constrain
the structure and parameterization of the model. Specifi-
cally, we consider the phase of the model in response to a
sinusoidal visual stimulus as a function of the frequency
and amplitude of the stimulus. Indeed, system identifica-
tion customarily involves both gain and phase. Numer-
ous studies (Peterka and Benolken 1995; Oie et al. 2002)
have found that phase for human subjects in response to a
sinusoidal stimulus, across different frequencies, is roughly
constant as a function of amplitude of the stimulus. At
the parameter values we tested we found that the model
of Van der Kooij et al. (2001) did not reproduce the phase
response seen with human subjects as a function of ampli-
tude. Specifically, at the values of the parameters we tested,
we found a substantial increase in phase (of about 150
◦
with a 0.2-Hz stimulus between amplitudes of 0.2
◦
and 5
◦
).
To address the issue of proper phase behavior, we were
led to consider simpler models than the ones studied in
Van der Kooij et al. (2001). Because of their simplicity,
our models were easier to understand and faster to simu-
late. With formulas that allowed us to approximate each
model’s transfer function in less than a second, we were
149
able to thoroughly explore each model’s parameter space.
We believe that different insights can come from studying
simpler models than from studying more realistic ones and
that both approaches should be taken together.
The models we present in this paper are simpler than
the model of Van der Kooij et al. (2001) in four respects.
First, the support base is fixed, and we do not include an
external force: the only variation in the environment is the
motion of the visual scene. Second, instead of a full set of
five sensors, we implement only two: a visual sensor and a
nonvisual sensor. Third, we do not model sensory dynam-
ics. The transfer function of each sensor (with respect to
velocity) is 1. Fourth, there is no time delay in our model.
As described below, our models also differ in several other
respects from the model of Van der Kooij et al. (2001).
Because we have simplified the model substantially, we
do not expect the model to agree quantitatively with exper-
imental data. Instead, we test for qualitative agreement.
Specifically, we seek to reproduce the experimental result
that across different frequencies of sinusoidal stimulation
the gain with respect to the stimulus drops as the amplitude
of the stimulus increases, but the phase remains roughly
constant. This constraint proved to be difficult to meet.
After considering the behavior of our simple postural
model with a random walk internal model of the envi-
ronment, we compare this behavior to its behavior with
two more sophisticated internal models. While these more
sophisticated internal models do not produce qualitative
agreement with experimental results, their behavior yields
interesting insights into how a postural control system is
theoretically capable of performing in response to sinusoi-
dal stimulation.
Finally, we compare the behavior of the three postural
models with internal models simulating the visual envi-
ronment in order to estimate its state to the behavior of a
postural model with no internal model simulating the envi-
ronment. This nonsimulating and nonestimating scheme
is the only one we found that produces qualitative agree-
ment with experimental data. We have found that, with
a stochastic process internal model simulating the visual
environment to estimate its state, substantial drops in gain
with increasing stimulus amplitude tend to come with sub-
stantial rises in phase at some stimulus frequencies. This
tendency could be reduced by tuning the parameters, but
we were unable to eliminate it.
2 Methods
2.1 Postural models
We modeled the standing human body as a linearized
inverted pendulum whose stability was maintained by a
proportional-derivative controller that depended upon an
estimate of the body’s position and velocity.
The modeled nervous system had two sensors. The first
sensor was visual (it output the body’s velocity relative to
the visual scene plus noise); the second sensor was nonvi-
sual (it output the body’s true velocity plus noise). Thus
neither sensor included dynamics: the transfer function of
both, with respect to velocity, was 1. We chose velocity
coupling for our sensors rather than position or acceler-
ation coupling (or some combination of the three) as a
simplification based on the hypothesis that velocity is the
most accurate form of information for postural control
(Jeka et al. 2004; Kiemel et al. 2002).
We considered three postural models that based their
response to the motion of the visual scene on the esti-
mates of progressively more sophisticated internal models
of the visual environment. In addition, we considered one
nonestimating postural model whose response to visual
stimulation was not based on estimates of the properties
of visual environment. We expressed each of the four sys-
tems as a stochastic differential equation.
Each model described both the computations of the
neural controller as well as the dynamics of the body. In
other words, each model’s stochastic differential equation
contained some components describing the body dynam-
ics as well as some components describing the dynam-
ics of the neural computations. An important constraint
on a well-stated postural model is that the computations
performed by the neural controller depend on the body
dynamics and the noise in the system only through a func-
tion that defines the sensory measurement.
To make this statement precise, the state of the body
was described by a vector x whose dynamics were given
by the following stochastic differential equation in x, the
ankle torque applied by the controller u, and the process
noise ξ
p
:
˙x(t)=F x(t) +Gu(t) +ξ
p
(t) . (1)
The components of x were the angular position and angu-
lar velocity of the body. The sensory measurement z was
given in terms of x, the visual scene position x
v
, and mea-
surement noise ξ
m
by the equation:
z(t) =H x(t) +E ˙x
v
(t) +ξ
m
(t) . (2)
The first component of z represented the visual measure-
ment, the second component the nonvisual measurement.
We assumed ξ
p
(t) and ξ
m
(t) were independent Gaussian
white noise processes with spectral densities Q and R,
respectively. For each of the four postural models we added
components (a neural controller) to (1) that determined
the dynamics of u(t). To satisfy the constraint that the
postural model be well stated, we only allowed the neu-
ral controller components to depend upon the quantities
in (1) through the measurement z(t). More precisely, the
neural controller had access to both z(t) and coefficients
of the matrices F , G, Q, H , E, and R, but it did not have
direct access to the processes x(t), ξ
p
(t), x
v
(t), and ξ
m
(t).
The matrices in (1) and (2) were given by
F =
01
γ 0
,G=
0
1
,Q=
00
0 σ
2
,H=
01
01
,
E =
−1
0
,R=
σ
2
21
0
0 σ
2
22
, (3)
where γ , σ , σ
21
, and σ
22
are parameters of the model.
The parameter γ is related to the torsional stiffness of
the system. Specifically, γ is the torsional stiffness divided
150
by the moment of inertia, which is the proportionality
factor that relates body angle to the angular acceleration
caused by that angle. The parameter γ can be expressed
as
γ =
mgh
J
−γ
p
,
where m is the mass of the body, g is the acceleration due
to gravity, h is the height of the center of mass above the
ankles, J is the moment of inertia of the body about the
ankles, and γ
p
is the passive component to γ . The first
term of this expression can be calculated for each sub-
ject from anthropometric quantities that can be measured
precisely. Values of this term used by postural models in
the literature have included 9.83 s
−2
(Peterka 2000) and
7.6 s
−2
(Kiemel et al. 2002). On the other hand, there is
no agreement about the value of the passive component
γ
p
. It is clear that γ
p
is positive (which bounds γ from
above by mgh/J ), but a lower bound on γ is less clear.
Indeed Winter et al. (1998) proposed that γ
p
was greater
than mgh/J , which would make γ negative and the system
marginally stable without active control. While more re-
cent authors have argued against this proposition (Loram
and Lakie 2002; Morasso and Schieppati 1999; Morasso
and Sanguineti 2002; Peterka 2002), estimates of γ
p
have
ranged from 91% of mgh/J (Loram and Lakie 2002) to
10% of mgh/J (Peterka 2002). As explained below we con-
strained our parameter space so that γ lay in the interval
(0, 9)s
−2
, and we considered a value of γ less than 0.5 s
−2
to be unreasonably small.
The other parameters that appear in this plant model,
σ , σ
21
, and σ
22
, represent noise levels: respectively, the
process, visual measurement, and nonvisual measurement
noise levels.
Note that in (1), u(t) can only be interpreted as the
applied ankle torque if the (2, 1) element of G, G
21
=1/J .
However, as will become apparent below, u(t) can be
rescaled by changing the gains on the proportional deriv-
ative controller, and these gains are also parameters of the
model. To avoid introducing J as a superfluous parameter,
we chose G
21
=1 in (3) and reinterpreted u(t ) as the scaled
applied ankle torque.
We coupled this plant model to four different neu-
ral controllers, each utilizing a different adaptive scheme.
Each postural model depended upon the four parameters
defining the plant (γ , σ , σ
21
, and σ
22
) as well as two gains
associated with the proportional-derivative control. To
compare models, we varied these six parameters to achieve
the best fit to experimental data. However, goodness of fit
at a particular set of parameter values was determined
for all experimental conditions simultaneously. Thus we
strove to find a model, together with just one set of values
for its parameters, that tested well under many different
experimental conditions, without the need for the modeler
to adjust parameters between conditions. In testing a suc-
cessful model only the visual scene position x
v
(t) would be
under experimental control. In other words, x
v
(t) would be
the only input to the system. As we have defined the prob-
lem, the only output to be compared with experimental
data would be the position of the center of mass, x
1
(t). The
results in this paper were derived from sinusoidal inputs
at a range of frequencies and amplitudes. However, in this
scenario, stochastic inputs and inputs depending upon the
output (i.e., sway referencing) would also be possible.
2.2 Estimating models
The three estimating schemes included progressively more
sophisticated internal models of the visual environment.
Each of these internal models depended upon parame-
ters (referred to below as adaptive parameters) that were
allowed to take on a large but finite number of possible val-
ues. Each possible set of adaptive parameter values gave
a different internal model. The scheme performed state
estimation with all of these models separately and simul-
taneously. In addition, it computed the likelihood that
each set of adaptive parameter values correctly described
the visual stimulus. The response was based on an aver-
age of the state estimates, weighted by their corresponding
likelihoods. Thus the modeled nervous system adapted to
changes in environmental conditions by recalculating the
likelihood-based weights associated with its various inter-
nal models.
Each estimating model implicitly assumed that the
motion of the environment could be described by an nth-
order linear stochastic process of the form
˙x
env
(t) =Ax
env
(t) +ξ
env
(t) ,
where ξ
env
is a Gaussian white noise process with spectral
density matrix D. Each scheme assumed that the vector
x
env
had n components, which it interpreted as follows: the
first represented the velocity of the visual scene, ˙x
v
, and
each subsequent component represented its next deriva-
tive. Each scheme expressed A and D in terms of adaptive
parameters (d
1
, d
2
, a
1
, and/or a
2
). Adaptive parameters
are those that the nervous system assumes are unknown
or subject to change.
The three internal models of the visual environment
differed in how they represented A and D. We called the
vector of adaptive parameters θ. For the first estimating
model (the random walk model) we assumed
A =
[
0
]
,D=
[
d
1
]
2
.
Thus n =1 and θ =d
1
. For the second estimating model
(the first-order model) we assumed
A =
[
−a
1
]
,D=
[
d
1
]
2
.
Thus n =1 and θ =(d
1
,a
1
)
T
. For the last estimating model
(the second-order model) we assumed
A =
01
−a
1
−a
2
,D=
d
1
d
2
d
1
d
2
.
Thus n =2 and θ =(d
1
,d
2
,a
1
,a
2
)
T
. The form we chose for
the internal model of the second-order scheme expresses
in a canonical way a general linear second-order stochastic
differential equation with one observed variable.
To determine the adaptive parameters θ we used a
scheme based on likelihood rather than the Myers and
Tapley scheme used by Van der Kooij et al. (2001). We
made this change because the Myers and Tapley scheme
can only be applied to discrete systems – it does not have
151
a useful continuous time limit. Implementing this likeli-
hood scheme in the dynamic model involved discretizing
the adaptive parameter space. Thus we allowed θ to only
take on m discrete values:
θ ∈{θ
1
,θ
2
,...,θ
m
}.
For each possible value θ
k
of the estimated parameters the
modeled nervous system conducted state estimation under
the assumption that the true value of θ was θ
k
. In our
implementation this process involved m separate Kalman
filters. We consider the limit as m becomes large and the
discretization becomes dense as the biologically relevant
one. While in this limit the dynamic model is intractable
to simulate on a serial computer, these computations may
be feasible to the human brain given its massively parallel
architecture.
Each of the m Kalman filters had a different internal
model of the environment as well as the same internal
model of the body and sensors. In our implementation,
this internal model of the body and sensors was a cor-
rect model in the sense that it used the known matrices F ,
G, Q, H , E, and R from (1) and (2). On the other hand,
the internal models of the environment (each involving
the matrices A(θ
k
) and D(θ
k
) for some θ
k
) attempted to
model the unknown stimulus x
v
(t) and thus could not be
expected to be correct.
To write down the equations for the Kalman filter corre-
sponding to the estimate θ
k
, we combine the body and sen-
sor internal model with the environment internal model.
Thus the state variables that the nervous system estimates
include both the body’s state, x, and the environment’s
state, x
env
. We write x
∗
=(x
T
,x
T
env
)
T
. The nervous system’s
internal model of the body, sensors, and environment was
given by:
˙x
∗
(t) =F
∗
(θ
k
)x
∗
(t) +G
∗
u(t) +ξ
∗
p
(t) ,
z(t) =H
∗
x
∗
(t) +ξ
∗
m
(t) ,
where ξ
∗
p
and ξ
∗
m
are independent Gaussian white noise
processes with spectral densities Q
∗
(θ
k
) and R, respec-
tively. These matrices were given by
F
∗
(θ
k
) =
F 0
2×n
0
n×2
A(θ
k
)
,G
∗
=
G
0
n×1
,
Q
∗
(θ
k
) =
Q 0
2×n
0
n×2
D(θ
k
)
,H
∗
=
HE0
2×(n−1)
, (4)
where the notation 0
i
1
×i
2
indicates a i
1
×i
2
zero matrix.
Using this internal model, the Kalman filter associated
with the kth possible value of the adaptive estimate, θ
k
,
maintained an estimate ˆx
∗
(t, θ
k
) of x
∗
(t). The equations
describing the corresponding computations were given by
˙
ˆx
∗
(t, θ
k
) = F
∗
(θ
k
) ˆx
∗
(t, θ
k
) +G
∗
u(t)
+K(θ
k
)
[
z(t) −H
∗
ˆx
∗
(t, θ
k
)
]
, (5)
where
K(θ
k
) =P(θ
k
)H
∗T
R
−1
(6)
and P(θ
k
) is the unique symmetric positive definite solu-
tion to the Ricatti equation
F
∗
(θ
k
)P (θ
k
) +P(θ
k
)F
∗
(θ
k
)
T
+Q
∗
(θ
k
)
−P(θ
k
)H
∗T
R
−1
H
∗
P(θ
k
) =0 . (7)
For a derivation of the Kalman filter equations see Bryson
and Ho (1975). Bryson and Ho (1975) noticed that the
matrices P(θ
k
) and K(θ
k
) did not depend on any dy-
namic quantities. Our method of solving (7) required
that D be positive definite. To enforce this requirement,
we restricted our parameter space to the region where
|d
j
|> 5 ×10
−7
deg s
−3/2
for each j .
In addition to maintaining state estimates for each
θ
k
, the modeled nervous system also calculated log-like-
lihoods q(t,θ
k
) of the observed measurements up to time
t given that the internal model represented by θ
k
correctly
modeled the environment. The equation describing this
calculation was given by
˙q(t,θ
k
) =ˆx
∗
(t, θ
k
)
T
H
∗T
R
−1
z(t) −
1
2
H
∗
ˆx
∗
(t, θ
k
)
−q(t,θ
k
). (8)
For a derivation of this equation [without the term −q
(t, θ
k
)], see Balakrishnan (1973, p. 195). We added the term
−q(t,θ
k
) as a heuristic to make the system forget past
information. Such forgetting was desirable under chang-
ing environmental conditions. We called the parameter
the rate constant.
Given the state estimates and the likelihoods, how did
the neural controller decide what ankle torque to apply? It
computed u(t) in two steps. First it computed an average
state estimate ˆx
av
from the state estimates of all the Kalman
filters. This average was weighted by the likelihoods
ˆx
av
(t) =
m
k=1
exp
(
q(t,θ
k
)
)
ˆx
∗
(t, θ
k
)
m
k=1
exp
(
q(t,θ
k
)
)
.
Exponentials were used because q(t,θ
k
) represented log-
likelihood. The second step based the proportional deriv-
ative control on ˆx
av
:
u(t) =−C
∗
ˆx
av
, where C
∗
=
[
c
1
c
2
0
1×n
]
. (9)
Here c
1
and c
2
are, respectively, the proportional and deriv-
ative gains, which together with γ , σ , σ
21
, and σ
22
, intro-
duced above, comprised the six parameters we varied with
the models.
In summary, the stochastic differential equation that
defined the estimating models was given by (1), (5), and
(8), where (5) and (8) were repeated m times, once for each
θ
k
. Thus the equation had dimension 2 +m(3 +n) where
n is the order of the internal model and m is the number
of possible discrete values for the adaptive parameter θ.
2.3 Nonestimating model
The three estimating schemes presented above all involved
a neural controller that included internal models of the
visual environment. These schemes accomplished adap-
tation to changing environmental conditions by recalcu-
lating the likelihood-based weights associated with their
various internal models. We now present a postural model
that lacks internal models of the environment. We called
this scheme nonestimating because it did not estimate the
dynamics of the environmental motion. Instead it adjusted
152
sensory weights to minimize the mean square of the con-
trol signal without making any specific assumptions about
the dynamic properties of the environmental motion.
A second important difference between the estimating
and nonestimating schemes was the region of parameter
space that they searched for an improved response. The
estimating schemes conducted global searches in the sense
that they simultaneously compared likelihoods over a wide
range of possible adaptive parameter values. In contrast,
the nonestimating model made a local search: it consid-
ered only the effects of small adaptive parameter changes.
We constructed the nonestimating model by first spec-
ifying a linear model with one parameter, then allowing
the parameter to slowly vary. Using the same notation as
before
˙x(t) = F x(t) +Gu(t) +ξ
p
(t) , (10)
˙
ˆx(t) = (F −GC) ˆx(t)+K(θ)[z(t) −H ˆx(t)] , (11)
where θ is a scalar parameter. As before, ξ
p
(t) and ξ
m
(t)
are Gaussian white noise processes with spectral densities
Q and R, respectively, and the measurement vector, z(t),
is given by
z(t) =H x(t) +E ˙x
v
(t) +ξ
m
(t), (12)
where x
v
is the position of the visual scene. The matrices
F , G, Q, H , E, and R are defined in (3). And finally the
control signal u and the matrix C aregivenby
u(t) =−C ˆx(t), where C =
[
c
1
c
2
]
. (13)
Note that (11) has the form of a Kalman filter. Many
different choices of the matrix-valued function K(θ) were
possible as long as the parameterization allowed the rela-
tive weighting of the two measurements, z
1
and z
2
, to vary
over a wide range. Our choice of K(θ)was the steady-state
Kalman gain matrix for a model of the plant that was
correct except in two respects. First, the plant model was
incorrect in the sense that it assumed the environment to
be stationary: from this assumption came the name non-
estimating. The second incorrect assumption was meant
to mitigate the first incorrect assumption. In particular,
the noise level associated with the visual modality was
assumed to be θ rather than its veridical value σ
21
. The
idea behind this incorrect assumption was to allow the
nervous system to choose an increasingly large value of θ
as the amplitude of visual stimulation increased. A larger
value of θ implied a plant model that assumed vision to be
more noisy – hence one that downweighted vision more.
The plant model was correct in every other respect. In par-
ticular, the matrices F , G, Q, H , and all other entries of
R (besides the upper-left element, σ
21
) were known and
used.
To make these statements precise, K(θ) could be found
by solving the following Ricatti equation for P :
0 =FP +PF
T
+Q −PH
T
˜
R(θ)
−1
HP ,
where
˜
R(θ)=
θ
2
0
0 σ
2
22
,
then using P(θ) to find K(θ) as follows:
K(θ)=P(θ)H
T
˜
R(θ)
−1
.
In practice it was simpler to use the following closed-form
expression for K(θ):
K(θ)=
2
4γ +σ
2
(1/θ
2
+1/σ
2
22
)
σ
2
22
θ
2
+σ
2
22
θ
2
θ
2
+σ
2
22
.
(14)
This expression makes apparent the reweighting of the two
measurements. When the rightmost matrix factor multi-
plies z, the result is an average of the two measurements
z
1
and z
2
, where each is weighted by the square of the
assumed noise level corresponding to the other. The left-
most factor then scales this average separately for the po-
sition and velocity components.
One interpretation of our choice of K(θ) is that the
nervous system assumes the visual scene to be station-
ary with all sensory conflicts arising from measurement
noise and estimates an appropriate noise level θ to describe
this noise. This interpretation is misleading because the
scheme does not change θ to improve its estimation of
the noise level, whose true value is σ
21
. Instead it changes
θ to minimize the mean square of the control signal. If
the visual scene is indeed stationary, θ does not in general
converge to σ
21
.
To derive the equation describing the dynamics of θ ,we
considered how x and ˆx in the linear model [given by (10)–
(13)] would change if the parameter θ were changed by a
small amount. Differentiating (10) and (11) with respect
to θ and using (12) and (13), we obtained
˙x
θ
(t) = Fx
θ
(t) −GC ˆx
θ
(t), (15)
˙
ˆx
θ
(t) = (F −GC) ˆx
θ
(t) +K
θ
[z(t) −H ˆx(t)]
+KH[ x
θ
(t) −ˆx
θ
(t)], (16)
where the subscript θ denotes the partial derivative with
respect to θ. The matrix K
θ
can be computed by direct
calculation from (14). Note that the calculation of x
θ
(t)
and ˆx
θ
(t) from (15) and (16) involved only the measure-
ment z(t) and other quantities accessible to the nervous
system. Thus even though x(t) was not knowable, the neu-
ral controller could integrate the equation for its partial
derivative with respect to θ, x
θ
(t).
In terms of ˆx
θ
we could compute
∂
∂θ
u(t)
2
=2u(t)u
θ
(t) =2C ˆx(t)C ˆx
θ
(t) . (17)
In the previous calculations we assumed that θ was
fixed. We now let θ slowly vary with the goal of mini-
mizing u
2
, where · denotes time average. Equation (17)
suggests the gradient descent rule:
˙
θ(t)=−C ˆx(t)C ˆx
θ
(t) , (18)
where is a small positive parameter that we called the
adaptation gain. Simulations showed that (18) did, in
fact, tend to minimize u
2
(Fig. 2). (A parameter also
appeared in the estimating models and was called the rate
153
constant. Its interpretation is different for the nonestimat-
ing scheme. Whereas for the estimating schemes the rate
constant determined the rate of forgetting of past sen-
sory information, for the nonestimating scheme the adap-
tation gain was related to the rate at which the adaptive
parameter changed.)
In summary, the stochastic differential equation defin-
ing the nonestimating model was given by (10), (11), (15),
(16), and (18). Thus the dimension of the equation was 9.
The parameters we varied with the nonestimating model
were the same as those we varied with the estimating
schemes, that is, we varied γ , σ , σ
21
, σ
22
, c
1
, and c
2
, which
defined the matrices F , Q, R, and C in (10)–(13).
2.4 Computing transfer functions
The response to sinusoidal stimuli could be character-
ized by the transfer function from x
v
to x
1
. We computed
this transfer function, T
x
v
x
1
, using two methods. Our first
method was the standard one involving the discrete Fou-
rier transform of a simulated model trajectory. Unfortu-
nately, the time required to simulate the model made an
extensive search of the parameter space, using this method
to evaluate transfer functions, computationally prohibi-
tive. Our second method involved applying an approxi-
mate formula for the transfer function, which we derived
for each adaptive scheme. This approximate formula was
defined in terms of the parameters of the model and the
amplitude and frequency of the sinusoidal stimulus. The
approximation became accurate in a limit that we state
below.
To compute the transfer function, T
x
v
x
1
, from simulated
trajectories, we calculated the discrete Fourier transforms
X
v
(f ) and X
1
(f ) of, respectively, x
v
(t) and x
1
(t), then
evaluated both at the stimulus frequency f
s
and divided:
T
x
v
x
1
(f
s
) =
X
1
(f
s
)
X
v
(f
s
)
.
For each value of the parameters tested and for each
experimental condition, we repeated this calculation on
ten trajectories each generated with a different seed for
the pseudorandom noise. To generate these trajectories,
we simulated the model for 2000 s using Euler’s method
with a time step of 0.005 s, then discarded the first 1000 s
to lessen the effect of transients. We computed mean gain
and phase as, respectively, the absolute value and angle
in degrees of the mean of the ten transfer functions in the
complex plane. We obtained a 95% confidence ellipse in the
complex plane for the transfer function mean by assuming
a bivariate normal distribution. We obtained confidence
intervals for gain and phase from bounds on the magni-
tude and angle of the transfer function derived from the
95% confidence ellipse.
This method for estimating the transfer function
required long computer calculations. For the random walk
model, with only one adaptive parameter, calculating a
confidence ellipse as described above for one set of param-
eter values and one experimental condition took approx-
imately 1.5 h. The time required for the other estimating
models increased exponentially with the number of adap-
tive parameters. Even for the nonestimating model this
method required more than 9 min of computer time. To
allow a thorough exploration of the parameter space, we
derived a formula that approximated the transfer function
of each model. The approximation became accurate in the
limit as (the rate constant or adaptation gain) tended to
0, the transient had decayed, and for the estimating mod-
els the discretization was dense in the parameter space.
Applying the transfer function formula required less than
1 s of computer time for each model, and simulations ver-
ified its accuracy (Figs. 1 and 2).
Three properties common to all four models made these
transfer function formulas possible. First, the adaptive
parameters tended to approach a steady-state value under
the condition that (the rate constant or adaptation gain)
was small. Note that, for the estimating models, what we
mean when we say that θ converges to a steady state or
asymptotic value is that, as time progresses, only one θ
k
in
the discretization has a nonnegligible likelihood weight for
the computation of ˆx
av
(Appendix B). Second, the asymp-
totic value of the vector of adaptive parameters coincided
with the minimum or maximum of a function that could
be explicitly written down and computed in terms of the
parameters of the model and the frequency and ampli-
tude of the stimulus. Finally, once the vector of adaptive
parameters reached its asymptotic value, the postural sys-
tem was linear. Thus from the asymptotic value we could
compute the approximate transfer function of the model
with a straightforward computation.
Unlike computing the transfer function from simula-
tions, there was no pseudorandom element involved in
applying the formula. Thus the results of the transfer
function formula did not involve confidence ellipses. How-
ever, in applying the formula, an optimization step was
involved. In consequence, multiple local optima could
make the formula fail, depending on the initial condition
of the optimization. We mitigated this problem by using
multiple initial conditions.
For the estimating models, the asymptotic value of the
vector of adaptive parameters approximately coincided
with the value
ˆ
θ that maximized the following functional:
−
1
2
Tr
R
−1
H
∗
d
e
(·,
ˆ
θ)d
e
(·,
ˆ
θ)
T
H
∗T
, where d
e
(t,
ˆ
θ)
=x
∗
(t) −ˆx
∗
(t,
ˆ
θ). (19)
The angled brackets denote time average. This function
could be computed directly from the parameters of the
model and the amplitude and frequency of the sinusoi-
dal stimulus. See Appendix B for the derivation of this
formula and Appendix C for an explanation of how to
calculate it. For later reference we call the functional of
(19) the likelihood correspondence function . As explained
in Appendix B, the estimating models increasingly select
the internal model whose parameters maximize this func-
tional. In this sense these systems estimate θ.
Similarly for the nonestimating model, the asymptotic
value of the adaptive parameter approximately coincided
with the value θ
†
that minimized the functional
u
2
(·,θ
†
)
.
154
0.2 0.5 1 2 5 10
0
0.2
0.4
0.6
0.8
1
gain
0.2 0.5 1 2 5 10
0
180
phase (deg)
amplitude (deg)
Fig. 1. Comparison of gain and phase calculated
by the transfer function formula for the random
walk model to estimates of these quantities
derived from simulations. The response seen is to
0.2 Hz sinusoidal stimulation as a function of
amplitude. The curves represent the calculations
of the transfer function formula. Gain and phase
are, respectively, the absolute value and angle in
degrees of the complex valued transfer function.
Each error bar represents a 95% confidence
interval for gain and phase derived from
simulations (Sect. 2.4). We used standard
parameter values (Sect. 2.6), and for the
simulations the rate constant was given by
=0.002 s
−1
, and the adaptive parameter space
was discretized as follows:
d
1
∈{0.01, 0.05, 0.10, 0.15,...,2.0}
0.2 0.5 1 2 5 10
0
0.2
0.4
0.6
0.8
1
gain
0.2 0.5 1 2 5 10
0
180
phase (deg)
amplitude (deg)
Fig. 2. Comparison of gain and phase calculated
by the transfer function formula for the
nonestimating model with estimates of these
quantities derived from simulations. The response
seen is to 0.2-Hz sinusoidal stimulation as a
function of amplitude. The curves represent the
calculations of the transfer function formula.
Gain and phase are, respectively, the absolute
value and angle in degrees of the complex valued
transfer function. Each error bar represents a 95%
confidence interval for gain and phase derived
from simulations (Sect. 2.4). We used standard
parameter values (Sect. 2.6), and for the
simulations the adaptation gain was given by
=0.002 s
2
Again this functional could be computed directly from the
parameters of the model and the amplitude and frequency
of the sinusoidal stimulus. See Appendix D for an explana-
tion of how to calculate this functional. Note that we have
used a different notation for the asymptotic value of the
adaptive parameter for the nonestimating model because
this asymptotic value could not be considered an estimate,
unlike for the estimating models.
In each case, once the asymptotic value of the vector of
adaptive parameters was found, the model was linear and
its transfer function from x
v
to x
1
could be calculated by
the method described in Appendix A. Figure 1 compares
confidence intervals for the gain and phase of the random
walk model derived from simulations with the approxima-
tions of the transfer function formula. Figure 2 plots the
corresponding comparison for the non-estimating model.
The slight discrepancy between the formula and simula-
tions, especially apparent at higher frequencies, is mostly
due to the fact that changes to θ happen slowly when θ
is large (that is, when the modeled nervous system deems
vision unreliable – see Discussion). The dynamics of θ pro-
duce very long transients for large-amplitude stimuli. For
our parameters and stimulus amplitudes greater than 0.5
◦
,
these effects are still significant after 1000 s.
2.5 Search for appropriate response to sinusoidal visual
stimulation
We searched the parameter space of each model for
the appropriate response to sinusoidal visual stimulation
155
across different stimulus frequencies and amplitudes. We
first attempted a-trial-and-error approach, using different
parameters and making many manipulations to the struc-
ture of the models. These manipulations included many
features not previously described such as sensory dynam-
ics, time delays, position and acceleration coupling, differ-
ent adaptation times, and state-dependent noise. Only
when the unsystematic search for realistic gain and phase
behavior failed for each model did we attempt a system-
atic search. For the systematic search we used the models
as described in Sect. 2.1 (i.e., without the manipulations
just listed). To allow a thorough exploration of the param-
eter space, we used the transfer function formulas. Thus
we assumed =0, the transient was given enough time
to decay, and for the estimating models that the discret-
ization was dense in the adaptive parameter space. After
restricting each model’s (nonadaptive) parameter space,
we defined a cost function to quantify goodness-of-fit to
experimental data. The domain of this cost function was
the tested model’s restricted parameter space. The cost was
defined in terms of the model’s transfer function at differ-
ent frequencies and different amplitudes of stimulation.
We emphasize that all of the transfer functions in a single
evaluation of the cost function occur at the same value of
the restricted model parameters. Finally, we conducted a
systematic search for parameters that optimized our cost
function with an algorithm involving simulated annealing.
To restrict each model’s parameter space, we assumed
that γ was between 0 and 9 s
−2
. We assumed c
1
=γ +
ω
2
0
and c
2
=α, where ω
0
and α were two of the elements
of stochastic structure reported in Jeka et al. (2004) for
the fixed surface condition (Table 1). This identification is
explained in Kiemel et al. (2002). This restriction left four
parameters: γ , σ , σ
21
, and σ
22
. We then reduced the dimen-
sion of the parameter space one further by rescaling the
noise terms so that sway standard deviation σ
COM
equaled
the mean sway standard deviation reported in Jeka et al.
(2004) for normal subjects with a fixed surface: 0.334
◦
.
This rescaling involved initially assuming σ =1
◦
s
−3/2
, cal-
culating σ
COM
, then dividing each of σ , σ
21
, and σ
22
by
σ
COM
/0.334
◦
.
For the systematic searches we measured the gain and
phase at each of two amplitudes (0.2
◦
and 5
◦
) and at each
Table 1. The experimental stochastic structure of quiet stance (means
and standard errors for the fixed surface condition) reported in Jeka
et al. (2004). Also shown is the models’ stochastic structure in quiet
stance at the standard parameter values. (The stochastic structure of
all four models in quiet stance at standard parameter values agree to
three significant digits.)
Measure Mean SE Model
β(s
−1
) 0.081 0.028 0.0518
α(s
−1
) 2.188 0.437 2.19
ω
0
(s
−1
) 2.139 0.187 2.17
σ
COM
(
◦
) 0.334 0.038 0.334
κ
r
/κ
tot
0.899 0.031 0.889
2|κ
c
|/κ
tot
0.136 0.040 0.145
of three frequencies (0.1, 0.2, and 0.5 Hz). This range of
amplitudes and set of frequencies coincided with those
studied experimentally in Peterka and Benolken (1995).
We imposed the following criteria on the model: phase
should not change between the two amplitudes at each
frequency, gain should drop by at least 85% between the
two amplitudes at each frequency, and, finally, the gain at
low amplitude (0.2
◦
) should be between 0.25 and 1.5.
We made these criteria precise with a cost function that
depended upon nine measures: three related to changes in
phase, three related to gains at low amplitude, and three
related to changes in gain. We defined these measures, m
p
0.1
,
m
p
0.2
, m
p
0.5
, m
g
0.1
, m
g
0.2
, m
g
0.5
, m
d
0.1
, m
d
0.2
, m
d
0.5
,by
m
p
f
=
|
p
f
|
64
◦
,
m
g
f
=
0.25−l
f
0.25
if l
f
< 0.25
0if0.25 ≤l
f
≤1.5
l
f
−1.5
1.5
if l
f
> 1.5
,
m
d
f
=
g
f
−0.85 l
f
0.85 l
f
if g
f
≥−0.85 l
f
0ifg
f
< −0.85 l
f
≤0
,
where the subscript f denotes frequency (f ∈{0.1, 0.2,
0.5}),
g
f
and p
f
are the model’s changes in, respec-
tively, gain and phase (in degrees) between the two tested
amplitudes (0.2
◦
and 5
◦
) at frequency f , and l
f
is the gain
at low amplitude (0.2
◦
) at frequency f .
We defined the cost function to be the maximum of the
nine measures times 100. A cost of zero meant that the
model satisfied all our criteria, at that particular value
of parameters. The cost could not be less than zero.
A cost greater than 100 meant that the phase changed
by more than 64
◦
[twice the largest standard error for
phase reported by Peterka and Benolken (1995)], the low-
amplitude gain was greater than three, or the gain in-
creased rather than decreased over the tested interval. The
cost also approached 100 when the low-amplitude gain
approached zero. Thus [0, 100] was a typical range for the
cost function.
We defined the cost function as a maximum of the mea-
sures to insure that the optimization sought to reduce only
on the currently largest measure(s). This property insured
that at the optimal parameter values the maximal mea-
sures could not be simultaneously reduced at the expense
of increasing any of the lesser measures.
We performed the optimization with a Nelder–Mead
simplex search algorithm combined with a Metropolis
(simulated annealing) procedure whereby uphill steps were
accepted with a probability that decreased to zero as the
optimization proceeded (Press et al. 1992). We repeated
the search 50 times starting from parameter values ran-
domly chosen over a wide range.
2.6 Standard parameter values
The need to carefully tune parameters to make a model
perform realistically (in this case with constant phase
across amplitudes) would raise our suspicions if there were
no reason to believe that the parameters of the human
156
postural control system would be so tuned. To compare
how well each model performed without parameter tun-
ing we defined standard parameter values for each model
to be relevant to the behavior of the human postural
control system but without reference to the response to
sinusoidal stimuli. Specifically, we defined the standard
parameter values to be those that best reproduced the sto-
chastic structure of experimental sway in quiet stance. The
stochastic structure is a characterization of the dynam-
ical properties of a stochastic process. In particular, a
stationary linear process is completely characterized by
the autocovariance function of its trajectories. For a linear
system, this autocovariance function can, in turn, be com-
pletely described by 2n numbers, where n is the order of
the system. Since these numbers completely characterize
the dynamical properties of the linear stochastic process,
they comprise its stochastic structure.
To simplify the analysis of the models’ stochastic struc-
ture, we used their slow-adaptation linear approximation
in the quiet-stance condition after the decay of the tran-
sient. For a given value of the nonadaptive parameters, all
four models had approximately the same stochastic struc-
ture in quiet stance.
We used six measures from Jeka et al. (2004) to com-
pare the stochastic structure of the models to that of the
experimental data. These measures are defined in terms of
the autocovariance function of the sway trajectories:
E[ x
1
(t)x
1
(t +τ)] =κ
1
e
λ
1
|τ |
+···+κ
p
e
λ
p
|τ |
with terms on the right-hand side arranged so that
|κ
1
e
λ
1
h
|≥···≥|κ
1
e
λ
p
h
|, where h =0.1 s is the time step
used to analyze the experimental data. The parameters
λ
1
,... ,λ
p
are the eigenvalues of the system. We denoted
the first real-valued eigenvalue by λ
r
and its correspond-
ing coefficient by κ
r
, and we denoted the first pair of
complex-conjugate eigenvalues by λ
c
and
¯
λ
c
and their
corresponding coefficients by κ
c
and ¯κ
c
. Then, κ
r
e
λ
r
|τ |
is
a first-order decay component of the autocovariance func-
tion, and κ
c
e
λ
c
|τ |
+¯κ
c
e
¯
λ
c
|τ |
is a damped-oscillatory compo-
nent of the autocovariance function. The other terms of
the autocovariance function are typically small for exper-
imental trajectories (Kiemel et al. 2002). For the mod-
els, we required that |κ
i
e
λ
i
h
|≤0.01κ
tot
for these remaining
terms, where κ
tot
=κ
1
+···+κ
p
is the sway variance. The
six measures of Jeka et al. (2004) are (i) the slow-decay rate
β =−λ
r
, (ii) the damping α =−(λ
c
+
¯
λ
c
), (iii) the eigenfre-
quency ω
0
=
λ
c
¯
λ
c
, (iv) the center-of-mass standard devi-
ation σ
COM
=
√
κ
tot
, (v) the slow-decay fraction κ
r
/κ
tot
,
and (vi) the damped-oscillatory fraction 2|κ
c
|/κ
tot
.
The standard parameter values for a model were chosen
as those that minimized the cost function
C =
6
i=1
m
i
−m
∗
i
δ
i
2
,
subject to the constraint that |κ
i
e
λ
i
h
|≤0.01κ
tot
for λ
i
∈
{λ
r
,λ
c
,
¯
λ
c
}, where m
∗
1
,... ,m
∗
6
and δ
1
,... ,δ
6
are the exper-
imental means and standard errors, respectively, of the
measures as given in Table 1 and where m
1
,... ,m
6
are the
measures for the model. The measures for the model were
computed as described in Kiemel et al. (2002).
All four models had the same parameters: γ , c
1
, c
2
, σ ,
σ
21
, and σ
22
. However, the measurement of velocity and
the measurement of velocity relative to the visual scene
were indistinguishable in quiet stance. Thus the stochastic
structure did not constrain the relative values of σ
21
and
σ
22
. To make such a constraint we arbitrarily set σ
21
=σ
22
.
The stochastic structure of all the estimating models
were approximately equal in quiet stance. This implied that
standard parameters for all the estimating models were
approximately equal. These parameter values were given
by γ =0.19 s
−2
, c
1
=4.9s
−2
, c
2
=2.2s
−1
, σ =0.25
◦
s
−3/2
,
σ
21
=σ
22
=0.095
◦
s
−1/2
. Each standard parameter for the
estimating model differed from its corresponding standard
parameter for the nonestimating model by less than 2%.
Thus the standard parameters for all models were approxi-
mately equal. At these parameter values all measures com-
prising the stochastic structure of the model were within
the standard errors given in Jeka et al. (2004) except the
slow-decay rate β. At the standard parameter values β
was about 36% too small. This discrepancy, as well as the
unrealistically low value of γ , may reflect the fact that
no computation noise was present in our models (Kiemel
et al. 2002).
3 Results
All of the results presented in this section were based on
application of the approximate transfer function formu-
las rather than on simulations of corresponding stochastic
differential equations.
3.1 Random walk model
After considerable effort we concluded that the postural
control model with the random walk internal model of the
visual environment did not behave like the human postural
control system in its response to sinusoidal stimuli. Fig-
ure 3 shows the gain and phase as a function of stimulus
amplitude for three different stimulus frequencies at the
standard parameter values. The desired substantial drops
in gain came with undesired substantial rises in phase.
Using the approximate transfer function formula we found
that over the interval [0.2, 5] in amplitude of stimulation
the maximum change in phase at the three frequencies
tested was 34
◦
at 0.1 Hz. However, at this frequency and
over this interval the gain only dropped by 0.13. Greater
changes in both gain and phase occurred at lower ampli-
tudes. Specifically, the phase increased by 135
◦
and the
gain dropped by 0.65 at 0.1 Hz over the interval [0.1, 5]
in stimulus amplitude. Moreover, the model behaved very
strangely at low amplitudes. Both gain and phase were
initially constant while the estimated parameter d
1
lay at
its minimum allowed value: 5 ×10
−7
deg s
−3/2
. At a criti-
cal amplitude, which depended on stimulus frequency, all
three measures jumped discontinuously.
Changes in the likelihood correspondence function as
the amplitude increased explained the first constant then
discontinuous behavior of the model. Recall that our
adaptive scheme estimated d
1
to be the global maximum
157
0.1 1.0 10.0
0
10
20
d
1
(deg s )
0.1 Hz
0.2 Hz
0.5 Hz
0.1 1.0 10.0
0
0.5
1
gain
0.1 1.0 10.0
180
0
180
phase (deg)
0.1 1.0 10.0
0
0.1
amplitude of sinusiodal stimulus (deg)
response amp.
(deg)
Fig. 3. Estimated d
1
, gain, phase, and response
amplitude for the random walk model as a
function of stimulus amplitude at three different
frequencies. Calculations were performed with
the transfer function formula. The parameters of
the model were those that best reproduced the
stochastic structure of postural sway in quiet
stance (Sect. 2.6). The response amplitude
represents the amplitude of sway at the driving
frequency; it is calculated by multiplying the
amplitude of the stimulus by the gain. Note that
the scale of the response amplitude plot is
different here than in Figs. 5 and 6
0.05
0.1
0.15 0.2
0.25
5
5
d
1
(deg s
3/2
)
value of likelihood correspondence function
0.080 deg stimulation
0.095 deg stimulation
0.110 deg stimulation
global maxima
non
global maxima
Fig. 4. Likelihood correspondence as a function of d
1
for three differ-
ent amplitudes of visual stimulation of the random walk model, each
at 0.2 Hz
of the likelihood correspondence function. Figure 4 plots
this function for three different amplitudes, all at a fre-
quency of 0.2 Hz. If the amplitude was low, its only maxi-
mum lay at the endpoint 5 ×10
−7
deg s
−3/2
. As the ampli-
tude increased a new local maximum appeared. Initially
the new maximum lay below the one at 5 ×10
−7
,butat
a higher amplitude it overtook the first maximum and
became global. At this bifurcation point the estimated
parameter, and hence the transfer function, jumped dis-
continuously.
The postural model with the random walk internal
model of the visual environment behaved similarly with
different parameter values as well as with many other
changes to the structure of the model. We found that it
was possible to make the phase nearly constant, as desired,
but only at the expense of also making the gain nearly con-
stant, as not desired. The manipulations to the structure
of the model that we tried included adding sensory dynam-
ics, coupling the sensory measurements to position and/or
acceleration instead of or in addition to velocity, short
adaptation time, time delay, and state-dependent noise.
This unsystematic search failed to yield realistic behavior.
With all these manipulations we consistently observed that
with a random walk internal model of the environment
significant drops in gain tended to accompany significant
rises in phase.
The systematic search produced results that were bet-
ter, but still not acceptable. At the best-fitting parameters
(γ =0.0022 s
−2
, σ =0.069
◦
s
−3/2
, σ
21
=0.0032
◦
s
−1/2
, σ
22
=
0.23
◦
s
−1/2
) three measures were maximal (approximately
equaling 22.4). Of these three measures, one corresponded
to the low-amplitude gain at 0.1 Hz being high (1.83), one
corresponded to the low amplitude gain at 0.5 Hz being
low (0.19), and one corresponded to the change in phase
over the amplitude interval [0.2, 5] at 0.1 Hz being large
relative to the criteria quantified by the other measures
(14.3
◦
). Thus making the phase relatively constant pro-
duced unacceptable gains.
Although this behavior may not seem especially far
from the desired behavior, we were not satisfied with these
results for two reasons. First, while the model’s behav-
ior did lie near the boundary of what we would consider
acceptable, it was the best that the model could do for all
values of its parameters. In particular, it was the best the
model could do without regard to its stochastic structure.
For all other values the model would do worse. Thus our
optimization procedure found a region in parameter space
in which the phase was relatively constant over the interval
[0.2, 5] degrees in stimulus amplitude. If the human pos-
tural control system implemented the mechanism of the
random walk model, there would be no reason to believe
158
that its parameters would be so tuned. We could see no rea-
son that the human postural control system would evolve
to make phase constant over this interval. Mechanisms
requiring careful tuning for qualitative agreement with
experiment raise our suspicions.
The second reason we were not satisfied with our results
is that they were based on a cost function that only looked
at the model’s behavior for stimulus amplitudes of 0.2
◦
and
5
◦
. The low end of this scale (corresponding to a stimulus
of about 5 mm at the eye level of an average person) was
high enough to avoid the discontinuous behavior of the
model for all frequencies tested. A response to a 2-mm
stimulus (about 0.08
◦
) at 0.2 Hz was reported in Oie et al.
(2002) without evidence of an abrupt change in the transfer
function for larger amplitudes. The random walk model’s
discontinuity occurs at a larger amplitude for a 0.2-Hz
stimulus at standard parameter values. Our failure to make
the random walk model behave in a realistic way led us to
search for a different mechanism.
3.2 First-order model
A pattern of gain and phase similar to the one seen with
the random walk internal model also appeared with the
general first-order internal model. Specifically, drops in
gain tended to coincide with rises in phase. This tendency
is shown in Fig. 5, which plots the gain and phase as a
function of amplitude at three frequencies at the standard
parameter values. The figure also shows the two adaptive
parameters, d
1
and a
1
. The constant then discontinuous
behavior of the random walk model did not appear. Indeed
the likelihood correspondence function, now a function
of two rather than one adaptive parameter, had a single
local maximum (the global maximum) for all parame-
ters tested. However, we observed that for small ampli-
tudes this global maximum lay on a very broad peak. For
small amplitudes all pairs (d
1
,a
1
) with d
1
small and a
1
had
approximately the maximum value of the likelihood cor-
respondence function. In this situation we would expect
the weights corresponding to the different Kalman filters
to take a long time to converge to their asymptotic values.
We did not investigate the consequences of this property
on the dynamics of the model.
On the other hand, restricting the likelihood correspon-
dence function to the line determined by the condition
a
1
=0 reproduced the likelihood correspondence function
for the random walk model. However, neither of the local
maxima for the random walk model were local maxima for
the general first-order model because likelihood increased
as a
1
increased for small d
1
.
The systematic search for optimal behavior produced
slightly better results for the first-order model than for
the random walk model, but these results were still not
satisfactory. At the optimal parameter values for the first-
order model (γ =5.5 ×10
−5
s
−2
, σ =6.9 ×10
−4
deg s
−3/2
,
σ
21
=0.0088
◦
s
−1/2
, σ
22
=0.25
◦
s
−1/2
), three measures were
approximately maximal (equaling about 19.6). These max-
imal measures resulted from the low-amplitude gain at
0.1 Hz being high (1.79), the low-amplitude gain at 0.5 Hz
being low (0.20), and the change in phase between low and
high amplitudes at 0.1 Hz being large relative to the criteria
quantified by the other measures (12.5
◦
). Note that the set
of measures that were maximal for the first-order model
was the same as the set of measures that were maximal for
the random walk model, indicating that both mechanisms
constrained the postural model in a similar way.
Because the first-order model did not display the dis-
continuities present in the random walk model, its low-
amplitude behavior did not differ substantially from its
behavior over the interval [0.2, 5] degrees. Thus tuning the
parameters for realistic gain and phase behavior below
0.2
◦
was more successful for the first-order model than for
the random walk model. Nevertheless, tuning was neces-
sary to give realistic behavior over this entire range, and
even with tuning the tendency for rises in phase to accom-
pany drops in gain could not be removed. As with the
random walk model, tuning produced model behavior on
the boundary of what we considered acceptable, but we
saw no reason why the parameters of the human postural
control system would be so tuned. Thus our results cast
doubt on the proposition that the human postural control
system estimates the state of visual environment with a
first-order internal model. This doubt led to a search for a
different mechanism that would not lead to a tendency for
substantial rises in phase to accompany substantial drops
in gain.
3.3 Second-order model
Could a more sophisticated internal model of the visual
environment produce more realistic behavior? We found
the answer to be a striking no: with a general second- or
higher-order linear stochastic process internal model the
system responded to any sinusoidal stimulus with a gain
that approached zero.
This curious property follows from the fact that a
second-order differential equation can correctly model
a sinusoidal stimulus. Recall that x
env
(t) is the modeled
nervous system’s representation of the state of the envi-
ronment. For the second-order model this representation
consists of the estimated velocity and estimated accelera-
tion of the visual scene. If
A =
01
−4π
2
f
2
s
0
,
then the solution of the differential equation
˙x
env
(t) =Ax
env
(t)
with the initial condition
x
env
(0) =
2πf
s
0
coincides with the true velocity and acceleration of the
visual scene [ ˙x
v
(t), ¨x
v
(t)]
T
, provided the stimulus is sinu-
soidal with amplitude and frequency f
s
(more precisely,
the true position of the visual scene is given by x
v
(t) =
sin(2πf
s
t)). This differential equation is realized as the
internal model of the environment with the second-order
model if a
1
=4π
2
f
2
s
, and a
2
=d
1
=d
2
=0.
159
0.1 1.0 10.0
0
10
20
d
1
(deg s )
0.1 Hz
0.2 Hz
0.5 Hz
0.1 1.0 10.0
0
2
4
a
1
(s
1
)
0.1 1.0 10.0
0
0.5
1
gain
0.1 1.0 10.0
180
0
180
phase (deg)
0.1 1.0 10.0
0
0.06
response amp.
(deg)
amplitude of sinusoidal stimulus (deg)
Fig. 5. Estimated parameters d
1
,anda
1
and gain,
phase, and response amplitude for the first-order
model as a function of stimulus amplitude at
three different frequencies. Calculations were
performed with the transfer function formula.
The parameters of the model were those that best
reproduced the stochastic structure of postural
sway in quiet stance (Sect. 2.6). The response
amplitude represents the amplitude of sway at the
driving frequency; it is calculated by multiplying
the amplitude of the stimulus by the gain. Note
that the scale of the response amplitude plot is
different here than in Figs. 3 and 6
Not surprisingly, the likelihood correspondence for the
second-order model approaches its maximum as
[d
1
,d
2
,a
1
,a
2
]
T
→[0, 0, 4π
2
f
2
s
, 0]
T
.
We called this point in the adaptive parameter space θ
†
.
We found that as θ approached θ
†
, the transfer function
computed by the formula tended to zero.
As θ
k
approached θ
†
, the kth Kalman filter esti-
mated the state of the environment with increasingly high
fidelity. (However, this happened only after a transient
that decayed slower as θ
k
approached θ
†
.) The modeled
nervous system then compensated for the environmen-
tal motion by subtracting the estimate from its visual
measurement. Moreover, for all amplitudes the Kalman
gain remained the same as in quiet stance. Thus the mod-
eled postural control system extracted the same stabilizing
information from the visual modality as it did when the
visual environment was stationary.
3.4 Nonestimating model
Unlike the models that estimated the state of the envi-
ronment, the nonestimating model produced realistic gain
and phase behavior according to most measures even with-
out tuning the parameters. At standard parameter values,
all but one measure was small. Figure 6 shows the gain,
phase, and estimated parameter, θ , as a function of ampli-
tude. As evident in the figure, the phase did depend on the
frequency of stimulation but did not show a strong depen-
dence on amplitude. Over the interval [0.2, 5] degrees in
stimulus amplitude, phase increased by 2.5
◦
at 0.1 Hz,
decreased by 0.63
◦
at 0.2 Hz, and decreased by 0.29
◦
at
0.5 Hz. At the same time the gain dropped by more than
85% at each of these frequencies. This behavior, relatively
constant phase together with substantial drops in gain,
was apparent even without tuning the parameters.
While the match of the nonestimating model to our
criteria was good at the typical parameters, it was not
perfect. While the gain with a stimulus amplitude of 0.2
◦
was reasonable at low stimulus frequencies (0.7 and 0.6 at
0.1 Hz and 0.2 Hz, respectively) it was only 0.05 at 0.5 Hz.
Thus the low-amplitude gains at the lower frequencies
were well within our desired range, but the low-amplitude
gain at 0.5 Hz was much too low. Nevertheless, tuning
the parameters removed this problem. At the parame-
ters that best matched our criteria found by the optimiza-
tion (approximately γ =5.7s
−2
, σ =8.2 ×10
−6
deg s
−3/2
,
σ
21
=0.29
◦
s
−1/2
, σ
22
=0.18
◦
s
−1/2
), the cost function evalu-
ated to less than the tolerance of the optimization scheme:
the phase difference over the tested amplitude interval at
all three frequencies was zero to numerical error and the
six cost measures related to gain were zero.
Based on the tendency of the estimating models to have
phase increases accompanying gain drops, and the lack of
such a tendency with the nonestimating model, we favor
the nonestimating model over the estimating models.
4 Discussion
We tested four postural models: three containing progres-
sively more sophisticated internal models that estimated
the state of the visual environment and one that made
no such estimates. We considered the behavior of all four
models in response to a sinusoidal stimulus as a function
of the stimulus amplitude and frequency. We found that
the estimating schemes tended, as a function of increasing
stimulus amplitude, to either have substantially increasing
phase leads or have no substantial changes in either gain or
phase. The nonestimating model, on the other hand, was
the only scheme that reproduced the well-known experi-
mental result that across frequencies the gain substantially
drops but the phase remains roughly constant as a func-
tion of increasing stimulus amplitude.
160
0.1 1.0 10.0
0
20
40
(deg s )
0.1 Hz
0.2 Hz
0.5 Hz
0.1 1.0 10.0
0
0.5
1
gain
0.1 1.0 10.0
180
0
180
phase (deg)
0.1 1.0 10.0
0
0.5
response amp.
(deg)
amplitude of sinusoidal stimulus (deg)
Fig. 6. Asymptotic value of θ, gain, phase, and
response amplitude for the nonestimating model
as a function of stimulus amplitude at three
different frequencies. Calculations were
performed with the transfer function formula.
The parameters of the model were those that best
reproduced the stochastic structure of postural
sway in quiet stance (Sect. 2.6). The response
amplitude represents the amplitude of sway at the
driving frequency; it is calculated by multiplying
the amplitude of the stimulus by the gain. Note
that the scale of the response amplitude plot is
different here than in Figs. 3 and 5
The estimating models, which did not behave appro-
priately, simulated dynamics of the visual scene while
calculating likelihoods corresponding to possible values
of the parameters that described the simulations. On the
other hand, the nonestimating model, which behaved
more appropriately, did not simulate the environment.
Instead, the nonestimating model adapted to changing
environmental conditions by adjusting sensory weights to
minimize the mean square of the control signal. These re-
sults suggest that an internal model of the environment is
not involved in the nervous system’s response to sinusoidal
visual stimuli.
Ravaioli et al. (2004) reached a different conclusion
concerning the response of the postural control system to
translatory visual scene motion. They found that the ner-
vous system extracts stabilizing information from a trans-
lating visual scene, and they interpreted this result as evi-
dence that the nervous system uses an internal model’s rep-
resentation of the translation to compensate. The results
of Ravaioli et al. (2004) together with ours suggest that
the nervous system may handle translatory stimuli differ-
ently than sinusoidal stimuli. We leave to future work the
task of finding a model of the postural control system that
appropriately responds to both sinusoidal stimuli as well
as translation-oscillations.
The absence of an internal model in the nonestimating
scheme is not the only characteristic that distinguishes it
from the other three schemes. The nonestimating scheme
uses a local search for changing its adaptive parameter. On
the other hand, the three estimating schemes base their
estimates on a global search of the adaptive parameter
space. In particular, the modeled nervous system with the
estimating schemes maintains a separate internal model
for each possible set of values of its adaptive parameters.
The chosen estimates, upon which the control is based,
depend on the calculated likelihoods corresponding to all
allowable sets of adaptive parameter values.
The local versus global character of the nonestimating
scheme has consequences for the behavior of the system
that we consider worthy of a more systematic examination
in the future. With the global likelihood scheme, if new val-
ues of the adaptive parameters become more likely than
the current estimates, the model will immediately adjust.
Thus the rate of adjustment to changing environmental
conditions will be independent of the current estimates.
This property does not hold with the local scheme. In par-
ticular, for the local scheme, we have observed that if θ is
small (meaning the system considers vision reliable), the
system makes changes to θ relatively quickly if conditions
change. On the other hand, if this parameter is high (the
system considers vision unreliable), it makes changes to θ
relatively slowly. The result of this difference is an appar-
ent asymmetry in the system’s response to an increase in
stimulus amplitude (from low to high) versus a return
(from high to low). This statement follows because after
an extended period when the amplitude is low, the sys-
tem deems vision a reliable source of postural information
(makes θ small), then changes θ quickly when the ampli-
tude becomes high. On the other hand, after an extended
period when the amplitude is high, the system deems vision
unreliable (makes θ large), then changes θ slowly when
amplitude returns to low. Unfortunately, there is very lit-
tle experimental data on the dynamics of reweighting to
address this prediction.
Is the failure of the estimating schemes a consequence
of their global character and not a consequence of their
internal models? No. Consider the random walk model. It
contains one adaptive parameter d
1
that varies monotoni-
cally with stimulus amplitude. In a local scheme, d
1
would
have different dynamics, but any reasonable scheme would
still estimate a larger d
1
for a larger stimulus amplitude.
Gain and phase of the model for large trial times depend
on stimulus amplitude only through the asymptotic value
d
1
and not through the mechanism through which it is
161
estimated. We have observed substantial increases in phase
accompanying substantial drops in gain as the asymptotic
value of d
1
increased with the random walk model. With a
local adaptive scheme the random walk model would still
have the same property.
The failure of the second-order process model is more
interesting. This model is capable of completely compen-
sating for sinusoidal stimuli, extracting the same amount
of stabilizing information from the visual modality as
when the visual scene is stationary. It shows that the
human postural control system does not respond opti-
mally to such stimuli, even though a control scheme that
does would evidently not be hard to implement in the ner-
vous system. One can only surmise that sinusoidal stimuli
were not common in the environment in which humans
evolved. The results of Ravaioli et al. (2004) suggesting
the presence of an internal model that compensates for
a translating environment may reflect the fact that trans-
lating environments were indeed common. It should be
pointed out that the second-order process model is only
optimal for stimuli that can be described as a linear sec-
ond-order stochastic process. The model would not behave
optimally for a stimulus that consisted of the sum of two
sine waves of different frequencies. Such a stimulus would
require a fourth-order model for complete compensation.
The results of this paper suggest a different mecha-
nism for adaptation to changing environmental conditions
than previously considered: a search for sensory weights
to minimize some criterion such as the applied ankle
torque. Based on our presented results this model makes
one prediction that could prove it false. Note that the re-
sponse amplitude for the nonestimating model (Fig. 6)
rises as a function of increasing amplitude, reaches a
maximum, then falls. Likewise, for normal subjects Pet-
erka and Benolken’s (1995) data suggest that the response
amplitude rises and then saturates as the stimulus ampli-
tude increases. Does the response amplitude also eventu-
ally decrease as the stimulus amplitude gets large? For a
fixed surface with normal subjects, Peterka and Benolken’s
(1995) data do not tell a compelling story. If the response
amplitude of normal subjects does not decrease, then the
nonestimating model, like the second-order model, out-
performs the human postural control system in its re-
sponse to sinusoidal stimuli because it more successfully
down-weights the visual modality as it provides more
unreliable information. That the human postural control
system does not respond in this way would suggest that
it was not performing optimally at least with respect to
minimizing ankle torque.
Acknowledgements. Funding for this research was provided by
National Institutes of Health grant NIH 1RO1NS046065 as
part of the NSF/NIH Collaborative Research in Computational
Neuroscience Program, John J. Jeka, PI.
References
Arnold L (1974) Stochastic differential equations: theory and
applications. Wiley, New York
Balakrishnan AV (1973) Stochastic differential systems: I. Fil-
tering and control. Springer, Berlin Heidelberg New York
Brandt T, Dichgans J, Koenig E (1973) Differential effects of
central versus peripheral vision on egocentric and exocentric
motion perception. Exp Brain Res 16:476–491
Brandt T, Wist ER, Dichgans J (1975) Foreground and back-
ground in dynamic spatial orientation. Percept Psychophys
17:497–503
Bryson AE, Ho YC (1975) Applied optimal control. Hemi-
sphere, Washington, DC
Horak FB, Macpherson JM (1996) Postural orientation and
equilibrium In: Shepard J, Rowell L (eds) Handbook of phys-
iology. Oxford University Press, New York pp 255–292
Jeka JJ, Kiemel T, Creath R, Horak F, Peterka R (2004)
Controlling human upright stance: velocity information is
more accurate than position or acceleration. J Neurophysi-
ol 92:2368–2379
Kiemel T, Oie KS, Jeka JJ (2002) Multisensory fusion and the
stochastic structure of postural sway. Biol Cybern 87:262–
277
Loram ID, Lakie M (2002