Page 1
Physica 16D (1985)285317
NorthHolland, Amsterdam
DETERMINING LYAPUNOV EXPONENTS FROM A TIME SERIES
Alan WOLF~, Jack B. SWIFT, Harry L. SWINNEY and John A. VASTANO
Department of Physics, University of Texas, Austin, Texas 78712, USA
Received 18 October 1984
We present the first algorithms that allow the estimation of nonnegative Lyapunov exponents from an experimental time
series. Lyapunov exponents, which provide a qualitative and quantitative characterization of dynamical behavior, are related to
the exponentially fast divergence or convergence of nearby orbits in phase space. A system with one or more positive Lyapunov
exponents is defined to be chaotic. Our method is rooted conceptually in a previously developed technique that could only be
applied to analytically defined model systems: we monitor the longterm growth rate of small volume elements in an attractor.
The method is tested on model systems with known Lyapunov spectra, and applied to data for the BelousovZhabotinskii
reaction and CouetteTaylor flow.
Contents
1. Introduction
2. The Lyapunov spectrum defined
3. Calculation of Lyapunov spectra from differential equations
4. An approach to spectral estimation for experimental data
5. Spectral algorithm implementation*
6. Implementation details*
7. Data requirements and noise*
8. Results
9. Conclusions
Appendices*
A. Lyapunov spectrum program for systems of differential
equations
B. Fixed evolution time program for ~'1
1. Introduction
Convincing evidence for deterministic chaos has
come from a variety of recent experiments [16]
on dissipative nonlinear systems; therefore, the
question of detecting and quantifying chaos has
become an important one. Here we consider the
spectrum of Lyapunov exponents [710], which
has proven to be the most useful dynamical di
agnostic for chaotic systems. Lyapunov exponents
are the average exponential rates of divergence or
tPresent address: The Cooper Union, School of Engineering,
N.Y., NY 10003, USA.
*The reader may wish to skip the starred sections at a first
reading.
convergence of nearby orbits in phase space. Since
nearby orbits correspond to nearly identical states,
exponential orbital divergence means that systems
whose initial differences we may not be able to
resolve will soon behave quite differentlypredic
tive ability is rapidly lost. Any system containing
at least one positive Lyapunov exponent is defined
to be chaotic, with the magnitude of the exponent
reflecting the time scale on which system dynamics
become unpredictable [10].
For systems whose equations of motion are ex
plicitly known there is a straightforward technique
[8, 9] for computing a complete Lyapunov spec
trum. This method cannot be applied directly to
experimental data for reasons that will be dis
cussed later. We will describe a technique which
for the first time yields estimates of the nonnega
tive Lyapunov exponents from finite amounts of
experimental data.
A less general procedure [6, 1114] for estimat
ing only the dominant Lyapunov exponent in ex
perimental systems has been used for some time.
This technique is limited to systems where a well
defined onedimensional (lD) map can be re
covered. The technique is numerically unstable
and the literature contains several examples of its
improper application to experimental data. A dis
cussion of the 1D map calculation may be found
01672789/85/$03.30 © Elsevier Science Publishers
(NorthH01!and Physics Publishing Division)
Page 2
286
A. Wolf et al. / Determining Lyapunov exponents from a time series
in ref. 13. In ref. 2 we presented an unusually
robust 1D map exponent calculation for experi
mental data obtained from a chemical reaction.
Experimental data inevitably contain external
noise due to environmental fluctuations and limited
experimental resolution. In the limit of an infinite
amount of noisefree data our approach would
yield Lyapunov exponents by definition. Our abil
ity to obtain good spectral estimates from experi
mental data depends on the quantity and quality
of the data as well as on the complexity of the
dynamical system. We have tested our method on
model dynamical systems with known spectra and
applied it to experimental data for chemical [2, 13]
and hydrodynamic [3] strange attractors.
Although the work of characterizing chaotic data
is still in its infancy, there have been many ap
proaches to quantifying chaos, e.g., fractal power
spectra [15], entropy [1618, 3], and fractal dimen
sion [proposed in ref. 19, used in ref. 35, 20, 21].
We have tested many of these algorithms on both
model and experimental data, and despite the
claims of their proponents we have found that
these approaches often fail to characterize chaotic
data. In particular, parameter independence, the
amount of data required, and the stability of re
suits with respect to external noise have rarely
been examined thoroughly.
The spectrum of Lyapunov exponents will be
defined and discussed in section 2. This section
includes table I which summarizes the model sys
tems that are used in this paper. Section 3 is a
review of the calculation of the complete spectrum
of exponents for systems in which the defining
differential equations are known. Appendix A con
tains Fortran code for this calculation, which to
our knowledge has not been published elsewhere.
In section 4, an outline of our approach to estimat
ing the nonnegative portion of the Lyapunov
exponent spectrum is presented. In section 5 we
describe the algorithms for estimating the two
largest exponents. A Fortran program for de
termining the largest exponent is contained in
appendix B. Our algorithm requires input parame
ters whose selection is discussed in section 6. Sec
tion 7 concerns sources of error in the calculations
and the quality and quantity of data required for
accurate exponent estimation. Our method is ap
plied to model systems and experimental data in
section 8, and the conclusions are given in
section 9.
2. The Lyapunov spectrum defined
We now define [8, 9] the spectrum of Lyapunov
exponents in the manner most relevant to spectral
calculations. Given a continuous dynamical sys
tem in an ndimensional phase space, we monitor
the longterm evolution of an infinitesimal nsphere
of initial conditions; the sphere will become an
nellipsoid due to the locally deforming nature of
the flow. The ith onedimensional Lyapunov expo
nent is then defined in terms of the length of the
ellipsoidal principal axis pi(t):
h~ = lim 1 log 2 pc(t)
t,oo t pc(O)'
(1)
where the )h are ordered from largest to smallestt.
Thus the Lyapunov exponents are related to the
expanding or contracting nature of different direc
tions in phase space. Since the orientation of the
ellipsoid changes continuously as it evolves, the
directions associated with a given exponent vary in
a complicated way through the attractor. One can
not, therefore, speak of a welldefined direction
associated with a given exponent.
Notice that the linear extent of the ellipsoid
grows as 2 htt, the area defined by the first two
principal axes grows as 2 (x~*x2)t, the volume de
fined by the first three principal axes grows as
2 (x'+x2+x~)t, and so on. This property yields
another definition of the spectrum of exponents:
tWhile the existence of this limit has been questioned [8, 9,
22], the fact is that the orbital divergence of any data set may
be quantified. Even if the limit does not exist for the underlying
system, or cannot be approached due to having finite amounts
of noisy data, Lyapunov exponent estimates could still provide
a useful characterization of a given data set. (See section 7.1.)
Page 3
A. Wolf et aL / Determining Lyapunov exponents from a time series
287
the sum of the first j exponents is defined by the
long term exponential growth rate of a jvolume
element. This alternate definition will provide the
basis of our spectral technique for experimental
data.
Any continuous timedependent dynamical sys
tem without a fixed point will have at least one
zero exponent [22], corresponding to the slowly
changing magnitude of a principal axis tangent to
the flow. Axes that are on the average expanding
(contracting) correspond to positive (negative) ex
ponents. The sum of the Lyapunov exponents is
the timeaveraged divergence of the phase space
velocity; hence any dissipative dynamical system
will have at least one negative exponent, the sum
of all of the exponents is negative, and the post
transient motion of trajectories will occur on a
zero volume limit set, an attractor.
The exponential expansion indicated by a posi
tive Lyapunov exponent is incompatible with mo
tion on a bounded attractor unless some sort of
folding process merges widely separated trajecto
ries. Each positive exponent reflects a "direction"
in which the system experiences the repeated
stretching and folding that decorrelates nearby
states on the attractor. Therefore, the longterm
behavior of an initial condition that is specified
with any uncertainty cannot be predicted; this is
chaos. An attractor for a dissipatiVe system with
one or more positive Lyapunov exponents is said
to be "strange" or "chaotic".
The signs of the Lyapunov exponents provide a
qualitative picture of a system's dynamics. One
dimensional maps are characterized by a single
Lyapunov exponent which is positive for chaos,
zero for a marginally stable orbit, and negative for
a periodic orbit. In a threedimensional continuous
dissipative dynamical system the only possible
spectra, and the attractors they describe, are as
follows: (+,0,), a strange attractor; (0,0,), a
twotoms; (0, , ), a limit cycle; and (, , ),
a fixed point. Fig. 1 illustrates the expanding,
"slower than exponential," and contracting char
acter of the flow for a three,dimensional system,
the Lorenz model [23]. (All of the model systems
that we will discuss are defined in table I.) Since
Lyapunov exponents involve longtime averaged
behavior, the short segments of the trajectories
shown in the figure cannot be expected to accu
rately characterize the positive, zero, and negative
exponents; nevertheless, the three distinct types of
behavior are clear. In a continuous fourdimen
sional dissipative system there are three possible
types of strange attractors: their Lyapunov spectra
are (+, +,0,), (+,0,0,), and (+,0,,).
An example of the first type is Rossler's hyper
chaos attractor [24] (see table I). For a given
system a change in parameters will generally
change the Lyapunov spectrum and may also
change both the type of spectrum and type of
attractor.
The magnitudes of the Lyapunov exponents
quantify an attractor's dynamics in information
theoretic terms. The exponents measure the rate at
which system processes create or destroy informa
tion [10]; thus the exponents are expressed in bits
of information/s or bits/orbit for a continuous
system and bits/iteration for a discrete system.
For example, in the Lorenz attractor the positive
exponent has a magnitude of 2.16 bits/s (for the
parameter values shown in table I). Hence if an
initial point were specified with an accuracy of one
part per million (20 bits), the future behavior
could not be predicted after about 9 s [20 bits/(2.16
bits/s)], corresponding to about 20 orbits. After
this time the small initial uncertainty will essen
tially cover the entire attractor, reflecting 20 bits of
new information that can be gained from an ad:
ditional measurement of the system. This new
information arises from scales smaller than our
initial uncertainty and results in an inability to
specify the state of the system except to say that it
is somewhere on the attractor. This process is
sometimes called an information gain reflecting
new information from the heat bath, and some
times is called an information lossbits shifted
out of a phase space variable "register" when bits
from the heat bath are shifted in.
The average rate at which information con
tained in transients is lost can be determined from
Page 4
288
A. Wolf et al. / Determining Lyapunov exponents from a time series
• ° • o • ° ° ° . ° • ° • ° °
"." :....
, :.~ "'.•'.
• • ". . t
".•
•... . • . • ... ....
...,.::.:::.........
. • .,:'..~....:~::..:..:'..:..:..
...•..
"" ":'" :"::"'"
~'"" """
".':':"'( (,~'."~,m
.. "•.. ....
.
• .. .. ,~ .,.:..
•.'~"."
:'.."
""
•
•
start
N~
.
" ~"
•
~" " ":'"'"'""
. •

'.:..""%V4";':'"
..,..:/~
"' ".'... " .... ~ " .
•
. • : • .... "... ..........
i IIIIIl[lll
time~
• • • • • . . ° • . ° . ° °
• •
•
.. o,:... ..
•. • . • . . . . . .. ,:,,.::. ".•.
(b) .
..',.<'."~::.:.':..
• . ~ ..... .:'~.?',',;~ "x ~il~
" "~,~'.:~'~",~,~'~xxx
"... "..
~.~'
,...~.'~:':,~:..~ "r.. ::
•
I  "
.'.• .:
. ..
" :::,.!...":"""...
• "
• ". ":::,.f,~,_,,,~.~
• $:L~.
.'.~...~.;.;.,....
" ".:,v':.:~
,., ,.,. ...: ......
..:.:..." .... ::.:..
............
"'"...
.... ~"~.'...__'••.~..
;
: •
•
.... ...•
, .:" "~.'. . . " " :
~start
[l,,m,,,,l,,,,,llli,,lllli,,dll
time ~
• . ........ :; .. ~.. : ..
• ... ;;.~.....~.::........... s,,,, ~.~,.~;,..':..::.~:"~:.v
•
:'"
",',...:~.'.2~'W~'".~... ".".'.. "'::...':"'.:
" ". " : "
• ., , • " •~,'•••• .• •" •
.........
time
Fig. 1. The short term evolution of the separation vector between three carefully chosen pairs of nearby points is shown for the
Lorenz attractor, a) An expanding direction (~1 > 0); b) a "slower than exponential" direction (~'2 = 0); C) a contracting direction
(X3 < 0).
the negative exponents• The asymptotic decay of a
perturbation to the attractor is governed by the
least negative exponent, which should therefore be
the easiest of the negative exponents to estimatet.
tWe have been quite successful with an algorithm for de
termiuing the dominant (smallest magnitude) negative expo
nent from pseudoexperimental data (a single time series ex
tracted from the solution of a model system and treated as an
experimental observable) for systems that are nearly integer
dimensional. Unfortunately, our approach, which involves mea
suring the mean decay rate of many induced perturbations of
the dynamical system, is unlikely to work on many experimen
tal systems. There are several fundamental problems with the
calculation of negative exponents from experimental data, but
For the Lorenz attractor the negative exponent is
so large that a perturbed orbit typically becomes
indistinguishable from the attractor, by "eye", in
less than one mean orbital period (see fig. 1).
of greatest importance is that posttransient data may not
contain resolvable negative exponent information and per
turbed data must refl~t properties of the unperturbed system,
that is, perturbations must only change the state of the system
(current values of the dynamical variables). The response of a
physical system to a nondelta function perturbation is difficult
to interpret, as an orbit separating from the attractor may
reflect either a locally repelling region of the attractor (a
positive contribution to the negative exponent) or the finite rise
time of the perturbation.
Page 5
A. Wolf et al. / Determining Lyapunov exponents from a time series
289
Table I
The model systems considered in this paper and their Lyapunov spectra and dimensions as computed from the equations of motion
Lyapunov
spectrum
(bits/s)t
Lyapunov
dimension* System Parameter
values
H~non: [25]
~1 = 0.603
h 2 =  2.34
(bits/iter.)
X. +1 = 1  aX;. + Yn
Y. + 1 = bX.
{ b = 1.4
= 0.3
Rosslerchaos: [26]
)( =  (Y + Z)
)'= X+ aY
= b + Z(X
[ a = 0.15
I b = 0.20
c = 10.0
)k 1 = 0.13
~2 =0.00
h 3 =  14.1 c)
Lorenz: [23]
)(= o(Y X)
~'= X( R Z)
= XY  bZ
[ o = 16.0
I R=45.92
b = 4.0
h 1 = 2.16
X 2 =0.00
;k 3 =  32.4
Y
Rosslerhyperchaos: [24]
Jr'=  (Y+ Z)
)'= X+ aY+ W
= b + XZ
if" = cW  dZ
( a = 0.25
[ b= 3.0
 c = 0.05
k d = 0.5
A t = 0.16
X 2 =0.03
h 3 = 0.00
h4 =  39.0
MackeyGlass: [27]
( a = 0.2 h t = 6.30E3
j( = aX(t + s )
1 + [ X(t + s)] c
 bX(t)
/ b = 0.1
) c = 10.0
s = 31.8
)~2 = 2.62E3
IX31 < 8.0E6
)'4 =  1.39E2
1.26
2.01
2.07
3.005
3.64
tA mean orbital period is well defined for Rossler chaos (6.07 seconds) and for hyperchaos (5.16 seconds) for the parameter values
used here. For the Lorenz attractor a characteristic time (see footnote section 3) is about 0.5 seconds. Spectra were computed for
each system with the code in appendix A.
~As defined in eq. (2).
The Lyapunov spectrum is closely related to the
fractional dimension of the associated strange at
tractor. There are a number [19] of different frac
tionaldimensionlike quantities, including the
fractal dimension, information dimension, and the
correlation exponent; the difference between them
is often small. It has been conjectured by Kaplan
and Yorke [28, 29] that the information dimension
d r is related to the Lyapunov spectrum by the
equation
Ei 1~i
df=J+ I?~j+il ' (2)
where j is defined by the condition that
j j+l
E)~i> 0
i1 i1
The conjectured relation between d r (a static
property of an attracting set) and the Lyapunov
and EX,<O.
(3)
Page 6
290
,4. 14/olf et aL / Determining Lyapunov exponents from a time series
exponents appears to be satisfied for some model
systems [30]. The calculation of dimension from
this equation requires knowledge of all but the
most negative Lyapunov exponents.
3. Calculation of Lyapunov spectra from differential
equations
Our algorithms for computing a nonnegative
Lyapunov spectrum from experimental data are
inspired by the technique developed indepen
dently by Bennetin et al. [8] and by Shimada and
Nagashima [9] for determining a complete spec
trum from a set of differential equations. There
fore, we describe their calculation (for brevity, the
ODE approach) in some detail.
We recall that Lyapunov exponents are defined
by the longterm evolution of the axes of an infini
tesimal sphere of states. This procedure could be
implemented by defining the principal axes with
initial conditions whose separations are as small as
computer limitations allow and evolving these with
the nonlinear equations of motion. One problem
with this approach is that in a chaotic system we
cannot guarantee the condition of small sep
arations for times on the order of hundreds of
orbital periodst, needed for convergence of the
spectrum.
This problem may be avoided with the use of a
phase space plus tangent space approach. A "fidu
cial" trajectory (the center of the sphere) is defined
by the action of the nonlinear equations of motion
on some initial condition. Trajectories.of points on
the surface of the sphere are defined by the action
of the linearized equations of motion on points
infinitesimally separated from the fiducial trajec
tory. In particular, the principal axes are defined
by the evolution via the linearized equations of an
initially orthonormal vector frame anchored to the
fiducial trajectory. By definition, principal axes
defined by the linear system are always infinitesimal
relative to the attractor. Even in the linear system,
principal axis vectors diverge in magnitude, but
this is a problem only because computers have a
limited dynamic range for storing numbers. This
divergence is easily circumvented. What has been
avoided is the serious problem of principal axes
finding the global "fold" when we really only want
them to probe the local "stretch."
To implement this procedure the fiducial trajec
tory is created by integrating the nonlinear equa
tions of motion for some posttransient initial
condition. Simultaneously, the linearized equa
tions of motion are integrated for n different ini
tial conditions defining an arbitrarily oriented
frame of n orthonormal vectors. We have already
pointed out that each vector will diverge in magni
tude, but there is an additional singularityin a
chaotic system, each vector tends to fall along the
local direction of most rapid growth. Due to the
finite precision of computer calculations, the col
lapse toward a common direction causes the tan
gent space orientation of all axis vectors to become
indistinguishable. These two problems can be
overcome by the repeated use of the Gram
Schmidt reorthonormalization (GSR) procedure on
the vector frame:
Let the linearized equations of motion act on
the initial frame of orthonormal vectors to give a
set of vectors
vector to align itself along the ~1 direction, and
the orientationpreserving properties of GSR mean
that the initial labeling of the vectors may be done
arbitrarily.) Then GSR provides the following or
thonormal set { ~ ..... v,' }:
{v 1 ..... Vn). (The desire of each
1D 1
v~ = IIv, ll '
v2 <v2,~>~
v~=
tlv~  <v~, ~>~ll '
tShould the mean orbital period not be welldefined, a
characteristic time can be either the mean time between inter
sections of a Poincar6 section or the time corresponding to a
dominant power spectral feature.
v.  <~., ~._,>~._, ..... <v.,~>~
(4)
Page 7
A. Wolf et al./ Determining Lyapunoo exponents from a time series
291
where (,)
frequency of reorthonormalization is not critical,
so long as neither the magnitude nor the orienta
tion divergences have exceeded computer limita
tions. As a rule of thumb, GSR is performed on
the order of once per orbital period.
We see that GSR never affects the direction of
the first vector in a system, so this vector tends to
seek out the direction in tangent space which is
most rapidly growing (components along other
directions are either growing less rapidly or are
shrinking). The second vector has its component
along the direction of the first vector removed, and
is then normalized. Because we are changing its
direction, vector v 2 is not free to seek out the most
rapidly growing direction. Because of the manner
in which we are changing it, it also is not free to
seek out the second most rapidly growing direc
tiont. Note however that the vectors ~ and if2
span the same twodimensional subspace as the
vectors v x and v 2. In spite of repeated vector
replacements, the space these vectors define continu
ally seeks out the twodimensional subspace that is
most rapidly growing. The area defined by these
vectors is proportional to 2 (x~+x2)t [8]. The length
of vector v t is proportional to 2 x~t so that monitor
ing length and area growth allows us to determine
both exponents. In practice, as ~
orthogonal, we may determine h 2 directly from
the mean rate of growth of the projection of vector
v 2 on vector 4. In general, the subspace spanned
by the first k vectors is unaffected by GSR so that
the longterm evolution of the kvolume defined
by these vectors is proportional to 2 ~ where # =
~.ki_ 1 ~ i t. Projection of the evolved vectors onto the
new orthonormal frame correctly updates the rates
of growth of each of the first kprincipal axes in
signifies the inner product. The
and if2 are
tThis is clear when we consider that we may obtain different
directions of vector 02 at some specified time if we exercise our
freedom to choose the intermediate times at which GSR is
performed. That is, beginning with a specified v 1 and 02 at
time ti, we may perform replacements at times t~+ x and ti+2,
obtaining the vectors ~, t~ and then v~', v~' or we may
propagate directly to time ti+ 2, obtaining vl*, v~. t~' and v~
are not parallel; therefore, the details of propagation and
replacement determine the orientation of 0 2 .
turn, providing estimates of the k largest Lyapunov
exponents. Thus GSR allows the integration of the
vector frame for as long as is required for spectral
convergence.
Fortran code for the ODE procedure appears in
appendix A. We illustrate the use of this procedure
for the Rossler attractor [26]. The spectral calcula
tion requires the integration of the 3 equations of
motion and 9 linearized equations for on the order
of 100 orbits of model time (a few cpu minutes on
a VAX 11/780) to obtain each exponent to within
a few percent of its asymptotic value. In practice
we consider the asymptotic value to be attained
when the mandatory zero exponent(s) are a few
orders of magnitude smaller than the smallest
positive exponent. The convergence rate of zero
and positive exponents is about the same, and is
much slower than the convergence rate of negative
exponents. Negative exponents arise from the
nearly uniform attractiveness of the attractor which
can often be well estimated from a few passes
around an attractor, nonnegative exponents arise
from a onceperorbit stretch and fold process that
must be sampled on the order of hundreds of
times (or more) for reasonable convergence.
The method we have described for finding
Lyapunov exponents is perhaps more easily under
stood for a discrete dynamical system. Here we
consider the H6non map [25] (see table I). The
linearization of this map is
[,sx.
=L/By. ,
(5)
where
10] ,6,
and X~ is the (n 1)st iterate of an arbitrary
initial condition X 1.
An orthonormal frame of principal axis vectors
such as ((0,1), (1,0)) is evolved by applying the
product Jacobian to each vector. For either vector
Page 8
292
A. Wolf et al./ Determining Lyapunov exponents from u tune series
the operation
ways. For example, for the vector (0,l) we have
may be written in two different
or, by regrouping the terms,
In eq. (7) the latest Jacobi matrix multiplies
each current axis vector, which is the initial vector
multiplied by all previous Jacobi matrices. The
magnitude of each current axis vector diverges,
and the angular separation between the two vec
tors goes to zero. Fig. 2 shows that divergent
behavior is visible within a few iterations. .GSR
corresponds to the replacement of each current
axis vector. Lyapunov exponents are computed
Fig.
orthonormal
initial
second
angular
tions
was uncommonly
2. The action
vector
(2)
of
frame
first
the divergence
of the frame
so that
slow.
the product
is illustrated
iterate;
Jacobian
for the H&non
(3) second
in vector
are quite apparent.
the angular
on an initially
map:
iterate.
magnitude
Initial
collapse of the vectors
(1)
the
the
frame;
iteration
collapse
were
and By
and
condi
chosen
from the growth
vector and the growth rate of the area defined by
both vectors.
In eq. (8) the product Jacobian acts on each of
the initial axis vectors. The columns of the product
matrix converge to large multiples of the eigenvec
tor of the biggest eigenvalue, so that elements of
the matrix diverge and the matrix becomes singu
lar. Here GSR corresponds to factoring out a large
scalar multiplier of the matrix to prevent the mag
nitude divergence, and doing row reduction with
pivoting to retain the linear independence
columns. Lyapunov exponents are computed from
the eigenvalues of the longtime product matrix?.
We emphasize that Lyapunov exponents are not
local quantities in either the spatial or temporal
sense. Each exponent arises from the average, with
respect to the dynamical motion, of the local de
formation of various phase space directions. Each
is determined by the longtime evolution
singZe volume element. Attempts to estimate expo
nents by averaging local contraction
sion rates of phase space are likely to fail at the
point where these contributions
are combined. In fig. 3a we show vector vi at each
renormalization step for the Lorenz attractor over
the course of several hundred
apparent multivaluedness
growing direction (in some regions of the attrac
tor) shows that this direction
function of position on the attractor.
direction is often nearly parallel to the flow on the
Lorenz attractor (see fig. 3b) it is usually nearly
transverse to the flow for the Rossler attractor. We
conclude that exponent calculation by averaging
local divergence estimates is a dangerous proce
dure.
rate of the length of the first
of the
of a
and expan
to the exponents
orbits [32]. The
of the most rapidly
is not simply a
While this
+We are aware
experimental
matrices
[31]. This
matrix
of an attempt
data
and
calculation
notation
problems
and resolution
to estimate
direct
of
is essentially
by diagonalizing
of sensitivity
Lyapunov
estimation
long
the same as ours
the system
to external
required
spectra
of
product
from
Jacobian
matrix
avoid
and has the same
to the amount
estimates.
through
formation
local
the time
(we
at each step)
noise,
for accurate
and
of data
Page 9
,4. Wolf et al. / Determining Lyapunoo exponents from a time series
293
• .
(b)
_  .__ ,,,, ._ 
J leo #1 •
Fig. 3. A modification to the ODE spectral code (see appendix A) allows us to plot the running direction of greatest growth (vector
v~ ) in the Lorenz attractor. In (a), infrequent renormalizations confirm that this direction is not singlevalued on the attractor. In (b),
frequent renormalizations show us that this direction is usually nearly parallel to the flow. In the Rossler attractor, this direction is
usually nearly orthogonal to the flow.
4. An approach to spectral estimation for
experimental data
Experimental data typically consist of discrete
measurements of a single observable. The well
known technique of phase space reconstruction
with delay coordinates [2, 33, 34] makes it possible
to obtain from such a time series an attractor
whose Lyapunov spectrum is identical to that of
the original attractor. We have designed a method,
conceptually similar to the ODE approach, which
can be used to estimate nonnegative Lyapunov
exponents from a reconstructed attractor. To un
derstand our method it is useful to summarize
what we have discussed thus far about exponent
calculation.
Lyapunov exponents may be defined by the
phase space evolution of a sphere of states. At
tempts to apply this definition numerically to
equations of motion fail since computer limita
tions do not allow the initial sphere to be con
structed sufficiently small. In the ODE approach
one avoids this problem by working in the tangent
space of a fiducial trajectory so as to obtain always
infinitesimal principal axis vectors. The remaining
divergences are easily eliminated with Gram
Schmidt reorthonormalization.
The ODE approach is not directly applicable to
experimental data as the linear system is not avail
able. All is not lost provided that the linear ap
proximation holds on the smallest length scales
defined by our data. Our approach involves
working in a reconstructed attractor, examining
orbital divergence on length scales that are always
as small as possible, using an approximate GSR
procedure in the reconstructed phase space as
Page 10
294
A. Wolf et al. / Determining Lyapunov exponents from a time series
necessary. To simplify the ensuing discussion we
will assume that the systems under consideration
possess at least one positive exponent.
To estimate X1 we in effect monitor the longterm
evolution of a single pair of nearby orbits. Our
reconstructed attractor, though defined by a single
trajectory, can provide points that may be consid
ered to lie on different trajectories. We choose
points whose temporal separation in the original
time series is at least one mean orbital period,
because a pair of points with a much smaller
temporal separation is characterized by a zero
Lyapunov exponent. Two data points may be con
sidered to define the early state of the first prin
cipal axis so long as their spatial separation is
small. When their separation becomes large we
would like to perform GSR on the vector they
define (simply normalization for this single vector),
which would involve replacing the nonfiducial
data point with a point closer to the fiducial point,
in the same direction as the original vector. With
finite amounts of data, we cannot hope to find a
replacement point which falls exactly along a
specified line segment in the reconstructed phase
space, but we can look for a point that comes
close. In effect, through a simple replacement pro
cedure that attempts to preserve orientation and
minimize the size of replacement vectors, we have
monitored the longterm behavior of a single prin
cipal axis vector. Each replacement vector may be
evolved until a problem arises, and so on. This
leads us to an estimate of X1. (See fig. 4a.)
The use of a finite amount of experimental data
does not allow us to probe the desired infinitesimal
length scales of an attractor. These scales are also
inaccessible due to the presence of noise on finite
length scales and sometimes because the chaos
producing structure of the attractor is of negligible
spatial extent. A discussion of these points is de
ferred until section 7.1.
An estimate of the sum of the two largest expo
nents X1 + X 2 is similarly obtained. In the ODE
procedure this involves the longterm evolution of
a fiducial trajectory and a pair of tangent space
vectors. In our procedure a triple of points is
evolved in the reconstructed attractor. Before the
area element defined by the triple becomes com
parable to the extent of the attractor we mimic
GSR by keeping the fiducial point, replacing the
remainder of the triple with points that define a
smaller area element and that best preserve the
element's phase space orientation. Renormaliza
tions are necessary solely because vectors grow too
large, not because vectors will collapse to indis
tinguishable directions in phase space (this is un
likely with the limited amounts of data usually
available in experiments). The exponential growth
rate of area elements provides an estimate of X1
+ X 2. (See fig. 4b.)
Our approach can be extended to as many non
negative exponents as we care to estimate: k + 1
points in the reconstructed attractor define a k
volume element whose longterm evolution is pos
sible through a data replacement procedure that
attempts to preserve phase space orientation and
probe only the small scale structure of the attrac
tor. The growth rate of a kvolume element pro
vides an estimate of the sum of the first k
Lyapunov exponents.
In principle we might attempt the estimation of
negative exponents by going to higherdimensional
volume elements, but information about contract
ing phase space directions is often impossible to
resolve. In a system where fractal structure can be
resolved, there is the difficulty that the volume
elements involving negative exponent directions
collapse exponentially fast, and are therefore
numerically unstable for experimental data (see
section 7.1).
5. Spectnd algorithm implementation
We have implemented several versions of our
algorithms including simple "fixed evolution time"
programs for ~'1 and X1 ÷ hE, "variable evolution
time" programs for XI+~:, and "interactive"
programs that are used on a graphics machinet.
tThe interactive program avoids the profusion of input
parameters required for our increasingly sophisticated expo
Page 11
A. Wolf et al./ Determining Lyapunov exponents from a time series
295
In appendix B we include Fortran code and
documentation for the h 1 fixed evolution time
program. This program is not sophisticated, but it
is concise, easily understood, and useful for learn
ing about our technique. We do not include the
fixed evolution time code for )~x + )~2 (though it is
briefly discussed at the end of appendix B) or our
other programs, but we will supply them to inter
ested parties. We can also provide a highly effi
cient data base management algorithm that can be
used in any of our programs to eliminate the
expensive process of exhaustive search for nearest
neighbors. We now discuss the fixed evolution
time program for A t and the variable evolution
time program for ~x + h2 in some detail.
5.1. Fixed evolution time program for )~1
Given the time series x(t), an mdimensional
phase portrait is reconstructed with delay coordi
nates [2, 33, 34], i.e., a point on the attractor is
given by {x(t), x(t + ~') ..... x(t + [m  1]~')}
where z is the almost arbitrarily chosen
delay time. We locate the nearest neighbor (in
the Euclidean sense) to the initial point
{x(t o) ..... X(to + [m 1]~)} and denote the dis
tance between these two points L(to). At a later
time tt, the initial length will have evolved to
length L'(tx). The length element is propagated
through the attractor for a time short enough so
that only small scale attractor structure is likely to
be examined. If the evolution time is too large we
nent programs. This program allows the operator to observe:
the attractor, a length or area element evolving over a range of
times, the best replacement points available over a range of
times, and so forth. Each of these is seen in a two or three
dimensional projection (depending on the graphical output
device) with terminal output providing supplementary informa
tion about vector magnitudes and angles in the dimension of
the attractor reconstruction. Using this information the oper
ator chooses appropriate evolution times and replacement
points. The program is currently written for a Vector General
3405 but may easily be modified for use on other graphics
machines. A 16mm movie summarizing our algorithm and
showing the operation of the program on the Lorenz attractor
has been made by one of the authors (A.W.).
may see L' shrink as the two trajectories which
define it pass through a folding region of the
attractor. This would lead to an underestimation
of hi We now look for a new data point that
satisfies two criteria reasonably well: its sep
aration, L(tl), from the evolved fiducial point is
small, and the angular separation between the
evolved and replacement elements is small (see fig.
4a). If an adequate replacement point cannot be
found, we retain the points that were being used.
This procedure is repeated until the fiducial trajec
tory has traversed the entire data file, at which
point we estimate
M
Y'~ log 2
k=l
L,(tk)
, (9)
)k I = tM_ t o L(tt,_x)
where M is the total number of replacement steps.
In the fixed evolution time program the time step
A = tk+ 1 t k (EVOLV in the Fortran program)
between replacements is held constant. In the limit
of an infinite amount of noisefree data our proce
dure always provides replacement vectors of infini
tesimal magnitude with no orientation error, and
)k 1 is obtained by definition. In sections 6 and 7 we
discuss the severity of errors of orientation and
finite vector size for finite amounts of noisy experi
mental data.
5.2. Variable evolution time program for )~1 + )~ 2
The algorithm for estimating h x + 1~2 is similar
in spirit to the preceeding algorithm, but is more
complicated in implementation. A trio of data
points is chosen, consisting of the initial fiducial
point and its two nearest neighbors. The area
A(to) defined by these points is monitored un
til a replacement step is both desirable and possi
ble the evolution time is variable. This mandates
the use of several additional input parameters: a
minimum number of evolution steps between re
placements (JUMPMN), the number of steps to
evolve backwards (HOPBAK) when a replacement
site proves inadequate, and a maximum length or
area before replacement is attempted.
Page 12
296
A. Wolf et aL / Determining Lyapunov exponents from a time series
(a)
/~
tLI m i
s %/
Llt~ t ! t 2
tiqtu¢lol t
(b, .
"t iI
M t o ) ~ r t I   t 2 ~itluci*l f ' ~I ..~'tec t°'¥
'o
Fig. 4. A schematic representation of the evolution and replacement procedure used to estimate Lyapunov exponents from
experimental data. a) The largest Lyapunov exponent is computed from the growth of length elements. When the length of the vector
between two points becomes large, a new point is chosen near the reference trajectory, minimizing both the replacement length L and
the orientation change ~. b) A similar procedure is followed to calculate the sum of the two largest Lyapunov exponents from the
growth of area elements. When an area element becomes too large or too skewed, two new points are chosen near the reference
trajectory, minimizing the replacement area A and the change in phase space orientation between the original and replacement area
elements.
Evolution continues until a "problem" arises. In
our implementation the problem list includes: a
principal axis vector grows too large or too rapidly,
the area grows too rapidly, and the skewness of
the area element exceeds a threshold value.
Whenever any of these criteria are met, the triple
is evolved backwards HOPBAK steps and a re
placement is attempted. If replacement fails, we
will pull the triple back another HOPBAK steps,
and try again. This process is repeated, if neces
sary, until the triple is getting uncomfortably close
to the previous replacement site. At this point we
take the best available replacement point, and
jump forward at least JUMPMN steps to start the
next evolution. At the first replacement time, tl,
the two points not on the fiducial trajectory are
replaced with two new points to obtain a smaller
area A(tt) whose orientation in phase space is
most nearly the same as that of the evolved area
A'(tl). Determining the set of replacement points
that best preserves area orientation presents no
fundamental difficulties.
Propagation and replacement steps are repeated
(see fig. 4b) until the fiducial trajectory has
traversed the entire data file at which point we
estimate
"
E l o g 2 
k=l
1__ (10)
~1 + ~2 = tM  to A(tk,x ) ,
where t k is the time of the k th replacement step.
It is often possible to verify our results for X~
through the use of the h 1 +h 2 calculation. For
Page 13
A. Wolf et aL / Determining Lyapunov exponents from a time series
297
attractors that are very nearly two dimensional
there is no need to worry about preserving orienta
tion when we replace triples of points. These ele
ments may rotate and deform within the plane of
the attractor, but replacement triples always lie
within this same plane. Since X 2 for these attrac
tors is zero, area evolution provides a direct esti
mate for h 1. With experimental data that appear
to define an approximately twodimensional at
tractor, an independent calculation of df from its
definition (feasible for attractors of dimension less
than three [35]) may justify this approach to esti
mating hx.
6. Implementation details
6.1. Selection of embedding dimension and delay
time
In principle, when using delay coordinates to
reconstruct an attractor, an embedding [34] of the
original attractor is obtained for any sufficiently
large m and almost any choice of time delay ~, but
in practice accurate exponent estimation requires
some care in choosing these two parameters. We
should obtain an embedding if m is chosen to be
greater than twice the dimension of the underlying
attractor [34]. However, we find that attractors
reconstructed using smaller values of m often
yield reliable Lyapunov exponents. For example,
in reconstructing the Lorenz attractor from its
xcoordinate time series an embedding dimension
of 3 is adequate for accurate exponent estimation,
well below the sufficient dimension of 7 given by
ref. [3411". When attractor reconstruction is per
formed in a space whose dimension is too low,
"catastrophes" that interleave distinct parts of the
attractor are likely to restflt. For example, points
fWe have found that it is often possible to ignore several
components of evolving vectors in computing their average
exponential rate of growth: keeping two or more components
of the vector often suffices for this purpose. As our discussion
of "catastrophes" will soon make clear, the search for replace
ment points most often requires that all of the delay coordi
nates be used.
on separate lobes of the Lorenz attractor may be
coincident in a twodimensional reconstruction of
the attractor. When this occurs, replacement ele
ments may contain points whose separation in the
original attractor is very large; such elements are
liable to grow at a dramatic rate in our recon
structed attractor in the short term, providing an
enormous contribution to the estimated exponent.
As these elements tend to blow up almost im
mediately, they are also quite troublesome to re
place,.
If m is chosen too large we can expect, among
other problems, that noise in the data will tend to
decrease the density of points defining the attrac
tor, making it harder to find replacement points.
Noise is an infinite dimensional process that, un
like the deterministic component of the data, flUs
each available phase space dimension in a re
constructed attractor (see section 7.2). Increasing
m past what is minimally required has the effect of
unnecessarily increasing the level of contamination
of the data.
Another problem is seen in a threedimensional
reconstruction of the Htnon attractor. The recon
structed attractor looks much like the original
attractor sitting on a twodimensional sheet, with
this sheet showing a simple twist in threespace.
We expect that this behavior is typical; when m is
increased, surface curvature increases ~. Increasing
m therefore makes it increasingly difficult to satisfy
orientation constraints at replacement time, as the
attractor is not sufficiently flat on the smallest
length scales filled out by the fixed quantity of
data. It is advisable to check the stationarity of
*If two points lie at opposite ends of an attractor, it is
possible that their separation vector lies entirely outside of the
attractor so that no orientation preserving replacement can be
found. If this goes undetected, the current pair of points is
likely to be retained for an orbital period or longer, until these
points are accidentally thrown close together.
*A simple study for the Htnon system showed that for
reconstructions of increasing dimension the mean distance be
tween the points defining the attractor rapidly converged to an
attractor independent value. The fold put in each new phase
space direction by the reconstruction process tended to make
the concept of "nearby point in phase space" meaningless for
this finite data set.
Page 14
298
A. Wolf et al./ Determining Lyapunov exponents from a time series
Fig. 5. The strange attractor in the BelousovZhabotinskii reaction is reconstructed by the use of delay coordinates from the bromide
ion concentration time series [2]. The delays shown are a) ~; b) ½; and c) ~ of a mean orbital period. Notice how the folding region of
the attractor evolves from a featureless "pencil" to a large scale twist.
results with m to ensure robust exponent esti
mates.
Choice of delay time is also governed by the
necessity of avoiding catastrophes. In our data [2]
for the BelousovZhabotinskii chemical reaction
(see fig. 5) we see a dramatic difference in the
reconstructed attractors for the choices T = 1/12,
~"  1/2 and I" = 3/4 of the mean orbital period.
In the first case we obtain a "pencillike" region
which obscures the folding region of the attractor.
This structure opens up and grows larger relative
to the total extent of the attractor for the larger
values of ~', which is clearly desirable for our
algorithms. We choose • neither so small that the
attractor stretches out along the hne x =yz =
..., nor so large that m z is much larger than the
orbital period. A check of the stationarity of expo
nent estimates with t is again recommended.
6.2. Evolution times between replacements
Decisions about propagation times and replace
ment steps in these calculations depend on ad
ditional input parameters, or in the case of the
interactive program, on the operator's judgement.
(The stationarity of )~l values over ranges of all
algorithm parameters is illustrated for the Rossler
attractor in figs. 6a6d.) Accurate exponent calcu
lation therefore requires the consideration of the
following interrelated points: the desirability of
maximizing evolution times, the tradeoff between
minimizing replacement vector size and minimiz
ing the concomitant orientation error, and the
manner in which orientation errors can be ex
pected to accumulate. We now discuss these points
in turn.
Maximizing the propagation time of volume ele
ments is highly desirable as it both reduces the
frequency with which orientation errors are made
and reduces the cost of the calculation consider
ably (element propagation involves much less
computation than element replacement). In our
variable evolution time program this is not much
of a problem, as replacements are performed only
when deemed necessary (though the program has
been made conservative in such judgments). In the
interactive algorithm this is even less of a problem,
as an experienced operator can often process a
large file with a very small number of replace
ments. The problem is severe, however, in our
fixed evolution time program, which is otherwise
desirable for its extreme simplicity. In this pro
gram replacements are attempted at fixed time
steps, independent of the behavior of the volume
element.
Page 15
A. Wolf et all Determining Lyapunov exponents from a time series
299
o21
I (a)
~0. t
)..
.._i
o. 8
o: ' ' 'o: ' ' ' : '
4 8 12
TAU (ORBITS)
I.B
. /~^
^
l I l I l
1
EVOLUTION TINE (ORglT~
* l l I I I I I I I I I I I
2 3
0,2
e
J
(c)
~  ~ ~" k
°'2f(d)
J
>
~q~ '"~ ....
WAXINUN LEN&~ CUTOFF
(% OF HORIZONThL EXTENT)
tb"
....... ~ ' " '
~S ' ' ' '
NINIB LENGTH
(% OF HORIZONTAL BCI'ENT)
'
5
. . . . ,b . . . .
15 25 15
Fig. 6. Stationarity of ~t for Rossler attractor data (8192 points spanning 135 orbits) for the fixed evolution time program is shown
for the input parameters: a) Tau (delay time); b) evolution time between replacement steps; c) maximum length of replacement vector
length allowed; and d) minimum length of replacement vector allowed. The correct value of the positive exponent is 0.13 bits/s and is
shown by the horizontal line in these figures.
Our numerical results on noisefree model sys
tems have produced the expected results: too fre
quent replacements cause a dramatic loss of phase
space orientation, and too infrequent replacements
allow volume elements to grow overly large and
exhibit folding. For the Rossler, Lorenz, and the
BelousovZhabotinskii attractors, each of which
has a onceperorbit chaos generating mechanism,
we find that varying the evolution time in the
range ½ to 1½ orbits almost always provides stable
exponent estimates. In systems where the mecha
nism for chaos is unknown, one must cheek for
exponent stability over a wide range of evolution
times. For such systems it is perhaps wise to
employ only the variable evolution time program
or the interactive program.
There are other criteria that may affect replace
ment times for variable evolution time programs
such as avoiding regions of high phase space veloc
ity, where the density of replacement points is
likely to be small. Such features are easily in
tegrated into our programs.
In the Lorenz attractor, the separatrix between
the two lobes of the attractor is not a good place
to find a replacement dement. An element chosen
here is likely to contain points that will almost