Quantifying Uncertainty for Non-Gaussian Ensembles in Complex Systems.

Full text

Page 1

QUANTIFYING UNCERTAINTY FOR NON-GAUSSIAN
ENSEMBLES IN COMPLEX SYSTEMS∗
RAFAIL V. ABRAMOV†AND ANDREW J. MAJDA†
SIAM J. SCI. COMPUT.
Vol. 26, No. 2, pp. 411–447
c ? 2004 Society for Industrial and Applied Mathematics
Abstract. Many situations in complex systems require quantitative estimates of the lack of
information in one probability distribution relative to another. In short-term climate and weather
prediction, examples of these issues might involve a lack of information in the historical climate
record compared with an ensemble prediction, or a lack of information in a particular Gaussian en-
semble prediction strategy involving the first and second moments compared with the non-Gaussian
ensemble itself. The relative entropy is a natural way to quantify this information. Here a recently
developed mathematical theory for quantifying this lack of information is converted into a practical
algorithmic tool. The theory involves explicit estimators obtained through convex optimization, prin-
cipal predictability components, a signal/dispersion decomposition, etc. An explicit computationally
feasible family of estimators is developed here for estimating the relative entropy over a large dimen-
sional family of variables through a simple hierarchical strategy. Many facets of this computational
strategy for estimating uncertainty are applied here for ensemble predictions for two “toy” climate
models developed recently: the Galerkin truncation of the Burgers–Hopf equation and the Lorenz
’96 model.
Key words. predictability, relative entropy, ensemble predictions
AMS subject classifications. 82C, 65C, 86A10
DOI. 10.1137/S1064827503426310
1. Introduction. Complex systems with many spatial degrees of freedom arise
in environmental science in diverse contexts such as atmosphere/ocean general cir-
culation models (GCMs) for climate or weather prediction [7], pollution models, and
models for the spread of hazardous biological, chemical, or nuclear plumes. These
nonlinear models are intrinsically chaotic over many time scales with sensitive depen-
dence on initial conditions. In this paper, such models are represented discretely as a
large system of ODEs for a vector?X ∈ RNgiven by
d?X
dt
with N ? 1.
Given both the uncertainty in a deterministic initial condition,?X0, as well as the
intrinsic chaos in solutions of (1.1), it is natural to consider an ensemble of initial data
characterized by a probability density, p0(?X), satisfying p0≥ 0,
mean given by?X0, i.e.,
?
The idea is to utilize the ensemble of solutions of (1.1) drawn from the initial data
p0(?X) to quantify the uncertainty and measure the confidence interval and predictive
=?F(?X,t),(1.1)
?p0= 1 and with
RN
?Xp0(?X)d?X =?X0. (1.2)
∗Received by the editors April 23, 2003; accepted for publication (in revised form) March 25, 2004;
published electronically December 22, 2004. The work of the second author was partially supported
by National Science Foundation grant DMS-9972865, Office of Naval Research grant N00014-96-1-
0043, and National Science Foundation CMG Research grant DMS-0222133. The work of the first
author was supported as a postdoctoral fellow through these research grants.
http://www.siam.org/journals/sisc/26-2/42631.html
†Courant Institute of Mathematical Sciences and Center for Atmosphere/Ocean Science, New
York University, New York, NY 10012 (abramov@cims.nyu.edu, jonjon@cims.nyu.edu).
411

Page 2

412
RAFAIL V. ABRAMOV AND ANDREW J. MAJDA
power of the deterministic solution beginning at?X0. Theoretically, this is straight-
forward because it is easy to establish that the initial probability density, p0(?X),
evolves to a new probability density, pt(?X), at later times which satisfies the Liouville
equation,
∂
∂tpt(?X) + div? X(?Fpt) = 0,
pt(?X)
???
t=0= p0(?X).
(1.3)
The Liouville equation is a linear PDE in a very large spatial dimension, RN, with
N ? 1, and is impractical to solve directly. In practice, instead a finite ensemble
of individual solutions of (1.1), {?Xr
initial distribution, p0(?X). Then pt(?X) is approximated by the empirical probability
distribution for ensemble prediction
t, 1 ≤ r ≤ R}, is constructed by sampling the
pE
t=1
R
R
?
r=1
δ? Xr
t(?X),(1.4)
where δ? X0(?X) is the Dirac delta measure at?X0. The probability density in (1.4) is
an explicit solution of the Liouville equation in (1.3).
The empirical ensemble prediction in (1.4) is the central probability measure of
interest in this paper. How can one quantify and estimate the uncertainty in such
a prediction? In the remainder of this introduction, some of the important scientific
issues associated with empirical ensemble prediction are discussed, as is the potential
use of information theory [6] in quantifying many aspects of uncertainty. The rest
of the paper focuses on demonstrating the fashion in which a recently developed
mathematical framework [16] can be converted into a practical tool for assessing
uncertainty in complex systems.
The first practical issue to confront is that the empirical ensemble size, R, in
(1.4) is often very small in environmental science, on the order of R ≤ 50 for weather
prediction and short-term climate prediction in GCMs [24, 21], while, for example,
R = 500 for intermediate models for predicting El Ni˜ no [10, 11]. This is due to the
computational cost in these extremely complex systems in an operational setting re-
quiring real time prediction. Furthermore, the idealized ensemble distribution, p0(?X),
is also unknown in practice. Thus, in this paper we intentionally focus on the em-
pirical distribution, pE
t, rather than perfect predictability scenarios. In the perfect
predictability scenario, the initial distribution, p0(?X) from (1.3), is known exactly
and the approximation by an ensemble R ? 1 is the main emphasis. These studies in
idealized systems are very useful in determining the features governing predictability
in a given system [12, 9].
What are available to quantify the uncertainty from the empirical ensemble pre-
diction pE
tand readily evaluated are some of the moments,
?
r=1
with?Xα= Xα1
N
moments, where L = 2 or 4, are utilized for a judicious restricted set of variables
RN(?X −¯?X)αpE
t(?X)d?X =1
R
R
?
(?Xr
t−¯?X
r)α,0 ≤ |α| ≤ 2L,(1.5)
1...XαN
[7], |α| =?αi. Of course, in practice only the low order
?Xi,1 ≤ i ≤ M.(1.6)

Page 3

UNCERTAINTY FOR NON-GAUSSIAN ENSEMBLES
413
Some of the main important issues in quantifying the uncertainty of an ensemble
prediction are listed below.
(A) How much of a lack of information is contained in a given prediction strategy,
for example, measuring only the second moments in (1.5), compared with
pE
t?
(B) When does the ensemble prediction distribution exhibit bimodality in some
variable so that there are at least two different scenarios of significant change
predicted in a given variable for times of interest? How can this be quantified
in a computationally tractable fashion?
(C) In a given dynamical system in (1.1), which subsets of variables X1...XM
are more predictable that the others? How can this be quantified?
(D) For long-term climate prediction, when is an ensemble prediction useful at
all beyond the historical climate record? How can one estimate the lack of
information in the climate record beyond the ensemble prediction? What
features control the variation of this predictive utility in a given dynamical
system?
(E) What features characterize the rare ensemble predictions with more than
typical information beyond the climate record?
All of the above issues involve a lack of information in one probability distribution,
Π(?X), compared with another probability distribution, p(?X), over some subset of vari-
ables,?X1...XM. In information theory, this lack of information content is quantified
by the relative entropy,
?
P(p,Π) =
RMpln
?p
Π
?
(1.7)
(see [6]). Recently, in an important paper, Kleeman [12] has suggested the use of
relative entropy to address the issue in (D) for long-term climate prediction. In this
case, Π is the historical record climate distribution on a given set of variables, while
p = pE
main issues in (D) were raised earlier by Anderson and Stern [4]. In the situation
in (A), monitoring only the first and second moments of the prediction as is usually
done in practice [24, 7] leads to the Gaussian prediction, pG; so in this case P(pE
should be estimated. Recently, Roulston and Smith [22] have suggested a similar use
of information theory as a “score” or predictability measure for ensemble forecasts.
Below we show how to develop rigorous computationally feasible tests for bimodality
to address (B) through non-Gaussian estimators for the empirical prediction ensemble.
With regard to (C), for the special case when both p and Π are Gaussian distributions,
Schneider and Griffies [23] have developed the idea of principal predictability compo-
nents utilizing the entropy difference; this interesting concept has been generalized by
Majda, Kleeman, and Cai [16] to one typical practical situation where Π is Gaussian
while p is non-Gaussian with (1.7) as the more precise measure of lack of information
in Π. Finally, to address the important issue in (E) for long-term prediction as well as
the key factors that determine variability in predictive utility, Kleeman [12] has intro-
duced and applied the concept of the signal/dispersion decomposition for the special
case when both Π and p are Gaussian distributions. Recently, Kleeman, Majda, and
Timofeyev [8] have determined the controlling factors of this facet of predictability
in a simple model with statistical features of the atmosphere in a Gaussian setting.
This decomposition for suitable non-Gaussian measures, p and Π, has been developed
by Majda, Kleeman, and Cai in [16], and section 2.3 below contains a new, more
tis the empirical ensemble prediction, so P(pE
t,Π) should be estimated. The
t,pG)

Page 4

414
RAFAIL V. ABRAMOV AND ANDREW J. MAJDA
precise decomposition into a signal, dispersion, and cross-term. Both the signal and
cross-term are directly and cheaply evaluated from the moments of p. Thus, they are
readily available in determining variations in utility in a given dynamical system.
With the background discussion presented above, once one adopts the relative
entropy in (1.7) as the theoretical tool for addressing the important issues in (A)–(E),
the following practical issues arise.
(F) How can one numerically compute or approximate the relative entropy P(p,Π)
for a complex system with many degrees of freedom and a subset of variables
with M ? 1?
(G) Are there computationally feasible and mathematically rigorous strategies to
estimate P(p,Π) from below, i.e., P(p,Π) ≥ P(p∗,Π), where P(p∗,Π) can be
evaluated either analytically or by a rapid numerical procedure?
The objective of this paper is to illustrate computationally feasible strategies to ad-
dress the practical issues in (F) and (G) so that the rigorous mathematical theory
developed recently [16] can be applied to address the important practical questions
for quantifying uncertainty in (A)–(E). Section 2 contains an overview of these com-
putational strategies for estimating (1.7) as well as a brief summary of the key theo-
retical results [16]. Two different toy models for statistical features of the atmosphere
[15, 14, 17, 18] and the statistical properties of the climate distribution are introduced
in section 3 for an interesting range of parameters. These toy models are used to il-
lustrate, in a simple context, how the tools discussed in this paper can be applied in
more practical situations.
2. A rigorous computational framework for estimating predictive infor-
mation content. Here we develop a mathematically rigorous and computationally
feasible framework for estimating the relative entropy in (1.7) which we also call the
predictive utility or utility. This framework addresses the key computational issues
listed in (F) and (G) in section 1 in a straightforward fashion. The material presented
below summarizes the mathematical theory developed recently by Majda, Kleeman,
and Cai [16] in a convenient fashion for applications; the interested reader can con-
sult that paper for the rigorous proofs. In section 4 of this paper, we address all of
the issues listed in (A)–(E) in section 1 for quantifying uncertainty by applying this
computational framework to “toy” climate models.
The relative entropy in (1.7) for the probability measures p and Π has several
attractive features listed below.
(A) P(p,Π) measures the average lack of information in Π compared with p;
(B) P(p,Π) > 0 unless p ≡ Π, with P(Π,Π) ≡ 0;
(C) P(p,Π) remains unchanged through an arbitrary nonlinear change of
variables.
Often a Gaussian measure is utilized as an estimate of an ensemble prediction as in
the application in (A) in section 1. Somewhat surprisingly, in many applications in
atmospheric science [25] as well as in simplified dynamical models for geophysical flows
[19], the probability distribution for the climate of a suitable collection of variables is
essentially Gaussian. Thus, for all the diverse applications for estimating predictive
utility listed in (A)–(E) in section 1, it is interesting to consider the important special
case of studying P(p,Π) when Π is a Gaussian measure.
Gaussian distribution,
Recall that for such a
Π(X1,... ,XM) = (2π)−M/2(detC)−1/2exp
?
−1
2((?X −¯?X),C−1(?X −¯?X))
?
,(2.1)

Page 5

UNCERTAINTY FOR NON-GAUSSIAN ENSEMBLES
415
where¯?X ∈ RMis the mean,
?
XjΠ =¯ Xj,1 ≤ j ≤ M,(2.2)
and C = (Cij) is the positive definite, C > 0, symmetric, M × M covariance matrix,
?
Since the predictive utility in (1.7) is invariant under changes of variables, for a
Gaussian measure Π, defined above in (2.1)–(2.3), consider the change of variables
(Xi−¯ Xi)(Xj−¯ Xj)Π = Cij. (2.3)
?X =¯?X + C1/2?Y .
(2.4)
In (2.4), C1/2is the square root of the positive definite matrix C; the matrix C1/2is
also symmetric and positive definite, commutes with C, and satisfies C1/2C−1C1/2=
I.Since C is positive definite symmetric, there is an orthonormal basis, {? eM
called empirical orthogonal functions (EOFs) in the atmosphere/ocean community
and corresponding positive eigenvalues λ1≥ λ2≥ ··· ≥ λM> 0 so that the covariance
matrix C is diagonalized, i.e.,
C?f =
i
i=1},
?
λi? ei(?f,? ei). (2.5)
The eigenvector ? e1is called the first EOF, ? e2the second EOF, and so forth. With
(2.5), the matrix C1/2is defined by
C1/2?f =
i
?
λ1/2
i
? ei(?f,? ei).(2.6)
From (2.1) it follows that in the coordinate system defined by (2.4), the Gaussian
distribution Π has the simplified form
Π(?Y ) = (2π)−M/2e−|?Y |2/2.(2.7)
With (2.4), given p(?X), one gets the transformed distribution,
pΦ(?Y ) = p(¯?X + C1/2?Y )det(C1/2). (2.8)
Thus, without loss of generality, we assume below that the Gaussian measure Π(?X)
has the canonical form
Π(?X) = (2π)−M/2exp
?
−1
2|?X|2
?
.(2.9)
2.1. Explicit estimators for predictive utility. For simplicity in exposition,
here we assume one-dimensional probability distributions to avoid cumbersome nota-
tion. All of the results presented below extend immediately to multivariable distribu-
tions with more complex notation. We assume a given prior distribution Π(λ), for a
scalar variable which is centered so that it has zero mean,
?
λΠ(λ)dλ = 0.(2.10)