ChapterPDF Available

Hydrologic Persistence and The Hurst Phenomenon

Authors:

Abstract

Unlike common random series like those observed for example in games of chance, hydrological (and other geophysical) time series have some structure, that is, consecutive values of hydrological time series are dependent to each other. A special kind of dependence observed at large timescales has been discovered by Hurst half a century ago and has been known by several names such as long-range dependence, long-term persistence or simply the Hurst phenomenon. Since then, it has been verified that this behaviour is almost omnipresent in several processes in nature (e.g. hydrology), technology (e.g. computer networks) and society (e.g. economics). The consequences of this behaviour are very significant, as it increases dramatically the uncertainty of the related processes. However, even today its importance and its consequences are not widely understood or are ignored, its nature is regarded as difficult to understand, and its reproduction in hydrological simulation is considered a hard task or not necessary. It is shown in this article that the Hurst phenomenon can have easy explanation and easy stochastic representation, and that simple algorithms can generate time series exhibiting long-term persistence.
The Encyclopedia of Water
Hydrological Persistence and the Hurst Phenomenon (SW-434)
Demetris Koutsoyiannis
Department of Water Resources, Faculty of Civil Engineering, National Technical University, Athens,
Heroon Polytechneiou 5, GR-157 80 Zographou, Greece (dk@itia.ntua.gr)
Abstract Unlike common random series like those observed for example in games of chance,
hydrological (and other geophysical) time series have some structure, that is, consecutive
values of hydrological time series are dependent to each other. A special kind of dependence
observed at large timescales has been discovered by Hurst half a century ago and has been
known by several names such as long-range dependence, long-term persistence or simply the
Hurst phenomenon. Since then, it has been verified that this behaviour is almost omnipresent
in several processes in nature (e.g. hydrology), technology (e.g. computer networks) and
society (e.g. economics). The consequences of this behaviour are very significant, as it
increases dramatically the uncertainty of the related processes. However, even today its
importance and its consequences are not widely understood or are ignored, its nature is
regarded as difficult to understand, and its reproduction in hydrological simulation is
considered a hard task or not necessary. It is shown in this article that the Hurst phenomenon
can have easy explanation and easy stochastic representation, and that simple algorithms can
generate time series exhibiting long-term persistence.
Keywords climate change; fractional Gaussian noise; Hurst phenomenon; hydrological
persistence; hydrological statistics; hydrological estimation; hydrological persistence;
hydrological prediction; long-range dependence; scaling; uncertainty
Introduction
Unlike common random series like those observed for example in games of chance (dice,
roulette, etc.), hydrological (and other geophysical) time series have some structure, that is,
consecutive values of hydrological time series are dependent to each other. It is easy to
understand that, for instance in a monthly river flow series, a month with very high flow is
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
2
likely to be followed by a month with high flow, too. Since the river flow is interrelated to the
groundwater storage, the high flow indicates that the groundwater storage will be high too,
and given that the groundwater flow is a slow process, it is expected that its contribution to
the river flow in the next month will be significant. This explains the dependence of
consecutive values of hydrological time series, which has been known as short-range (or
short-term) dependence, persistence, or memory.
Interestingly, however, there is another kind of dependence observed at larger timescales,
known as long-range (or long-term) dependence, persistence, or memory. This has been
discovered by Hurst (1951), while investigating the discharge time series of the Nile River in
the framework of the design of the Aswan High Dam, and found in many other hydrological
and geophysical time series. This behaviour is essentially the tendency of wet years to cluster
into multi-year wet periods or of dry years to cluster into multi-year drought periods. The
terms ‘Hurst phenomenon’ and ‘Joseph effect’ (due to Mandelbrot, 1977, from the biblical
story of the ‘seven years of great abundance’ and the ‘seven years of famine’) have been used
as alternative names for the same behaviour. Since its original discovery, the Hurst
phenomenon has been verified in several environmental quantities such as (to mention a few
of the more recent studies) in wind power (Haslett and Raftery, 1989); global or point mean
temperatures (Bloomfield, 1992; Koutsoyiannis, 2003a, b); flows of several rivers such as
Nile (Eltahir, 1996; Koutsoyiannis, 2002), Warta, Poland (Radziejewski and Kundzewicz,
1997), Boeoticos Kephisos, Greece (Koutsoyiannis, 2003a), and Nemunas, Lithuania
(Sakalauskienė, 2003); inflows of Lake Maggiore, Italy (Montanari et al., 1997); indexes of
North Atlantic Oscillation (Stephenson et al., 2000); and tree-ring widths, which are
indicators of past climate (Koutsoyiannis, 2002). In addition, the Hurst phenomenon has
gained new interest today due to its relation to climate changes (e.g. Evans, 1996;
Koutsoyiannis, 2003a, c; Koutsoyiannis and Efstratiadis, 2004).
The possible explanation of the long-term persistence must be different from that of the
short-term persistence discussed above. This will be discussed later. However, its existence is
easy to observe even in a time series plot, provided that the time series is long enough. For
example, in Figure 1 (up) we have plotted one of the most well-studied time series, that of the
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
3
annual minimum water level of the Nile river for the years 622 to 1284 A.D. (663
observations), measured at the Roda Nilometer near Cairo (Toussoun, 1925, p. 366-385;
Beran, 1994). In addition to the plot of the annual data values versus time, the 5-year and 25-
year averages are also plotted versus time. For comparison we have also plotted in the lower
panel of Figure 1 a series of white noise (consecutive independent identically distributed
random variates) with statistics same with those of the Nilometer data series. We can observe
that the fluctuations of the aggregated processes, especially for the 25-year average, are much
greater in the real world time series than in the white noise series. Thus, the existence of
fluctuations in a time series at large scales distinguishes it from random noise.
800
900
1000
1100
1200
1300
1400
1500
600 700 800 900 1000 1100 1200 1300
Year
Nilometer inde
x
Annual value Average, 5 years Average, 25 years
800
900
1000
1100
1200
1300
1400
1
5
00
600 700 800 900 1000 1100 1200 1300
Ye a r
Annual value Average, 5 years Average, 25 years
Figure 1 (Up) Plot of the Nilometer series indicating the annual minimum water level of the Nile River for the
years 622 to 1284 A.D. (663 years); (down) a white noise series with same mean and standard deviation, for
comparison.
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
4
Stochastic representation of the Hurst phenomenon
The quantification of the long-term persistence is better expressed mathematically using the
theory of stochastic processes. Let X
i
denote a stochastic representation of a
hydrometeorological process with i = 1, 2, …, denoting discrete time with time step or scale
which for the purposes of this article is annual or multi-annual. It is assumed that the process
is stationary, a property that does not hinder to exhibit multiple scale variability. The
stationarity assumption implies that its statistics are not functions of time. Therefore, we can
denote its statistics without reference to time, i.e. its mean as µ := E[X
i
], its autocovariance as
γ
j
:= Cov[X
i
, X
i + j
] (j = 0, ±1, ±2, …), its autocorrelation ρ
j
:= Corr[X
i
, X
i + j
] = γ
j
/ γ
0
, and its
standard deviation σ := γ
0
. Further, we assume ergodicity, so that these statistics can be
estimated from a unique time series substituting time averages for expected values.
Let k be a positive integer that represents a timescale larger than the basic timescale of the
process X
i
. The aggregated stochastic process on that timescale is denoted as
Z
(k)
i
:=
l = (i – 1) k + 1
i k
X
l
(1)
The statistical characteristics of Z
(k)
i
for any timescale k can be derived from those of X
i
. For
example, the mean is
E[Z
(k)
i
] = k µ (2)
whilst the variance and autocovariance (or autocorrelation) depends on the specific structure
of γ
j
(or ρ
j
). In a process that exhibits the Hurst phenomenon, the variance γ
(k)
0
for timescale k
is related to that of the basic scale γ
0
by
γ
(k)
0
:= Var[Z
(k)
i
] = k
2H
γ
0
(3)
where H is a constant known as the Hurst coefficient with values in the interval (0.5, 1). The
value H = 0.5 corresponds to random noise whereas values in the interval (0, 0.5) are
mathematically possible but without interest in hydrology. Consequently, the standard
deviation is a power law of the scale or level of aggregation k with exponent H, i.e.,
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
5
σ
(k)
:=
(
γ
(k)
0
)
1/2
= k
H
σ (4)
This simple power law can be easily used both for detecting whether a time series exhibits
the Hurst phenomenon and for determining the coefficient H, which is a measure of the long-
term persistence. Equation (4) calls for a double logarithmic plot of standard deviation σ
(k)
of
the aggregated process Z
(k)
i
versus timescale k. In such a plot, called the aggregated standard
deviation plot, the Hurst behaviour is manifested as a straight line arrangement of points
corresponding to different timescales, whose slope is the Hurst coefficient. An example is
depicted in Figure 2 for the Nilometer series of Figure 1. Clearly, the plot of the empirical
estimates of standard deviation is almost a straight line on the logarithmic diagram with slope
0.85. For comparison we have also plotted the theoretical curve for the white noise with slope
equal to 0.5, significantly departing from the historical data.
1.8
2
2.2
2.4
2.6
2.8
3
3.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4
Lo
g
(
Scale
)
Log(Standard deviation)
Empirical
White noise
FGN
H= 0.85
1.8
2
2.2
2.4
2.6
2.8
3
3.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4
Lo
g
(
Scale
)
Log(Standard deviation)
Empirical
White noise
FGN
1.8
2
2.2
2.4
2.6
2.8
3
3.2
0 0.2 0.4 0.6 0.8 1 1.2 1.4
Lo
g
(
Scale
)
Log(Standard deviation)
Empirical
White noise
FGN
H= 0.85
Figure 2 Aggregated standard deviation plot of the Nilometer series.
By virtue of (3), it can be shown that the autocorrelation function, for any aggregated
timescale k, is independent of k and given by
ρ
(k)
j
= ρ
j
= (1 / 2) (|j + 1|
2H
+ |j – 1|
2H
) – |j|
2H
) H (2 H – 1) |j|
2H – 2
(5)
which shows that autocorrelation is a power function of lag. Consequently, the autocovariance
γ
(k)
j
= γ
(k)
0
ρ
(k)
j
is a power law of both the scale k (with exponent 2H) and the lag j (with
exponent 2H – 2).
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
6
The power spectrum of the process
s
(k)
γ
(ω) := 2
j = –
γ
(k)
j
cos (2
π
j
ω) (6)
is given approximately by
s
(k)
γ
(ω) 4 (1 – H) γ
(k)
0
(2 ω)
1 – 2 H
(7)
which is a power law of both the scale k (with exponent 2H) and the frequency ω (with
exponent 1 – 2H).
The power law equations (5) and (7) can be used, in addition or alternatively to (3) and (4),
to detect the Hurst behaviour of a time series. It should be mentioned here that Hurst’s (1951)
original formulation to detect this behaviour was based on another quantity, the so called
rescaled range, which corresponds to the cumulated process of inflow minus outflow of a
hypothetical infinite reservoir.
Equations (3)-(7) describe essentially second order properties of the process Z
(k)
i
. A
generalization is possible, if we assume that the process of interest exhibits scale invariant
properties in its (finite dimensional joint) distribution function, i.e.,
(Z
(k)
i
k µ) =
d
k
l
H
(Z
(l)
j
l µ) (8)
where the symbol =
d
stands for equality in distribution. In this case, (3) can be obtained from
(8) setting i = j = l = 1 and taking the variance of both sides. Equation (8) defines X
i
and Z
(k)
i
as stationary increments of a self-similar process. If, in addition, X
i
(and hence Z
(k)
i
) follows
the normal distribution, then X
i
(and Z
(k)
i
) is called fractional Gaussian noise (FGN;
Mandelbrot, 1965). Our interest here includes processes that may be not Gaussian, so we will
limit the scaling property (8) to second-order properties only and call the related process a
simple scaling signal (SSS).
Physical explanations of the Hurst phenomenon
As described in the Introduction, the concept of short-term persistence in hydrological
processes is ease to explain, whereas long-term persistence and the Hurst phenomenon are
more difficult to understand. Mesa and Poveda (1993) classify the Hurst phenomenon as one
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
7
of the most important unsolved problems in hydrology and state that “something quite
dramatic must be happening from a physical point of view”. However, several explanations
have been proposed. These can be classified in two categories: physically-based and
conceptual.
Klemeš (1974) proposed an explanation that may be classified in the first category.
According to this, the Hurst behaviour of hydrological records can be explained by
representing the hydrological cycle by a ‘circular cascade of semi-infinite storage reservoirs’
where the output from one reservoir constitutes an essential part of input into the next. He
showed that, even with an originally uncorrelated Gaussian forcing, the outputs grew
progressively more Hurst-like with an increasing complexity of the system, e.g. with the
number of reservoirs in the cascade. Another example of such hydrological system was
suggested in Klemeš (1978).
Beran (1994, pp. 16-20) describes two physically-based model types that lead to system
evolution (in time or space) with long-range dependence. The first model type applies to
critical phenomena in nature such as phase transition (transition from liquid to gaseous phase,
or spontaneous magnetization of ferromagnetic substances). For some critical system
temperature the correlation of the system state at any two points decays slowly to zero, so the
correlation in space can be represented by (5). The second type is related to models based on
stochastic partial differential equations, which, under certain conditions, result in solutions
with long-range dependence. These models provide sound links of long-range dependence
with physics but are very complex.
A simple model of this category was studied by Koutsoyiannis (2003b). This model
assumes a system with purely deterministic dynamics in discrete time, which however results
in time series with irregular appearance exhibiting the Hurst phenomenon. The system
dynamics is based on the simple map
x
i
= g(x
i – 1
; α) := (2 – α) min (x
i – 1
, 1 – x
i – 1
)
1 – α min (x
i – 1
, 1 – x
i – 1
) (9)
where x
i
is the system state, assumed to be scalar, at time i and α < 2 is a parameter. This map,
known as generalized tent map, has been used in the study of dynamical systems. For
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
8
example, the map approximates the relation between successive maxima in the variable x(t)
from the Lorenz equations that describe climatic dynamics (Lasota and Mackey, 1994, p.
150). Koutsoyiannis (2003b) demonstrated that this model can describe a system subject to
the combined action of a positive and a negative feedback. If the parameter α is assumed to
vary in time following the same map, i.e., α
i
= g(α
i – 1
; λ), then we obtain the double tent map
u
i
= G(u
i – 1
, α
i – 1
; κ, λ) := g(u
i – 1
; κ α
i – 1
) (10)
Both parameters κ and λ should be < 2 whereas the domain of u
t
is the interval [0, 1]. If the
system domain is the entire line of real numbers, we can apply an additional transformation to
shift from [0, 1] to (–, ), e.g.,
x
t
= ln[u
t
/ (1 – u
t
)] (11)
0
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1
λ
κ
0
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1
λ
κ
Long-range
dependence
(0.5 < H< 1)
Short-range
dependence
(H= 0.5)
Runaway
area
(H1)
Figure 3 Schematic of the general behaviour of the double tent map in terms of the ranges of its parameters κ
and λ.
The model behaviour with respect to parameters κ and λ is depicted in Figure 3. We
observe that for small values of the parameter κ, the time series synthesized by the model
exhibit short-range dependence with Hurst coefficient around 0.5. For large values of κ, the
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
9
model yields a runaway behaviour. However, in between the two non-interesting areas, there
is an area of parameter values, shaded in Figure 3, in which the resulting time series exhibit
the long-range dependence. All three types of behaviour are observed for negative values of λ
(not shown in Figure 3) whereas for λ > 1 only the runaway behaviour is observed regardless
of the value of κ.
The second category, labelled here ‘conceptual’, does not aim at explaining the physical
mechanism leading to Hurst behaviour of historical records of some natural or other
processes, but examines different stochastic mechanisms that might produce realizations
resembling the patterns of the observed empirical time series. For example, Klemeš (1974)
analyzed several variants of the ‘changing mean’ mechanism which assumes that the mean of
the process is not a constant determined by the arithmetic mean of the record, but varies
through time. Specifically, he performed numerical experiments with the following Gaussian
random processes:
1. a process with the mean alternating periodically between two values after constant
time intervals called ‘epochs’;
2. a process with a monotonic linear trend in the mean, throughout the entire series
length;
3. a process similar to 1 but with epoch lengths taking two different values with
probabilities p and 1 – p; and
4. a process with a Gaussian-distributed mean randomly varying from epoch to epoch,
the epoch length also varying randomly and following either a uniform, exponential,
or (single parameter) Pareto distribution.
The processes behaved increasingly Hurst-like as their structure changed from 1 to 4. This
behaviour was most influenced by the distribution of epoch lengths, while the distribution of
the mean itself had little effect.
The effect of periodical patterns, which are extensions of those of model 1, have been
thoroughly studied by Montanari et al. (1999), who however note that such patterns are
unusual in real data. The effect of monotonic deterministic trends, which are extensions of
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
10
model 2, was studied by Bhattachara et al. (1983), who showed mathematically that a trend of
the form f(t) = c (m + t)
H – 1
with t denoting time, c a nonzero constant, m a positive constant
and H a constant in the interval (0.5, 1), results in time series exhibiting the Hurst
phenomenon with Hurst coefficient precisely equal to H. We may note, however, that this
kind of nonstationarity with a monotonic deterministic trend spanning the whole length of a
time series can hardly represent a long time series of real data, even though in short time
series seem to be realistic. For example, to refer to the Nilometer series of Figure 1, if one had
available only the data of the period 700-800 one would detect a ‘deterministic’ falling trend
of the Nile level; similarly, one would detect a regular rising trend of the Nile level between
the years 1000-1100. However, the complete picture of the series suggests that these trends
are parts of large-scale random fluctuations rather than deterministic trends.
Based on this observation, Koutsoyiannis (2002) proposed a conceptual explanation, which
can be essentially regarded as an extension of Klemeš’s model 4 and is also similar to other
proposed conceptual models as it will be discussed later. More specifically, Koutsoyiannis
(2002) demonstrated that superimposition of three processes with short-term persistence
results in a composite process that is practically indistinguishable from an SSS process.
This demonstration is reproduced here in Figures 4-5. It starts assuming a Markovian
process U
i
, like the one graphically demonstrated in Figure 4(a), with mean µ := E[U
i
],
variance γ
0
and lag one autocorrelation coefficient ρ = 0.20. The specific form of this process
is an AR(1) one, i.e. U
i
:= ρ U
i – 1
+ E
i
, where E
i
is white noise, and its autocorrelation is
Corr[U
i
, U
i + j
] = ρ
j
(12)
The autocorrelation function is shown in Figure 5(a) along with the autocorrelation function
of SSS with same lag one autocorrelation coefficient (0.20). We observe the large difference
of the two autocorrelation functions: that of the Markovian process practically vanishes off at
lag 4 whereas that of SSS has positive values for lags as high as 100.
In the next step, a second process V
i
is constructed by superimposing another Markovian
process M
i
, i.e.,
V
i
= U
i
+ M
i
µ (13)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
11
Here the process M
i
is constructed in a different way, rather than using the AR(1) model, yet
without loosing its Markovian behaviour, so that its autocorrelation is
Corr[M
i
, M
i + j
] = φ
j
(14)
for φ > ρ. More specifically, a continuous time process M (see explanatory sketch on Figure
4(b)) with the following properties was assumed: (a) it has mean µ and some variance Var[M];
(b) any realization m of M lasts N years and is independent from previous realizations; (c) N is
a random variable exponentially distributed with mean λ = –1 / ln φ. (This means that N can
take non-integer values). In other words, M takes a value m
(1)
that lasts n
1
years, then it
changes to a value m
(2)
that lasts n
2
years, etc. (where the values m
(1)
, m
(2)
, … can be generated
from any distribution). The exponential distribution of N indicates that the points of change
are random points in time. If we denote M
i
the instance of the M process at discrete time i, it
can be shown that indeed M
i
is Markovian with lag one autocorrelation φ. This way of
constructing M
i
allows us to interpret V
i
as a process similar to U
i
but with mean M
i
that varies
randomly in time (rather than being constant, µ) shifting among randomly determined values
m
(1)
, m
(2), …,
each lasting a random time period with average λ. It can be easily shown from
(13) that the autocorrelation of V
i
for lag j is
Corr[V
i
, V
i + j
] = (1 – c)ρ
j
+ c φ
j
(15)
where c := Var[M
i
] / (Var[M
i
] + Var[U
i
]). Setting for instance λ = 7.5 years (φ = 0.875) and c
= 0.146 we get the autocorrelation function shown in Figure 5(b), which has departed from
the AR(1) autocorrelation and approached the SSS autocorrelation.
In a third step, another process W
i
is constructed by superimposing V
i
with a third
Markovian process P
i
, i.e.,
W
i
= V
i
+ P
i
µ = U
i
+ M
i
+ P
i
– 2 µ (16)
P
i
is constructed in a way identical to that of M
i
, but with lag one autocorrelation ξ > φ, so that
the mean time between changes of the value of P is ν = –1 / ln ξ. Working as in the previous
step we find
Corr[W
i
, W
i + j
] = (1 – c
1
c
2
)ρ
j
+ c
1
φ
j
+ c
2
ξ
j
(17)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
12
where c
1
and c
2
are positive constants (with c
1
+ c
2
< 1). Setting for instance λ = 7.5 years (φ =
0.875), ν = 200 years (ξ = 0.995), c
1
= 0.146 and c
2
= 0.036 we get the autocorrelation
function shown in Figure 5(c), which has now become almost indistinguishable from the SSS
autocorrelation for time lags from 1 to 1000.
Tim e, i –>
Value, u
i
–>
Small sc ale (annual) random fluc tuat ion
Mean
(a)
Tim e, i –>
Value, v
i
–>
Small sc ale (annual) random fluc tuat ion
Medium s cal e random fluct uation
Mean
(b)
Tim e, i –>
Value, w
i
–>
Small sc ale (annual) random fluc tuat ion
Medium s cal e random fluct uation
Large scale random fluct uation
Mean
(c)
Figure 4 Illustrative sketch for multiple timescale random fluctuations of a process that can explain the Hurst
phenomenon: (a) a time series from a Markovian process with constant mean; (b) the same time series
superimposed to a randomly fluctuating mean on a medium timescale; (c) the same time series further
superimposed to a randomly fluctuating mean on a large timescale (from Koutsoyiannis, 2002).
In conclusion, a Markovian underlying process can result in a nearly SSS process if there
occur random fluctuations of the mean of the process on two different scales (e.g., 7.5 and
200 years), yet the resulting composite process being stationary. If we consider that
fluctuations occur on a greater number of timescales, the degree of approximation of the
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
13
composite process to the SSS process will be even better and can cover time lags greater than
1000 (although the extension to lags beyond 1000 may not have any practical interest in
hydrology). In conclusion, the irregular changes of climate that, according to National
Research Council (1991, p. 21), occur on all timescales can be responsible for, and explain,
the Hurst phenomenon.
0
0.1
0.2
0.3
0.4
1 10 100 1000
Autocorrelation, ρ
j
Proc ess U (= AR(1))
SSS
(a)
0
0.1
0.2
0.3
0.4
1 10 100 1000
Autocorrelation, ρ
j
Proces s V
SSS
AR(1)
(b)
0
0.1
0.2
0.3
0.4
1 10 100 1000
Lag, j
Autocorrelation, ρ
j
Proces s W
SSS
AR(1)
(c)
Figure 5 Plots of the example autocorrelation functions of (a) the Markovian process U with constant mean; (b)
the process U superimposed to a randomly fluctuating mean on a medium timescale (process V); (c) the process
V further superimposed to a randomly fluctuating mean on a large timescale (process W). The superimposition of
fluctuating means increases the lag one autocorrelation (from ρ
1
= 0.20 for U to ρ
1
= 0.30 and 0.33 for V and W
respectively) and also shifts the autocorrelation function from the AR(1) shape (also plotted in all three panels)
towards the SSS shape (also shown in all three panels) (from Koutsoyiannis, 2002).
This demonstration, in fact bridges several ideas that had been proposed to explain the
Hurst phenomenon, rather than being a novel explanation. As already discussed, it is similar
with Klemeš’s model 4, except for the setting of multiple timescales of fluctuation of mean
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
14
and the emphasis on the stationarity of the composite process. Here, it should be mentioned
that Klemeš referred to all his ‘changing mean’ models as models with nonstationarity in their
mean, even though this is strictly true only for models 1 and 2. While he did point out that his
final models in group 4 were in fact stationary, and that he kept the term ‘nonstationary’ for
all changes in the mean to communicate the fact (elaborated in more detail in Klemeš, 1976)
that one cannot tell the difference from the pattern of a single ‘nonstationary-looking’ time
series (which even a stationary model is designed to mimic), his explanation has sometimes
been missed and lead to a misconception about his work by some authors (including this one,
who expresses his apology).
The idea of irregular sporadic changes in the mean of the process appeared also in Salas
and Boes (1980), but not in connection with SSS and not in the setting of multiple timescales.
The idea of composite random processes with two timescales of fluctuation appeared in
Vanmarcke (1983, p. 225). The idea of an explanation of the Hurst phenomenon as a mixture
of scales appears in Mesa and Poveda (1993). The idea of representing SSS as an aggregation
of short-memory processes is the principle of the well known fast fractional Gaussian noise
algorithm (FFGN, Mandelbrot, 1971) and is also studied, as a possible physical explanation of
the Hurst phenomenon, by Beran (1994, p. 14). The difference of the above described
explanation is the aggregation of only three short-memory processes.
Importance of the Hurst phenomenon
The presence of the Hurst phenomenon increases dramatically the uncertainty of climatic and
hydrological processes. If such a process were random and our information on this was based
on a sample of size n, then the uncertainty on the long term, which can be expressed in terms
of the variance of the estimator of the mean, X
, would be:
var[X
] = σ
2
n (18)
This offers good approximation for a process with short-term persistence, as well, but it is not
valid for a process with long-term persistence. Instead, the following relation holds
(Adenstedt, 1974; Beran, 1994, p. 54; Koutsoyiannis, 2003a):
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
15
var[X
] = σ
2
n
2–2H
(19)
The difference between equations (18) and (19) becomes very significant for large values of
H. For example, in a time series of n = 100 years of observations and standard deviation σ,
according to the classical statistics (equation (18)), the standard estimation error, i.e. the
square root of var[X
], is σ/10. However, for H = 0.8 the correct standard error, as given by
equation (19), is σ/2.5, i.e. four times larger. To have an estimation error equal to σ/10, the
required length of the time series would be 100 000 years! Obviously, this dramatic difference
induces substantial differences in other common statistics as well (Koutsoyiannis, 2003a).
A demonstration of the difference in estimations related to climate is given in Figure 6.
Here, a long climatic time series (992 years) was used, which represents the Northern
Hemisphere temperature anomalies with reference to 1961–1990 mean (Figure 8, up) This
series was constructed using temperature sensitive palaeoclimatic multi-proxy data from 10
sites worldwide that include tree rings, ice cores, corals, and historical documents (Jones et
al., 1998a, b). The time series was studied in relation to the Hurst phenomenon by
Koutsoyiannis (2003a) and it was found that the estimate of the Hurst coefficient is 0.88. In
the upper panel of Figure 6 the point estimates and the 99% confidence limits of the quantiles
of the temperature anomalies have been plotted for probability of nonexceedence, u, ranging
from 1 to 99%, assuming a normal distribution, as verified from the time series, and using the
classical statistical estimators. This is done for two timescales, the basic one (k = 1) that
represents the annual variation of temperature anomaly, and the 30-year timescale, which
typically is assumed to be sufficient to smooth out the annual variations and provide values
representative of the climate. (For the latter, the averaged rather than aggregated time series,
i.e. z
i(30)
/30, has been used.)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
16
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Probability of nonexceedence
Temperature anomaly
Point estimates , annual
99% confidenc e limits , annual
Point estimates , 30-y ear average
99% confidenc e limits , 30-year average
.01 .02 .05 .1 .2 .5 .8 .9 .95 .98.99
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Probability of nonexceedence
Temperature anomaly
Point estimates, annual
99% confidenc e lim its , annual
Point estimates, 30-year average
99% confidenc e lim its , 30-y ear average
.01 .02 .05 .1 .2 .5 .8 .9 .95 .98.99
Figure 6 Point estimates of quantiles and 99% confidence limits thereof at the basic timescale (annual values, k
= 1) and the 30-year timescale (30-year averages, k = 30), for the Jones’s time series of the Northern Hemisphere
temperature anomalies: (up) using classical statistics; (down) using adapted statistics.
If classical statistics is used (Figure 6, upper panel), then it is observed that, due to the
large length of the series, the confidence band is very narrow and the point estimates for the
basic and the aggregated timescale differ significantly. The variability of climate, as
expressed by the distribution of the average at the 30-year timescale, is very low, despite the
much higher variability at the annual scale. This justifies the saying “Climate is what you
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
17
expect, weather is what you get”. Things change dramatically, if the statistics based on the
hypothesis of long-term persistence (Koutsoyiannis, 2003a) are used with H = 0.88. This is
depicted in the lower panel of Figure 6, where it is observed that the variation of the 30-year
average is only slightly lower than that of the annual values and the confidence band has
dramatically widened for both timescales. This could be expressed by paraphrasing the above
proverb to read “Weather is what you get, climate is what you get – if you keep expecting for
many years”. The consequences of these differences in estimating the climatic uncertainty due
to natural variability are obviously very significant.
The consequences in the water resources engineering and management are even more
significant. Particularly, since the notion of climate implies long timescales, it is to be
expected that the practical importance of the Hurst phenomenon increases in projects whose
operation cycles span across long periods of time. As a typical example may serve large
reservoirs with multi-year flow regulation (Klemeš et al., 1981; see also the entry SW-776,
Reliability Concepts in Reservoir Design). For small-to-middle range reservoirs it is generally
regarded that the effect of the Hurst phenomenon appears to be within the margin of error of
hydrological data used for their design and operation. However, even in hydrosystems with
small reservoirs or no reservoirs at all, as it becomes obvious from the above discussion, the
effect on the Hurst phenomenon is significant if the uncertainty (not only the expected value)
of water availability is to be assessed.
Simple algorithms to generate time series respecting the Hurst phenomenon
Several algorithms have been developed to generate time series that respect the Hurst
phenomenon. Among these, we discuss here the simplest ones that can be applied even in a
spreadsheet environment. These are based on the above-discussed properties of SSS and can
be used to provide approximations of SSS good for practical hydrological purposes.
A first, rather “quick and dirty” algorithm can be very easily formulated based on the
deterministic double tent map (equations (10) and (11)). The problem with the resulting time
series is that consecutive generated values are too regularly and smoothly related. This can be
avoided by discarding some of the generated values x
i
and holding only the values x
ν j
, for
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
18
some ν > 1 and for j = 1, 2, …, n, where n is the required series length. Figure 7 depicts the
attained Hurst coefficient in a time series generated from the double tent map (equation (10))
either untransformed or transformed (equation (11)) for ν = 1 and 4. This figure can serve as a
tool to estimate the parameter κ required to achieve a certain Hurst coefficient H (assuming λ
= 0.001). A time series so generated can then be transformed linearly to acquire the required
mean and standard deviation. By appropriately choosing the initial values α
0
and u
0
one can
obtain a time series that can have a presumed general shape; this needs a random search
optimization technique to be applied. An example of the application of this algorithm to the
Jones data set already discussed above is depicted in Figure 8.
ν = 1
ν = 4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
κ
H
Transformed
Untransformed
Figure 7 Hurst coefficient of a time series generated from the double tent map (equations (10) and (11)) for
parameter values λ = 0.001, κ ranging from 0 to 2 and ν = 1 and 4. For κ approaching 2 the double tent map has a
runaway behaviour.
As we saw earlier, the weighted sum of three exponential functions of the time lag
(equation (17)) can give an acceptable approximation of the SSS autocorrelation function on
the basic timescale. This observation leads to an easy algorithm to generate SSS. The
following equations (from Koutsoyiannis, 2002) can be used to estimate the parameters ρ, φ
and ξ:
ρ = 1.52 (H – 0.5)
1.32
, φ = 0.953 – 7.69 (1 – H)
3.85
(20)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
19
ξ =
0.932 + 0.087 HH 0.76
0.993 + 0.007 HH > 0.76
(21)
-1.2
-0.6
0
0.6
1000 1200 1400 1600 1800 2000
Year A.D.
Temperature anomaly (
o
C)
Historical data series, annual scale
Historical data series, 50-
y
ear movin
g
avera
g
e
-1.2
-0.6
0
0.6
1000 1200 1400 1600 1800 2000
Year A.D.
Temperature anomaly (
o
C)
Synthetic data series, annual scale
S
y
nthetic data series, 50-
y
ear movin
g
avera
g
e
-1.2
-0.6
0
0.6
1000 1200 1400 1600 1800 2000
Year A.D.
Temperature anomaly (
o
C)
Historical data series, 50-year moving averag
e
S
y
nthetic data series, 50-
y
ear movin
g
avera
g
e
Figure 8 (Up) Plot of the Jones data series indicating the North Hemisphere temperature anomalies with
reference to 1961–1990 mean; (middle) a synthetic time series generated by the double tent map fitted to the
Jones data set and assuming ν = 4; (down) comparison of the synthetic and original time series in terms of their
50-year moving averages.
The remaining parameters c
1
and c
2
can be then estimated such that the approximate
autocorrelation function (17) match the exact function (5) for two lags, e.g. lags 1 and 100.
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
20
(Their values are obtained by solving two linear equations). Comparison plots of approximate
autocorrelation functions based on equations (17) and (20)-(21) versus the exact SSS
autocorrelation functions (equation (5)) for various values of the Hurst exponent H are shown
in Figure 9. Equations (16) and (17) may be interpreted as representing the sum of three
independent AR(1) processes, with lag one correlation coefficients ρ, φ, and ξ, and variances
(1 – c
1
c
2
)
γ
0
, c
1
γ
0
, and c
2
γ
0
, respectively. Thus, the generation algorithm is as simple as the
generation of three AR(1) series and their addition.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1 10 100 1000
Lag, j
Autoc orrelation, ρ
j
Exact
Approximation
H = 0.6
H = 0.7 H= 0.8
H= 0.9
Figure 9 Approximate autocorrelation functions based on equations (17) and (20)-(21) vs. the exact SSS
autocorrelation functions (equation (5)) for various values of the Hurst exponent H (from Koutsoyiannis, 2002).
The simple expressions of the statistics of the aggregated SSS process make possible a
disaggregation approach for generating SSS (Koutsoyiannis, 2002). Specifically, let us
assume that the desired length n of the synthetic series to be generated is 2
m
where m is an
integer (e.g., n = 2, 4, 8, , …); if not, we can increase n to the next power of 2 and then
discard the redundant generated items. We first generate the single value of Z
(n)
1
knowing its
variance n
2H
γ
0
(from (3)). Then we disaggregate Z
(n)
1
into two variables on the timescale n / 2,
i.e. Z
(n / 2)
1
and Z
(n / 2)
2
and we proceed this way until the series Z
(1)
1
X
1
, …, Z
(1)
n
X
n
is generated
(see explanatory sketch in Figure 10).
We consider the generation step in which we disaggregate the higher-level amount Z
(k)
i
(1 <
i < n / k) into two lower-level amounts Z
(k / 2)
2 i – 1
and Z
(k / 2)
2 i
such that
Z
(k / 2)
2 i – 1
+ Z
(k / 2)
2 i
= Z
(k)
i
(22)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
21
Thus, it suffices to generate Z
(k / 2)
2 i – 1
and then obtain Z
(k / 2)
2 i
from (22). At this generation step we
have available the already generated values of previous lower-level time steps, i.e., Z
(k / 2)
1
, …,
Z
(k / 2)
2 i – 2
and of next higher-level time steps, i.e., Z
(k)
i + 1
, …, Z
(k)
n / k
(see Figure 10). Theoretically, it
is necessary to preserve the correlations of Z
(k / 2)
2 i – 1
with all previous lower-level variables and
all next higher-level variables. However, we can get a very good approximation if we
consider correlations with only one higher-level time step behind and one ahead. Under this
simplification, Z
(k / 2)
2 i – 1
can be generated from the linear relationship
Z
(k / 2)
2 i – 1
= a
2
Z
(k / 2)
2 i – 3
+ a
1
Z
(k / 2)
2 i – 2
+ b
0
Z
(k)
i
+ b
1
Z
(k)
i + 1
+ V (23)
where a
2
, a
1
, b
0
and b
1
are parameters given by
a
2
a
1
b
0
b
1
=
1ρ
1
ρ
2
+ ρ
3
ρ
4
+ ρ
5
ρ
1
1ρ
1
+ ρ
2
ρ
3
+ ρ
4
ρ
2
+ ρ
3
ρ
1
+ ρ
2
2(1 + ρ
1
)ρ
1
+ 2ρ
2
+ ρ
3
ρ
4
+ ρ
5
ρ
3
+ ρ
4
ρ
1
+ 2ρ
2
+ ρ
3
2(1 + ρ
1
)
–1
ρ
2
ρ
1
1 + ρ
1
ρ
2
+ ρ
3
(24)
with ρ
j
given by (5), and V is an innovation with variance
Var[V] = γ
(k / 2)
0
(1 – [ρ
2
, ρ
1
, 1 + ρ
1
, ρ
2
+ ρ
3
] [a
2
, a
1
, b
0
, b
1
]
T
) (25)
where the superscript T denotes the transpose of a vector.
All parameters are independent of i and k and therefore they can be used in all steps. When
i = 1 there are no previous time steps and thus the first two rows and columns of the above
matrix and vectors are eliminated. Similarly, when i = n / k, there is no next time step and thus
the last row and column of the above matrix and vectors are eliminated.
The power law of the power spectrum of SSS allows the generation of an SSS time series
X
i
filtering a series of white noise V
i
by the symmetric moving average (SMA) scheme
(Koutsoyiannis, 2000):
X
i
=
j = –q
q
a
|j|
V
i + j
= a
q
V
iq
+ … + a
1
V
i – 1
+ a
0
V
i
+ a
1
V
i + 1
+ … + a
q
V
i + q
(26)
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
22
Z
(k / 2)
2 i + 2
Z
(
/
2)
1
Z
(
/
2)
2
Z
(
)
1
Z
(
/
2)
2 i – 3
Z
(
/
2)
2 i – 2
Z
(
/
2)
2 i – 1
Z
(
/
2)
2 i
Z
(
/
2)
2 i + 1
Z
(
/
2)
2 i + 2
Z
(
)
i – 1
Z
(
)
i
Z
(
)
i + 1
Z
(
/
2)
2 n / k – 1
Z
(
/
2)
2 n / k
Z
(
)
n / k
Z
(n
/
2)
1
Z
(n
/
2)
2
Z
(n)
1
L
L L
L
M M
Current step
Figure 10 Explanation sketch of the disaggregation approach for generation of SSS. Grey boxes indicate random
variables whose values have been already generated prior to the current step and arrows indicate the links to
those of the generated variables that are considered in the current generation step (from Koutsoyiannis, 2002).
where q theoretically is infinity but in practice can be restricted to a finite number, as the
sequence of weights a
j
tends to zero for increasing j. Koutsoyiannis (2002) showed that the
appropriate sequence of a
j
is
a
j
(2 – 2 H) γ
0
3 – 2H (|j + 1|
H + 0.5
+ |j – 1|
H + 0.5
– 2 |j|
H + 0.5
) (27)
The sequence length q must be chosen at least equal to the desired number of autocorrelation
coefficients m that are to be preserved. In addition, the ignored terms a
j
beyond a
q
must not
exceed an acceptable tolerance β
σ. These two conditions result in
q max
m
2 β
H
2
– 0.25
1 / (H – 1.5)
(28)
Thus, q can be very large (on the order of thousands to hundreds of thousands) if H is large
(e.g. > 0.9) and β is small (e.g. < 0.001). Approximate autocorrelation functions based on
equations (26) and (27) versus the exact SSS autocorrelation functions (equation (5)) for
various values of H and q are shown in Figure 11.
This method can also generate non Gaussian series with skewness ξ
Χ
by appropriately
choosing the skewness of the white noise ξ
V
. The relevant equations for the statistics of V
i
,
which are direct consequences of (26), are
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
23
0.0001
0.001
0.01
0.1
1
1 10 100 1000 10000
Lag, j
Autocorrelation, ρ
j
Exact
Approx imation
H = 0.6,
q = 10 000
H = 0.7,
q = 10 000
H = 0.8,
q = 50 124
H = 0.9,
q = 250 000
Figure 11 Approximate autocorrelation functions based on equations (26) and (27) vs. the exact SSS
autocorrelation functions (equation (5)) for various values of the Hurst exponent H and of the number of weights
q (from Koutsoyiannis, 2002).
a
0
+ 2
j = 1
s
a
j
E[V
i
] = µ, Var[V
i
] = 1,
a
0
3
+ 2
j = 1
q
a
j
3
ξ
V
= ξ
Χ
γ
0
3/2
(29)
Concluding remarks
More than half a century after its discovery, the Hurst phenomenon, has been verified to be
almost omnipresent in several processes in nature (e.g. hydrology), technology (e.g. computer
networks) and society (e.g. economics). However, still its consequences are not widely
understood or are ignored; to quote Klemeš (1974), still it is regarded by many as “a ghost to
be conjured away”.
For example, in stochastic hydrological simulations that are used in hydrosystem
modeling, the Hurst phenomenon is generally not reproduced. The most widespread
stochastic hydrology packages, have not implemented any types of models that respect the
Hurst phenomenon. However, today there exist methodologies, implemented into software
packages, which can reproduce the Hurst phenomenon even in complicated situations, such as
in multivariate setting with multiple timescales and asymmetric probability distributions
(Koutsoyiannis, 2000; Koutsoyiannis and Efstratiadis, 2001; Langousis and Koutsoyiannis,
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
24
2003). In addition, as described above, the reproduction of the Hurst phenomenon in
univariate problems is quite simple.
In hydrological analysis, it has been a common practice to detect falling or rising
monotonic ‘trends’ in the available records, assume that these are deterministic components,
and then ‘subtract’ them from the time series to obtain a ‘detrended’ time series, which is
finally used in subsequent analyses. This common technique, which is described in several
hydrological texts, is obviously a disrespect of the Hurst phenomenon. The ‘trends’ are in fact
large scale fluctuations, i.e. the basis of the Hurst phenomenon. There could be regarded as
deterministic components if a sound, physically-based model could capture them and also
predict their evolution in the future. This, however, is not the case. The a posteriori fitting of
a regression curve (e.g. a linear equation) on historical data series has no relation with
deterministic modelling. The subtraction of the ‘trends’ from the time series results in a
reduction of standard deviation, i.e. artificial decrease of uncertainty. This is exactly opposite
to the real meaning of the Hurst phenomenon, which, as analyzed above, increases uncertainty
substantially.
Even without adopting this ‘detrending’ technique, hydrological statistics, the branch of
hydrology that deals with uncertainty, in its current state is not consistent with the Hurst
phenomenon. Typical statistics used in hydrology such as means, variances, cross- and auto-
correlations and Hurst coefficients, and the variability thereof, are based on classical statistical
theory, which describe only a portion of natural variability and thus its results may
underestimate dramatically the natural uncertainty and the implied risk.
The situation is even worse in climatology, which again uses the classical statistical
framework but on longer timescales (e.g. 30 years). As demonstrated above, the consequences
of the Hurst phenomenon in the natural variability increase as the timescale increases. Recently,
many researchers are involved in the detection of anthropogenic climatic changes mostly using
classical statistical tests, i.e. without taking into account the Hurst phenomenon. If statistical
estimators respecting the Hurst phenomenon are used, which is a choice more consistent with
nature, it is more unlikely that such tests will result in statistically significant changes.
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
25
Acknowledgment
The enlightening general discussions of Vit
Klemeš
about the nature of the Hurst phenomenon and
related issues, as well his detailed suggestions and comments on this particular article are gratefully
appreciated.
References
Adenstedt, R. K. (1974), On large sample estimation for the mean of a stationary random sequence, Ann. Statist.
2, 1095–1107.
Beran, J. (1994), Statistics for Long-Memory Processes, vol. 61 of Monographs on Statistics and Applied
Probability, Chapman and Hall, New York, USA.
Bhattacharya, R, N., Gupta, V. K. and Waymire, E. (1983) The Hurst effect under trends, J. Appl. Prob. 20, 649–
662.
Bloomfield, P. (1992), Trends in global temperature, Climatic Change 21, 1–16.
Eltahir, E. A. B. (1996), El Niño and the natural variability in the flow of the Nile River, Wat. Resour. Res.
32(1), 131–137.
Evans, T. E. (1996), The effects of changes in the world hydrological cycle on availability of water resources. In:
Global Climate Change and Agricultural Production: Direct and Indirect Effects of Changing
Hydrological, Pedological and Plant Physiological Processes (ed. by F. Bazzaz and W. Sombroek),
Chapter 2, FAO and John Wiley, Chichester, West Sussex, UK.
Haslett, J. and Raftery, A. E. (1989), Space–time modelling with long-memory dependence: assessing Ireland’s
wind power resource, Appl. Statist. 38(1), 1–50.
Hurst, H. E. (1951), Long term storage capacities of reservoirs, Trans. ASCE, 116, 776-808.
Jones, P. D., Briffa, K. R., Barnett, T. P. and Tett, S. F. B (1998a), Millennial Temperature Reconstructions.
IGBP PAGES/World Data Center-A for Paleoclimatology Data Contribution Series #1998-039,
NOAA/NGDC Paleoclimatology Program, Boulder, Colorado, USA (ftp.ngdc.noaa.gov/paleo/
contributions_by_author/jones1998/)
Jones, P. D., Briffa, K. R., Barnett, T. P. and Tett, S. F. B. (1998b), High-resolution paleoclimatic records for the
last millennium: interpretation, integration and comparison with General Circulation Model control-run
temperatures, Holocene 8(4), 455–471.
Klemeš, V. (1974), The Hurst phenomenon: a puzzle? Wat. Resour. Res. 10(4), 675–688.
Klemeš, V. (1976), Geophysical time series and catastrophism, Catastrophist Geology, 1(1), 43-48.
Klemeš, V. (1978), Physically based stochastic hydrologic analysis, Advances in Hydroscience, 11, 285-356.
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
26
Klemeš, V., Sricanthan, R. and McMahon T.A. (1981), Long-memory flow models in reservoir analysis: What is
their practical value? Water Resources Research, l7(3), 737-75l.
Koutsoyiannis, D. (2000), A generalized mathematical framework for stochastic simulation and forecast of
hydrologic time series, Water Resources Research, 36(6), 1519-1533.
Koutsoyiannis, D. (2002), The Hurst phenomenon and fractional Gaussian noise made easy, Hydrol. Sci. J.,
47(4), 573-596.
Koutsoyiannis, D. (2003a), Climate change, the Hurst phenomenon, and hydrological statistics, Hydrological
Sciences Journal, 48(1), 3-24.
Koutsoyiannis, D. (2003b), A toy model of climatic variability with scaling behaviour, Hydrofractals '03, An
international conference on fractals in hydrosciences, Monte Verita, Ascona, Switzerland, August 2003,
ETH Zurich, MIT, Université Pierre et Marie Curie (http://www.itia.ntua.gr/g/docinfo/585/).
Koutsoyiannis, D. (2003c), Hydrological statistics for engineering design in a varying climate, EGS-AGU-EUG
Joint Assembly, Geophysical Research Abstracts, Vol. 5, Nice, April 2003, European Geophysical
Society, American Geophysical Union (http://www.itia.ntua.gr/g/docinfo/565/).
Koutsoyiannis, D., and Efstratiadis, A. (2001), A stochastic hydrology framework for the management of
multiple reservoir systems, 26th General Assembly of the European Geophysical Society, Geophysical
Research Abstracts, Vol. 3, Nice, March 2001, European Geophysical Society,
(http://www.itia.ntua.gr/e/docinfo/54/).
Koutsoyiannis, D., and Efstratiadis, A. (2004), Climate change certainty versus climate uncertainty and
inferences in hydrological studies and water resources management, 1st General Assembly of the
European Geosciences Union, Geophysical Research Abstracts, Vol. 6, Nice
(http://www.itia.ntua.gr/g/docinfo/606/).
Langousis, A., and Koutsoyiannis, D. (2003), A stochastic methodology for generation of seasonal time series
reproducing overyear scaling, Hydrofractals '03, An international conference on fractals in
hydrosciences, Monte Verita, Ascona, Switzerland, August 2003, ETH Zurich, MIT, Université Pierre et
Marie Curie (http://www.itia.ntua.gr/g/docinfo/586/).
Lasota, A., and Mackey, M.C. (1994), Chaos, Fractals and Noise, Stochastic Aspects of Dynamics, Springer-
Verlag.
Mandelbrot, B. B. (1965), Une classe de processus stochastiques homothétiques a soi: application à la loi
climatologique de H. E. Hurst, C. R. Acad. Sci. Paris 260, 3284–3277.
Mandelbrot, B. B. (1971), A fast fractional Gaussian noise generator, Wat. Resour. Res. 7(3), 543–553.
Mandelbrot, B. B. (1977), The Fractal Geometry of Nature, Freeman, New York, USA.
Mesa, O. J. and Poveda, G. (1993), The Hurst effect: the scale of fluctuation approach, Wat. Resour. Res. 29(12),
3995–4002.
D. Koutsoyiannis, Hydrological Persistence and the Hurst Phenomenon (SW-434)
27
Montanari, A., Rosso, R. and Taqqu, M. S. (1997), Fractionally differenced ARIMA models applied to
hydrologic time series. Wat. Resour. Res. 33(5), 1035–1044.
Montanari, A., Taqqu, M.S. and Teverovsky, V. (1999), Estimating long-range dependence in the presence of
periodicity: an empirical study. Mathematical and Computer Modeling, 29, 217-228.
National Research Council (Committee on Opportunities in the Hydrologic Sciences) (1991), Opportunities in
the Hydrologic Sciences, National Academy Press, Washington DC, USA.
Radziejewski, M. and Kundzewicz, Z. W. (1997), Fractal analysis of flow of the river Warta, J. Hydrol. 200,
280–294.
Sakalauskienė, G. (2003), The Hurst Phenomenon in Hydrology, Environmental Research, Engineering and
Management, 3(25), 16-20.
Salas, J. D. and Boes, D. C. (1980), Shifting level modelling of hydrologic time series, Adv. Wat. Resour. 3, 59–
63.
Stephenson, D. B., Pavan, V. and Bojariu, R. (2000), Is the North Atlantic Oscillation a random walk? Int. J.
Clim. 20, 1–18.
Toussoun, O. (1925), Mémoire sur l’histoire du Nil, in: Mémoires a l’Institut d’Egypte, vol. 18, 366–404.
Vanmarcke, E. (1983), Random Fields, The MIT Press, Cambridge, Massachusetts, USA.
... An additional difficulty is the dryness at the beginning of the rainy season, because non-flow states in January and February clearly hamper the streamflow forecasting for the remaining season from March to June, using the aforementioned time series models. Therefore, in the case of nonflow states for the predictors, we added a new assumption for the k-NN approach: the nearest neighbours to the predicand were those also closer in time (the hydrologic persistence; Koutsoyiannis, 2005), besides the Euclidean distance of the traditional k-NN approach. ...
... The seasonal-scale streamflow forecasting from April to June was a much better forecast for IS than SS. This outstanding result of the seasonal-scale streamflow forecasting at IS may be explained by the use of the hydrologic persistence assumption (Koutsoyiannis, 2005), which was quite often presented for IS. This assumption worked always when the beginning of the streamflow season (January and February) showed non-flow, which was the case for 12 out of 30 a in the validation set of each seasonal-scale predicand (April, May and June) at IS. ...
Article
Streamflow forecasting in drylands is challenging. Data are scarce, catchments are highly human-modified and streamflow exhibits strong nonlinear responses to rainfall. The goal of this study was to evaluate the monthly and seasonal streamflow forecasting in two large catchments in the Jaguaribe River Basin in the Brazilian semi-arid area. We adopted four different lead times: one month ahead for monthly scale and two, three and four months ahead for seasonal scale. The gaps of the historic streamflow series were filled up by using rainfall-runoff modelling. Then, time series model techniques were applied, i.e., the locally constant, the locally averaged, the k-nearest-neighbours algorithm (k-NN) and the autoregressive (AR) model. The criterion of reliability of the validation results is that the forecast is more skillful than streamflow climatology. Our approach outperformed the streamflow climatology for all monthly streamflows. On average, the former was 25% better than the latter. The seasonal streamflow forecasting (SSF) was also reliable (on average, 20% better than the climatology), failing slightly only for the high flow season of one catchment (6% worse than the climatology). Considering an uncertainty envelope (probabilistic forecasting), which was considerably narrower than the data standard deviation, the streamflow forecasting performance increased by about 50% at both scales. The forecast errors were mainly driven by the streamflow intra-seasonality at monthly scale, while they were by the forecast lead time at seasonal scale. The best-fit and worst-fit time series model were the k-NN approach and the AR model, respectively. The rainfall-runoff modelling outputs played an important role in improving streamflow forecasting for one streamgauge that showed 35% of data gaps. The developed data-driven approach is mathematical and computationally very simple, demands few resources to accomplish its operational implementation and is applicable to other dryland watersheds. Our findings may be part of drought forecasting systems and potentially help allocating water months in advance. Moreover, the developed strategy can serve as a baseline for more complex streamflow forecast systems.
... To obtain the 15-minute data intervals for those periods, a linear interpolation was performed between the known data points to fill those gaps. The linear interpolation method was identified as being sufficient for this purpose as it relies on the 'Hurst phenomenon', "that is, consecutive values of hydrological time series are dependent to each other" (Koutsoyiannis, 2005). ...
Thesis
Full-text available
Water resources researchers have advanced our understanding of sediment transport in karst aquifers and cave systems in recent years. However, we find knowledge gaps for sediment transport in karst including knowledge of processes controlling sediment hysteresis patterns from sensor datasets, and knowledge of dominant sediment transport processes that lead to sediment transport formulas for prediction. To address these gaps, we analyze a new suspended sediment, electrical conductivity and water flowrate datasets from sinking streams and a phreatic cave system for an epigenetic karst basin in the inner bluegrass region of Kentucky USA. Three sediment patterns are common including: pulses of sediment arriving before or after water flowrate peaks indicating hysteresis; a direct and near linear dependence of sediment concentration as a function water flowrate; and an order of magnitude shift in sediment transport flux and time lags during spring 2018. The hysteresis patterns occur from external sediment entering the cave from the sinking streams. Variability of hysteresis reflect the time-varying sediment concentration of water entering via the sinking streams and pre-event water storage in the karst aquifer. Sediment pulses from the sinking streams are attenuated as they arrive at the cave’s spring. The effect is evidenced from clockwise Q-TSS patterns in the sinking streams shifting to linear and counterclockwise patterns in the cave, clockwise EC-TSS patterns for all sites, and clockwise Q-TSS patterns for all sites when surface water is isolated via mixing models and used in the analyses. The effect is further evidenced with statistical moments of hysteresis indices and numerical modelling of sediment transport, which are novel features of this contribution. The timing of the sediment peak relative to the water flowrate peak at the cave’s spring agrees well with pre-event water storage in the karst aquifer. The magnitude of pre-event water storage can cause clockwise or counterclockwise hysteresis as evidenced by both data and numerical modelling results. Findings are contrary to interpretation of hysteresis in previous karst studies where researchers suggest resuspension of internal cave sediment cause hysteresis. We find no sediment mechanics available to predict such interpretation. The direct and near linear dependence of sediment concentration as a function water flowrate (i.e., no hysteresis) reflects resuspension of internal cave sediment. Sediment pulses out of phase with water flowrate do not occur from this process. Rather, the system is monotonic with water discharge and is evidenced by the fact that it is well described by the fluvial sediment transport rate formula of Partheniades. Sediment hysteresis results in the cave are also indicative of disturbances across the landscape surface. We find a six-month time period of very high disturbance in the basin increases sediment loads by one order of magnitude at the cave’s spring, shifts hysteresis patterns to highly influenced by streambank erosion, and require shifting the sediment storage parameter in the Partheniades formula.
... In hydrology and hydroclimatology, the concept has been invoked to quantify the persistence of a signal in signatures of hydrological processes, such as the influence of precipitation on flood (Khanal et al., 2019), storage-recession characteristics (Ye et al., 2014), and the influence of soil moisture signals on evapotranspiration and streamflow (Orth & Seneviratne, 2013) or of snowpack on streamflow (Hammond et al., 2018). Persistence may be short-term (hours) or long-term (e.g., the Hurst phenomenon, which quantifies persistence over the long term; Hirpa et al., 2010;Koutsoyiannis, 2005). ...
Article
Full-text available
How precipitation (P) is translated into streamflow (Q) and over what timescales (i.e., “memory”) is difficult to predict without calibration of site‐specific models or using geochemical approaches, posing barriers to prediction in ungauged basins or advancement of general theories. Here, we used a data‐driven approach to identify regional patterns and exogenous controls on P–Q interactions. We applied an information flow analysis, which quantifies uncertainty reduction, to a daily time series of P and Q from 671 watersheds across the conterminous United States. We first demonstrated that information transfer from P to Q primarily reflects the quickflow component of water‐budgets, based on a watershed model. Readily quantifiable information flows show a functional relationship with model parameters, suggesting utility for model calibration. Second, applied to real watersheds, P–Q information flows exhibit seasonally varying behavior within regions in a manner consistent with dominant runoff generation mechanisms. However, the timing and the magnitude of information flows also reflect considerable subregional heterogeneity, likely attributable to differences in watershed size, baseflow contributions, and variation in aerial coverage of preferential flow paths. A regression analysis showed that a combination of climate and watershed characteristics are predictive of P–Q information flows. Though information flows cannot, in most cases, uniquely determine dominant runoff mechanisms, they provide a means to quantify the heterogeneous outcomes of those mechanisms within regions, thereby serving as a benchmarking tool for models developed at the regional scale. Last, information flows characterize regionally specific ways in which catchment connectivity changes from the wet to dry season.
... The Hurst coefficient measures the persistence in the timeseries; high persistence means 332 consecutive values are more dependent upon one another (Koutsoyiannis, 2005) and thus multi-333 year droughts will tend to be longer, as will multi-year runs of wet conditions. Note, during 334 perturbation the Hurst coefficient value can be directly perturbed as one of the five stressors in 335 the stress test, in which case the perturbed value will be preserved rather than the historic (see 336 Section 2.4.3). ...
Article
“Bottom-up” methods are increasingly used to assess the vulnerability of water systems to climate change. Central to these methods is the climate “stress test”, where the system is subjected to various climatic changes to test for unacceptable outcomes. We present a framework for climate stress testing on a monthly timestep, suitable for systems whose dominant dynamic is seasonal or longer (eg. water resource systems with carry-over storage). The framework integrates multi-site stochastic climate generation with perturbation methods and in-built rainfall runoff modelling. The stochastic generation includes a low frequency component suitable for representing multi-annual fluctuations. Multiple perturbation options are provided, ranging from simple delta change through to altered seasonality and low frequency dynamics. The framework runs rapidly, supporting comprehensive multi-dimensional stress testing without recourse to supercomputing facilities. We demonstrate the framework on a large water resource system in southern Australia. The Matlab/Octave framework is freely available for download from https://doi.org/10.5281/zenodo.5617008.
... The amplitude parameter is chosen for each sub-area separately so that the observed historic value of the Hurst coefficient (Hurst, 1951) is replicated in the baseline precipitation data. The Hurst coefficient measures the persistence in the timeseries; high persistence means consecutive values are more dependent upon one another (Koutsoyiannis, 2005) and thus multiyear droughts will tend to be longer, as will multi-year runs of wet conditions. Note, during perturbation the Hurst coefficient value can be directly perturbed as one of the five stressors in the stress test, in which case the perturbed value will be preserved rather than the historic (see Section 2.4.3). ...
Preprint
Full-text available
“Bottom-up” methods are increasingly used to assess the vulnerability of water systems to climate change. Central to these methods is the climate “stress test”, where the system is subjected to various climatic changes to test for unacceptable outcomes. We present a framework for climate stress testing on a monthly timestep, suitable for systems whose dominant dynamic is seasonal or longer (eg. water resource systems with carry-over storage). The framework integrates multi-site stochastic climate generation with perturbation methods and in-built rainfall runoff modelling. The stochastic generation includes a low frequency component suitable for representing multi-annual fluctuations. Multiple perturbation options are provided, ranging from simple delta change through to altered seasonality and low frequency dynamics. The framework runs rapidly, supporting comprehensive multi-dimensional stress testing without recourse to supercomputing facilities. We demonstrate the framework on a large water resource system in southern Australia. The Matlab/Octave framework is freely available for download from https://doi.org/10.5281/zenodo.5617008.
... On the other hand, the Rescaled Range (R/S) statistical analysis was applied to the SPEI time series in order to quantify the longterm persistence of trends. We ascertained whether the drought trends observed to be statistically significant in the study period (past) persist in time (future), since this is related to the predictability of droughts and climate change (Koutsoyiannis, 2005(Koutsoyiannis, , 2003. This analysis was introduced by Hurst (1956). ...
Article
Full-text available
Study region Mozambique. Study focus Mozambique does not currently have the necessary tools for systematic monitoring and forecasting of drought at a subnational scale. The purpose of this study was to characterize drought conditions and trends throughout the country and to evaluate the influence of major climatic drivers on drought events (period 1950–2019). Drought conditions were studied by means of the Standardized Precipitation and Evapotranspiration Index (SPEI) and run theory. The principal component analysis technique and the k-means clustering method were applied for defining homogenous drought regions. The Mann-Kendall trend test and Rescaled Range statistical analysis were used for defining the temporal characteristics of drought. The cross-correlation method, a spectral analysis based on the Fast Fourier Transform and a cross-wavelet analysis, were used to identify possible climate drivers. The results are ultimately intended to contribute to the development of a drought monitoring system in this country. New hydrological insights for the region Three homogeneous drought regions can be defined in Mozambique. The South and Centre regions showed more intense and severe drought events. In all regions, a significant trend towards a higher incidence of droughts and long-term a persistence were found. El Niño-Southern Oscillation and Darwin Sea Level Pressure anomalies were identified as significant drivers of drought variability, especially in the southern regions. These climate indices can be used as predictors in drought forecasting models.
... Hydrological time series exhibit some structure, unlike a common random series. For instance, consecutive values of hydrological time series are dependent on each other [33]. Future climate trends can be predicted using the R/S analysis method. ...
Article
Full-text available
Citation: Kalyan, A.; Ghose, D.K.; Thalagapu, R.; Guntu, R.K.; Agarwal, A.; Kurths, J.; Rathinasamy, M. Multiscale Spatiotemporal Analysis of Extreme Events in the Gomati River Basin, India. Atmosphere 2021, 12, 480. https://doi.
Article
Full-text available
Drought is a prolonged period of low precipitation that negatively impacts agriculture, animals, and people. Over the last decades, gradual changes in drought indices have been observed. Therefore, understanding and forecasting drought is essential to avoid its economic impacts and appropriate water resource planning and management. This paper presents a recent literature review, including a brief description of data pre-processing, data-driven modelling strategies (i.e., univariate or multivariate), machine learning algorithms (i.e., advantages and disadvantages), hybrid models, and performance metrics. Combining various prediction methods to create efficient hybrid models has become the most popular use in recent years. Accordingly, hybrid models have been increasingly used for predicting drought. As such, these models will be extensively reviewed, including preprocessing-based hybrid models, parameter optimisation-based hybrid models, and hybridisation of components combination-based with preprocessing-based hybrid models. In addition, using statistical criteria, such as RMSE, MAE, NSE, MPE, SI, BIC, AIC, and AAD, is essential to evaluate the performance of the models.
Article
This work applied implicit stochastic optimization (ISO) refined by long-term mean inflow forecasting and instance-based learning for the operation of the Sobradinho reservoir, Brazil. For efficiency assessment, the reservoir was also operated by perfect-forecast deterministic optimization, the standard operating policy, stochastic dynamic programming and two parameterization-simulation-optimization models, which were compared in terms of vulnerability, reliability and resilience found in each of the 100 synthetic inflow scenarios they were applied to. Evidence of long-term persistence was found in Sobradinho's records and this was replicated in the scenarios. The ISO model was employed with forecast horizons of 0, 1, 3, 6, 9, 12, 18 and 24 months. The operations demonstrated that the model with forecast horizons of 3 months or more was less vulnerable than all other models, revealing that it may be used efficiently for reservoir operation.
Conference Paper
Full-text available
In generating synthetic time series of hydrologic processes at sub-annual scale it is important to preserve seasonal characteristics and short-term persistence. At the same time, it is equally important to preserve annual characteristics and over year scaling behaviour. This scaling behaviour, which is equivalent to the Hurst phenomenon, has been detected in a large number of hydroclimatic series and affects seriously planning and design of hydrosystems. However, when seasonal models are used, the preservation of annual characteristics and overyear scaling is a difficult task and is often ignored. Disaggregation techniques are the only way to produce synthetic series that are consistent with historical series in several time scales, from seasonal to multiyear, simultaneously. Such techniques involve two or more steps, where in the first step annual series are generated, which are subsequently disaggregated to finer scales. However, disaggregation involves several difficulties (e.g. in parameter estimation), inaccuracies and is a slow procedure. As an alternative, a new methodology is proposed that directly operates on seasonal time scale, avoiding disaggregation, and simultaneously preserves annual statistics and the scaling properties on overyear time scales thus respecting the Hurst phenomenon.
Conference Paper
Full-text available
It is demonstrated that a simple deterministic model in discrete time can capture the scaling behaviour of hydroclimatic processes at time scales coarser than annual. This toy model is based on a generalized "chaotic tent map", which may be considered as the compound result of a positive and a negative feedback mechanism, and involves two degrees of freedom. The model is not a realistic representation of a climatic system, but rather a radical simplification of real climatic dynamics. However, its simplicity enables easy implementation, even on a spreadsheet environment, and convenient experimentation. Application of the toy model gives traces that can resemble historical time series of hydroclimatic variables, such as temperature, rainfall and runoff. In particular, such traces exhibit scaling behaviour with a Hurst exponent greater than 0.5 and density function similar to that of observed time series. Moreover, application demonstrates that large-scale synthetic "climatic" fluctuations (like upward or downward trends) can emerge without any specific reason and their evolution is unpredictable, even when they are generated by this simple fully deterministic model with only two degrees of freedom. Obviously, however, the fact that such a simple model can generate time series that are realistic surrogates of real climatic series does not mean that a real climatic system involves that simple dynamics.
Article
Necessary and sufficient conditions for the so-called Hurst effect are given in the case of a weakly dependent stationary sequence of random variables perturbed by a trend. As a consequence of this general result it is shown that the Hurst effect is present in the case of weakly dependent random variables with a small monotonic trend of the form f(n) = c(m + n) ß , where m is an arbitrary non-negative parameter and c is not 0. For – ½ < ß < 0 the Hurst exponent is shown to be precisely given by 1 + ß. For ß ≦ – ½ and for ß = 0 the Hurst exponent is 0.5, while for ß > 0 it is 1. This simple mathematical model, motivated by empirical evidence in various geophysical records, demonstrates the presence of the Hurst effect in a direction not explored before.
Article
The concept of stochastic hydrology as a theory seeking to deduce statistical and stochastic properties of a hydrologic phenomenon from their physical causes, rather than only to describe them and manipulate their characteristics or, in other words, the concept of a physically based stochastic theory of the hydrologic cycle, has not yet been clearly formulated.
Article
By design fast fractional Gaussian noises (ffGn) have the following characteristics: The number of operations needed to generate them is relatively small. The long run statistical dependence that they exhibit is strong and has the form required for self similar hydrology. Their short run properties, as expressed by the correlations between successive or nearly successive yearly averages, are adjustable (within bounds) and can be fitted to the corresponding short run properties of records. Extension to the multidimensional (multisite) case should raise no essential difficulty. Finally, their definition, as sums of Markov-Gauss and other simple processes, fits the intuitive ideas that climate can be visualized as either unpredictably inhomogeneous or ruled by a hierarchy of variable regimes.
Article
The potential of applying shifting level (SL) models to hydrologic processes is discussed in light of observed statistical characteristics of hydrologic data. An SL model and an ARMA (1, 1) model are fitted to an actual hydrologic series. Computer simulation experiments with these models are carried out to compare maximum accumulated deficit and run properties. Results obtained indicate that the mean maximum accumulated deficit, mean longest negative run length, and mean largest negative run sum for both models are similar while there are differences in their corresponding variances.