ArticlePDF Available

Abstract

It is demonstrated that extremization of entropy production of stochastic representations of natural systems, performed at asymptotic times (zero or infinity) results in constant derivative of entropy in logarithmic time and, in turn, in Hurst-Kolmogorov processes. The constraints used include preservation of the mean, variance and lag-1 autocovariance at the observation time step, and an inequality relationship between conditional and unconditional entropy production, which is necessary to enable physical consistency. An example with real world data illustrates the plausibility of the findings.
Hurst-Kolmogorov dynamics as a result of extremal entropy production
Demetris Koutsoyiannis
Department of Water Resources and Environmental Engineering, National Technical
University of Athens, Greece (dk@itia.ntua.gr – http://www.itia.ntua.gr/dk)
Abstract. It is demonstrated that extremization of entropy production of stochastic
representations of natural systems, performed at asymptotic times (zero or infinity) results in
constant derivative of entropy in logarithmic time and, in turn, in Hurst-Kolmogorov
processes. The constraints used include preservation of the mean, variance and lag-1
autocovariance at the observation time step, and an inequality relationship between
conditional and unconditional entropy production, which is necessary to enable physical
consistency. An example with real world data illustrates the plausibility of the findings.
Keywords. Stochastic processes, Long-term persistence, Hurst-Kolmogorov dynamics,
Entropy, Uncertainty, Extremal entropy production
1. Introduction
Variational principles, such as the principle of extremal time for determining the path of light
(a generalization of Hero’s and Fermat’s principles) and the principle of extremal action for
determining the path of simple physical systems, are well established in physics. In complex
systems, entropy maximization is a principle that can determine the thermodynamic
equilibrium of a system. Historically, in classical thermodynamics, entropy was introduced as
a conceptual and rather obscure notion via a rather circular logic
*
. Later, within statistical
thermophysics, entropy has been given an austere and clear probabilistic meaning by
Boltzmann and Gibbs, while Shannon used essentially the same entropy definition to describe
the information content, which he also called entropy at von Neumann’s suggestion [1].
Despite having the same name, information entropy and thermodynamic entropy are
commonly regarded as two distinct notions. One of the strongest proponents of this view is E.
T. Jaynes [2], who, notably, introduced the principle of maximum entropy as a principle for
*
Cf. the classical definition dS = dQ/T, where S, Q and T denote entropy, heat and temperature, respectively; the
definition is applicable in a reversible process, which in turn is one in which dS = dQ/T.
This was also suggested by a reviewer of this paper.
Jaynes [2] wrote: We must warn at the outset that the major occupational disease of this field is a persistent
failure to distinguish between the information entropy, which is a property of any probability distribution, and
the experimental entropy of thermodynamics, which is instead a property of a thermodynamic state as defined,
for example by such observed quantities as pressure, volume, temperature, magnetization, of some physical
system. They should never have been called by the same name; the experimental entropy makes no reference to
any probability distribution, and the information entropy makes no reference to thermodynamics. Many
textbooks and research papers are flawed fatally by the author's failure to distinguish between these entirely
different things, and in consequence proving nonsense theorems”.
2
inference. However, this distinction has been disputed by others: the “information entropy [is]
shown to correspond to thermodynamic entropy” [1] and the two become precisely identical,
if Boltzmann’s constant is taken equal to unity. Perhaps the fact that the information entropy
is a dimensionless quantity, while the thermodynamic entropy has units (J/K), has been a
major obstacle to see that the two are identical notions*. But this is an historical accident,
related to the arbitrary introduction of temperature scales, which Boltzmann respected, thus
multiplying the probabilistic entropy by the constant bearing his name. This has been made
crystal clear now [3] and it has been recognized that, given Shannon’s entropy definition, in
principle we do not need a separate base unit for temperature, the kelvin and in essence,
thermodynamic entropy, like the information entropy, is dimensionless [4].
Here we regard the thermodynamic and information entropy identical notions, for which
we use the term Boltzmann-Gibbs-Shannon (BGS) entropy whose definition is given in
Equation (3) below. Likewise, we regard the principle of maximum entropy both as a physical
principle to determine equilibrium thermodynamic states of natural systems and as a logical
principle to make inference about natural systems (in an optimistic view that our logic in
making inference about natural systems could be consistent with the behavior of the natural
systems).
Application of the principle of maximum entropy is rather simple when time is not
involved in the problem studied, which is the case in systems in equilibrium. In studying
systems out of the equilibrium, time should necessarily be involved [e.g., 5] and the paths of
the systems should be inferred. The more recently developed, within non-equilibrium
thermodynamics, extremal principles of entropy production attempt to predict the most likely
path of system evolution [6,7,8]. Central among the latter are Prigogine’s minimum entropy
production principle [9] and Ziegler’s maximum entropy production principle [10], which are
not in contradiction [11].
Extremization of entropy or its production rate involves a set of constraints, which impose
macroscopic physical restrictions in the probabilistic concept of entropy. For example, in the
kinetic theory of gases [e.g.,12], if f(p)dp is the number of molecules with momenta between
p and p + dp in a certain macroscopic volume, then the zeroeth moment (the integral of f(p)
over the entire volume) is the total number of molecules and should be constant due to
*
The fact the kelvin is just a convention rather than a fundamental physical unit is reflected in the currently
discussed proposal for redefining the kelvin scale indirectly, based on Boltzmann's constant k (specifically, the
kelvin, unit of thermodynamic temperature, is such that the Boltzmann constant is exactly 1.380 65 × 10
−23
J/K
[4]); had k been chosen equal to one, without dimensions, the kelvin scale would not be needed at all).
Atkins [3] writes: “Although Boltzmann’s constant k is commonly listed as a fundamental constant, it is
actually only a recovery from a historical mistake. If Ludwig Boltzmann had done his work before Fahrenheit
and Celsius had done theirs, then … we might have become used to expressing temperatures in the units of
inverse joules… Thus, Boltzmann’s constant is nothing but a conversion factor between a well-established
conventional scale and the one that, with hindsight, society might have adopted”.
3
conservation of mass. Also, the first moment is constrained to a constant value due to the law
of conservation of (macroscopic) momentum, and the second marginal moment is constrained
by the law of conservation of energy (since ||p||
2
is proportional to the kinetic energy).
Likewise, joint moments of momenta represent deviatoric stresses [13].
In studying very complex systems, e.g. the Earth climatic system or parts thereof, it may be
difficult to establish a relationship of macroscopic and microscopic quantities. However, the
entropy can still be determined in terms of the probability density function of merely
macroscopic quantities, and constraints in entropy extremization could also be formulated on
the basis of assigning a conceptual meaning to macroscopic properties (moments) of the
process of interest. The values of these moments can be determined from a statistical sample,
rather than deduced from fundamental physical laws. Thus, the first and second marginal
moments (mean and variance) represent a mean level of the process and the intensity of
fluctuations around the mean, respectively, whereas a joint moment of time lagged variables
(autocovariance) represents time dependence. In natural processes, for short lags, the
dependence (expressed in terms of autocovariance) must necessarily be positive because
values at adjacent times should be close to each other.
In geophysical processes, the available length of observation records are usually short and
restrict the reliable statistical estimation to the lowest marginal moments (mean, variance) and
the smallest lags for autocovariances. The asymptotic stochastic properties of the processes,
i.e. the marginal distribution tails and the tails of the autocovariance functions, can hardly be
estimated from available records. However, these asymptotic properties are crucial for the
quantification of future uncertainty, as well as for planning and design purposes. This work
tries to connect statistical physics (the extremal entropy production concept, in particular)
with stochastic representations of natural processes, which are used as modeling tools in
planning and design purposes. Extremal entropy production may provide a theoretical
background in such stochastic representations, which are otherwise solely data-driven. The
focus of the study is the asymptotic behavior of the autocovariance function. The principal
question is whether the autocovariance decays following an exponential or a power-type law.
The two different behaviors are also known as short-term and long-term persistence,
respectively. A typical representative of the former behavior is a Markov process, while one
of the latter is the Hurst-Kolmogorov (HK) process (named so after Hurst [14], who pioneered
its detection in geophysical time series, and Kolmogorov [15], who proposed the
mathematical process), else known as fractional Gaussian noise [16]. Both these processes
will be fully described below.
2. Basic notions
Let x
i
be a stationary stochastic process at discrete time i = 0, 1, … (where time 0 denotes the
present), representing a natural process x
i
. (Notice that underlined quantities denote random
variables, whereas regular mathematical quantities are not underlined). As natural processes
4
evolve in continuous time, it is natural to assume that x
i
are stationary increments, at time
steps of equal size , of a cumulative process z(t) in continuous time t, i.e., x
i
:= z(i∆) –
z((i – 1)). Observing the natural process x
1
i
at time step, say, = 1 we can estimate from a
statistical sample the following quantities:
E[x
1
i
] =: µ, Var[x
1
i
] := E[(x
1
i
µ)
2
] =: γ, Cov[x
1
i
, x
1
i + 1
] := E[(x
1
i
µ) (x
1
i + 1
µ)] = ρ γ (1)
where E[ ] denotes expectation and ρ is the lag-1 autocorrelation coefficient. The three
equalities in (1) form the simplest possible set of constraints for multivariate entropy
extremization. Alternatively, these constraints could be formulated in terms of the cumulative
process z(t) (assuming z(0) = 0) as:
E[z(1)] = µ, Var[z(1)] = γ, Var[z(2)] = 2(1 + ρ) γ (2)
The problem we investigate here is the determination of the multivariate (joint) distribution
of the process x
i
(or equivalently, z(t)) by extremizing a proper entropic metric. It is known
[e.g., 17] that, for a single random variable x (rather than a stochastic process), extremization
of the BGS entropy using the first two of the above constraints results in the normal
(Gaussian) distribution of x. Also, for many random variables x
i
with given correlation matrix,
the BGS entropy extremizing distribution is the multivariate Gaussian [17]. We note that the
BGS entropy definition may be insufficient for certain phenomena, where generalized
entropies are necessary to consider, which result in different entropy extremizing distributions
[18,19]; again, however, an appropriate nonlinear transformation (derived from the
generalized entropy definition) may make the process Gaussian. For simplicity, in this study
we adhere to the BGS entropy and use the above classical results to assert that the joint
distribution of z(t) (as well as x
i
) of any order is Gaussian. The focus of the study is not the
marginal distribution but the dependence structure (autocovariance) of x
i
and this can be
readily inferred if the variance of z(t) is known for any time t. We recall that the BGS entropy
of a single random variable z with probability density function f(z) is defined as*:
Φ[z] := Ε[–ln f(z)] = –
–∞
f(z) ln f(z) dz (3)
and that for a Gaussian variable the entropy depends on its variance γ only and is given as
[17]
Φ[z] = (1/2) ln(2πe γ) (4)
*
Here we have used the older notation of entropy by Φ instead of the newer S and H, which in stochastics are
reserved for standard deviation and Hurst coefficient, respectively. The quantity ln f(z) in (3) is meant, more
precisely, as ln [f(z)/m(z)] (e.g. Ref. [2], p. 375) where m(z) is meant as the Lebesgue density, i.e. equal to 1
times the unit [z]
-1
, so that Φ[z] is dimensionless.
5
Furthermore, for the stochastic process z(t) we can define entropy production as the time
derivative Φ΄[z(t)] := dΦ[z(t)]/dt. For a Gaussian process, by virtue of (4), the entropy
production is
Φ΄[z(t)] = [dγ(t)/dt] / 2γ(t) (5)
We will use the notion of extremal entropy production to infer the variance of z(t), which in
turn determines the entire dependence structure of x
i
, using the constraints (2) (or (1)). We
clarify that, while other studies make an assumption on the type of dependence (e.g. long-
range dependence [20]), here we attempt to derive, rather than assume, the dependence
structure.
3. Methodology
We start our investigation from a linear Markovian process in continuous time, because of its
simplicity and mathematical convenience, also assuming (without loss of generality) zero
mean µ (we will remove this assumption in section
5). We note that any dependence structure
of a stochastic process can be obtained by the sum of an infinite number of Markovian
processes independent to each other [cf. 21]. Furthermore, the sum of a finite number of
independent Markovian processes can provide good approximations of any dependence
structure on a finite range of time scales. For example, in practical problems, an HK process
can be approximated using as few as three Markovian processes [22].
A linear Markovian process q(t) in continuous time, else known as Ornstein–Uhlenbeck
(OU) process [17,23], with zero mean is defined by
(1/k) dq + q dt = σ v dt (6)
where k > 0 and σ > 0 are parameters, and v is the white noise process (i.e., v dt = dw where w
is the Wiener process) with mean E[v(t)] = 0 and covariance function Cov[v(t), v(s)] = δ(ts),
whereas δ( ) is the Dirac delta function. The quantity 1/k has units of time and represents a
characteristic time scale of the process, whereas the quantity σ
2
represents a characteristic
variance. The linear differential equation (6) is easily solved to give
q(t) = q(0) e
kt
+ k σ e
kt
0
t
v(ξ) e
dξ (7)
and has mean and variance, respectively,
Ε[q(t)]
=
e
kt
Ε[q(0)], Var[q(t)] = Var[q(0)] e
–2kt
+ (k σ
2
/2) (1 – e
–2kt
) (8)
The cumulative OU process is
z(t) =
0
t
q(ξ) dξ = q(0) (1 – e
kt
)/k + σ
0
t
v(ξ) [1 – e
k(t ξ)
] dξ (9)
6
and has mean and variance, respectively,
Ε[z(t)]
=
Ε[q(0)] (1 – e
kt
)/k (10)
Var[z(t)] = Var[q(0)] (1 – e
kt
)
2
/ k
2
+ σ
2
[2kt + 1 – (2 – e
kt
)
2
] / (2k) (11)
Two cases are most interesting to consider: the unconditional case, in which no initial
condition is known, and the conditional case where the present and the entire past of q(t), t
0, are known (observed). In the former case, stationarity in q(t) demands that E[q(t)] = Ε[q(0)]
= 0 and Var[q(t)] = Var[q(0)]. Substituting this into (8), we find Var[q(0)] = k σ
2
/2 =: ω, so
that in the unconditional (stationary) case we have (by virtue of (11); see also [17]):
Ε[q(t)]
= Ε[z(t)]
= 0, Var[q(t)] = ω, γ(t) := Var[z(t)] = 2(kt + e
kt
– 1)
k
2
ω (12)
In the conditional case, since the process is Markovian, only the most recent observed value
q(0) = q(0) matters in conditioning on the past, and also Var[q(0)] = 0. Thus,
Ε[q(t)|q(0)]
= q(0) e
kt
, Ε[z(t)|q(0)]
=
q(0) (1 – e
kt
)/k, Var[q(t)|q(0)] = (1 – e
–2kt
) ω,
γ
C
(t) := Var[z(t)|q(0)] = 2kt + 1 – (2 – e
kt
)
2
k
2
ω
(13)
By means of γ(t) and γ
C
(t) in (12) and (13) and by virtue of (4) we can determine, at any
time t, the unconditional and conditional entropy of z(t). Likewise, by virtue of (5), we can
determine the unconditional and conditional entropy production, which are, respectively,
Φ΄[z(t)] = k(1 – e
kt
)
2(kt + e
kt
– 1), Φ
C
΄[z(t)] = k(1 – e
kt
)
2
2kt + 1 – (2 – e
kt
)
2
(14)
We observe that in both cases the entropy production is proportional to k, the inverse
characteristic time scale, depends on the dimensionless time kt, and is independent of the
characteristic variance ω. We also observe that the limit of both Φ΄[z(t)] and Φ΄
C
[z(t)] for t
0 is ∞ and for t → ∞ is 0. Consequently, any composite process determined as a sum of
Markovian processes will also have the same limits. This does not enable comparison of the
asymptotic behaviour of different processes (dependence structures). On the other hand, the
asymptotic values of entropy production for t 0 and are the most important for model
comparisons and eventually for extremization of entropy production. Otherwise, comparisons
of different models at any specified finite time t would involve a degree of subjectivity and
arbitrariness for the choice of t. In a similar situation, Koutsoyiannis [24] used an average
entropy over a range of time scales.
Here, we follow a different method to tackle the infinite and zero entropy production. We
first observe that for any time t an inequality relationship between any two stochastic
processes 1 and 2, i.e. Φ΄[z
1
(t)] Φ΄[z
2
(t)] is preserved if we replace entropy production Φ΄
with the following quantity, which we will call entropy production in logarithmic time
(EPLT):
7
φ[z(t)] := Φ΄[z(t)] t ≡ dΦ[z(t)] / d(lnt) (15)
Therefore, extremization of entropy production is equivalent to extremization of EPLT. In the
OU model the unconditional and conditional EPLTs are, respectively,
φ[z(t)] = kt (1 – e
kt
)
2(kt + e
kt
– 1), φ
C
[z(t)] = kt (1 – e
kt
)
2
2kt + 1 – (2 – e
kt
)
2
(16)
These quantities depend on the dimensionless time kt only and their limits are finite, i.e.
φ[z(0)] = 1, φ
C
[z(0)] = 3/2, and φ[z(∞)] = φ
C
[z(∞)] = 1/2.
As stated above, we can materialize a stochastic process with an arbitrary dependence
structure as the sum of a number m of OU processes independent to each other, i.e. z(t) :=
Σ
m
j = 1
z
j
(t), where z
j
(t) has characteristic variance ω
j
and characteristic time scale 1/k
j
. The
variances of z(t) will then be
γ(t) =
j = 1
m
2(k
j
t + e
k
j
t
– 1) ω
j
/k
2
j
, γ
C
(t) =
j = 1
m
[2k
j
t + 1 – (2 – e
k
j
t
)
2
] ω
j
/k
2
j
(17)
and the EPLTs
φ[z(t)] = t
γ(t)
j = 1
m
(1– e
k
j
t
) ω
j
/k
j
, φ
C
[z(t)] = t
γ
C
(t)
j = 1
m
(1– e
k
j
t
)
2
ω
j
/k
j
(18)
Clearly, the characteristic variances ω
j
are weights of each of the constituents z
j
(t) of the
composite process z(t) and the EPLTs of z(t) depend in a nonlinear manner on all these
weights.
4. Application
The EPLT extremization problem is formulated as
extremize φ[z(t)] or φ
C
[z(t)] for t → 0 or t → ∞
(19)
subject to constraints of Equation (2)
The control variables (unknown quantities) to be determined are the weights ω
j
for specified
inverse characteristic time scales k
j
. By appropriate choice of k
j
, analytical solutions may be
possible, but here we tackle the optimization problem numerically. The numerical procedure
is quite general, so that it can host constraints more than those contained in (2) (e.g. more than
one autocovariance value), but here we adhere to the simplest possible set of constraints for
parsimony. At the same time, the framework is quite simple and, thus, the extremization could
be performed by widespread all-purpose optimization tools (here we used the generalized
reduced gradient tool by Frontline Systems, v3.5, implemented as a Solver in Excel).
In our numerical framework we investigate times t in the range 2
–10
to 2
10
(a range
spanning approximately 6 orders of magnitude) and we use inverse characteristic time scales
8
k
j
in the range k
min
= k
1
= 2
–15
to k
max
= 2
15
, with k
j
= 2 k
j – 1
(a range spanning approximately 9
orders of magnitude). The unknown quantities are the m = 31 characteristic variances ω
j
.
These are determined by extremizing (maximizing or minimizing) the EPLTs for the lowest
and highest times of the range considered (2
–10
to 2
10
). These are supposed to approximate the
limits for t 0 and t ∞, respectively. We use the three equality constraints in (2) and, to
force determination of a limit rather than a local optimum for time 2
–10
or 2
10
, we set an
additional inequality constraint that the standard deviation of the EPLT values at five
consecutive time scales nearest to 2
–10
or 2
10
is lower than a small value ε.
We performed several numerical optimizations of this type and we generalized the results
which proved to be very simple. An example is shown in Figure 1 in terms of the extremizing
weights ω
j
for each inverse characteristic time scale k
j
, and in Figure 2 in terms of the
resulting EPLTs. This example corresponds to constraints µ = 0, γ = 1 and ρ = 0.543; the latter
value was chosen so as to correspond to an OU process with k = 1. Two characteristic
processes emerge by EPLT extremization. The first is a single OU process, in which all ω
j
are
zero except one, namely that corresponding to k = 1. The single OU process maximizes both
EPLTs for t → 0, with φ[z(0)] = 1 and φ
C
[z(0)] = 3/2. The same process is the one that
minimizes both EPLTs for t → ∞, with φ[z(∞)] = φ
C
[z(∞)] = 1/2.
The second extremizing process, denoted in the figures as the Hybrid process, has again all
ω
j
zero except two, i.e. those corresponding to k
min
= 2
–15
and k
max
=2
15
. This minimizes both
EPLTs for t0, with φ[z(0)] = φ
C
[z(0)] = 1/2 and maximizes EPLTs for t∞, with φ[z(∞)]
= 1 and φ
C
[z(∞)] > 1.
0.001
0.01
0.1
1
10
100
1000
10000
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1000
10000
100000
k
j
ω
j
Markov
Hybrid
HK
Figure 1 Entropy extremizing characteristic variances (weights of the composite processes) ω
j
versus the inverse
characteristic time scales k
j
, as obtained by the numerical framework for the three solutions discussed in text.
9
0
0.5
1
1.5
2
0.001 0.01 0.1 1 10 100 1000
t
φ(t)
Markov, unconditional
Markov, conditional
Hybrid, unconditional
Hybrid, conditional
HK, unconditional+conditional
0
0.5
1
1.5
2
0.001 0.01 0.1 1 10 100 1000
t
φ(t)
Markov, unconditional
Markov, conditional
Hybrid, unconditional
Hybrid, conditional
HK, unconditional+conditional
Figure 2 Entropy production in logarithmic time φ(t) φ[z(t)] versus time t in the three extremizing solutions
discussed in text and shown in Figure 1; (upper) as determined by the numerical framework; (lower) analytical
models.
However, it is readily understood that the latter numerical solution is artificially affected
because of the use of finite minimum and maximum scales. We can thus infer that the precise
analytical solution would be the one in which k
min
0 and k
max
∞. It can be derived from
(12) and (13) that, as k
max
∞, γ(t) = γ
c
(t) → 2ω
max
t/k
max
= at (where a = 2ω
max
/k
max
),
whereas, as k
min
0, γ(t) ω
min
t
2
= bt
2
(where b = ω
min
) and γ
c
(t) = 0. Thus, in the
analytical extremizing solution, the variance of z(t) should be the sum γ(t) = at + bt
2
and the
10
conditional variance should be γ
C
(t) = at. Given constraints (2), we can calculate the
parameters a and b as a = 2γ(1) γ(2)/2 = (1 ρ)γ and b = γ(1) + γ(2)/2 = ργ. Its EPLTs are
thus φ[z(t)] = (1 ρ + 2ρt) / (2 – 2ρ + 2ρt) (so that φ[z(0)] = 1/2 and φ[z(∞)] = 1), and φ
C
[z(t)]
= 1/2 (constant). These analytical solutions have been depicted in Figure 2 (lower panel),
where it can be seen that they are indistinguishable from the numerical ones except for
φ
C
[z(t)] for very large times (t > 2
5
), for which the numerical solution indicates values > 1/2
while the analytical solution is exactly 1/2.
Are these mathematically optimal results physically meaningful? It seems that both have
problems. The OU process heavily depends on the time scale (here assumed equal to 1) in
which the constraints in (2) were formulated; a different OU process would perhaps be
obtained if the three constraints were formulated at a different time scale. The problem here
emerges because the constraints were quantified based on induction (observational data as
explained in the introduction) rather than deduction. It is not reasonable to expect that the
actual dependence structure of a process might be affected by the choice of the time scale of
observation. The Hybrid process has a more fundamental problem: As shown in Figure 2, the
conditional entropy production is lower that the unconditional one at all times t. This
contradicts physical realism: Naturally, by observing the present state of a process (at t = 0),
the future entropy is reduced. In other words, the conditional entropy should be lower than the
unconditional for t > 0, whereas as t → ∞ conditional and unconditional entropies should tend
to be equal. However, this cannot happen if the entropy production is consistently lower in the
conditional than in the unconditional case. The problem arises here because of the fact that
noises, such as the constituents of the Hybrid process, are just mathematical constructs,
whereas Nature does not produce noises—rather it produces uncertainty, i.e. entropy [25]. As
will be seen in section
5, the Hybrid process has additional problems related to its
observability.
For this reason, to reinstate physical plausibility we impose an inequality constraint
φ
C
[z(t)] ≥ φ[z(t)], in addition to the equality constraints (2). In this case, the Hybrid model is
replaced by one in which the inequality constraint becomes binding. Our numerical
framework resulted in an extremizing solution whose weights ω
j
are all nonzero as shown in
Figure 1 (the solution marked as “HK”) and whose resulting EPLTs are shown in Figure 2.
The same solution is extremizing for both t 0 and t and for both the conditional and
unconditional cases. In this solution,
φ
C
[z(t)] = φ[z(t)] = H (20)
(independent of time). Despite being constant, this solution minimizes both EPLTs for t 0
and maximizes them for t → ∞. The small roughness of the curve shaped by the weights ω
j
, as
a function of the inverse scale k
j
, in Figure 1 may be due to numerical effects, but these effects
do not seem to disturb the constancy of EPLTs in Figure 2.
11
Under constant EPLT equal to H, combining (5) and (15) we obtain (1/2) d(ln γ)/d(ln t) =
H, or dγ/γ = 2H dt/t. This results in
γ(t) = t
2H
γ(1) (21)
which is readily recognized as the variance function of the cumulative HK process. In effect,
(21) can serve as a definition of the cumulative HK process, while the constant H is the well
known Hurst coefficient. The fact that H in (21) is identical to the EPLT gives it a sound
physical interpretation. Based on a numerical approximation from [24], the conditional
variance of the HK process is
γ
C
(t) = t
2H
c γ(1), c := 1 – (2H – 1)
2
[0.72(H – 1) + 1] (22)
Unlike the OU and Hybrid processes, the HK extremizing solution does not involve apparent
inconsistencies with physical reality.
5. A real world example
To examine the consistency with reality of each of the three stochastic processes extremizing
entropy production we present an example with real world data. We chose one of the longest
available instrumental geophysical records, the mean annual temperature of Vienna, Austria
(WMO station code: 11035 WIEN/HOHE_WAR; coordinates: 48.25ºN, 16.37ºE, 209 m)
which comprises 235 years of data (1775-2009) available online from the Royal Netherlands
Meteorological Institute (http://climexp.knmi.nl). Part of the record (141 years; 1851-1991)
has been included in the Global Historical Climatology Network (GHCN). Comparison of the
GHCN time series (available from the same web site), which has undergone consistency
checking and correction, with the original one shows that adjustments are minor (less than
±0.25º at the annual level), which increases confidence to the longer original data set. Both
time series are shown in Figure 3 (upper panel). The Gaussian distribution fits very well the
time series, whose skewness is almost zero.
The EPLT cannot be calculated directly from the time series but the entropy Φ[z(t)] can,
for a certain range of t, based on (4) and using sample estimates g(t) of γ(t). Denoting the total
observation period as T (235 years) and assuming a varying time step we form samples of
x
i
with size n = T/ (more precisely, the floor of T/). Here we varied from 1 to 23 years,
so that the sample size be at least 10. For each , the sample mean is
x
z(T) (/T) (23)
which is an unbiased estimator of the true mean µ, and the sample variance is
g() = 1
T/ – 1
l = 1
T/
(x
l
- x
)
2
(24)
12
6
7
8
9
10
11
12
1770 1790 1810 1830 1850 1870 1890 1910 1930 1950 1970 1990 2010
Θ (ºC)
Original Adjusted 30-year average
Slope = 0.5
Slope —> 1
Slope = 0.74
0
1
2
3
4
5
6
1
1.5
2
2.5
3
3.5
4
4.5
0 0.5 1 1.5 2 2.5 3 3.5 4
ln
Φ[x
]
ln γ(),
ln g(),
ln E[g()]
Empirical
White noise
Markov
Hybrid, theoretical
Hybrid, adapted
HK, theoretical
HK, adapted
_
Figure 3 (Upper) The time series of the annual temperature (Θ) in Vienna, Austria; (lower) the entropy Φ[x
],
along with the corresponding variances at time step , versus ln , as estimated from the time series using the
classical statistical variance estimate g() and as predicted by the three extremizing models theoretically (γ())
and after adaptation for bias (E[g[]).
In classical statistics, where x
l
are independent identically distributed, g() is an unbiased
estimator of the true variance γ(). However, if consecutive x
l
are dependent, the following
equation holds true:
13
E[g()] = T/
T/ – 1{Var[x
i
] – Var[x
]}
(25)
where Var[x
]
=
(/T)
2
Var[z(T)] and Var[x
i
]
=
γ(). This results in E[g()] =
c(,
T)
γ(),
where
c(, T) := 1 – (/T)
2
γ(T)/γ()
1 –/T
(26)
represents a bias adaptation factor, which equals 1 (no bias) only when γ()/γ(T) = /T, i.e. in
white noise. Depending on the dependence structure, the bias adaptation factor may be quite
smaller than 1 (although this is often missed in the literature). This is particularly the case in
the Hybrid model (Figure 3, lower panel), where E[g()] (plotted as a continuous line with
triangles) is almost indistinguishable from the true variance γ() of the white noise instead of
being close to the true variance of the Hybrid model (notice that there is a one-to-one
association between variance and entropy, shown in figure). This constitutes the observability
problem of the Hybrid model mentioned above.
In contrast, in the Markovian (OU) model (also shown in Figure 3, lower panel), the bias
adaptation is negligible (the two curves before and after adaptation are indistinguishable to
each other), i.e. the factor c(, T) is very close to unity. On the other hand, the OU model
implies the lowest theoretical entropy for large scale . After adaptation for bias, as shown in
Figure 3 (lower panel), the Hybrid model entails even lower observed entropy, despite that it
entails the highest theoretical entropy.
In the HK model the adaptation factor is closer to unity than in the Hybrid model (the
theoretical and adapted entropy curves are close to each other) but not as close as in the OU
model. In this respect, the HK model represents a good balance of observability on the one
hand, expressed by high c(, T), and, on the other hand, high entropy for large scales,
expressed by γ() or even by the ratio γ(T)/γ() (since T >> ). In the HK model the
adaptation factor c(, T) is maximized for H = 0.5 and the ratio γ(T)/γ() is maximized for H
= 1, in which however c(, T) = 0. The aforesaid balance is expressed by the fact that the
product of the two (i.e., c(, T) γ(T)/γ()) is maximized for H between 0 and 1 (specifically,
for H = 1 – (1/2) ln 2/ln(T/), as can be easily verified).
To compare the three models with reality, the proximity of empirical points (corresponding
to sample estimates g()) with the model should be assessed on the basis of the predicted
curves E[g()] of the models, rather than their theoretical variances γ(). The former are
designated in Figure 3 (lower panel) as “adapted” whereas the latter are designated as
“theoretical”. It is seen in the figure that the HK model is the only one that agrees with reality.
The curves E[g()] of the other two models lie far apart from empirical points. It is noted that
the consistency of the HK model with reality has been detected in numerous studies using
geophysical, biological, technological and even economical time series (e.g. [22-25,26,27,28,
29,30,31] and references therein) and this makes the findings of this study more physically
plausible.
14
6. Conclusion and discussion
It is demonstrated that extremization of entropy production of stochastic representations of
natural systems, performed at asymptotic times (zero of infinity) and using simple constraints
referred to finite times, at which a process is observed, results in constant derivative of
entropy in logarithmic time, which in turn results in Hurst-Kolmogorov processes and long-
term persistence. One eminent characteristic of the derivation is its parsimony, in terms of
both constraints and physical principles used. Specifically, it was demonstrated that no other
notions (e.g. self-organized criticality, scale invariance, etc.) in addition to entropy
extremization are necessary to explain the emergence of the Hurst-Kolmogorov behavior. An
example with real world data, which is in agreement with a large body of studies that have
detected long-term persistence in long time series of a wide range of processes, illustrates the
plausibility of the findings.
These findings connect statistical physics (the extremal entropy production concept, in
particular) with stochastic representations of natural processes. Extremal entropy production
may provide a theoretical background in such stochastic representations, which otherwise are
solely data-driven. A theoretical background in stochastic representations is important also for
the reason that, as noted by Koutsoyiannis and Montanari [31], merely statistical arguments
do not suffice to verify or falsify the presence of long-term persistence in natural processes.
The practical consequences of these findings may be significant because stochastic
representations of processes are used as modeling tools particularly for the estimation of
future uncertainty for planning and design purposes with long time horizons. The emergence
of maximum entropy (i.e., maximum uncertainty) for large time horizons, as demonstrated
here, should be considered seriously in planning and design studies, because otherwise the
uncertainty would be underestimated and the constructions undersized. The relevance of the
last point may be even wider, given the current scientific and public interest on long-term
predictions.
Acknowledgments I gratefully thank three reviewers for their encouraging comments and
constructive suggestions, which helped me to substantially improve the presentation.
References
[1] H. S. Robertson, Statistical Thermophysics (Prentice Hall, Englewood Cliffs, NJ, 1993).
[2] E. T. Jaynes, Probability Theory: The Logic of Science (Cambridge Univ. Press, 728 pp., 2003).
[3] P. Atkins, Four Laws that Drive the Universe (Oxford Univ. Press, 131 pp., 2007).
[4] J. Fischer et al., Report to the CIPM on the implications of changing the definition of the base unit Kelvin
(International Committee for Weights and Measures, http://www.bipm.org/wg/CCT/TG-
SI/Allowed/Documents/Report_to_CIPM_2.pdf, Retrieved 2010-11-28).
[5] A. Porporato et al., Irreversibility and fluctuation theorem in stationary time series, Phys. Rev. Lett., 98,
094101 (2007).
15
[6] D. Kondepudi and I. Prigogine, Modern Thermodynamics (Wiley, Chichester, 1998).
[7] T. Pujol and J. E. Llebot, Q. J. R. Meteorol. Soc., 125, 79-90 (1999) .
[8] H. Ozawa et al., Rev. Geophys., 41(4), 1018, doi:10.1029/2002RG000113 (2003).
[9] I. Prigogine, Bulletin de la Classe des Sciences, Academie Royale de Belgique, 31, 600-606 (1945).
[10] H. Ziegler, in Progress in Solid Mechanics ed. by I.N. Sneddon and R. Hill, vol. 4 (North-Holland,
Amsterdam, 1963).
[11] L.M. Martyushev and V.D. Seleznev, Phys. Rep., 426, 1-45 (2006).
[12] W. Pauli, Statistical Mechanics (Dover, New York, 1973).
[13] I. Müller, in Entropy, ed. by A. Greven, G. Keller and G. Warnecke, Ch. 5, 79–105 (Princeton University
Press, Princeton, New Jersey, USA, 2003).
[14] H.E. Hurst, Trans. Am. Soc. Civil Engrs., 116, 776–808 (1951).
[15] A.N. Kolmogorov, Dokl. Akad. Nauk URSS, 26, 115–118 (1940).
[16] B.B. Mandelbrot and J. W. Van Ness, SIAM Rev, 10 (4), 422-437 (1968).
[17] A. Papoulis, Probability, Random Variables and Stochastic Processes, 3rd edn. (McGraw-Hill, New York,
1991).
[18] C. Tsallis, Journal of Statistical Physics, 52(1), 479-487 (1988).
[19] C. Tsallis, Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World, Springer
(2009).
[20] F. Bouchet et al., Physica A, 389, 4389-4405 (2010).
[21] B. B. Mandelbrot, A fast fractional Gaussian noise generator. Wat. Resour. Res. 7 (3), 543–553 (1971).
[22] D. Koutsoyiannis, Hydrol. Sci. J., 47 (4), 573–595 (2002).
[23] S. Karlin and H. M. Taylor, A Second Course in Stochastic Processes (Academic Press, Boston, 1981).
[24] D. Koutsoyiannis, Hydrol. Sci. J., 50 (3), 405–426 (2005).
[25] D. Koutsoyiannis, Hydrol. Earth Sys. Sci., 14, 585–601 (2010).
[26] T. Lux, Appl. Econ. Lett., 1996, 3, 701–706 (1996).
[27] E. Koscielny-Bunde et al., Phys. Rev. Lett., 81(3), 729–732 (1998).
[28] W.E. Leland et al., IEEE/ACM Trans. Networking, 2(1), 1-15 (1994).
[29] A. Bunde et al., Phys. Rev. Lett., 85(17), 3736- 3739 (2000).
[30] A. Bunde and S. Havlin, Physica A, 314, 15-24 (2002).
[31] D. Koutsoyiannis and A. Montanari, Water Resour. Res., 43 (5), doi:10.1029/2006WR005592 (2007).
... Hence, in complex systems the quantity that gets extremized is the entropy-or it could be the entropy production when time is involved (Koutsoyiannis, 2011b). As will be clarified in section 2.9, entropy is a fully stochastic concept. ...
... The next important step was made by Shannon (1948), who used a definition essentially similar to Planck's to describe the information content, which he also called entropy at von Neumann's suggestion (Robertson, 1993;Brissaud, 2005;Koutsoyiannis, 2011b). According to the latter definition, entropy is a probabilistic concept, a measure of information or, equivalently, uncertainty. ...
... In a stochastic process the change of uncertainty in time can be quantified by the entropy production, i.e. the time derivative of the entropy [ ( )] of the cumulative process ( ) (Koutsoyiannis, 2011b): ...
Preprint
Full-text available
This is a working draft of a book in preparation. Current version 0.4 – uploaded on ResearchGate on 25 January 2022. (Earlier versions: 0.3 – uploaded on ResearchGate on 17 January 2022. 0.2 – uploaded on ResearchGate on 3 January 2022. 0.1 (initial) – uploaded on ResearchGate on 1 January 2022.) Some stuff is copied from Koutsoyiannis (2021, https://www.researchgate.net/ publication/351081149). Comments and suggestions will be greatly appreciated and acknowledged.
... The HK dynamics are also linked to the entropy maximization principle, and thus, to robust physical justification [8]. It is worth noting that the stochastic simulation of the HK dynamics is still a mathematical challenge [9] since it requires the explicit preservation of high-order moments in a vast range of scales [10,11], affecting both the intermittent (fractal) behavior in small scales [12] and the dependence in extremes [13]. ...
... depicts the scaled images at scales k = 2,4,8,16,20,25,40,50, 80, 100 and 200, which are used to calculate the 2D climacogram. Encyclopedia 2021, 1, Benchmark of image analysis, the evolution of the universe[68]: (a) 500 million years after Big Bang image with faint clustering, average brightness 0.45; (b) 5000 million years after Big Bang image with clustering, average brightness 0.37; (c) 10,000 million years after Big Bang image with intense clustering, average brightness 0.33. ...
Article
Full-text available
The stochastic analysis in the scale domain (instead of the traditional lag or frequency domains) is introduced as a robust means to identify, model and simulate the Hurst–Kolmogorov (HK) dynamics, ranging from small (fractal) to large scales exhibiting the clustering behavior (else known as the Hurst phenomenon or long-range dependence). The HK clustering is an attribute of a multidimensional (1D, 2D, etc.) spatio-temporal stationary stochastic process with an arbitrary marginal distribution function, and a fractal behavior on small spatio-temporal scales of the dependence structure and a power-type on large scales, yielding a high probability of low- or high-magnitude events to group together in space and time. This behavior is preferably analyzed through the second-order statistics, and in the scale domain, by the stochastic metric of the climacogram, i.e., the variance of the averaged spatio-temporal process vs. spatio-temporal scale.
... Data from [5,6]. 58 59 Figure 2. Relative frequency of appearances of the indicated key phrases in the article title, ab- 60 stract and keywords of about 70 million articles written in English, which are contained in the 61 Scopus database [7] up to year 2020. 62 Out of its physical and stochastic context, the term "entropy" is typically used meta- 63 phorically and hence its meaning becomes ambiguous or diverse. ...
... 195 The next important step was made by Shannon in 1948 [58]. Shannon used an essen-196 tially similar, albeit more general, entropy definition to describe the information content, 197 which he also called entropy at von Neumann's suggestion [59][60][61]. According to the latter 198 definition, entropy is a probabilistic concept, a measure of information or, equivalently, 199 uncertainty. ...
Preprint
Full-text available
While entropy was introduced in the second half of the 19th century in the international vocabulary as a scientific term, in the 20th century it became common in colloquial use. Popular imagination has loaded “entropy” with almost every negative quality in the universe, in life and in society, with a dominant meaning of disorder and disorganization. Exploring the history of the term and many different approaches on it, we show that entropy has a universal stochastic definition which is not disorder. The accompanying principle of maximum entropy, which lies behind the Second Law, gives explanatory and inferential power to the concept and promotes entropy as the mother of creativity and evolution. As social sciences are often contaminated by subjectivity and ideological influences, we try to explore whether the maximum entropy, applied to the distribution of wealth quantified by annual income, can give an objective description. Using publicly available income data, we show that the income distribution is consistent with the principle of the maximum entropy. The increase of entropy is associated to increase of society’s wealth yet a standardized form of en-tropy can be used to quantify inequality. Historically, technology has played a major role in de-velopment and increase of the entropy of income. Such findings are contrary to the theories of ecological economics and other theories which use the term entropy in a Malthusian perspective.
... Whether or not long-range dependence occurs in hydrological time series, and if it is pre-asymptotic behaviour only or whether any physical processes exist that can explain such a statistical model, remains a frequently discussed topic (e.g. Klemes 1974;Salas et al. 1979;Mesa and Poveda 1993;Beran 1994;Koutsoyiannis 2011). ...
Article
Full-text available
Previous studies suggest that flood-rich and flood-poor periods are present in many flood peak discharge series around the globe. Understanding the occurrence of these periods and their driving mechanisms is important for reliably estimating future flood probabilities. We propose a method for detecting flood-rich and flood-poor periods in peak-over-threshold series based on scan-statistics and combine it with a flood typology in order to attribute the periods to their flood-generating mechanisms. The method is applied to 164 observed flood series in southern Germany from 1930 to 2018. The results reveal significant flood-rich periods of heavy-rainfall floods, especially in the Danube river basin in the most recent decades. These are consistent with trend analyses from the literature. Additionally, significant flood-poor periods of snowmelt-floods in the immediate past were detected, especially for low-elevation catchments in the alpine foreland and the uplands. The occurrence of flood-rich and flood-poor periods is interpreted in terms of increases in the frequency of heavy rainfall in the alpine foreland and decreases of both soil moisture and snow cover in the midlands.
... Natural processes are expected to exhibit specific stochastic similarities because they are all bounded by the law of entropy extremization (e.g., see the physical justification of the LRD in [9]). This concept has been confirmed in recent studies, such as the one by Dimitriadis et al. [6], where stochastic similarities (including LRD) are traced in all key hourly and daily-scale hydrological-cycle processes (i.e., surface air temperature, relative humidity, dew-point, close-to-surface wind speed, atmospheric pressure, streamflow, and precipitation) by analysing the most comprehensive available databases (i.e., including tens of billions of values from several thousands of globally distributed stations). ...
Article
Full-text available
The identification of the second-order dependence structure of streamflow has been one of the oldest challenges in hydrological sciences, dating back to the pioneering work of H.E Hurst on the Nile River. Since then, several large-scale studies have investigated the temporal structure of streamflow spanning from the hourly to the climatic scale, covering multiple orders of magnitude. In this study, we expanded this range to almost eight orders of magnitude by analysing small-scale streamflow time series (in the order of minutes) from ground stations and large-scale streamflow time series (in the order of hundreds of years) acquired from paleoclimatic reconstructions. We aimed to determine the fractal behaviour and the long-range dependence behaviour of the stream-flow. Additionally, we assessed the behaviour of the first four marginal moments of each time series to test whether they follow similar behaviours as suggested in other studies in the literature. The results provide evidence in identifying a common stochastic structure for the streamflow process, based on the Pareto-Burr-Feller marginal distribution and a generalized Hurst-Kolmogorov (HK) dependence structure.
... This finding suggests how complexity, which captures the balance between emergence and self-organization, can explain the dynamics of Hu. The association can be attributed to the extremization of entropy production which results in persistence of the process (Koutsoyiannis 2011). It is observed that for low-snow-dominated catchments, the correlation between Hu and C is significant. ...
Article
In this study, catchments are considered as complex systems, and information-theoretic measures are used to capture temporal streamflow characteristics. Emergence and self-organization are used to quantify information production and order in streamflow time series, respectively. The measure complexity is used to quantify the balance between emergence and self-organization in streamflow variability. The complexity measure is found to be effective in distinguishing streamflow variability for high and low snow-dominated catchments. The state of persistence-reflecting the memory of streamflow time series, is shown to be related to the complexity of streamflow. Moreover, it is observed that conventional causal detection methods are constrained by the state of persistence, and more robust methods are needed in hydrological applications considering persistence.
... The estimators and models applied for both the marginal and the second-order dependence structures are part of the stochastic framework of the HK dynamics, with a focus on the LRD behavior [30][31][32][33][34][35], and they have been applied to turbulent and key hydrologicalcycle processes of global networks with resolutions spanning from small scales (relevant to the fractal behavior) to climatic scales (for a review, see [26]). ...
Article
Full-text available
The stochastic structures of potential evaporation and evapotranspiration (PEV and PET or ETo) are analyzed using the ERA5 hourly reanalysis data and the Penman–Monteith model applied to the well-known CIMIS network. The latter includes high-quality ground meteorological samples with long lengths and simultaneous measurements of monthly incoming shortwave radiation, temperature, relative humidity, and wind speed. It is found that both the PEV and PET processes exhibit a moderate long-range dependence structure with a Hurst parameter of 0.64 and 0.69, respectively. Additionally, it is noted that their marginal structures are found to be light-tailed when estimated through the Pareto–Burr–Feller distribution function. Both results are consistent with the global-scale hydrological-cycle path, determined by all the above variables and rainfall, in terms of the marginal and dependence structures. Finally, it is discussed how the existence of, even moderate, long-range dependence can increase the variability and uncertainty of both processes and, thus, limit their predictability.
... This approach is further enhanced to assess the uncertainty in the nonstationary parameters through Bayesian Inference (Cheng and AghaKouchak, 2015;Sarhadi and Soulis, 2017). Some studies argue that the incorporation of non-stationarity in extreme event modeling (Koutsoyiannis, 2011;Koutsoyiannis and Montanari, 2014) adds to the uncertainty, since the nonstationary modeling includes the changes merely based on the trend in the hydrologic variable. These changes may also be due to physical processes (Ragno et al., 2019;Serinaldi and Kilsby, 2015), and hence, incorporating the relevant physical processes in extreme event modeling is beneficial (Montanari and Koutsoyiannis, 2014;Ragno et al., 2019). ...
Article
The overwhelming increase and variations in the extreme rainfall events demand the use of a nonstationary Intensity-Duration-Frequency (IDF) curve for the design and management of water resource infrastructure. Generally, nonstationary IDF curves were developed by incorporating the trend in the distribution parameter using Generalized Extreme Value distribution (GEV) with time as a covariate. The physical process influencing the variations in a hydrologic variable can be captured by utilizing relevant climatic variables (climate-informed) as covariates, since time alone cannot be the best covariate. Hence this study investigates the potential climate-informed covariates influencing the extreme rainfall and incorporates the best covariates to develop a realistic nonstationary IDF relationship. Unlike previous studies, a Time Sliding Window (TSW) approach is employed to detect the changing distribution parameters before performing Nonstationary Modeling (NSM). The proposed covariate based TSW-NSM is effectively used to construct IDF curves for seven major metropolitan cities of India. Several models are generated based on the detected changing parameters and combinations of covariates. Then, Bayesian Differential Evolutionary Monte Carlo (DE-MC) algorithm is employed to estimate the uncertainty bound of the nonstationary parameters, and the best model is chosen using the Deviance Information Criterion (DIC). The results reveal that the best covariate combinations for short duration events are dominated by local processes (i.e., Local temperature changes and diurnal temperature changes), whereas the same for longer duration events are dominated by the global process (global warming, ENSO Modoki cycle, and IOD). However, the acceptable nonstationary models reveal that all the temperature-based covariates are capable of capturing the dynamic behavior. It is also observed that the local processes carry the signature of global processes. Finally, the return levels computed through the best nonstationary model show that the return periods are decreasing, and the short duration events have undergone drastic changes than the longer duration events. Therefore, employing climate-informed covariates based nonstationary IDF curves is indispensable for devising long-term strategies to address the changing climate.
... The next important step was made by Shannon in 1948 [69]. Shannon used an essentially similar, albeit more general, definition describing information content, which he also called entropy, at von Neumann's suggestion [70][71][72]. According to the latter definition, entropy is a probabilistic concept, a measure of information or, equivalently, uncertainty. ...
Article
Full-text available
While entropy was introduced in the second half of the 19th century in the international vocabulary as a scientific term, in the 20th century it became common in colloquial use. Popular imagination has loaded “entropy” with almost every negative quality in the universe, in life and in society, with a dominant meaning of disorder and disorganization. Exploring the history of the term and many different approaches to it, we show that entropy has a universal stochastic definition, which is not disorder. Hence, we contend that entropy should be used as a mathematical (stochastic) concept as rigorously as possible, free of metaphoric meanings. The accompanying principle of maximum entropy, which lies behind the Second Law, gives explanatory and inferential power to the concept, and promotes entropy as the mother of creativity and evolution. As the social sciences are often contaminated by subjectivity and ideological influences, we try to explore whether maximum entropy, applied to the distribution of a wealth-related variable, namely annual income, can give an objective description. Using publicly available income data, we show that income distribution is consistent with the principle of maximum entropy. The increase in entropy is associated to increases in society’s wealth, yet a standardized form of entropy can be used to quantify inequality. Historically, technology has played a major role in the development of and increase in the entropy of income. Such findings are contrary to the theory of ecological economics and other theories that use the term entropy in a Malthusian perspective.
... The statistical bias in the climacogram estimator can be calculated as follows. As shown in Koutsoyiannis's study [15], assuming that we have n = T/∆ observations of the averaged process x i (∆) , where the observation period T is an integer multiple of ∆, the time-resolution, the expected value of the empirical (sample) climacogramγ(∆) is: ...
Article
Full-text available
We investigate the impact of time's arrow on the hourly streamflow process. Although time asymmetry, i.e., temporal irreversibility, has been previously implemented in stochastics, it has only recently attracted attention in the hydrological literature. Relevant studies have shown that the time asymmetry of the streamflow process is manifested at scales up to several days and vanishes at larger scales. The latter highlights the need to reproduce it in flood simulations of fine-scale resolution. To this aim, we develop an enhancement of a recently proposed simulation algorithm for irreversible processes, based on an asymmetric moving average (AMA) scheme that allows for the explicit preservation of time asymmetry at two or more timescales. The method is successfully applied to a large hourly streamflow time series from the United States Geological Survey (USGS) database, with time asymmetry prominent at time scales up to four days.
Article
Full-text available
According to the traditional notion of randomness and uncertainty, natural phenomena are separated into two mutually exclusive components, random (or stochastic) and deterministic. Within this dichotomous logic, the deterministic part supposedly represents cause-effect relationships and, thus, is physics and science (the "good"), whereas randomness has little relationship with science and no relationship with understanding (the "evil"). We argue that such views should be reconsidered by admitting that uncertainty is an intrinsic property of nature, that causality implies dependence of natural processes in time, thus suggesting predictability, but even the tiniest uncertainty (e.g., in initial conditions) may result in unpredictability after a certain time horizon. On these premises it is possible to shape a consistent stochastic representation of natural processes, in which predictability (suggested by deterministic laws) and unpredictability (randomness) coexist and are not separable or additive components. Deciding which of the two dominates is simply a matter of specifying the time horizon of the prediction. Long horizons of prediction are inevitably associated with high uncertainty, whose quantification relies on understanding the long-term stochastic properties of the processes.
Article
Full-text available
We study the temporal correlations in the atmospheric variability by 14 meteorological stations around the globe, the variations of the daily maximum temperatures from their average values. We apply several methods that can systematically overcome possible nonstationarities in the data. We find that the persistence, characterized by the correlation C(s) of temperature variations separated by s days, approximately decays C\(s\)~s-gamma, with roughly the same exponent gamma≅0.7 for all stations considered. The range of this universal persistence law seems to exceed one decade, and is possibly even larger than the range of the temperature series considered.
Book
This book focuses on nonextensive statistical mechanics, a current generalization of Boltzmann-Gibbs (BG) statistical mechanics, one of the greatest monuments of contemporary physics. Conceived more than 130 years ago by Maxwell, Boltzmann and Gibbs, the BG theory exhibits many impressive successes in physics, chemistry, mathematics, and computational sciences. Presently, several thousands of publications by scientists around the world have been dedicated to its nonextensive generalization. A variety of applications have emerged in complex systems and its mathematical grounding is by now well advanced. A pedagogical introduction to its concepts - nonlinear dynamics, extensivity of the nonadditive entropy, global correlations, and extensions of the standard central limit theorems, among others - is presented in this book, as well as a selection of paradigmatic applications in various sciences and diversified experimental verifications of some of its predictions. Introduction to Nonextensive Statistical Mechanics is suitable for students and researchers with an interest in complex systems and statistical physics. © Springer Science+Business Media, LLC 2009. All rights reserved.
Article
By design fast fractional Gaussian noises (ffGn) have the following characteristics: The number of operations needed to generate them is relatively small. The long run statistical dependence that they exhibit is strong and has the form required for self similar hydrology. Their short run properties, as expressed by the correlations between successive or nearly successive yearly averages, are adjustable (within bounds) and can be fitted to the corresponding short run properties of records. Extension to the multidimensional (multisite) case should raise no essential difficulty. Finally, their definition, as sums of Markov-Gauss and other simple processes, fits the intuitive ideas that climate can be visualized as either unpredictably inhomogeneous or ruled by a hierarchy of variable regimes.
Article
Prologue 1. THE ZEROTH LAW: the concept of temperature 2. THE FIRST LAW: the conservation of energy 3. THE SECOND LAW: the increase in entropy 4. FREE ENERGY: the availability of work 5. THE THIRD LAW: the unattainability of zero Conclusion