Page 1

arXiv:0812.4370v1 [physics.data-an] 23 Dec 2008

EPJ manuscript No.

(will be inserted by the editor)

Nonadditive entropy: the concept and its use⋆

Constantino Tsallisa

Centro Brasileiro de Pesquisas Fisicas

and National Institute of Science and Technology for Complex Systems

Xavier Sigaud 150, 22290-180 Rio de Janeiro-RJ, Brazil

and

Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA

Received: date / Revised version: date

Abstract. The thermodynamical concept of entropy was introduced by Clausius in 1865 in order to con-

struct the exact differential dS = δQ/T, where δQ is the heat transfer and the absolute temperature T

its integrating factor. A few years later, in the period 1872-1877, it was shown by Boltzmann that this

quantity can be expressed in terms of the probabilities associated with the microscopic configurations of

the system. We refer to this fundamental connection as the Boltzmann-Gibbs (BG) entropy, namely (in

its discrete form) SBG = −kPW

i=1pilnpi, where k is the Boltzmann constant, and {pi} the probabilities

corresponding to the W microscopic configurations (hencePW

cussed by Gibbs, von Neumann and Shannon, and constituting the basis of the celebrated BG statistical

mechanics, is additive. Indeed, if we consider a system composed by any two probabilistically independent

subsystems A and B (i.e., pA+B

ij

= pA

system is constituted by N equal elements which are are either independent or quasi-independent (i.e., not

too strongly correlated, in some specific nonlocal sense), this additivity guarantees SBG to be extensive

in the thermodynamical sense, i.e., that SBG(N) ∝ N in the N >> 1 limit. If, on the contrary, the cor-

relations between the N elements are strong enough, then the extensivity of SBG is lost, being therefore

incompatible with classical thermodynamics. In such a case, the many and precious relations described in

textbooks of thermodynamics become invalid. Along a line which will be shown to overcome this difficulty,

and which consistently enables the generalization of BG statistical mechanics, it was proposed in 1988 the

entropy Sq = k[1 −PW

theory, this and similar forms have in fact been repeatedly introduced before 1988. The entropic form

Sq is, for any q ?= 1, nonadditive. Indeed, for two probabilistically independent subsystems, it satisfies

Sq(A + B)/k = [Sq(A)/k] + [Sq(B)/k] + (1 − q)[Sq(A)/k][Sq(B)/k] ?= Sq(A)/k + Sq(B)/k. This form will

turn out to be extensive for an important class of nonlocal correlations, if q is set equal to a special value

different from unity, noted qent (where ent stands for entropy). In other words, for such systems, we verify

that Sqent(N) ∝ N (N >> 1), thus legitimating the use of the classical thermodynamical relations. Stan-

dard systems, for which SBG is extensive, obviously correspond to qent = 1. Quite complex systems exist in

the sense that, for them, no value of q exists such that Sq is extensive. Such systems are out of the present

scope: they might need forms of entropy different from Sq, or perhaps — more plainly — they are just

not susceptible at all for some sort of thermostatistical approach. Consistently with the results associated

with Sq, the q-generalizations of the Central Limit Theorem and of its extended L´ evy-Gnedenko form

have been achieved. These recent theorems could of course be the cause of the ubiquity of q-exponentials,

q-Gaussians and related mathematical forms in natural, artificial and social systems. All of the above, as

well as presently available experimental, observational and computational confirmations — in high energy

physics and elsewhere —, are briefly reviewed. Finally, we address a confusion which is quite common in

the literature, namely referring to distinct physical mechanisms versus distinct regimes of a single physical

mechanism.

i=1pi = 1). This entropic form, further dis-

ipB

j, ∀(i,j)), we verify that SBG(A + B) = SBG(A) + SBG(B). If a

i=1pq

i]/(q − 1)(q ∈ R; S1 = SBG). In the context of cybernetics and information

PACS. 05.20.-y Classical statistical mechanics; 02.50.Cw Probability theory; 05.90.+m Other topics in

statistical physics, thermodynamics, and nonlinear dynamical systems; 05.70.-a Thermodynamics

⋆To appear in Statistical Power-Law Tails in High Energy

Phenomena, ed. T.S. Biro, Eur. Phys. J. A (2009).

atsallis@cbpf.br

1 Introduction

The concept of entropy S, as well as its name, were intro-

duced in thermodynamics by Clausius in 1865 [1]. It was

Page 2

2Constantino Tsallis: Nonadditive entropy: the concept and its use

done on a purely macroscopic basis (the possible existence

of a microscopic world was merely speculative at the time,

with just a few incipient scientific indications), with the

purpose to associate with the heat transfer δQ an exact

differential. This was indeed achieved through the cele-

brated relation dS = δQ/T, where dS is the differential

entropy, and the absolute temperature T the integrating

factor.

A decade later, in the period 1872-1877, it was shown

by Boltzmann [2,3] that this quantity can be expressed

in terms of the probabilities associated with the micro-

scopic configurations of the system. We refer to this con-

nection — one of the deepest ever done in physics — as the

Boltzmann-Gibbs (BG) entropy. In its present (discrete)

form, it is written as follows

SBG= −k

W

?

i=1

pilnpi, (1)

where k is the Boltzmann constant (or some other conve-

nient value, e.g. k = 1, in areas outside physics, such as

information theory, cybernetics and others), and the prob-

abilities {pi} corresponding to the W microscopic config-

urations satisfy

W

?

i=1

pi= 1.(2)

This entropic form, further discussed by Gibbs [4], von

Neumann [5] and Shannon [6], constitutes the basis of

the BG statistical mechanics, one of the monuments of

contemporary physics. Eq. (1) satisfies a variety of conve-

nient mathematical properties (non-negativity, concavity,

expansibility, Lesche-stability, composability, Topsoe fac-

torizability, finite entropy production per unit time satis-

fying the Pesin identity). For equal probabilities, i.e., for

pi= 1/W (∀i), it attains its maximal value, namely

SBG= k lnW .(3)

In the present context, let us focus on its additivity prop-

erty. An entropy S({pi}) is said additive [7] if, for a system

composed by any two probabilistically independent subsys-

tems A and B (i.e., satisfying pA+B

1,2,...,WA; j = 1,2,...,WB), we verify

ij

= pA

ipB

j, ∀(i,j); i =

S(A + B) = S(A) + S(B),(4)

where S(A + B) ≡ S({pA+B

S(B) ≡ S({pB

SBG({pi}) given by Eq. (1) is additive. Due to this prop-

erty, the BG entropy of any system made of N equal and

independent elements satisfies

ij

}), S(A) ≡ S({pA

i}) and

j}). It is straightforward to verify that

SBG(N) = NSBG(1). (5)

This fact obviously complies with the classical thermody-

namical requirement for the entropy S to be extensive,

i.e., such that

S(N)

N

lim

N→∞

< ∞. (6)

Indeed, in such a case,

lim

N→∞

SBG(N)

N

= SBG(1) ≤ kln[W(1)], (7)

where W(1) is the number, assumed finite, of possible con-

figurations of one element.

If the system is constituted by N equal elements which

are not strictly independent, but quasi-independent in-

stead (i.e., not too strongly correlated, in some nonlo-

cal sense to be further clarified later on; typically for a

Hamiltonian many-body system whose elements interact

through short-range interactions, or which are weakly quan-

tum -entangled), the additivity of SBGguarantees its ex-

tensivity in the thermodynamical sense, i.e., that Eq. (6)

is satisfied.

If, on the contrary, the correlations between the N ele-

ments are strong enough (a feature which might typically

occur for nonergodic states, e.g., in Hamiltonian many-

body systems with long-range interactions, or which are

strongly quantum-entangled), then the extensivity of SBG

might be lost (at least at the level of a large subsystem

of a much larger system), being therefore incompatible

with classical thermodynamics. In such a case, many of

the useful relations described in textbooks of thermody-

namics may become invalid. It is precisely this pathologi-

cal class of systems the one which is addressed within the

thermostatistical theory usually referred to as nonexten-

sive statistical mechanics [8,18,19], described in the next

Section.

2 Nonadditive entropy and nonextensive

statistical mechanics

2.1 Nonadditive entropy Sq

As an attempt to generalize BG statistical mechanics, and

possibly provide a frame for handling some of the above

mentioned pathological systems, it was postulated in 1988

[8] the following entropy:

Sq= k1 −?W

i=1pq

i

q − 1

(q ∈ R; S1= SBG), (8)

A simple manner to obtain S1= SBGis through the use of

pq−1

i

= e(q−1)lnpi∼ 1+(q−1)lnpi. If q < 0, the sum must

be done only over configurations which have nonzero prob-

ability to occur. Entropy (8) can be conveniently rewritten

in the following alternative forms:

Sq= k

W

?

i=1

pilnq(1/pi) (9)

= −k

W

?

i=1

pq

ilnqpi

(10)

= −k

W

?

i=1

piln2−qpi, (11)

Page 3

Constantino Tsallis: Nonadditive entropy: the concept and its use3

where the q-logarithmic function is defined as follows:

lnqz ≡z1−q− 1

1 − q

(q ∈ R; z ≥ 0; ln1z = lnz). (12)

Sq attains its extremum (maximum for q > 0, and min-

imum for q < 0) for equal probabilities, and its value is

given by

Sq= k lnqW .(13)

It can be shown to satisfy, for independent subsystems,

Sq(A + B)

k

=Sq(A)

k

+Sq(B)

k

+(1−q)Sq(A)

k

Sq(B)

k

. (14)

Therefore, this entropy is generically nonadditive. It satis-

fies, nevertheless, most other properties (mentioned previ-

ously) of the entropy SBG. In other words, it constitutes a

sort of minimalistic generalization of SBG. From Eq. (14)

we obtain

Sq(A + B) = Sq(A) + Sq(B) +1 − q

k

Sq(A)Sq(B), (15)

which exhibits the equivalence between (q − 1) → 0 and

k → ∞. Since for stationary states (e.g., thermal equi-

librium), k appears multiplicatively accompanied by the

temperature T (i.e., in the form kT), k → ∞ turns out to

be equivalent to T → ∞. We may consider that it is here

where the fact emerges that, at the T → ∞ limit, all the

microcanonical, canonical, grand-canonical ensembles of

classical or quantum (Fermi-Dirac, Bose-Einstein, Gentile

parastatistics) statistics, q-statistics (as will become evi-

dent later on)), coincide, and coincide with the hypothesis

of equal probabilities for an isolated system.

Eq. (14) can be generalized in the presence of arbitrary

correlations between two sysbsystems A and B of a given

system. It becomes [9]

Sq(A + B)

k

=Sq(A)

k

+Sq(B|A)

k

+ (1 − q)Sq(A)

+ (1 − q)Sq(A|B)

k

Sq(B|A)

k

Sq(B)

k

=Sq(A|B)

k

+Sq(B)

kk

, (16)

where Sq(A + B) is to be calculated with the joint prob-

abilities

?pA+B

?pA

Sq(A|B) with the conditional probabilities

(analogously for Sq(B|A)): see [9] for full details. Eq. (16)

straightforwardly recovers Eq. (14) if A and B are inde-

pendent, hence Sq(A|B) = Sq(A) and Sq(B|A) = Sq(B).

It is precisely this nonadditivity the property which

enables thermodynamical extensivity. More precisely, if

both A and B are very large (i.e., NA>> 1 and NB>>

1), then a value of q, noted qent, might exist for which

Sqent(A + B) ∼ Sqent(A) + Sqent(B). In other words, if

a system has N >> 1 equal elements, it becomes possi-

ble that a special value of q exists such that generically

0 < limN→∞

?Sqent(N)/N?< ∞.

This interesting feature can be easily illustrated in the

case of equal probabilities, for which Eq. (13) holds. If

ij

?, Sq(A) with the marginal probabilities

j=1pA+B

ij

?(analogously for Sq(B)), and

i

?≡??WB

?pA+B

ij

/pB

j

?

W(N) ∼ CµN(with C > 0 and µ > 1), then qent = 1,

i.e., SBG(N) ∝ N

tions forbid many (typically most) microscopic configu-

rations to occur, then it might happen that the num-

ber Weff(N) of effective (or admissible) configurations

satisfies Weff(N) << W(N). If we have, in particular,

Weff∼ DNρ(with D > 0 and ρ ∈ R), then

qent= 1 −1

(N >> 1). But if strong correla-

ρ.

(17)

This type of highly restricted phase space may occur in

various systems, as has been numerically or analytically

illustrated in various examples. Let us briefly mention here

two of them that are analytically tractable, namely an

abstract probabilistic one and a physical one.

The probabilistic model consists in N correlated dis-

tinguishable binary variables [10]. The probabilities of the

2Nstates vanish excepting for ∼ (d+1)N of them (which

can be seen, in the classical Pascal-like triangular repre-

sentation, as a N-long “strip” whose width is (d + 1),

d being non-negative). This model asymptotically satis-

fies probabilistic scale-invariance (Leibnitz triangle rule)

in the limit N → ∞. It can be verified that

qent= 1 −1

d.

(18)

The physical model corresponds to a long ring of N

1/2 spins with ferromagnetic first-neighbor interactions at

zero temperature. The interactions are of the anisotropic

XY ones in the presence of a transverse external magnetic

field (i.e., along the Z direction) at its critical value. Two

well known universality classes are contained within such a

system, namely the Ising universality class (corresponding

to a central charge c = 1/2), and the isotropic XY uni-

versality class (corresponding to a central charge c = 1).

We consider a block of L successive spins among those

N spins, and address the entropy Sq(L) of the N → ∞

quantum system. More precisely, we are interested in

Sq(L) = k1 − Tr(ρL)q

q − 1

,(19)

where ρL≡ limN→∞Tr{N−L}ρ(N), ρ(N) being the den-

sity matrix associated with the system of N spins, and

where we have traced over all but the L successive spins.

We define, in this case, qentas the value of q for which the

block entropy Sq(L) is extensive, i.e., such that Sqent(L) ∝

L. Such value does exist [11], and it is given by qent =

√37−6 ≃ 0.0828 for c = 1/2, and qent=√10−3 ≃ 0.1623

for c = 1. By using a recent result within conformal quan-

tum field theory [12], these two values for qentcan be gen-

eralized for the entire class of (1 + 1)-dimensional models

characterized by a generic central charge c. It is obtained

[11]

√9 + c2− 3

qent=

c

. (20)

As we see, qentmonotonically increases from zero to unity

(the BG value!) when c increases from zero to infinity. For

Page 4

4Constantino Tsallis: Nonadditive entropy: the concept and its use

Table 1. QSS stands for quasi-stationary state ([16,17] and

references therein).

SYSTEM ENTROPY SBG

(additive)

ENTROPY Sq (q < 1)

(nonadditive)

Short-range

interactions,

weakly

entangled

blocks, etc

EXTENSIVE

NONEXTENSIVE

Long-range

interactions

(QSS),

strongly

entangled

blocks, etcNONEXTENSIVE

EXTENSIVE

c = 4, one obtains q = 1/2, which has already emerged in

general relativistic problems [13]; the possible connection,

if there is one, remains however without explanation at

the present time. Also, for c = 26, which corresponds to

string theory [14], we obtain qent ≃ 0.8913. Finally, it

also remains presently without explanation the reason for

which c → ∞ leads to the BG result, i.e., qent= 1.

The previous system is a fermionic d = 1 one, for

which it is known that SBG(L) ∝ lnL. Results, though

only numerical, also exist for a bosonic d = 2 system [11].

Once again extensivity only occurs for qent< 1, whereas

SBG(L) ∝ L. Finally it is known that, for the black hole,

the BG entropy is proportional to the area L2, instead

of being proportional to the volume L3(see, for instance,

[15]); in fact, more generally, it is known, for d-dimensional

bosonic systems, the so-called area law, i.e., the fact that

SBG(L) ∝ Ld−1, which obviously violates classical ther-

modynamics. All these anomalies are believed to be a con-

sequence of strongly nonlocal quantum entanglement. In

fact all of the above results can be unified through the

following conjectural expression, ∀d:

SBG(L) ∝ ln2−dL ≡Ld−1− 1

d − 1

?= Ld∝ N .(21)

In all these cases, as for the d = 1 and the d = 2 above

described examples, it might well exist a value of qent<

1 such that thermodynamic extensivity is ensured, i.e.,

Sqent(L) ∝ Ld∝ N. This would of course mean that

Clausius entropy should be, for this class of anomalous

(sub)systems, identified with Sqentand not with SBG. The

plausible conjectural scenario is summarized in Table 1.

2.2 Nonextensive statistical mechanics

Since Sqgeneralizes SBGand maintains most of its math-

ematically convenient properties (e.g., concavity, Lesche

stability, among others), it is quite natural to attempt the

q-generalization of BG statistical mechanics itself. This

extended theory is usually referred to in the literature as

nonextensive statistical mechanics. It was first proposed

in 1988 [8], and later on connected to thermodynamics

[18,19]. It has since then received a considerable amount

of applications and verifications in natural, artificial and

social systems [20,21,22,23,24]. Some of its predictions

have been experimentally and observationally checked in

systems such as the motion of Hydra viridissima [25] and

cells [26,27,28], defect turbulence [29], solar wind [30,31,

32,33], cold atoms in optical dissipative lattices [34], dusty

plasma [35], silo drainage [36,37], high-energy physics (see,

e.g., [38,39,40,41,42]). They have also been checked an-

alytically and computationally in various nonlinear dy-

namical problems such as the edge of chaos of simple

unimodal dissipative maps [43,44,45,46,47,48,49,50,51,

52,53], and long-range-interacting many-body Hamilto-

nian systems [54,55,16,17]; also, various applications to

the so-called scale-free networks are available [56,57,58,

59,60,61,62,63,64]. In one way or another, most if not all

of these systems appear to share slow (power-law rather

than exponential) sensitivity to the initial conditions. In

other words, for classical systems, at the level of first prin-

ciples, BG statistical mechanical concepts are legitimate

and fruitful when the system exhibits a positive maximal

Lyapunov exponent (corresponding essentially to Boltz-

mann’s molecular chaos hypothesis), whereas vanishing

maximal Lyapunov exponent appears to be necessary (al-

though probably not sufficient) for the applicability of the

nonextensive statistical mechanical concepts. The mecha-

nisms that typically yield q-statistics involve, at the meso-

scopic level, non-Markovian processes [65], multiplicative

noise [66], nonlinear Fokker-Planck equations [67,68,69,

70], and similar ones.

Let us now briefly review, within the present theory,

two important stationary-state distributions, namely the

q-generalizationof the celebrated BG weight (discrete case),

and the q-generalization of the Gaussian distribution (con-

tinuous case).

To generalize the BG factor for the canonical ensemble

(i.e., a system in a stationary state due to its contact with

a “thermostat”) we follow [19]. We must extremize Sqas

given by Eq. (8) with the constraints

W

?

i=1

pi= 1, (22)

and

W

?

i=1

EiP(q)

i

= Uq, (23)

where the escort distribution {P(q)

i

} is defined through

[P(q)

i

]1/q

?W

P(q)

i

≡

pq

j=1pq

i

?W

j

, pi=

j=1[P(q)

j

]1/q, (24)

and Uq is a finite quantity characterizing the width of

the energy distribution {pi}; {Ei} are the eigenvalues of

the system Hamiltonian (with the chosen boundary condi-

tions). Notice, by the way, that constraint (22) can equiv-

alently be written as?W

i=1P(q)

i

= 1.

Page 5

Constantino Tsallis: Nonadditive entropy: the concept and its use5

Through the entropic optimization procedure, we ob-

tain straighforwardly

pi=e−βq(Ei−Uq)

q

¯Zq

, (25)

with

βq≡

β

?W

j=1pq

j

, (26)

and

¯Zq≡

W

?

i

e−βq(Ei−Uq)

q

, (27)

with the q-exponential function (inverse of the previously

defined q-logarithm) ez

(1 − q)z > 0, and zero otherwise (ez

Lagrange parameter associated with the constraint (23).

Eq. (25) makes explicit that the probability distribution

is, for fixed βq, invariant with regard to the arbitrary

choice of the zero of energies. The stationary state (or

(meta)equilibrium) distribution (25) can be rewritten as

follows:

pi=e

q≡ [1 + (1 − q)z]1/(1−q)if 1 +

1= ez), β being the

−β′

q

Z′q

qEi

,(28)

with

Z′

q≡

W

?

j=1

e

−β′

q

qEj

, (29)

and

β′

q≡

βq

1 + (1 − q)βqUq

.(30)

The form (28) is particularly convenient for many appli-

cations where comparison with experimental or computa-

tional data is involved. Also, it makes clear that piasymp-

totically decays like 1/E1/(q−1)

i

for q < 1, instead of the familiar exponential decay with

Eifor q = 1.

The connection to thermodynamics is established in

what follows. It can be proved that

for q > 1, and has a cutoff

1

T=∂Sq

∂Uq

, (31)

with T ≡ 1/(kβ). Also we can prove, for the free energy,

Fq≡ Uq− TSq= −1

βlnqZq, (32)

where

lnqZq= lnq¯Zq− βUq. (33)

This relation takes into account the trivial fact that, in

contrast with what is usually done in BG statistics, the

energies {Ei} are here referred to Uq in Eq. (25). It can

also be proved

Uq= −∂

∂βlnqZq, (34)

as well as relations such as

Cq≡ T∂Sq

∂T

=∂Uq

∂T

= −T∂2Fq

∂T2. (35)

In fact the entire Legendre transformation structure of

thermodynamics is q-invariant, which is both remarkable

and welcome.

As a final remark, let us stress an interesting feature

concerning q > 1 (power-law decay for pi). Let us assume

that the energy spectrum has a (quasi-continuous) den-

sity state g(E). The normalization condition (22) implies

that?dE g(E)p(E) is finite. Since, for q > 1, p(E) decays

as 1/E1/(q−1), it follows that g(E)/E1/(q−1)must be inte-

grable at infinity. This determines the maximal value of q

which is mathematically admissible in the present theory

(for instance, if g(E) is constant, then q < 2 must be satis-

fied). Let us now focus on the other constraint, namely Eq.

(23). We immediately see that Eg(E)[p(E)]qmust also be

integrable at infinity, i.e., Eg(E)/Eq/(q−1)must be inte-

grable, which implies the same limit for q as before! So,

for instance, if g(E) is a constant, both the normalization

and the energy constraints are finite for q < 2. For q ≥ 2

the entire theory becomes mathematically inadmissible.

Let us address now the continuous case which gener-

alizes the Gaussian distribution. We want to extremize

Sq= k1 −?dx[p(x)]q

q − 1

(36)

with the constraints

?

dxp(x = 1, (37)

and

?

dxx2P(q)(x) = σ2, (38)

where

P(q)(x) ≡

[p(x)]q

?dx[p(x)]q, (39)

σ2being a fixed (positive) quantity. We obtain

p(x) =

e−βx2

q

?dxe−βx2

q

, (40)

where β > 0 can be determined by using constraint (38).

The entire theory is valid for q < 3, above which both the

normalization and the q-variance (38) diverge. For q ≥ 1

the distribution is defined for all values of x; for q < 1 it

has a finite support. For q < 5/3 the standard variance

?dxx2p(x) is finite; for q ≥ 5/3 it diverges. See [71] for

a numerical comparison between variance and q-variance,

which exhibits the considerable convenience of the latter.

2.3 q-generalized central limit theorems

We focus here on the q-generalization of the Central Limit

Theorem (CLT). Let us remind what the standard CLT