Phase diagrams for an evolutionary prisoner's dilemma game on two-dimensional lattices.

Abstract

The effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary prisoner's dilemma game with players located on the sites of different two-dimensional lattices. This system exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only the defectors remain alive. Using Monte Carlo simulations and the generalized mean-field approximations we have determined the phase boundaries (critical points) separating the two phases on the plane of the temperature (noise) and temptation to choose defection. In the zero temperature limit the cooperation can be sustained only for those connectivity structures where three-site clique percolation occurs.

Full text

Page 1

arXiv:cond-mat/0506433v1 [cond-mat.stat-mech] 17 Jun 2005
Phase diagrams for an evolutionary Prisoner’s Dilemma game
on two-dimensional lattices
Gy¨ orgy Szab´ o1, Jeromos Vukov2, and Attila Szolnoki1
1Research Institute for Technical Physics and Materials Science P.O. Box 49, H-1525 Budapest, Hungary
2Department of Biological Physics, E¨ otv¨ os University, H-1117 Budapest, P´ azm´ any P. stny. 1/A., Hungary
The effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary
Prisoner’s Dilemma game with players located on the sites of different two-dimensional lattices. This system
exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only
the defectors remain alive. Using systematic Monte Carlo simulations and different levels of the generalized
mean-field approximations we have determined the phase boundaries (critical points) separating the two phases
on the plane of the temperature (noise) and temptation to choose defection. In the zero temperature limit this
analysis suggests that the cooperation can be sustained only for those connectivity structures where three-site
clique percolation occurs.
PACS numbers: 89.65.-s, 05.50.+q, 02.50.+Le, 87.23.Ge
In the original (two-player and one-shot) Prisoner’s
Dilemma (PD) game [1, 2] the players should simultaneously
choose between two options, called defection and coopera-
tion. The selfish players wish to maximize their own income
in the knowledge of payoffs dependent on their choices. The
curiosity of PD game is hidden in the fact that the choice of
defection yields higher income independently of the partner’s
choice. However, if both players choose defection then their
individual income is lower than those obtained for mutual co-
operation when the maximum total payoff is shared equally.
The rational (intelligent) players cannot resolve this dilemma
and both of them choose defection (this is the so-called Nash-
equilibrium in the PD game). At the same time we find many
examples in the nature where the mutual cooperation (altru-
ism, ethical norms, etc.) emerges spontaneously among the
selfish individuals[3]. In the last decades several mechanisms
(e.g., kin selection [4], applicationof retaliating strategies [5],
andvoluntaryparticipation[6])arereportedwhichenforcethe
appearance of cooperation in the societies.
Thespatialversions[7, 8]oftheevolutionaryPDgamescan
explain the maintenance of cooperation for the iterated games
with a limited range of interaction if the players follow one of
thetwosimpleststrategies. Forthetwosimpleststrategies,de-
noted shortly as D and C, the player choose always defection
and cooperation, respectively. In the evolutionary games the
players wish to maximize their total payoff, coming from PD
games with the neighbors, by adopting one of the more suc-
cessful strategies available in their neighborhood. This type
of dynamics describes the behavior of the ecological systems
controlled by the Darwinian selection [9, 10].
Following the pioneering work of Nowak et al. [7, 8] the
two-strategyspatial evolutionaryPD gameshave alreadybeen
studied by several authors using different evolutionary rules
on a large class of backgrounds including social networks
[11, 12, 13, 14] (for a survey of lattice models see the papers
[15, 16, 17] and further references therein). In the present pa-
per our attention is focused on the effect of noise built into
the dynamical rule. It is turned out that the effect of noise on
the stationary concentration of cooperators depends strongly
on the topological features of the neighborhood and the mea-
sure of cooperation can be enhanced by increasing the noise
in some cases.
Forthis purposewe consideran evolutionaryPD gamewith
players located on the sites x of a two-dimensional lattice.
The players follow one of the above mentioned two strategies
whose distribution is described by a two-state Potts model,
i.e., sx= C or D, where for later convenience the states are
denoted by the two-dimensional unit vectors,
D =
?1
0
?
and C =
?0
1
?
.
(1)
In this notation the total income of player x can be expressed
as
Ux=
?
δ
s+
xA · sx+δ,
(2)
wheres+
mation runs over those four neighbors who the player x plays
PD gamewith. FollowingNowaket al. [7]the rescaledpayoff
matrix is given as
xdenotesthetransposeofthestatevectorsx, thesum-
A =
?0b
1c
?
, 1 < b < 2 − c , c < 0 .
(3)
The evolutionary process is governed by random sequential
strategy adoptions, that is, the randomly chosen player x
adopts one of the (randomly chosen) neighboring strategy (at
site y) with a probability depending on the payoff difference
W[sx← sy] =
1
1 + exp[(Ux− Uy)/K],
(4)
where K is the measure of stochastic uncertainties (noise) al-
lowing the irrational choices [18, 19].
Our analysis will be restricted to two-dimensional lattices
where the topologically equivalent sites have four neighbors
(z = 4) as indicated by the edges in Fig. 1. By this way
we can avoid the undesired effects due to the variation of the
number of co-players [7, 8, 20, 21].
The investigated connectivity structures are the square (1)
and Kagome (3) lattices, and a square lattice of four-site

Page 2

2
123
FIG. 1: Three two-dimensional lattice structures on which an evolu-
tionary Prisoner’s Dilemma game is studied.
cliques (2). The latter structure consists of four-site cliques
(withinacliquethenodesarelinkedto eachother)whosesites
are connected only to one external site belonging to the near-
est clique. These structures can be distinguishedtopologically
by considering the connectivity between the triangles (three-
site cliques). In fact, the square lattice is free of triangles, that
is, its clustering coefficient C = 0. On the contrary, C = 1/2
and C = 1/3 for the structures 2 and 3. On the Kagome lat-
tice percolation of overlapping triangles takes place whereas
the overlappingtriangles formisolated four-sitecliques on the
structure 2.
In order to investigate the relevance of the mentioned topo-
logical features first we show the prediction of the classical
mean-field theory where the state is characterized by the con-
centrationρ of cooperators. In this case the average payofffor
the C and D strategies are
UC= z[ρ + (1 − ρ)c] and UD= zρb .
(5)
The presentdynamicalrule, representedby theadoptionprob-
ability (2), yields the following equation of motion for the
concentration of cooperators:
∂ρ
∂t
= ρ(1 − ρ)[W(D ← C) − W(C ← D)]
= −ρ(1 − ρ)tanh
?UD− UC
2K
?
.
(6)
According to this differential equation ρ tends to zero for
arbitrary value of K as UD > UC. Shortly, the cooperators
become extinct in those systems satisfying the conditions of
mean-field approximation, e.g., if the temporal co-players are
chosen randomly or in a system where all the possible pairs
play a game with each other (infinite range of interaction).
Here it is worth mentioning that the cooperators also die
out in the one-dimensional system [22] because for a con-
fronting cooperator-defector pair the maximum cooperator’s
payoff(1+c) is always less than the minimumdefector’s pay-
off (b).
For higher dimension, however, the cooperator can receive
support from more than one neighboring cooperators and its
total income can exceed the neighboring defector’s income.
For such a connectivity structure the cooperation can be sus-
tained within a region of b (and c) dependent on the value of
noise (K). This work is addressed to quantify the regions of
the b-K parameter plain where cooperation can emerge. For
sake of simplicity, our analysis will be restricted to the limit
c → −0 which is suggested by Nowak et al. in their pioneer-
ing work [7].
Figure 2 shows the concentration of cooperators on the
square lattice when increasing b for three different values of
K. These data are obtainedby Monte Carlo (MC) simulations
performed on a block of L × L sites under periodic bound-
ary conditions. The linear size is varied from L = 400 to
L = 2000. The larger sizes are used in the close vicinity
of the extinction of cooperators because this critical transition
belongs to the so called directed percolation (DP) universality
class [22, 23, 24, 25].
1.001.02 1.04
b
1.06 1.08
0.0
0.2
0.4
0.6
ρ
FIG. 2: Monte Carlo results for the concentration of cooperators vs.
b for three different temperatures: K = 0.1 (pluses), 0.4 (squares),
and 1.2 (diamonds) on the square lattice.
In the stationary state the concentration of cooperators is
independentof the initial state and decreases monotonouslyif
b is increased. Above a threshold value (b > bcr), however,
the C strategies always die out and the system remains in the
homogeneousD state for ever. The value of bcris determined
for many different values of K and the results of the system-
atic MC simulations are summarized in Fig. 3. Notice that bcr
reaches its maximum value at about K = 0.32 and bcr(K)
tends to 1 if K goes to either 0 or ∞. Henceforth this plot
is considered as a phase diagram because the cooperators can
survive only below the bcr(K) curve indicated by the solid
line connecting the MC data in Fig. 3.
This phase diagram differs significantly from those pre-
dicted by the above mentioned mean-field approximation
[b(mf)
cr
(K) = 1]. More adequate theoretical results are ex-
pected when using the pair approximation detailed in [15].
This approachis able to describe the coexistence of the C and
D strategies, however, the value of bcris significantly overes-
timated as shown by the dashed line (in Fig. 3) which goes to
2 in the limit K → 0. This serious shortage can be reduced
by using the more sophisticated extensions of this technique
when all possible configuration probabilities are determined
on larger clusters. The generalization is straightforward from
two-site clusters (pair approximation) to larger blocks (the
essence of this method is briefly described in [26, 27]). Ne-

Page 3

3
0.0 0.51.0
K
1.52.0
1.0
1.1
1.2
1.3
bcr
FIG. 3: Critical value of b as a function of temperature on the square
lattice. Symbols come from Monte Carlo simulations, the dashed,
dotted, and dashed-dotted lines represent the prediction of general-
ized mean-field approximation for 2-, 2×2-, and 3×3-site clusters.
glecting the technical details now we report only the results of
this calculation for the levels of 2×2- and 3×3-site clusters.
In both cases the calculations reproduce the main qualitative
features (see Fig. 3), that is, bcr(K) has a maximum at a finite
K and bcr = 1 in the limits K → 0 and ∞. Evidently, the
accuracy of this approach is improved when choosing larger
and larger clusters and the strikingly large deviations between
the prediction of different levels refer to the importance of the
complex short range order affected by the local topological
features.
0.00.5 1.0
K
1.52.0
1.0
1.1
1.2
1.3
b
C+D
D
FIG. 4: b − K phase diagram on the structure 2 illustrated in Fig. 1.
Symbols denote the MC data. The dotted and dashed lines illustrate
the phase boundary between the D and (C +D) phases as predicted
by generalized mean-field approximations for the four- and eight-site
clusters shown at the top.
This behavior indicates the existence of a noise level pro-
viding the highest measure of cooperation at a fixed b for a
square lattice connectivity structure. This means furthermore
that one can observe two subsequent phase transitions (both
are belonging to the DP universality class) if K is increased
from zero for a fixed value of temptation b < max(bcr).
On structure 2 the results of MC simulations are very sim-
ilar to those found on the square lattice (the differences are
comparable to the symbol size) as shown in Fig. 4. In con-
trary to the square lattice, the four-site approximation overes-
timates the results of MC simulations obtained on structure 2.
At the same time the prediction of the eight-site approxima-
tion fits very well to the MC data for low noises (K < 0.3).
It is suspected that the prediction of eight-site approximation
(particularly for large K vales) can be observed on such non-
spatial structures where four-site cliques are substituted for
the nodes of a random regular graph (or Bethe lattice) with a
degree of four. (Notice, that the eight-site cluster is equivalent
to a pair of four-site cliques and the pair approximationseems
to be more correct for the Bethe lattice due to the absence of
loops [28]).
In contraryto the abovephase diagrams, a qualitativelydif-
ferent behavior is observed on the Kagome lattice as illus-
trated in Fig. 5. The most striking feature is that here the crit-
ical value of b decreases monotonously if K is increased and
bcr(K = 0) = 3/2 in agreement with the prediction of the
three- and five-site approximations.
0.00.51.0
K
1.5 2.0
1.0
1.2
1.4
b
C+D
D
FIG. 5: Phase diagram on the Kagome lattice [structure 3 in Fig. 1].
Symbols denote the MC data. The dotted and dashed lines illustrate
the phase boundary suggested by the three- and five-site approxima-
tion on the clusters shown at the top.
In orderto deducea generalpicture aboutthe relevanttopo-
logical features supporting the maintenance of cooperation in
the low noise limit (for the suggested dynamics) we have be-
gun to study several other connectivity structures. Accord-
ing to the preliminary results the latest phase diagram (see
Fig. 5) is reproduced qualitatively on the square lattice with
first and second neighbor interactions (z = 8), on the trian-
gular lattice (z = 6), and on the body centered cubic lattice
(z = 8). In agreement with our expectation, the prediction
of the five-site approximation (shown in Fig. 5) is reproduced

Page 4

4
very well by the MC results obtained on the random regular
structure (z = 4) constructed from one-site overlapping trian-
gles. For all these structures the overlapping triangles (three-
site cliques) span the whole system. It would be interesting to
check the emergence of cooperation (in the K → 0 limit) on
other networks where clique percolation takes place [29, 30].
We have to emphasize, however, that the cooperation is not
favored within the large cliques according to the mean-field
arguments mentioned above. This might be another reason
why the cooperationvanishes on the structure 2 in the K → 0
limit. In agreement with the above conjecture the cooperators
die out for vanishing K on the cubic (z = 6) and honeycomb
(z = 3) lattices. Besides it, the one-dimensional lattice with
first and second neighbor interactions (z = 4) represents an
exception (because it inherits the one-dimensionalfeatures on
large scales) exhibiting a sharp transition between the homo-
geneous states (from C to D if b is increased for a fixed K).
In summary, we have studied systematically the effect of
noise K (allowing irrational strategy adoptions) and temp-
tation b to choose defection on the measure of cooperation
in an evolutionary Prisoner’s Dilemma game for such two-
dimensional lattice structures where the number of neighbors
is fixed, z = 4. For the investigated dynamical rule two ba-
sically different behaviors can be distinguished when varying
the connectivity structures. In the first case the cooperators
die out in the zero noise limit and the maintenance of coop-
eration can be optimized by choosing a suitable level of noise
for any fixed value of temptation if b < max(bcr). In the
second case the highest measure of cooperationoccurs for the
lowest temptation (b = 1) and noise K = 0 and the critical
value of b decreases if K is increased. It is conjectured that
the second behavior occurs for all the d-dimensional, d ≥ 2,
or non-spatial (e.g., Bethe lattice or random regular graphs)
connectivity structures where the overlapping triangles span
the whole system. This indicates that the percolation of the
overlapping triangle in the connectivity structure can provide
the optimum topological condition for the maintenance of co-
operationin the situations of multi-agent Prisoner’s Dilemma.
In the last years several algorithms were introduced to cre-
ate a large class of networks [31, 32, 33] and very recently the
gamesare also suggestedto control the evolutionof a network
[34, 35]. The above results raise the chance that similar evo-
lutionary PD games (in the zero noise limit) can be utilized to
control the creation of networks of percolating triangles.
Acknowledgments
This work was supported by the Hungarian National Re-
search Fund (Grant No. T-47003) and by the European Sci-
ence Foundation (COST P10).
[1] J. W. Weibull, Evolutionary Game Theory (MIT Press, Cam-
bridge, Mass., 1995).
[2] H. Gintis, Game Theory Evolving (Princeton University Press,
Princeton, 2000).
[3] M. A. Nowak, R. M. May, and K. Sigmund, Sci. Am. 272
(1995).
[4] W. D. Hamilton, J. Theor. Biol. 7, 17 (1964).
[5] R. Axelrod, The Evolution of Cooperation (Basic Books, New
York, 1984).
[6] C. Hauert, S. De Monte, J. Hofbauer, and K. Sigmund, Science
296, 1129 (2002).
[7] M. A. Nowak and R. M. May, Int. J. Bifurcat. Chaos 3, 35
(1993).
[8] M. A. Nowak, S. Bonhoeffer, and R. M. May, Int. J. Bifurcat.
Chaos 4, 33 (1994).
[9] J. Maynard Smith, Evolution and the theory of games (Cam-
bridge University Press, Cambridge, 1982).
[10] J. Hofbauer and K. Sigmund, Evolutionary Games and Pop-
ulation Dynamics (Cambridge University Press, Cambridge,
1998).
[11] G. Abramson and M. Kuperman, Phys. Rev. E 63, 030901
(2001).
[12] B. J. Kim, A. Trusina, P. Holme, P. Minnhagen, J. S. Chung,
and M. Y. Choi, Phys. Rev. E 66, 021907 (2002).
[13] N. Masuda and K. Aihara, Phys. Lett. A 313, 55 (2003).
[14] H. Ebel and S. Bornholdt, Phys. Rev. E 66, 056118 (2002).
[15] C. Hauert and G. Szab´ o, Am. J. Phys. 73, 405 (2005).
[16] M. Nakamaru, H. Matsuda, and Y. Iwasa, J. Theor. Biol. 184,
65 (1997).
[17] K. Lindgren and M. G. Nordahl, Physica D 75, 292 (1994).
[18] L. E. Blume, Games Econ. Behav. 5, 387 (1993).
[19] G. Szab´ o and C. Hauert, Phys. Rev. E 66, 062903 (2002).
[20] C. Hauert, Proc. R. Soc. Lond. B 268, 761 (2001).
[21] F. Schweitzer, L. Behera, and H. M¨ uhlenbein, Adv. Complex
Systems 5, 269 (2003).
[22] G. Szab´ o, T. Antal, P. Szab´ o, and M. Droz, Phys. Rev. E 62,
1095 (2000).
[23] G. Szab´ o and C. T˝ oke, Phys. Rev. E 58, 69 (1998).
[24] J. Marro and R. Dickman, Nonequilibrium Phase Transitions
in Lattice Models (Cambridge University Press, Cambridge,
1999).
[25] H. Hinrichsen, Adv. Phys. 49, 815 (2000).
[26] R. Dickman, Phys. Rev. A 38, 2588 (1988).
[27] G. Szab´ o, A. Szolnoki, and R. Izs´ ak, J. Phys. A: Math. Gen. 37,
2599 (2004).
[28] G. Szab´ o and J. Vukov, Phys. Rev. E 69, 036107 (2004).
[29] I. Der´ enyi, G. Palla, and T. Vicsek, Phys. Rev. Lett. 94, 160202
(2005).
[30] G. Palla, I. Der´ enyi, I. Farkas, and T. Vicsek, Nature 435, 814
(2005).
[31] R. Albert and A.-L. Barab´ asi, Rev. Mod. Phys. 74, 47 (2002).
[32] J. F. F. Mendes and S. N. Dorogovtsev, Evolution of Networks:
From Biological to the Internet and WWW (Oxford University
Press, New York, 2003).
[33] M. E. J. Newman, SIAM Rev. 45, 167 (2003).
[34] M. G.Zimmermann, V. M. Egu´ ıluz, and M. SanMiguel, inEco-
nomics and Heterogeneous Interacting Agents, edited by J. B.
Zimmermann and A. Kirman (Springer Verlag, Berlin, 2001),
pp. 73–86.
[35] C.Biely, K.Dragosits,
arXiv:physics/0504190.
andS.Thurner (2005),