Phase diagrams for an evolutionary prisoner's dilemma game on two-dimensional lattices.
ABSTRACT The effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary prisoner's dilemma game with players located on the sites of different two-dimensional lattices. This system exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only the defectors remain alive. Using Monte Carlo simulations and the generalized mean-field approximations we have determined the phase boundaries (critical points) separating the two phases on the plane of the temperature (noise) and temptation to choose defection. In the zero temperature limit the cooperation can be sustained only for those connectivity structures where three-site clique percolation occurs.
[show abstract] [hide abstract]
ABSTRACT: Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.PLoS ONE 01/2012; 7(2):e30689. · 4.09 Impact Factor
[show abstract] [hide abstract]
ABSTRACT: Social networks affect in such a fundamental way the dynamics of the population they support that the global, population-wide behavior that one observes often bears no relation to the individual processes it stems from. Up to now, linking the global networked dynamics to such individual mechanisms has remained elusive. Here we study the evolution of cooperation in networked populations and let individuals interact via a 2-person Prisoner's Dilemma--a characteristic defection dominant social dilemma of cooperation. We show how homogeneous networks transform a Prisoner's Dilemma into a population-wide evolutionary dynamics that promotes the coexistence between cooperators and defectors, while heterogeneous networks promote their coordination. To this end, we define a dynamic variable that allows us to track the self-organization of cooperators when co-evolving with defectors in networked populations. Using the same variable, we show how the global dynamics--and effective dilemma--co-evolves with the motifs of cooperators in the population, the overall emergence of cooperation depending sensitively on this co-evolution.PLoS ONE 01/2012; 7(2):e32114. · 4.09 Impact Factor
[show abstract] [hide abstract]
ABSTRACT: Coevolution of individual strategies and social ties, in which individuals not only adjust their strategies by social learning but also switch their adverse partners to search for potential beneficial ones, has attracted increasing attention very recently. It is found that the interplay of strategic updating and partner network adaptation can facilitate the escape from the stalemate of cooperation in social dilemmas. But the question how individual preferential partner choice shapes the dynamical and topological organization of cooperation has yet to be fully answered. Here we propose a simple evolutionary game model to address this problem. In our model, when severing a current disadvantageous partnership, individuals can choose a new partner, either among their friends of friends preferentially according to their reputation scores or randomly from the remaining population. In addition to partner switching, individuals also update their strategies by imitating social neighbors. The interplay between these two processes gives rise to rich evolutionary dynamics. We focus on both strategic and topological evolution. We find that reputation-based partner selection leads to highly heterogeneous and often disassortative partner networks. During the coevolutionary process, a few successful individuals who attain a large number of partners emerge as social hubs and thus directly influence periphery individuals of small degree, forming leader–follower hierarchical structures. Cooperation prevails because of the positive feedback effects: good guys attract more partnerships and “the rich get richer.” Our work sheds light on the emergence and maintenance of cooperation on dynamically changing social networks, where reputation plays a decisive role in the formation of social ties. KeywordsCoevolutionary games–Social networks–Reputation–Partner switching04/2012; 1(3):354-369.
arXiv:cond-mat/0506433v1 [cond-mat.stat-mech] 17 Jun 2005
Phase diagrams for an evolutionary Prisoner’s Dilemma game
on two-dimensional lattices
Gy¨ orgy Szab´ o1, Jeromos Vukov2, and Attila Szolnoki1
1Research Institute for Technical Physics and Materials Science P.O. Box 49, H-1525 Budapest, Hungary
2Department of Biological Physics, E¨ otv¨ os University, H-1117 Budapest, P´ azm´ any P. stny. 1/A., Hungary
The effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary
Prisoner’s Dilemma game with players located on the sites of different two-dimensional lattices. This system
exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only
the defectors remain alive. Using systematic Monte Carlo simulations and different levels of the generalized
mean-field approximations we have determined the phase boundaries (critical points) separating the two phases
on the plane of the temperature (noise) and temptation to choose defection. In the zero temperature limit this
analysis suggests that the cooperation can be sustained only for those connectivity structures where three-site
clique percolation occurs.
PACS numbers: 89.65.-s, 05.50.+q, 02.50.+Le, 87.23.Ge
In the original (two-player and one-shot) Prisoner’s
Dilemma (PD) game [1, 2] the players should simultaneously
choose between two options, called defection and coopera-
tion. The selfish players wish to maximize their own income
in the knowledge of payoffs dependent on their choices. The
curiosity of PD game is hidden in the fact that the choice of
defection yields higher income independently of the partner’s
choice. However, if both players choose defection then their
individual income is lower than those obtained for mutual co-
operation when the maximum total payoff is shared equally.
The rational (intelligent) players cannot resolve this dilemma
and both of them choose defection (this is the so-called Nash-
equilibrium in the PD game). At the same time we find many
examples in the nature where the mutual cooperation (altru-
ism, ethical norms, etc.) emerges spontaneously among the
selfish individuals. In the last decades several mechanisms
(e.g., kin selection , applicationof retaliating strategies ,
appearance of cooperation in the societies.
explain the maintenance of cooperation for the iterated games
with a limited range of interaction if the players follow one of
noted shortly as D and C, the player choose always defection
and cooperation, respectively. In the evolutionary games the
players wish to maximize their total payoff, coming from PD
games with the neighbors, by adopting one of the more suc-
cessful strategies available in their neighborhood. This type
of dynamics describes the behavior of the ecological systems
controlled by the Darwinian selection [9, 10].
Following the pioneering work of Nowak et al. [7, 8] the
two-strategyspatial evolutionaryPD gameshave alreadybeen
studied by several authors using different evolutionary rules
on a large class of backgrounds including social networks
[11, 12, 13, 14] (for a survey of lattice models see the papers
[15, 16, 17] and further references therein). In the present pa-
per our attention is focused on the effect of noise built into
the dynamical rule. It is turned out that the effect of noise on
the stationary concentration of cooperators depends strongly
on the topological features of the neighborhood and the mea-
sure of cooperation can be enhanced by increasing the noise
in some cases.
Forthis purposewe consideran evolutionaryPD gamewith
players located on the sites x of a two-dimensional lattice.
The players follow one of the above mentioned two strategies
whose distribution is described by a two-state Potts model,
i.e., sx= C or D, where for later convenience the states are
denoted by the two-dimensional unit vectors,
and C =
In this notation the total income of player x can be expressed
xA · sx+δ,
mation runs over those four neighbors who the player x plays
PD gamewith. FollowingNowaket al. the rescaledpayoff
matrix is given as
, 1 < b < 2 − c , c < 0 .
The evolutionary process is governed by random sequential
strategy adoptions, that is, the randomly chosen player x
adopts one of the (randomly chosen) neighboring strategy (at
site y) with a probability depending on the payoff difference
W[sx← sy] =
1 + exp[(Ux− Uy)/K],
where K is the measure of stochastic uncertainties (noise) al-
lowing the irrational choices [18, 19].
Our analysis will be restricted to two-dimensional lattices
where the topologically equivalent sites have four neighbors
(z = 4) as indicated by the edges in Fig. 1. By this way
we can avoid the undesired effects due to the variation of the
number of co-players [7, 8, 20, 21].
The investigated connectivity structures are the square (1)
and Kagome (3) lattices, and a square lattice of four-site
FIG. 1: Three two-dimensional lattice structures on which an evolu-
tionary Prisoner’s Dilemma game is studied.
cliques (2). The latter structure consists of four-site cliques
are connected only to one external site belonging to the near-
est clique. These structures can be distinguishedtopologically
by considering the connectivity between the triangles (three-
site cliques). In fact, the square lattice is free of triangles, that
is, its clustering coefficient C = 0. On the contrary, C = 1/2
and C = 1/3 for the structures 2 and 3. On the Kagome lat-
tice percolation of overlapping triangles takes place whereas
the overlappingtriangles formisolated four-sitecliques on the
In order to investigate the relevance of the mentioned topo-
logical features first we show the prediction of the classical
mean-field theory where the state is characterized by the con-
centrationρ of cooperators. In this case the average payofffor
the C and D strategies are
UC= z[ρ + (1 − ρ)c] and UD= zρb .
The presentdynamicalrule, representedby theadoptionprob-
ability (2), yields the following equation of motion for the
concentration of cooperators:
= ρ(1 − ρ)[W(D ← C) − W(C ← D)]
= −ρ(1 − ρ)tanh
According to this differential equation ρ tends to zero for
arbitrary value of K as UD > UC. Shortly, the cooperators
become extinct in those systems satisfying the conditions of
mean-field approximation, e.g., if the temporal co-players are
chosen randomly or in a system where all the possible pairs
play a game with each other (infinite range of interaction).
Here it is worth mentioning that the cooperators also die
out in the one-dimensional system  because for a con-
fronting cooperator-defector pair the maximum cooperator’s
payoff(1+c) is always less than the minimumdefector’s pay-
For higher dimension, however, the cooperator can receive
support from more than one neighboring cooperators and its
total income can exceed the neighboring defector’s income.
For such a connectivity structure the cooperation can be sus-
tained within a region of b (and c) dependent on the value of
noise (K). This work is addressed to quantify the regions of
the b-K parameter plain where cooperation can emerge. For
sake of simplicity, our analysis will be restricted to the limit
c → −0 which is suggested by Nowak et al. in their pioneer-
ing work .
Figure 2 shows the concentration of cooperators on the
square lattice when increasing b for three different values of
K. These data are obtainedby Monte Carlo (MC) simulations
performed on a block of L × L sites under periodic bound-
ary conditions. The linear size is varied from L = 400 to
L = 2000. The larger sizes are used in the close vicinity
of the extinction of cooperators because this critical transition
belongs to the so called directed percolation (DP) universality
class [22, 23, 24, 25].
FIG. 2: Monte Carlo results for the concentration of cooperators vs.
b for three different temperatures: K = 0.1 (pluses), 0.4 (squares),
and 1.2 (diamonds) on the square lattice.
In the stationary state the concentration of cooperators is
independentof the initial state and decreases monotonouslyif
b is increased. Above a threshold value (b > bcr), however,
the C strategies always die out and the system remains in the
homogeneousD state for ever. The value of bcris determined
for many different values of K and the results of the system-
atic MC simulations are summarized in Fig. 3. Notice that bcr
reaches its maximum value at about K = 0.32 and bcr(K)
tends to 1 if K goes to either 0 or ∞. Henceforth this plot
is considered as a phase diagram because the cooperators can
survive only below the bcr(K) curve indicated by the solid
line connecting the MC data in Fig. 3.
This phase diagram differs significantly from those pre-
dicted by the above mentioned mean-field approximation
(K) = 1]. More adequate theoretical results are ex-
pected when using the pair approximation detailed in .
This approachis able to describe the coexistence of the C and
D strategies, however, the value of bcris significantly overes-
timated as shown by the dashed line (in Fig. 3) which goes to
2 in the limit K → 0. This serious shortage can be reduced
by using the more sophisticated extensions of this technique
when all possible configuration probabilities are determined
on larger clusters. The generalization is straightforward from
two-site clusters (pair approximation) to larger blocks (the
essence of this method is briefly described in [26, 27]). Ne-
FIG. 3: Critical value of b as a function of temperature on the square
lattice. Symbols come from Monte Carlo simulations, the dashed,
dotted, and dashed-dotted lines represent the prediction of general-
ized mean-field approximation for 2-, 2×2-, and 3×3-site clusters.
glecting the technical details now we report only the results of
this calculation for the levels of 2×2- and 3×3-site clusters.
In both cases the calculations reproduce the main qualitative
features (see Fig. 3), that is, bcr(K) has a maximum at a finite
K and bcr = 1 in the limits K → 0 and ∞. Evidently, the
accuracy of this approach is improved when choosing larger
and larger clusters and the strikingly large deviations between
the prediction of different levels refer to the importance of the
complex short range order affected by the local topological
FIG. 4: b − K phase diagram on the structure 2 illustrated in Fig. 1.
Symbols denote the MC data. The dotted and dashed lines illustrate
the phase boundary between the D and (C +D) phases as predicted
by generalized mean-field approximations for the four- and eight-site
clusters shown at the top.
This behavior indicates the existence of a noise level pro-
viding the highest measure of cooperation at a fixed b for a
square lattice connectivity structure. This means furthermore
that one can observe two subsequent phase transitions (both
are belonging to the DP universality class) if K is increased
from zero for a fixed value of temptation b < max(bcr).
On structure 2 the results of MC simulations are very sim-
ilar to those found on the square lattice (the differences are
comparable to the symbol size) as shown in Fig. 4. In con-
trary to the square lattice, the four-site approximation overes-
timates the results of MC simulations obtained on structure 2.
At the same time the prediction of the eight-site approxima-
tion fits very well to the MC data for low noises (K < 0.3).
It is suspected that the prediction of eight-site approximation
(particularly for large K vales) can be observed on such non-
spatial structures where four-site cliques are substituted for
the nodes of a random regular graph (or Bethe lattice) with a
degree of four. (Notice, that the eight-site cluster is equivalent
to a pair of four-site cliques and the pair approximationseems
to be more correct for the Bethe lattice due to the absence of
In contraryto the abovephase diagrams, a qualitativelydif-
ferent behavior is observed on the Kagome lattice as illus-
trated in Fig. 5. The most striking feature is that here the crit-
ical value of b decreases monotonously if K is increased and
bcr(K = 0) = 3/2 in agreement with the prediction of the
three- and five-site approximations.
FIG. 5: Phase diagram on the Kagome lattice [structure 3 in Fig. 1].
Symbols denote the MC data. The dotted and dashed lines illustrate
the phase boundary suggested by the three- and five-site approxima-
tion on the clusters shown at the top.
In orderto deducea generalpicture aboutthe relevanttopo-
logical features supporting the maintenance of cooperation in
the low noise limit (for the suggested dynamics) we have be-
gun to study several other connectivity structures. Accord-
ing to the preliminary results the latest phase diagram (see
Fig. 5) is reproduced qualitatively on the square lattice with
first and second neighbor interactions (z = 8), on the trian-
gular lattice (z = 6), and on the body centered cubic lattice
(z = 8). In agreement with our expectation, the prediction
of the five-site approximation (shown in Fig. 5) is reproduced
very well by the MC results obtained on the random regular
structure (z = 4) constructed from one-site overlapping trian-
gles. For all these structures the overlapping triangles (three-
site cliques) span the whole system. It would be interesting to
check the emergence of cooperation (in the K → 0 limit) on
other networks where clique percolation takes place [29, 30].
We have to emphasize, however, that the cooperation is not
favored within the large cliques according to the mean-field
arguments mentioned above. This might be another reason
why the cooperationvanishes on the structure 2 in the K → 0
limit. In agreement with the above conjecture the cooperators
die out for vanishing K on the cubic (z = 6) and honeycomb
(z = 3) lattices. Besides it, the one-dimensional lattice with
first and second neighbor interactions (z = 4) represents an
exception (because it inherits the one-dimensionalfeatures on
large scales) exhibiting a sharp transition between the homo-
geneous states (from C to D if b is increased for a fixed K).
In summary, we have studied systematically the effect of
noise K (allowing irrational strategy adoptions) and temp-
tation b to choose defection on the measure of cooperation
in an evolutionary Prisoner’s Dilemma game for such two-
dimensional lattice structures where the number of neighbors
is fixed, z = 4. For the investigated dynamical rule two ba-
sically different behaviors can be distinguished when varying
the connectivity structures. In the first case the cooperators
die out in the zero noise limit and the maintenance of coop-
eration can be optimized by choosing a suitable level of noise
for any fixed value of temptation if b < max(bcr). In the
second case the highest measure of cooperationoccurs for the
lowest temptation (b = 1) and noise K = 0 and the critical
value of b decreases if K is increased. It is conjectured that
the second behavior occurs for all the d-dimensional, d ≥ 2,
or non-spatial (e.g., Bethe lattice or random regular graphs)
connectivity structures where the overlapping triangles span
the whole system. This indicates that the percolation of the
overlapping triangle in the connectivity structure can provide
the optimum topological condition for the maintenance of co-
operationin the situations of multi-agent Prisoner’s Dilemma.
In the last years several algorithms were introduced to cre-
ate a large class of networks [31, 32, 33] and very recently the
gamesare also suggestedto control the evolutionof a network
[34, 35]. The above results raise the chance that similar evo-
lutionary PD games (in the zero noise limit) can be utilized to
control the creation of networks of percolating triangles.
This work was supported by the Hungarian National Re-
search Fund (Grant No. T-47003) and by the European Sci-
ence Foundation (COST P10).
 J. W. Weibull, Evolutionary Game Theory (MIT Press, Cam-
bridge, Mass., 1995).
 H. Gintis, Game Theory Evolving (Princeton University Press,
 M. A. Nowak, R. M. May, and K. Sigmund, Sci. Am. 272
 W. D. Hamilton, J. Theor. Biol. 7, 17 (1964).
 R. Axelrod, The Evolution of Cooperation (Basic Books, New
 C. Hauert, S. De Monte, J. Hofbauer, and K. Sigmund, Science
296, 1129 (2002).
 M. A. Nowak and R. M. May, Int. J. Bifurcat. Chaos 3, 35
 M. A. Nowak, S. Bonhoeffer, and R. M. May, Int. J. Bifurcat.
Chaos 4, 33 (1994).
 J. Maynard Smith, Evolution and the theory of games (Cam-
bridge University Press, Cambridge, 1982).
 J. Hofbauer and K. Sigmund, Evolutionary Games and Pop-
ulation Dynamics (Cambridge University Press, Cambridge,
 G. Abramson and M. Kuperman, Phys. Rev. E 63, 030901
 B. J. Kim, A. Trusina, P. Holme, P. Minnhagen, J. S. Chung,
and M. Y. Choi, Phys. Rev. E 66, 021907 (2002).
 N. Masuda and K. Aihara, Phys. Lett. A 313, 55 (2003).
 H. Ebel and S. Bornholdt, Phys. Rev. E 66, 056118 (2002).
 C. Hauert and G. Szab´ o, Am. J. Phys. 73, 405 (2005).
 M. Nakamaru, H. Matsuda, and Y. Iwasa, J. Theor. Biol. 184,
 K. Lindgren and M. G. Nordahl, Physica D 75, 292 (1994).
 L. E. Blume, Games Econ. Behav. 5, 387 (1993).
 G. Szab´ o and C. Hauert, Phys. Rev. E 66, 062903 (2002).
 C. Hauert, Proc. R. Soc. Lond. B 268, 761 (2001).
 F. Schweitzer, L. Behera, and H. M¨ uhlenbein, Adv. Complex
Systems 5, 269 (2003).
 G. Szab´ o, T. Antal, P. Szab´ o, and M. Droz, Phys. Rev. E 62,
 G. Szab´ o and C. T˝ oke, Phys. Rev. E 58, 69 (1998).
 J. Marro and R. Dickman, Nonequilibrium Phase Transitions
in Lattice Models (Cambridge University Press, Cambridge,
 H. Hinrichsen, Adv. Phys. 49, 815 (2000).
 R. Dickman, Phys. Rev. A 38, 2588 (1988).
 G. Szab´ o, A. Szolnoki, and R. Izs´ ak, J. Phys. A: Math. Gen. 37,
 G. Szab´ o and J. Vukov, Phys. Rev. E 69, 036107 (2004).
 I. Der´ enyi, G. Palla, and T. Vicsek, Phys. Rev. Lett. 94, 160202
 G. Palla, I. Der´ enyi, I. Farkas, and T. Vicsek, Nature 435, 814
 R. Albert and A.-L. Barab´ asi, Rev. Mod. Phys. 74, 47 (2002).
 J. F. F. Mendes and S. N. Dorogovtsev, Evolution of Networks:
From Biological to the Internet and WWW (Oxford University
Press, New York, 2003).
 M. E. J. Newman, SIAM Rev. 45, 167 (2003).
 M. G.Zimmermann, V. M. Egu´ ıluz, and M. SanMiguel, inEco-
nomics and Heterogeneous Interacting Agents, edited by J. B.
Zimmermann and A. Kirman (Springer Verlag, Berlin, 2001),
 C.Biely, K.Dragosits,