Page 1
The Evolutionary Stability of Cooperation
Jonathan Bendor; Piotr Swistak
The American Political Science Review, Vol. 91, No. 2. (Jun., 1997), pp. 290-307.
Stable URL:
http://links.jstor.org/sici?sici=0003-0554%28199706%2991%3A2%3C290%3ATESOC%3E2.0.CO%3B2-U
The American Political Science Review is currently published by American Political Science Association.
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained
prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in
the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/journals/apsa.html.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
The JSTOR Archive is a trusted digital repository providing for long-term preservation and access to leading academic
journals and scholarly literature from around the world. The Archive is supported by libraries, scholarly societies, publishers,
and foundations. It is an initiative of JSTOR, a not-for-profit organization with a mission to help the scholarly community take
advantage of advances in technology. For more information regarding JSTOR, please contact support@jstor.org.
http://www.jstor.org
Mon Nov 12 16:57:55 2007
Page 2
American Political Science Review Vol. 91, No. 2 June 1997
The Evolutionary Stability of Cooperation
JONATHAN BENDOR Stanford University
PIOTR SWISTAK University of Maryland
I
exceptionally influential work yields a new picture. Part of Axelrod's evolutionary story turns out to be false.
But the main intuition, that retaliatory strategies of conditional cooperation are somehow advantaged,
proves correct in one specijic and sigrzijicant sense: Under a standard evolutionary dynamic these strategies
require the minimalffequency to stabilize. Hence, they support the most robust evolutionary equilibrium: the
easiest to reach and retain. Moreover, the less eficient a strategy, the larger is its minimal stabilizing
ffequency; Hobbesian strategies of pure defection are the least robust. Our main theorems hold for a large
class of games that pose diverse cooperation problems: prisoner's dilemma, chicken, stag hunt, and many
others.
I
1986). Even formal institutions rarely specify every-
thing by role or employment contract; hence informal
cooperation, when viable, is the glue of such systems
(Chisholm 1989, Heclo 1977, Matthews 1960, Shils and
Janowitz 1948). Yet, what do we really know about the
evolution of cooperation? In some ways we know less
than we think: Much of what is commonly believed is
not true. Many probably believe that Axelrod's seminal
work, The Evolution of Cooperation (1984), established
the merits of the simple reciprocity-based strategy of
Tit for Tat (TFT). Hence we know that TFT "is
superior to all other [strategies] in playing repeated
games of prisoner's dilemma" (Trivers 1985, 391).
Hence we know that TFT is evolutionarily stable and is
successful because it is nice (never the first to defect),
retaliatory (it will defect in period t + 1if its partner
defects in t), and forgiving (it will return to coopera-
tion in t + 1if its partner cooperated in t). Hence we
know that TFT gets the highest score, if the future
matters sufficiently and enough others play TFT.
In fact, we know none of these things, because the
above claims are either false or ambiguous-the seem-
ingly settled facts about the evolution of cooperation
are not settled at all. Indeed, the most important part
of the conventional wisdom, that TFT is evolutionarily
s cooperation without central authority stable? If so, how robust is it? Despite what might be the
conventional wisdom, The Evolution of Cooperation did not solve this problem deductively. In fact,
results obtained later by others seem to have contradicted the book's main message. Reexamining this
t has been ably and amply argued that cooperation
is vital for systems without central authority (Axel-
rod 1981; Milgrom, North, and Weingast 1990; Oye
Jonathan Bendor is Professor of Public Policy at the Graduate
School of Business, Stanford University, Stanford CA 94305 (e-mail
bendor~onathan@gsb.stanford.edu). Piotr Swistak is Associate Pro-
fessor of Political Science, University of Maryland, College Park,
MD 20742 (e-mail pswistak@bss2.umd.edu).
An earlier version of this article was presented at the American
Political Science Association meetings in August 1992. Some of the
results were reported (though not established) in two nontechnical
summaries: Bendor and Swistak (1995, 1996a). For their comments
we thank John Aldrich, Dave Baron, Randy Calvert, James Coleman,
Gordon Hines, Hugo Hopenhayn, Marek Kaminski, David Lalman,
Susanne Lohmann, Mancur Olson, Ed Schwartz, Jeroen Swinkels,
and Peyton Young. We would especially like to thank Bob Axelrod
for his useful comments on various drafts of this paper. Swistak
thanks the General Research Board at the University of Maryland
for summer support.
stable, has been contradicted by a Nature article with
the unequivocal title of "No Pure Strategy Is Evolu-
tionarily Stable in the Repeated Prisoner's Dilemma"
(Boyd and Lorberbaum 1987). So in what sense, if any,
is TFT stable? Do cooperative strategies enjoy an
evolutionary advantage over less cooperative ones? If
so, what is the difference? The stability of cooperation
is an issue of the foremost importance for the science
of politics. Yet, many readers will be surprised, we
believe, to find out how little we know about it.
And so our task is twofold: First we must show that
there is a problem and then we must solve it. We begin
by analyzing the claims and counterclaims. The inher-
ent logic of this inquiry generates a succession of
auxiliary findings leading to our main results, which are
presented near the end of the article. Since the path
that leads to them is essential to understand their
meaning, comprehension requires patience.
The article is organized as follows. First we set out
assumptions and give a brief overview of evolutionary
game theory. We then analyze the fundamental ques-
tion of whether cooperative strategies like TFT are,
indeed, evolutionarily stable. In the following section
we examine why and in what ways TFT's crucial
properties are evolutionarily important. Next, we gen-
eralize the problem of cooperation from the prisoner's
dilemma to a much more general class of games. We
then deliver the fundamental results, theorems 7-9, for
this class. Theorem 7 shows that conditionally cooper-
ative strategies are, indeed, evolutionarily advantaged:
Across all ecologies with a finite number of strategies
evolving under the standard replicator dynamic, these
strategies require the minimal populational frequency
for stability. In this sense they support the most robust
evolutionary equilibrium. Theorem 8 demonstrates
that only strategies which are sufficiently cooperative
and provocable can support this most robust equilib-
rium. At the other end of the spectrum, theorem 9 and
its corollary show that Hobbesian strategies-those
that always defect with their clones-are
fragile of all evolutionarily stable strategies. These
results are established for a large class of games that
involve many different kinds of cooperation problems
the most
Page 3
American Political Science Review
(e.g., some coordination games, chicken, stag hunt, and
others).
EVOLUTIONARY ANALYSIS: BASIC
CONCEPTS AND DEFINITIONS
Before diving into the heart of the issue, we must
emphasize one point about our assumptions. The evo-
lution of cooperation has been intensively studied over
the last decade.l Since this scholarship is not unified
and lacks a single central result that can serve as a
reference point, knowing what happens under the
standard set of assumptions is important. Our results
follow from these standard premises. These assump-
tions are as postulated by Axelrod and other participants
in the debate (Axelrod 1981, 1984; Axelrod and Ham-
ilton 1981; Boyd and Lorberbaum 1987; Farrell and
Ware 1989).
Specifically, we model the problem of cooperation by
a tournament structure (Axelrod 1984). Here, people
meet randomly to play a two-person prisoner's di-
lemma (PD). The PD is repeated: With a k e d proba-
bility, 6, two players will encounter each other again in
the next round (0 < 6 < 1). The standard PD assump-
tions govern the one-shot game: Each player can either
cooperate or defect, with resulting payoffs T, R, P, S
such that T > R > P > S, and 2R > T + S. (These
are standard labels: T is the temptation payoff to the
player who defects while his or her partner cooperates,
R is the reward for mutual cooperation, P is the
punishment for mutual defection, and S is the sucker's
payoff to the player who cooperates while his or her
partner defects.) These choices and payoffs hold in
every round. At the start of a new period each player
knows what his or her partner did in the previous
rounds. The "dilemma" is that in a single-period game,
defection is a dominant strategy-player 1is better off
defecting no matter what player 2 does-but
use their dominant strategies, then both are worse off
than if they had cooperated.
Since cooperation is strongly dominated in the one-
period game, the only equilibrium outcome is for both
players to defect. The same is true when the game is
iterated but 6 is low (the future benefits from mutual
cooperation have a lower expected value than the
benefits from defecting on the current move), so
defecting always remains the best response to all
strategies. Yet, if S is large enough, then the expected
payoff from future cooperation suffices to support
constant cooperation in Nash equilibrium; this is just
one of a great variety of equilibria that are possible
with sufficiently large 6. In fact, standard folk theorems
state that for sufficiently high 6, any "amount" of
if both
Fora review of this work see Axelrod and Dion (1988) and Axelrod
and D'Ambrosio (1994). Other relevant articles on evolutionary
games are summarized in our last section. Much of the recent work
investigates stability under "trembles" or other stochastic conditions
that alter the assumptions of the original debate on the evolution of
cooperation. Naturally, different assumptions describe different stra-
tegic ecologies; which set best describes a particular ecology is an
empirical question that cannot be decided a priori.
Vol. 91, No. 2
mutual cooperation (from 0 to 100%) can be sustained
in Nash equilibrium.
Since the repeated game is strategically identical to
the one-shot if 6 is low (the only equilibrium being
mutual defection), it makes sense to study the evolu-
tion of cooperation only when 6 is "sufficiently" big.
Hence, all our results assume that the "shadow of the
future" is sufficiently long.
Definitions of Strategies
In addition to TFT, we use several other strategies as
examples. (Recall that TFT cooperates in period one
and thereafter cooperates if its opponent cooperated in
the previous period, and defects otherwise.) For easy
reference, we now list full names, abbreviated names,
and definitions of the most commonly used strategies.
The text will use the abbreviations.
TIT FOR 2 TATS (TF2T): Cooperate in periods 1and
2. Thereafter, defect in any period k r 3 if and only
if your opponent defected in k - 1 and k - 2.
SUSPICIOUS TIT FOR TAT (STFT): Defect in pe-
riod l, and for any period k r 2 do what your
opponent did in k - 1.
ALWAYS DEFECT (ALL D): Defect unconditionally
in all periods.
ALWAYS COOPERATE (ALL C): Cooperate uncon-
ditionally in all periods.
GRIM TRIGGER (GT): Cooperate in period ?1,and
never defect first; if the opponent defects in any
period k, defect from k + 1 on.
Evolutionary Game Theory
All evolutionary analyses are based on the principle of
survival of the fittest. What survive or die in evolution-
ary games are strategies. Thus, as Axelrod remarked in
his characteristically succinct way, "the evolutionary
approach is based on a simple principle: whatever is
successful is likely to appear more often in the future"
(1984, 169). This is a general idea. A precise definition
requires a careful setup. Let an ecology or population
be a finite set of strategies, j,, . . . , j , ,
respective proportions p ,, . . . , p,, with Xpi= 1.(For
example, Axelrod's first tournament had 15 strategies;
each was one-fifteenth of the ecology.) Let V(jk)
denote a standardized tournament score of strategy jk.
V(jk)-which corresponds to the "fitness" of jk-is
defined as the standardized sum of jk7s payoffs across
all its painvise games with other strategies in the
population: V(jk) = p1v(jk, j,) + . . . + pNV(jk, j,).
(V(i, j) denotes i's payoff when i plays j.) The assump-
tion that the number of strategies in the population is
k e d and finite is important. It permits the analysis of
games with infinitely many pure strategies, such as
infinitely repeated games, as well as games in which
both pure and mixed strategies replicate. What we have
to keep in mind, however, is that every evolutionary
game must be interpreted as population-spec@: The
process unfolds in a space defined by a specific set of
and their
Page 4
I
The Evolutionan, Stability of Cooperation
strategies. (For a more extensive discussion see Bendor
and Swistak 1996c.)
We will call a dynamic an evolutionaly process if for
all strategies j, and j, the following conditions are
satisfied:
-< -iff V(jk) < V(jl)
Pk Pl
A ~ k A ~ l
A ~ k Apl
-
Pk
iff V(jk) = V(jl)
PI
where the V's denote tournament scores in generation
T, and assuming that p andp' denote proportions in
two consecutive generations T and T + 1, A refers to
p' -p. When changes per generation are sufficiently
small, a process that is discrete in time (i.e., time points
are generations 1, 2, 3,. . .) can be approximated by a
process that is a continuous function of time. Condi-
tions 1and 2 with dpldt's substituted for Ap's define
the continuous form of an evolutionary process. Con-
dition 1 represents the basic monotonicity postulate
that more fit strategies have higher relative growth
rates. Condition 2, together with 1, implies that the
process is in equilibrium only if all strategies in an
ecology are equally fit. (What we call here an evolu-
tionary process is often referred to as a monotone
process [cf. Bendor and Swistak 1996c; Friedman 1991,
1994; Nachbar 19901.)
To summarize: The heart of evolutionary game
theory is the basic dynamic postulate that the better a
strategy performs in the present, the faster it grows.
The three elements of an evolutionary game-a
egy's stability, fitness, and painvise payoffs against
other strategies-are all related. This obvious though
important point turns out to be very c~nsequential.~
With these concepts defined, we can move on to the
notion of evolutionary stability. In general, a system is
dynamically stable if, after being perturbed slightly, it
returns to its original state. To visualize this, imagine
several hills separated by valleys, as in Figure 1.A ball
perched on a hill is in equilibrium: As long as no force
acts on the ball, it will stay where it is. But this is not a
stable equilibrium, for a slight push will send the ball
into a valley. Once the ball is in a valley it is stable: A
small disturbance will move it only part way up a hill,
and it will then roll back down into the valley. Intu-
itively, the deeper the valley, the more stable is the
equilibrium, for deeper valleys require larger shocks if
the ball is to be dislodged. The deeper valleys represent
the more robust equilibria. This notion of robustness
will be crucial for our main result, the Characterization
Theorem.
strat-
2 Indeed, some readers of Maynard Smith have, despite his warnings
(1982, 204), come to regard the evolutionary stability of a strategy i
as equivalent to i satisfying the following painvise payoff condition
for all j # i,
V(i, i) 2 VG, i), and if V(i, i) = VQ, i), then V(i,j) > VQ,j)
Doing so is a mistake: while (3) implies stability under some
conditions, it does not imply it under others (Bendor and Swistak
1992, 1996c; Weibull 1995).
(3)
June 1997
I FIGURE 1. The Robustness of Equilibria
When the system is a population of strategies, the
original position is the ecology's initial composition of
strategies; a perturbation is a change in this composi-
tion. By condition 2, in an evolutionary equilibrium all
strategies are equally "fit." Suppose, then, that strategy
i has completely taken over an ecology. Obviously, the
system is then in equilibrium. We can analyze its
stability by subjecting it to small shocks-introducing
new strategies-and seeing whether the ecology re-
turns to the original situation wherein everyone played
i.3 ?
TFT AND THE IDEA OF EVOLUTIONARY ?
STABILITY ?
In what sense are TFT and other conditional coopera-
tors evolutionarily stable? As we shall see, different
scholars have perhaps unknowingly had different con-
cepts in mind when answering this question. Clarifying
related conceptual issues is a major task of this section.
The shortest path to this goal happens to lead
through the back door, via the notion of disequilib-
rium. Just as different equilibria can be more or less
robust, so can disequilibria be more or less destructive.
Consider the most fatal disequilibrium of all.
Fundamental Instability
Suppose that all decision makers use the saintly ALL
C. Although, by condition 2, the process is in equilib-
rium, the balance is utterly unstable: If even one person
were to switch to ALL D, then slhe would do much
better than the "native" ALL C. No matter how small
the invasion and no matter what evolutionary process
governs reproduction, ALL C will be driven to extinc-
tion. It is the paradigmatic example of what we call a
fitndamentally unstable strategy, which we define as
follows. Suppose that 1 - E of all players in an ecology
In this standard kind of dynamic analysis, one perturbs the system
and then lets it reach a new equilibrium (e.g., the ball comes to a new
resting place). Clearly, one may obtain different results if the system
is shocked repeatedly, without allowing it to reach equilibria in
between disturbances (cf. Young and Foster 1991).
Page 5
American Political Science Review
Vol. 91, No. 2
FIGURE 2. Problems with the Collective Stability Concept
Game A
Game B
X
Y
X
Y
6, 6
0, 6
X
3, 3 0,3
X
Y
6, 0
3, 3
Y
3, 0
6,6
Note: In these two one-shot games the pure strategy x is collectively stable (i.e., V(x, x) 2 VO, x) for all j) yet fundamentally unstable. With (1 - E ) x's and
E y's in Game A V(x) = 6(1 - E) whereas V(y) = 6(1 - E) + 38,' in Game B V(x) = 3(1 - E) whereas V(y) = 3(1 - E) + 68. So V(x) < V(y), for all values of
E. Game A shows how a dynamic can go from the efficient state of (x, x) to the inefficient (y, y); Game B shows the opposite. Together they show the
problems with collective stability.
play the native strategy i, while a few (E > 0) play a
mutant strategy j. Furthermore, assume that j beats i
(VG) > V(i)) for every value of E. Then, i's frequency
will fall across generations until i disappears, no matter
which evolutionary dynamic holds sway or how small
was the invasion. Such an i we call fundamentally
unstable.
The Necessary Condition for Stability
ALL C's problem is that it is not provocable and so
cannot defend itself against exploitation. In more gen-
eral terms, ALL C is not a best reply to itself-a failing
that makes it fundamentally unstable (Bendor and
Swistak 1992,1996~).~
STFT, for instance, suffers from
the same problem. Because STFT defects in all periods
when playing its clone, yet can cooperate indefinitely
with a more "patient" strategy, say, TF2T, it is not the
best reply to itself if 6 is large enough. Any invasion by
TF2T will wipe out the population of STFT, which is
fundamentally unstable.
If a strategy fails to be a best response to itself, the
dynamic effects are critical-not
prisoner's dilemma (IPD), but in all games. Hence, it is
good to introduce this condition formally. A strategy i
is a best reply to itself if for all strategies j,
just in the iterated
V(i, i) 2 V(j, i).
(4)
This, of course, is better known as a pair of strategies
(i, i) being in Nash equilibrium.
The Instability of "Collective Stability"
Axelrod referred to a strategy satisfying condition 4 as
collectively stable (1981, 310). While, as we have seen,
condition 4 is necessary for evolutionary stability, it is
not sufficient: Strategies that satisfy condition 4 may be
fundamentally unstable. (See Figure 2 for two exam-
In other words, two ALL Cs are not in a Nash equilibrium. For
related results, see Nachbar 1990 and Hirshleifer and Martinez Coll
1988 (379).
ples.) For this reason they should not be called (col-
lectively) stable; condition 4 does not imply, in general,
any dynamic stability (Boyd and Lorberbaum 1987,
Sugden 1988, Swistak 1989). Put simply, Axelrod's
stability concept was incorrect.
Does TFT pass the necessary test of condition 4?
TFT is, in fact, a best reply to itself if 6 is sufficiently big
(Axelrod and Hamilton 1981). The formal proof is
simple and agrees with an intuition that T W s provo-
cability and niceness together ensure that for large 8,
the best reply to TFT is constant cooperation. In
particular, the kind of invasion that overcame STFT
cannot destabilize TFT; nor can ALL D destroy a
population of TFT as it did ALL C.
No Strategies are Strongly Stable
Yet, TFT cooperates with many different strategies.
This causes problems for TFT's evolutionary stability
(Selten and Hammerstein 1984). Suppose in a popula-
tion in which everyone plays TFT someone switches to
TF2T. What will happen? Since both TFT and TF2T
are nice, everyone in this ecology will cooperate in
every period and TFT and TF2T will be equally fit.
Hence, the ecology will not return to its original state of
everyone playing TFT-the
dure under any dynamics. And this is not an arcane
technical property; it has a natural and important
empirical interpretation. When everyone is playing
TFT, everyone cooperates in every period. Hence,
TFT's retaliatory nature remains latent. And because
an ability that is never deployed may deteriorate, some
actors may start using softer strategies, such as TF2T.
Thus, there may be an invasion of TF2Ts. As long as
only nice strategies are in the ecology, however, this
invasion is inconsequential: learning cannot occur and
the "mutant" TF2T cannot be driven out. Indeed, the
mutant cannot even be detected; TF2T is a "neutral
mutant" of TFT, being behaviorally indistinguishable
from TFT if no "non-nice" strategies are in the popu-
lation. In general, we will say that j is a neutral mutant
invading strategy will en-
Page 6
The Evolutionary Stability of Cooperation
of i if they are behaviorally identical when only i and j
are in the ecology.5
Hence, as Selten and Hammerstein conclude, TFT
cannot be evolutionarily stable in a strong sense, for it
cannot eliminate neutral invaders and therefore cannot
restore the original equilibrium. The same holds for all
pure strategies in the IPD. Suppose the native is STFT.
This native can be invaded by ALL D, a neutral mutant
of STFT. Similarly, ALL D is vulnerable to an invasion
by STFT.6
Yet, in these examples, though the native strategy is
unable to drive the mutant out, the mutant will not
(absent other mutations) spread in the population.
This suggests a distinction between two types of evo-
lutionary stability: a native strategy is strongly stable if,
after it is invaded, its proportion increases; a strategy is
weakly stable if, after it is invaded, its proportion does
not further decrease. (The Appendix defines these
concepts formally. An extended discussion of these
ideas is given in Bendor and Swistak 1996c.) Thus, TFT
may yet be weakly stable, although we now know that it
is not strongly stable.7
Indeed, the concept of strong stability is generally
inapplicable to repeated games. The reason is that
neutral mutants exist for every pure strategy in all
repeated games. (The proof is easy.) And even in
one-shot games, neutral mutants exist whenever the
game has multiple stages (e.g., buying guns or butter,
followed by a decision whether to go to war). So for
pure strategies, the criterion of strong stability must be
restricted to one-period games with a single stage.
These are games of little substantive importance in
political science.
No Strategies Are Even Weakly Stable
under All Processes
In one sense, that TFT is not strongly stable seems to
have little bearing on the evolution of cooperation.
Although TF2T can invade TFT, the ensuing evolu-
tionary equilibrium is behaviorally identical to the
preinvasion state of universal cooperation. But this
conclusion may be premature, for now we must exam-
ine the stability of this new equilibrium. What happens
if there is another slight perturbation? Suppose a STFT
invades so that the ecology contains a small fraction of
TF2T's and STFTs among the native TFTs. This
invasion is more substantial. Since different strategies
More precisely, for any strategy i, j is a neutral mutant of i if in an
iterated game with i, i and j follow the same path as i does when
playing another i; nodes that distinguish i from j are never reached.
It is precisely for this reason that no pure strategy in the IPD can
satisfy Maynard Smith's payoff condition (3).
For processes that satisfy certain regularity conditions-e.g.,
tinuous in p's with continuous partial derivatives, or Lipschitz
continuous (cf. Weibull 1995)-weak
Lyapunov stability) and strong stability implies asymptotic stability. It
is important to note, however, that in general our concepts of weak
and strong stability differ from stability and asymptotic stability.
Others have used different names for weak stability: Selten (1983),
Sobel (1993) and Warneryd (1993). Bomze and van Damme (1992)
use the same name as we do.
con-
stability implies stability (or
June 1997
have different fitnesses, selection mechanisms have a
chance to work. How will such an ecology evolve?
The strategies do not do equally well. TFT continues
to do well with TF2T and vice versa. But TFT's
provocability gets it into trouble with STFT, for follow-
ing STFT's defection, TFT retaliates in period two,
which STFT then punishes, and so on, indefinitely.
TF2T avoids this vendetta with STFT. Because TF2T
forgives isolated defections, it cooperates in period
two, which ensures mutual cooperation with STFT
from period three onward. Since 2R > T + S, TF2T's
ability to elicit cooperation from STFT is superior to a
vendetta.8 Consequently, TF2T's fitness exceeds TFT's,
which exceeds STFT's.
What are the dynamic implications of TFT not being
maximally fit? Will the population of TFT become
destabilized (Boyd and Lorberbaum 1987)? Not neces-
sarily. What happens depends on which specific dy-
namic governs replication.
TFT can be destabilized by certain evolutionary
processes. Suppose, for example, that actors simply
switch to the strategy with the highest fitness.9 Then,
after STFT's invasion, the decision makers will adopt
TF2T.lO Hence, TFT would not be even weakly stable
under this "imitate-the-winner" process. (In general,
we will call an evolutionary process imitate-the-winnerif
only maximally fit strategies increase under it.) Fur-
thermore, in the new status quo everyone plays TF2T,
a fragile equilibrium. TF2T's willingness to forgive
isolated defections invites exploitation by, for example,
ALTERNATOR, which defects in odd periods and
cooperates in even ones. For any value of 6, an
arbitrarily small invasion of ALTERNATOR can drive
TF2T to extinction. In turn, ALTERNATOR can be
wiped out by an arbitrarily small invasion of ALL D.
Thus, the entire structure of cooperation can collapse.
The invadability of TFT and the subsequent erosion
of cooperation may occasion some surprise. Is not TFT
a best reply to itself? Does that not imply that TFT is
maximally fit, if sufficiently common in the ecology?
The answer to the first question is "yes," but as we have
just seen, the answer to the second is "no." For
convenience, we will say that a strategy which is
maximally fit in any ecology in which it is sufficiently
common is unbeatable. The difference between a best-
reply strategy and an unbeatable one turns on ecolog-
ical effects. A strategy being a best response to itself is
a painvise property. Being unbeatable in a tournament
8 More precisely, if 6 is big enough then the best response to STFT
is always to cooperate.
Axelrod (1984, 159) used a variant of this rule in his analysis of
territorial dynamics: in generation T + 1 a decision maker adopts
whichever strategy scored highest in her "neighborhood" in T. Thus
players imitate local winners.
10 If everyone imitated the winner, then every strategy not maximally
fit in T would become extinct in T + 1. Hence, all such strategies
would have equal growth rates, despite (possibly) unequal fitnesses,
and so the process would fail condition 1. To accommodate this we
may require, for instance, that a person using strategy j does not
change his strategy with probability p(V(j)), where p(.) is strictly
increasing in V(j). (For example, if the maximal fitness in T is V*,
thenp(V(j)) = e(V(j)/V*), with 0 < E s 1.) If a person does change
his strategy, he adopts the winning one.
Page 7
American Political Science Review
involves more complex ecological effects, depending
not only on how the native i does against j, but also on
how each fares against a third strategy k. Boyd and
Lorberbaum (1987) use this observation to prove that
no pure strategy is unbeatable in the IPD.11 Here is
their argument. Take any pure native i. If a neutral
mutant, it, invades, then V(i, i) = V(if, i) = V(i, it)
= V(it, if). Accordingly, if another mutant j appears,
the ranking of i's and i"s tournament scores will
depend on how well they do against j, that is, on V(i, j)
versus V(it, j). Boyd and Lorberbaum show that for
every pure native i, there always exist a neutral mutant
if and a third strategy j that will favor if over the native
i, that is, V(it, j) > V(i, j). Hence, no pure strategy in
the IPD is unbeatable.
This result generalizes to all substantively interesting
games, as shown by theorem 1. (We have proven this
result elsewhere-Bendor and Swistak 1992, 1996c.)
THEOREM 1:For suficiently high 6 , no pure strategy is
unbeatable in any repeated nontrivial game.
We say that a symmetric one-shot game is trivial if
each player has a strategy that gives him the biggest
payoff in the game, no matter what action the opponent
takes. Such games exhibit no strategic interdependence
whatsoever.
Note that if a strategy is beatable, then it can be
destabilized under imitate-the-winner dynamics. As
theorem 1 shows, this is true of all pure strategies in
virtually all repeated games.
But Stability under Some Processes Is Still
Possible
That TFT can be destabilized by imitate-the-winner
does not imply that it can be destabilized by all
evolutionary processes. Imitate-the-winner is only one
type of dynamics that satisfy the basic evolutionary
postulate. While imitate-the-winner processes are intu-
itively compelling, other dynamics also may have sound
behavioral interpretations.
A specific dynamic, adopted by Axelrod in the
"ecological analysis" of his tournaments, presumes
growth that is linearly proportional in fitness. Ifp, and
ptkdenote strategy jk's proportions in two consecutive
generations, then the discrete form of the process can
be formally defined as
where v is the population's average fitness in the
current generation. The continuous form of the PFR is
defined and discussed in Bendor and Swistak (1996c), for
example. This evolutionary process, both discrete
and continuous, is sometimes called the proportional
fitness rule (PFR), a term we also shall use. It is
important to note that VG,)/V is larger than one if
11 Boyd and Lorberbaum's (1987) result has been extended to finitely
mixed strategies (Farrell and Ware 1989) and to completely mixed
strategies (Lorberbaum 1994).
Vol. 91. No. 2
jk9s fitness, VG,), exceeds v,the average fitness in the
population. So, under the PFR, jk's relative fre-
quency will keep increasing as long as its fitness is
higher than average.
Consequently, TFT need not be unbeatable to flour-
ish under the PFR. Under what conditions will TFT
thrive under this growth rule? Before we address this
open question, we must examine the nature of the
PFR.
The PFR-more often called the "replicator dynam-
ics"-is almost universally used in evolutionary models
of social phenomena.12 The rule, however, has been
mechanically transplanted from biology, where it has
an important substantive interpretation,13 to the social
sciences, where it does not. Until recently no one has
explained why the PFR should be considered a plausi-
ble mechanism of behavior transmission, though the
need for some justification has been apparent (Fried-
man 1991, Mailath 1992). Below we propose a frame-
work that provides a behavioral interpretation of the
PFR and other evolutionary processes. (Cabrales
[1993]; Gale, Binmore, and Samuelson [1995]; and
Schlag [I9941 have recently proposed alternative expla-
nations of the PFR.)
Evolutionary Processes, PFR, and the
Meaning of Evolutionary Change
A key notion of evolutionary analysis is that players
tend to adopt successful strategies and discard weak
ones. There are two obvious measures of success. The
first is a strategy's current relative performance,
V('j,)lV*: the strategy's raw fitness score, Vu,), com-
pared to some fitness parameter of the population, V*,
such as mean population fitness or maximal fitness.
This indicates how well j, does in the current genera-
tion. The second measure, a strategy's proportion in
the current generation (p,), reflects j,'s performance
in past generations: With players lacking intergenera-
tional memory, this is the only available measure of
j,'s past success. Formally, then, strategy j,'s propen-
sity to switch to strategy j,, 'rrjjs, is a function, F, of j,'s
proportion in the population, p,, and j,'s relative
performance, which we call V*(j,). In short, 'rrjj, =
F(p,, VG,)). Since p, and V*(i,) both measure j,'s
success, past and present, it is natural to assume that
F is nondecreasing in both variables. We now con-
sider two other properties of F and their meaning for
the type of act& they describe.
One substantively important consideration is the
relative effects ofp, and V* us) on F. If a change of A
in frequency produces a larger change in F than the
same change in relative performance (i.e., F(x + A, y)
> F(x, y + A)), then this means that an individual's
propensity to switch is influenced more by the past
(measured by the current proportions) than by the
-
'2 We use PFR (a name occasionally used in biology) instead of the ?
more conventional name "replicator dynamics," which we believe has ?
misleading connotations outside biological contexts. ?
13 In genetics the PFR is implied by the classical selection equation, ?
which in turn follows from the Hardy-Weinberg law of population ?
genetics (Hofbauer and Sigmund 1988, Schuster and Sigmund 1983). ?
Page 8
The Evolutionary Stability of Cooperation
current performance. Hence, at one extreme we have
homo sociologicus, whose F is a function only of the
past or tradition. At the other extreme we have homo
economicus, whose F is a function only of strategies'
current performance. At this extreme are processes
like imitate-the-winner. Between the extremes is an
actor who weights the past and present equally:
F(x + A,y) =F(x,y + A).
(6)
This property turns out to be an important character-
istic of the PFR.
A second substantively important consideration
turns on how the propensity F(x, y) changes as a
function of the two variables. Consider, for example,
the first variable-the proportion. On the one hand,
imagine a process for which a fixed increase of A inp,
would, other factors equal, increase F more the smaller
the p, (i.e., a2~lap: < 0). This type of adjustment
would be observed, for instance, with elite imitation
processes, in which the smaller the elite, the higher is
its status, hence the stronger its influence. (We define
this process in detail in Bendor and Swistak 1996c.
Many other processes are described in, e.g., Weibull
1995.) If, on the other hand, the lar 8er the proportion
the more F increases (i.e., a2~/ap, > 0), then the
change would be typical of snowball processes: The
larger the group of j,'s the stronger its impact on other
strategies. Between these two extremes are processes
for which the change is constant for all values of p,:
In this case, the propensity to change is unaffected by
either snobbish or populist.considerations. This, as our
next result shows, is the second important characteris-
tic of the PFR.
THEOREM 2: A propensity function F that is increasing in
both variables satisfies conditions 6 and 7if and only if
the corresponding evolutionary dynamic is the PFR.
Hence, we have linked properties of the PFR to clear
and significant behavioral interpretations. This is crit-
ical to sustain a hope that Axelrod's conjectures apply
to more than sticklebacks (Milinski 1987) and vampire
bats (Wilkinson 1984). This section puts forward two
conclusions that seem to be beyond doubt: The PFR is
a plausible model of some behavioral processes, and it
is not a plausible model of all behaviorally important
processes. What happens under the PFR is the subject
of this paper; what happens under other processes is an
important subject of future research.
In this section we have shown that strong stability is
impossible in iterated games. So is stability under all
processes. Yet, weak evolutionary stability under the
PFR may be feasible. This is the type of stability
studied below. To avoid awkwardness, a strategy that is
weakly evolutionarily stable under the PFR will be
called "weakly evolutionarily stable" (weak ESS) or
just "evolutionarily stable" (ESS) or "stable." We shall
call the behavior of a population of ESSs an evolution-
June 1997
arily stable state. As we have shown elsewhere (Bendor
and Swistak 1992, 1996c), a strategy i being weakly
stable under the PFR is equivalent to i satisfying the
following payoff condition: For all j,
v(i, i)
VQ, i), and if v(i, i) = VQ, i),
thenV(i,j) r V(j,j) (8)
Condition 8 is useful for a ~ractical and a theoretical
reason. First, it provides a iimple test for establishing
stability under the PFR. Second, it can be proven
(Bendor and Swistak 1992, 1996c) that condition 8 is a
borderline condition: A strategy is fundamentally un-
stable if and only if it fails to satisfy condition 8.
In the next section we establish some basic facts
about weak ESSs in the IPD. These results will dis-
prove several famous conjectures about TFT's "evolu-
tionary advantage."
THE VARIETY OF STABLE STRATEGIES
To show that TFT has an evolutionary advantage over
other strategies, it does not suffice to show that TFT
has the desirable dynamic properties; it is equally
essential to show that other strategies do not. As
theorems 3-5 reveal, establishing that TFT has an
advantage and figuring out why it does is more complex
than a reading of Axelrod (1984) would suggest.
Evolutionarily Stable States Can Exhibit Any
Amount of Cooperation
Since Axelrod's results emphasize two stable strate-
gies,l4 ALL D and TFT, some researchers (Dawkins
1989) interpret his works as if only two evolutionarily
stable states were possible: universal defection and
universal cooperation. To clarify this issue we examine
the feasible stable states and their underlying weak
ESSs. We begin by asking how much cooperation can
be achieved in a stable state.
We will say that a stable population state exhibits x
amount of cooperation if x is the percentage of moves
in which both players cooperate. This index is used by
the next result.
THEOREM 3 (EVOLUTIONARY
IPD): An evolutionarily stable state in tournaments of
iterated prisoner's dilemma with suficiently high 8 can
exhibit any amount of cooperation.
Hence, we have a profusion of stable states: from the
least efficient state of pure defection to the most
efficient state of pure cooperation. If we interpret the
amount of cooperation in a stable state as the strategy's
degree of "niceness," theorem 3 implies that ESSs can
have any degree of niceness; thus niceness and evolu-
tionary stability are unrelated.
FOR
FOLK THEOREM
THE
l4 Because weak evolutionary stability is a sound stability concept,
unlike collective stability, and because most of Axelrod's results
remain valid when redescribed in terms of the former, in the text we
recast his results in terms of weak ESSs.
Page 9
American Political Science Review
Vol. 91. No. 2
Evolutionarily Stable Strategies Can Be
Infinitely Exploited
A more subtle problem is whether being retaliatoiy-
TFT's second crucial property-is related to evolution-
ary stability. Being retaliatory confers an important
benefit: Such strategies cannot be exploited in more
than one period in a row. Thus, TFT's cumulative score
against any j never falls below j's score by more than
S - T. We call any strategy with this property
~nexploitable.~~
On the other end of the spectrum, we
will call a strategy i infinitely exploitable if there is a
strategy j such that lim,,,[V(i,
Hence, an infinitely exploitable strategy loses to some
other strategy j (in the sense of V(i, j) - VU,i)) by an
arbitrarily large amount as 6 + 1. Intuitively, one
might think that infinitely exploitable strategies cannot
be evolutionarily stable. Yet our next theorem shows
that they can.
THEOREM 4: For any 0 <x 5
stable strategy in the iterated prisoner's dilemma which
is infinitely exploitable and supports x amount of
cooperation in the stable state.
Together, theorems 3 and 4 imply that neither
niceness nor unexploitability are related to evolution-
ary stability per se. The set of ESSs contains strategies
with vastly diverse properties: from purely cooperative
to purely defective, and from unexploitable to infinitely
exploitable. The mere fact that a strategy, say, TFT, is
an ESS is hardly meaningful: That implies little about
its other properties.
Thus, if invaders can cluster by freely setting the ?
probability of interacting with their clones, then virtu- ?
ally any strategy-even the fundamentally unstable
ALL C-can invade ALL D. This property does not
make TFT distinctive.
Part (ii) of theorem 5 addresses a more demanding
question: How diverse are the strategies which, like
TFT, not only require the least clustering to increase
among ALL D but also stabilize once common in the
ecology? The answer: They are diverse indeed, ranging
from the unexploitable TFT to strategies that are
infinitely exploitable.
Theorems 3-5 show that arguments made about
TFT's "evolutionary edge" can be made about a great
many strategies, some of which differ profoundly from
TFT. The evolutionary advantage of TFT and other
conditionally cooperative strategies remains an open
question. This question is answered in our next section.
j) - V(j, i)] = -cc.
1, there is an evolutionarily
Strategies with Minimal Clustering Can Be
lnfinitely Exploited
Axelrod (1984) claims that one reason TFT is superior
is that it can invade a stable population of ALL Ds
given some population structure (i.e., if TFT is more
likely to interact with its clones than with ALL D),
whereas ALL D cannot invade a stable population of
TFTs with or without population structure. He further
asserts that among all strategies which can invade ALL
D in clusters, TFT requires the least clustering. The
problem, again, is how unique are these properties of
TFT. Theorem 5 gives the answers.
(i) With suficient clustering, any pure strategy that is
not a neutral mutant of ALL D can invade a
population of ALL D.
(ii) Among strategies that require the least clustering to
grow in a population ofALL D,there are strategies
which are both evolutionarily stable and infinitely
exploitable.
Formally we call a strategy i unexploitable if for all strategies j, all
periods n in the game, and all values of 6, V(i, j) -
T, where V" is the payoff in the game's initial n periods. Unexploit-
ability in this sense means unexploitable in more than one period in
a row. Because "pure unexploitability" (V"(i, j) - V"(j,i) 2 0) is
an attribute only of nasty (never the first to cooperate) and retalia-
tory strategies such as ALL D, it makes cooperating impossible.
i) 2 S
P O ' , -
WHY COOPERATION IS THE ROBUST
STABLE STATE AND WHICH STRATEGIES
SUPPORT IT IN "GAMES OF
COOPERATION"
The Domain: Games of Cooperation
When thinking about the results on the evolutionary
advantages of conditional cooperation, one must won-
der whether they generalize beyond the prisoner's
dilemma. Though the PD paints the problem of coop-
eration in its sharpest form, many other games exhibit,
in varying degrees, problems of cooperation or coordi-
nation. Snidal (1991), for example, analyzes six such
2 X 2 games. Our main results, below, do indeed hold
for a class of games much larger than the PD. Our first
task, then, is to define these games, which we call
"games of cooperation." In a set of symmetric two-
person games with M actions, a,, . . . ,a, (M > I), a
game of cooperation is defined by the following two
conditions that generalize two important properties of
the PD: the value of cooperation and the danger of
exploitation. (Below, v(a, b) denotes player 1's payoff
in a stage-game when she plays action a against player
2's b.)
The Value of Cooperation. In the PD, mutual cooper-
ation is the unique socially efficient outcome: in addi-
tion to R > P, it is also assumed that 2R > T + S.
Similarly, we assume that in any game of cooperation
there is a cooperative action, a,, such that v(a,, a,) is
the unique efficient outcome in pure strategies, that is,
2v(ac, a,) 2, v(ak, a,) + v(a,, a,), for all m, k,
equality holding if and only if m = k = c.
The Danger of Exploitation. The other key feature of
the PD is the risk that a cooperating player may be
exploited if his partner defects. In general, it makes
sense to consider an action exploitive only if exploiting
is at least as good as being exploited. Hence, our
second assumption is that there exists an action, a,,
distinct from a,, such that v(a,, aj) r v(aj, a,) for all
j = 1, . . . , M. Together, the two assumptions imply
Page 10
The Evolutionary Stability of Cooperation
FIGURE 3. Two Games of Cooperation
low middle high
low
middle
high
Game 1: military investment
that the payoff to an exploited cooperator, v(a,, a,), is
less than the cooperative payoff, v(a,, a,).
Any stage-game that satisfies these two conditions of
cooperation and exploitation we call a cooperation
game.16 TO see how these conditions generalize the PD,
observe that in a game with two actions they imply only
that R > P and 2R > T + S (efficiency) and T r S
(exploitation), Hence, 2 X 2 games of cooperation
include both kinds of stag hunt (assurance) and the
kind of chicken in which compromise is Pareto-supe-
rior to alternately aggressing and conceding. (Indeed,
five of the six games in Snidal's classification are
included.)
Furthermore, because games of cooperation can
have any finite number of actions, going beyond the
2 X 2 case generates an explosive increase in the
number and variety of substantively important strategic
situations. It also produces games with qualitatively
new properties. Consider the two games in Figure 3.
Game 1 depicts a game of the military investment
needed to seize valued territory. Because investment is
costly, each side wants to avoid overspending. As a
result, the game has no pure strategy equilibria. Yet,
game 1is a game of cooperation. In comparison, all of
Snidal's games-indeed, all symmetric 2 X 2 games-
have pure strategy equilibria. Thus, the conjunction
between cooperation problems and the existence of
pure strategy equilibria turns out to be a contingent
feature of 2 X 2 games.
Game 2 represents a generic situation of bureau-
cratic politics in which each official can help, ignore, or
hurt the other. This game of cooperation captures an
important distinction between action and inaction
(March and Simon 1958, 175). The prototypical "de-
fection" in many organizational situations is inaction-
the only costless, hence very common, solution. Pun-
ishment is another matter, for it absorbs resources.
Because no 2 X 2 game can distinguish between helping,
A cooperation game has a single "cooperative" action, a,, but it
may have more than one "defection" action.
June 1997
help ignore hurt
help
ignore
hurt
Game 2: bureaucratic politics
ignoring, and hurting, games with only two actions
cannot possibly serve as plausible models of these and
many other similar situations.
Because a game of cooperation is a generalization of
the one-shot PD, when we turn to iterated games of
cooperation, properties of strategies are straightfor-
ward generalizations of definitions adopted for the
IPD. Thus, a strategy is nice if it never deviates from
cooperation (a,) first. A strategy is retaliatory if it
punishes (plays a,) any deviation a, (k f c) from a,
until j's benefits from deviating are made up for by j's
subsequent cooperation:
v(ak, a,) + Nv(a,, ad) 5 v(a,, ak) + Nv(ad, a,)
where N is the minimal positive integer for which
condition 9 holds for all k. We call a strategy nasty if it
never cooperates first. Similarly, definitions of strate-
gies in the IPD, such as TFT, extend directly to games
of cooperation.
(9)
The Main Results
The property that turns out to distinguish cooperative
ESSs like TFT harks back to the metaphor of balls
rolling on hills and valleys. Recall that a ball in a
deeper valley is more stable since it can absorb bigger
shocks. Analogously, native strategies that can with-
stand large invasions are more stable. Thus, now we
focus attention on this critical parameter of an evolu-
tionary game-the minimal frequency that ensures a
strategy's stability in the population. (See the Appendix
for its precise definition.)
Compare two strategies, i and j, where i requires a
frequency of at least 0.6 to be stable, whereas j requires
a frequency of at least 0.9. Strategy i has a clear
advantage over j. First, it is more likely than j to reach
a stable state: If each starts out as a rare mutant, it is
easier for i-other factors equal-to
threshold of 60% than for j to reach its threshold of
90%. Second, having become common in the popula-
reach its critical
Page 11
American Political Science Review
tion, i is more stable than j since it can resist invasions
by larger groups: 0.4 of the population as opposed to
0.1 for j.
Stabilizing frequencies are therefore of fundamental
importance for both the emergence and the stability of
strategies and their corresponding stable states. When
two strategies have different minimal stabilizing fre-
quencies, the one with the lower frequency is clearly
"more stable." The key problem considered in this
section is what strategies are the most stable (i.e., have
the lowest stabilizing frequency) and what stable states
(behavior) they support.
It is important to note that theorems 6-9 are formu-
lated in a nontechnical way. This may be appreciated
by some; others may find it insufficiently precise. The
Appendix provides definitions, notation, and the
"proper" formulation of theorems; the proofs remove
any remaining ambiguities.
"Minority" Strategies Will Die Out in Some Ecologies.
We begin our analysis by establishing the lower bound
of the minimal stabilizing frequency in evolutionary
games of cooperation with sufficiently high 6.
THEOREM 6: In evolutionary games of cooperation with
suficiently high 6 no strategy has a minimal stabilizing
fvequency smaller than 0.5 under any evolutionary
dynamic.
More precisely, consider any strategy i and an ecol-
ogy in which it has a frequency of p < 0.5. Consider
now a set of all evolytionary games with ap fraction of
i and a fixed 6. Theorem 6 says that for sufficiently high
6 this set includes ecologies in which i will decrease
across generations. Hence, no strategy that is in the
minority can be stable in all evolutionary games with
sufficiently high 6-stability requires majority status of
even the most robust strategy. The best we can expect
is that the stabilizing frequencies of the most robust
strategies come arbitrarily close to one-half.
Nice and Retaliatory Strategies Are the Most Robust. As
Axelrod suspected, TFT turns out to be superior to
many ESSs, not only in the IPD but also in the entire
class of games of cooperation. Our next theorem
establishes that all nice and retaliatory strategies (TFT,
grim trigger, etc.) are indeed the most robust: All they
need to stabilize is to exceed the threshold of one-half.
(Note that in contrast to theorem 6, which holds for all
evolutionary processes, theorem 7 holds only for the
PFR.)
THEOREM 7 (CHARACTERIZATION
CIENT CONDITION):?
Under the PFR, the minimal stabi-
lizingfrequency of any nice and retaliatory strategy in a
game of cooperation converges to 0.5 as 6 converges
to 1.
Theorem 7's dynamic implications are straightfor-
ward. Suppose a majority of the people in an ecology
initially play, for example, TFT. The rest can play any
mix of strategies. Theorem 7 then guarantees that the
native's relative frequency will not fall over successive
generations. Indeed, in Boyd and Lorberbaum's (1987)
THE SUFFI- THEOREM, ?
Vol. 91, No. 2
example, where the native is TFT and the mutants are
TF2T and STFT, TFT, though less fit than TF2T, does
better than average. Accordingly, if the evolutionary
dynamic is the PFR, TFT will increase. Of course,
because TF2T scores highest in this ecology, it will
increase faster than TFT. Only STFT will decrease, its
loss offsetting the others' growth. Once the population
restabilizes, only TFT and TF2T will remain; STFT will
have died out. Because only nice strategies persist in
the new equilibrium, the behavior of the old equilibri-
um-universal cooperation-is restored.
This point holds more generally. In fact, the proof of
theorem 7 implies that, when in the majority, a nice
and retaliatory native will score above average if the
ecology has any nonnice strategies. With nonnice
present, a nice and retaliatory native must increase.
This process stops only when all nonnice mutants are
wiped out. (See the proof of the theorem for one
technical proviso.) Hence, the following corollary of
theorem 7:
COROLLARY:
Under the conditions of theorem 7, the
population converges to a stable state in which all
strategies are nice. Hence, the resulting unique stable
population state is universal cooperation.
Thus, so long as mutants do not overwhelm a nice
and retaliatory native i, their invasion will have no
behavioral effect: If the future matters enough, the
status quo ante of perpetual cooperation among all
players will be reestablished. Of course, this does not
mean that the ecology's composition would remain
unaffected by the invasion: A nice strategy other than i
may survive in the new equilibrium. In the example of
{TFT, TF2T, STFT) the dynamics will stabilize with
both TFT and TF2T surviving. As the corollary implies,
the nonnice STFT will be eliminated.
In other situations, the nice and retaliatory native i
may wipe out all mutants. Suppose, for example, that
initially there are three strategies: i (which is in the
majority), plus ALL C and ALL D. Depending on the
game's parameters, the following trajectory is possible.
ALL D may initially be the most fit and increase more
quickly than i. (Theorem 7 guarantees i's increase.)
But their increase is offset by the decrease of ALL D's
prey, ALL C, which had given ALL D an evolutionary
edge over i. The disappearance of ALL C will subse-
quently kill ALL D, which-absent ALL C-is
than i. In equilibrium only i will survive: The popula-
tion will return to its original state.
Finally, it is most important to note one other
property of the cooperative equilibrium induced by
nice and retaliatory i. It may be difficult for a single
pure strategy, say, TFT, to reach the stabilizing fre-
quency of 50%. For instance, in Axelrod's simulations
(1984,51, Figure 2) no strategy grew beyond 15%. But
note that theorem 7 does not require that i be pure; i
may be a mixture of two or more nice and retaliatory
pure strategies, say any mix of TFT and GT. Formally,
i can be any mixed strategy, i = p j, + . . . + p$,, of
pure nice and retaliatory j's. And as long as the
frequency of this mix is above 50%, it will not decline.
(The composition of the mix may change, with some j's
less fit
Page 12
The Evolutionary Stability of Cooperation
increasing and others decreasing, but its overall pro-
portion in the ecology will be stable.) Thus, we can use
theorem 7 to make inferences not only about a single
strategy but also about sets of strategies. For example,
in his first computer tournament Axelrod had three
strategies that were nice and retaliatory: TFT, FRIED-
MAN (GT), and SHUBIK. (Several other strategies
were almost-nice and almost-retaliatory, as defined
later.) Theorem 7 tells us that if any ensemble of these
three strategies ever were to reach 50%, it would never
decline. We can thus take an inductive observation, like
that of Axelrod's Figure 2 (1984), and predict the
ecology's evolution under certain hypothetical scenar-
ios. Specifically, Theorem 7 implies that had any mix of
TFT, FRIEDMAN, and SHUBIK been more than half
of Figure 2's ecology, then, given sufficiently large 6,
this group never would have decreased over simulated
generations, and the stable state would have contained
no defecting strategies. Theorem 7 and its corollary
point to the importance of set-valued solutions in
evolutionary games: What may be stable in some
ecologies are sets of strategies rather than any single
stratew in the set.
So g r we have shown that all nice and retaliatory
strategies enjoy the lowest possible stabilizing fre-
quency in all games of cooperation with sufficiently
high 6. We should interpret this finding cautiously,
however, for it may be another "folk theorem" result:
The same property may hold for a diverse set of
strategies; worse, it may hold for all of them. Theorem
7 shows sufficient conditions for a strategy to be most
robust. To make our findings meaningful, we must
establish the necessary conditions as well. Another
characterization result is thus needed.
The Most Robust Strategies Must Be Almost-nice and
Almost-retaliatory. Consider a strategy (used in the
proof of theorem 6) called MACHO, which begins by
defecting. If its partner cooperates in period one, then
MACHO defects always, but if its partner shows
"toughness" by defecting in period one, MACHO
cooperates in period two and thereafter (unless its
partner ever defects again, whence MACHO retaliates
forever). While vastly different, MACHO and TFT
have one important property in common. When TFT
plays another TFT it cooperates in all periods of the
game; MACHO cooperates with its clone in every
period but the first. Intuitively, the one-period defec-
tion should not much impair MACHO'S robustness for
large enough 6. This would also be true of all strategies
which, when playing their clones, converge on uninter-
rupted cooperation. For this reason it makes sense to
extend the notion of "niceness." We say that a strategy
is almost-nice if it cooperates with its clone "almost
always," in the sense that the payoff of an almost-nice
strategy playing its clone converges (as 6 + 1) to the
efficient payoff garnered by two ni,ce strategies: lim,,,
V(i, i)(l - 6) = v(a,, a,) = R. Note that the set of
nice strategies is a subset of almost-nice ones; hence,
the latter concept generalizes the former.
Immediate provocability is yet another property of
TFT that can be easily generalized. Consider a class of
June 1997
nice strategies called Tit for n Unreciprocated Exploi-
tations (TFnUE). Any TFnUE cooperates in period
one, thereafter defecting against any strategy j in any
period k + 1if the number of periods in which TFnUE
was exploited (it cooperated, but its partner defected)
exceeds the number of times in which TFnUE ex-
ploited its partner in the previous k periods by n.
Operationally, then, TFnUE defects whenever its
partner has defected n more times than it has. A
common property of strategies like TFT and TFnUE
is "some" degree of retaliation and unexploitability.
This property of TFnUE ensures that V(j, TFnUE) -
V(TFnUE, j) 5 n(T - S); because no strategy j can
outscore TFnUE by more than n(T - S) for any value
of 6, TFnUE is "almost-retaliatory." For payoffs this
implies that, for all strategies j, V(TFnUE, j) con-
verges, as 6 goes to 1, to at least the value of V(j,
TFnUE). Formally we define strategy i as almost-
retaliatory if, for all strategies j, lim,,,
2 lim8+, V(j, i)(l - 6).
With these generalizations of TFT's two fundamen-
tal properties we can now formulate the conditions
needed for maximal robustness.
V(i, j)(l - 6)
THEOREM
SARY CONDITION):
eration and any evolutionary dynamic, if the minimal
stabilizing fvequency of strategy i converges to 0.5 as 6
converges to 1, then (i) i is almost-nice, and (ii) i is
almost-retaliatory.
THEOREM,
8 (CHARACTERIZATION
For any evolutionary game of coop-
THE NECES-
It follows that neither strategies which fail to coop-
erate "almost always" with their clones (i.e., strategies
which are not almost-nice) nor those which can be
infinitely exploited (i.e., strategies which are not al-
most-retaliatory) attain the minimal stabilizing fre-
quency. Maximally robust strategies, by virtue of being
almost-nice, secure "almost universal" cooperation in
the stable state. As 6 approaches 1, their payoff comes
arbitrarily close to the efficient payoff of universal
cooperation.
Our last two results, theorem 9 and its corollary, tell
us finally why defecting strategies are deficient. The
corollary shows that the Hobbesian state of universal
defection, supported by nasty strategies, is the most
vulnerable of all stable states. It follows from a more
general result, Theorem 9, which links a strategy's
minimal stabilizing frequency to its degree of cooper-
ativeness, or efficiency.
More specifically, for any game of cooperation de-
note P = minl,i5n v(ai, a,) and R = v(a,, a,) =
maxi,,,, v(ai, a,). Hence, for any strategy i, R r
lim,,, V(i, i)(l - 6) 2 P. Writing lim8+,
V(i, i)(l - 6) = xR + (1 - x)P, where 0 5 x 5 1,
we can meaningfully treat x as i's degree of efficiency.
For any value of x consider a maximally robust strategy
(one with the smallest minimal stabilizing frequency) in
the class of strategies with x degree of efficiency and in
games with 6 converging to 1.
THEOREM 9: For any evolutionary game of cooperation
with suficiently big 6 and any evolutionary dynamic,
the larger the degree of eficiency of the most robust
Page 13
American Political Science Review
strategy of this degree, the larger the maximal invasion
this strategy can repel.
COROLLARY:
For any evolutionary game of cooperation
and any evolutionary dynamic, the minimal stabilizing
frequency of any nasty strategy converges to I as 6
converges to I.
Thus, as 6 +1, nasty strategies can be invaded by an
arbitrarily small proportion of conditional cooperators,
with no clustering whatsoever. The suboptimal
Hobbesian state of pure defection is the least robust of
all evolutionarily stable states. More generally, theo-
rem 9 shows that this holds in a precise quantitative
sense for all degrees of efficiency: Ceteris paribus,
more cooperative natives can resist bigger invasions
than can less cooperative ones.
And so, the deductive conclusions of evolutionary
analysis are sanguine after all. Axelrod's insightful
inductive observations have proved to carry the seed of
a universal principle. The fully cooperative equilibrium
is indeed evolutionarily advantaged. The benefits of
this efficient result come, however, at the cost of
everlasting provocability; as in the dictum: "eternal
vigilance is the price of liberty" (John Philpot Curran,
speech on the right of election of the Lord Mayor of
Dublin, July 10, 1790).
CONCLUSION
The unique power of a properly constructed theory-a
formal theory, as some would say-is
defines all objects of its domain and their properties.
Consequently, we can see exactly why certain proper-
ties imply others. We have shown here in what sense
cooperative, retaliatory strategies are more stable than
others: The difference is in their minimal stabilizing
frequencies. These results strongly agree with our
empirical intuitions. In part this is because the minimal
stabilizing frequency is a property with obvious empir-
ical relevance. We believe that studying basins of
attraction of evolutionary equilibria is essential to any
evolutionary analysis, and we hope our article will
encourage this direction of research.
that it precisely
Related Results
Our analysis is limited by the assumptions of the
standard evolutionary model. New models are gener-
ated by altering one or more of these assumptions. In
general, these analyses show that a small change in
assumptions can easily affect equilibrium selection.
Such is the case, for instance, when noise (on actions,
strategies, or payoffs) is introduced into the analysis
(Bendor 1993; Foster and Young 1990; Kandori,
Mailath, and Rob 1993; Sugden 1986; Young and
Foster 1991.) Even models of the same type of noise
can yield sharply contrasting conclusions. For example,
although Boyd (1989) and Fudenberg and Maskin
(1990, 1993) both study trembles-noise
tions-their conclusions are profoundly different: Boyd
shows that for 6 sufficiently big an evolutionary folk
theorem holds, whereas Fudenberg and Maskin (1993)
affecting ac-
Vol. 91, No. 2
show that only "nearly" efficient states are stable! What
drives the difference is that Boyd fixes 6 and then
introduces invasions, while Fudenberg and Maskin do
the opposite.
Another set of assumptions that may profoundly
affect equilibrium selection concerns the evolutionary
costs of complexity. Some have suggested that since
more complex strategies are harder to remember and
implement, evolutionary dynamics will favor simpler
ones. Binmore and Samuelson (1992) capture this idea
via a lexicographic notion: If in a given ecology V(i) =
VQ) and i is simpler than j, then i is more fit than j; if
their payoffs differ, complexity is irrelevant. However,
measuring complexity is both controversial-there
several plausible ways to do it (Banks and Sundaram
1989)-and sensitive, since conclusions about evolu-
tionary stability depend on these alternative measures.
Because these results are relatively easy to present and
nicely illustrate what may be a widespread problem
with equilibrium selection, we now take a closer look at
these two works.
Binmore and Samuelson use Rubinstein's (1986)
measure: Take the simplest automaton required to
implement a strategy and count its states. So ALL D,
which needs only a one-state machine, is simpler than
TFT, which needs two. In contrast, Banks and
Sundaram argue that a complexity measure should
account for the amount of monitoring, measured by
how often an automaton transitions from one state to
another: One automaton is simpler than another if it is
at least as simple on both dimensions (number of states
and of transitions) and strictly simpler on at least one
of them. Using Rubinstein's measure, Binmore and
Samuelson show that with undiscounted payoffs only
efficient outcomes are evolutionarily stable in a large
class of games including the IPD. Using their more
comprehensive measure, Banks and Sundaram show
that no outcome is stablee17 Why the difference?
These authors agree on one point: Anything less
than 100% cooperation is unstable. What they disagree
about is the stability of full cooperation. Since a
cooperative native is protected from invasions by
"more efficient" strategies, it can only be invaded by
strategies that are as efficient but simpler.18 Binmore
and Samuelson construct a two-state strategy, TAT
FOR TIT, that is both stable and efficient, cooperating
with itself in all but the first period. But for Banks
and Sundaram, this strategy can be invaded by its
unconditional neutral mutant, "defect in period one
and cooperate thereafter." Thus for them, efficiency-
based invasions destabilize zero cooperation, and sim-
plicity-based invasions wreck everything else. A small
difference in measuring complexity can profoundly
affect equilibrium selection.
Above we chose to comment on a handful of papers
are
l7 This sharp difference persists when payoffs are discounted; see
Bendor and Swistak (1996b).
l8 The following illustrate these two types of invasions. (1) With
undiscounted payoffs, TFT can invade ALL D. This is an efficiency-
based invasion. (2) If the native is a GT, it can be invaded by ALL C,
which discards the unused punishment state. This is a simplicity-
based invasion.
Page 14
The Evolutionary Stability of Cooperation
on the evolution of cooperation. There are hundreds of
others.19 Our choice was motivated by two reasons.
First, we wanted to illustrate what we think is the most
important methodological lesson to be learned from
the research on evolutionary games: "The very fact that
varying the details in a dynamic model can alter the
equilibrium selected shows that the institutional envi-
ronment in which a game is learned and played can
matter" (Binmore and Samuelson 1994,l). Second, we
wanted to focus attention on models that are most
directly comparable with ours. Literally hundreds of
other papers on evolutionary equilibria build on as-
sumptions that are significantly different from the
standard ones. For instance, stochastic evolutionary
analysis has focused almost exclusively on one-shot
games (e.g., Ellison 1993; Foster and Young 1990;
Fudenberg and Harris 1992; Kandori, Mailath, and
Rob 1993; Young 1993a, b). Due to the profound
differences in assumptions, comparing results of these
models might confuse rather than illuminate our un-
derstanding of the evolution of cooperation.
Extensions
It is important to investigate how sensitive equilibria
in evolutionary games are to changes in the game's
standard parameter^.^^ We suspect, however, that the
most fascinating problems will arise along some rela-
tively unorthodox lines of research. Consider the fol-
lowing example. Like most of evolutionary game the-
ory, the general instability result of Bendor and Swistak
(1992, 1996c) and Boyd and Lorberbaum's (1987)
instability result for the IPD rely on a tacit assumption
about strategies: Player 1's moves against player 2
depend only on the history of moves between 1and 2.
Hence, in Boyd and Lorberbaum's example, TFT's
behavior vis-a-vis TF2T is based only on TF2T's con-
duct toward TFT. It turns out that if TFT can defect
against TF2T to penalize it for "unwarranted" cooper-
ation with STFT, then TFT will score highest in this
ecology. But this requires departing from dyadic be-
havior: Player 1 must base her actions toward 2 not
only on 2's history with 1but also on 2's history with
other players. Such strategies are norm-like in that 1
can punish 2 for 2's conduct toward third parties.
Elsewhere we have shown that nom-like strategies are
stable under all evolutionary processes (Bendor and
Swistak 1993), not only under the PFR. This suggests
that certain dynamics more typical of homo economi-
cus-such as replication by switching to the winning
strategy-may create a selective pressure for the evo-
lution of norms. (See also Boyd and Richerson 1992.)
The effect of other forms of social institutions on the
evolution of cooperation has also been studied. These
l9 In their annotated bibliography, Axelrod and D'Arnbrosio (1994)
list 209 references for 1988-94. An earlier bibliography for 1981-87
(Axelrod and Dion 1987) lists 236 entries; the major findings of this
period were summarized in Axelrod and Dion 1988.
ZOFor example, sustaining cooperation when social efficiency re-
quires specialization (e.g., 2R < T + S in the IPD) is probably
harder than when it requires coordinating on the same action, but
specialization is vital for complex institutions (Sugden 1986, 1989).
June 1997
include population structure (e.g., Myerson, Pollock
and Swinkels 1991; Pollock 1988), exit (e.g., Peck 1993,
Vanberg and Congleton 1992), ostracism (Hirshleifer
and Rasmusen 1989), and reputation (e.g., Raub and
Weesie 1990). These and related studies go beyond the
standard set of game parameters analyzed by econo-
mists and point up the importance of social and
political institutions for the efficiency of equilibria.
Whichever direction the field goes, the canonical
case would always serve as a reference point. If we do
not understand the standard model, results obtained in
other models are, to a large extent, meaningless.
Hence, the objective of this article was to take the
problem defined by the assumptions used in Axelrod's
pioneering work and solve it deductively. Our answers,
we believe, bring closure to a set of intriguing conjec-
tures posed by The Evolution of Cooperation.
APPENDIX
Notation and Definitions
Denote by g a symmetric (identical sets of actions and a
symmetric payoff matrix) two-player game, with a finite set of
actions A. For a,, a, E A, v(a,, a,) denotes the payoff to
the player moving a, against the other player's action a,,
where v is a real-valued function defined on all pairs of
actions:v : A X A +Re. An infinitely repeated game G(g,
8, S) of a stage game g is defined in a standard way; the third
argument, S, denotes a set of strategies in the repeated game.
We assume here games with discounting: V(i, j) = 2;"=,
Stv(a:, af) denotes a payoff to i when it plays j; a f and af, af,
af EA, are realized actions of i and j in period t. Parameter
8 can be interpreted as a discount parameter, a probability of
continuing the game, or a joint effect of both. An evolutionary
game EG, where EG = (G(g, 8, S), ET, F) consists of a
repeated game G(g, 8, S), an ecology of strategies playing
the repeated game ET = (GI, p,), . ..., GN, p,))
generation T (p, is a frequency of strategy], in the popula-
tion), and an evolutionary process F. Whenever the time
index T is inessential we skip it. F is defined as any dynamic
which for all strategies j, and j, satisfies Aphlph < Ap,lp, iff
VG,) < VG,) and Ap,/p, = Ap,lp, iff VG,) = VG,), where
(pi, . . . ,PL) = F((p1, .. . PN)) and Api = P: - Pi. V"S
denote strategies' fitness (VG,) = p,VG,,
pNV(jk, j,))
in generation T whereas p and p' denote
proportions in two consecutive generations T and T', respec-
tively. More specifically, for any ecology {GI,p,), . . . , GN,
p,)) wedefinesimplexSNasSN= {p E 3,: 2 z l p i = 1,
wherep, 2 0, for all 1 I i 5 N). We define an evolutionary
process as any function F : aN+3, that satisfies the two
conditions above and leaves SN and its faces invariant. This
means that F transforms a vector of frequencies of strategies
j,, .. . , j, from one generation (p,, .. . , p,)
(pi, . . . ,ph) = F((p,, . . . ,p,)) such that 2 pi = C pi =
1. F is a function of time (T), and as such it can be either
discrete or continuous. A point P E SN is strongly stable if
there is a neighborhood of P such that in any point of the
neighborhood the gradient of the process is strictly positive in
the direction of P. A point P is weakly stable if the gradient is
nonnegative. The concepts of weak and strong stability are
closely related to the concepts of stability and asymptotic
stability. I f a process is sufficiently "well behaved" in the
neighborhood of P (Lipschitz continuity is a sufficient condi-
tion), then strong stability implies asymptotic stability. If,
however, a process is not Lipschitz continuous then strong
in
j,) + . . . +
to another
Page 15
American Political Science Review
stability need not imply asymptotic stability. In contrast,weak
stability does imply stability for any process. Consider now
any evolutionary game EG = (G(g, 6, S), E, F) and any
strategy j, with frequencyp, in E: (j,, p,) E E. Denote by
Flk the truncation of F to its kth coordinate. The minimal
value ofpkosuch that for all ecologiesE wherep, 2 p, IFl,
is nondecreasing in time is defined as jk3s minimal stabifizing
frequency.
In all theorems below we assume that the set of strategies
S is finite. S is also restricted to strategies that are either pure
or finite mixes of pure strategies.
Theorems
We now proceed with the precise formulations of the theo-
rems and their proofs. We begin with a result which is a
simple consequence of theorems originally established in
Bendor and Swistak (1992, 1996~). Because this result is a
very useful characterization of strategies that are weakly
stable under the PFR, it is useful in many of the following
proofs.
THEOREM
(proportionalfitness rule) if and only if it satisfies condition
8.
strategy is weakly stable under the PFR Al: A
We now proceed to the proofs of theorems 2-9.
THEOREM
variables satisfies conditions 6 and 7 if and only if the
corresponding form of the evolutionary dynamic is the PFR.
2:A propensity function F that is increasing in both
Proof: I f F is the PFR then it clearly satisfiesconditions 6 and
7. What requires proof is the inference in the opposite
direction. Consider an ecology E = {(p,, j,), . . . , (p,,
j,)). njd,stands for the probability that strategyj, switches to
j, in the next generation. I f bypTwe denote the proportion of
strategyj, in the next generation, thenp: = EL,p,nj . Note
that since for any 1 r t 5 N, n,, =
- - ... = 7~~~~~ = 7~~ ,i.e., any strategy in the ecology has equaf
propensity to switch j,. This implies that p: = plnjS + . . +
p,7Tj,
= 9.
We will grst show that F(x,y) = Cxy, C > 0. Without loss
of generality we can assume that the domain ofF(x,y) is such
that 0 5 x 5 1 and y > 0, and that for x, y > 0 F(x, y) =
CxyG(x, y), where C is a constant not equal to zero.
From condition 7 we have F(Kx, y) = CKxyG(Kx, y) =
KCxyG(x, y), for all k > 0. This implies that for K > 0
G(Kx,y) = G(x,y), hence G is constant in x, and F(x,y) =
CxyGb). Now, given that F(x + A, y) = F(x, y + A), we
conclude that F(x, y) = C*xy. Moreover, C* > 0, since F
was assumed to be increasing in x and y.
Hence, we have n.. = F(ps, V*(j,)) = Kp,V*(j,) = p:.
Given that Ey=, p: 2s1 and given that
we conclude that V* = zy= p,V(j,) = V and K = 1 so that,
finally,p: = p,[V(j,)]/V.
THEOREM
FOLK THEOREM
3 (EVOLUTIONARY
For an evolutionary game of IPD EG = (IPD,E, PFR) an
evolutionarilystable state in games with sufficiently high 6 can
exhibit any amount of cooperation.
n,,
F(p,, v*(jSj),
+
0,) = [V(j,)]/V*,
Q.E.D.
FOR THE IPD):
Proof: Take a standard trigger strategy i (as in any proof of a
folk theorem) that supports r amount of cooperation. For
sufficientlyhigh 6 and any j which is not a neutral mutant of
i we have V(i,i) > V(j, i) which, given theorem A1 above,
proves that i is evolutionarily stable. Q.E.D.
THEOREM
PFR) and any 0 < E 5 1, there is an evolutionarily stable
= (ZPD,E,
4: For an evolutionarygame of IPD EG
Vol. 91. No. 2
strategy which is infinitely exploitable and supports E amount
of cooperation in the stable state.
Proof: Consider the following i. When playing a neutral
mutant i cooperates every nth period only, starting with
period 1. When i plays a nonneutralj, then there is a period
s whenj moves differentlythan i. In this case i defectsfor the
next k - 2 periods and cooperates in period s + k - 1,
thereafter it only cooperates in every kthperiod regardless of
j's moves in the periods following s. Note that strategy i is
infinitely exploitable by, e.g., ALL D. Take now any j which
is not neutral with i. The best j can do against i is to defect
in all periods. Thus V(ALLD, i) 2 V(j,i), for all strategies
j nonneutral with i. We will now determine avalue ofk which
for sufficiently high 8 will give V(i, i) > V(ALL D, i), and
thus V(i, i) > V(j, i), for all nonneutral j (which implies
evolutionary stability of i).
First, note that V(i, i) = R/(1 - 6") + P(l/(l - 6) -
1/(1 - 6")) and V(ALLD,i) = T/(l - Sk) + P(1/ (1 - 6)
- 1/(1 - 6,)). Consequently, we get V(i,i) > V(ALLD, i)
if and only if (T - P)/(R - P) < (1 - ijk)/(l- 6"). Take
now any natural number m such that (T - P)I(R - P) < m.
To show that there is an 60 such that for all 6 E (60, 11, (T -
P)/(R - P) < (1 - tik)/(l- En), denote 6" = t and take
k = n(m + 1). Then (1 - Sk)/(l- 6") = (1 - tm+l)/(l-
t) = 1 + t + . . . + tm. Since lim,,,(l
m + 1 and 1 + t + . . . + tm is an increasing function of t in
t E [0, I],there is a to,0 < to < 1 such that, for all t E (to,
11, m < 1 + t + . . . + tm. Thus, equivalently, there is an 6,
(namely, 6, = "6)
such that, for all 6 E (So, 11, (T -
P)/(R - P) < (1 - Sk)/(l- 6") which completes the proof
that i is evolutionarily stable for sufficiently high 6. Since n
can take any natural values, i, when universal in the popula-
tion, can generate an amount of cooperation which is arbi-
trarily close to zero, being infinitely exploitable as well.
+ t + . . . + tm) =
Q.E.D.
(i) With sufficient clustering any pure strategy in the evolu-
tionary IPD that is not a neutral mutant of ALL D can
invade a population of ALL D.
(ii)Among strategies that require the least clustering to grow
in a population of ALL D, there are strategies which are
both evolutionarily stable and infinitely exploitable.
Proof
Part (i). Consider an ecology with x i's and (1 - x) j's.
Denote:p = probability that i interactswith another i (note:
p 2 x), q = probability that j interactswith another j. Then
V(i) = pV(i, i) + (1 - p)V(i, j) and V(j) = qV(j, j ) +
(1 - q)V(j,i). Probabilitiesp and q are not independent and
it is easy to show that q = 1 - (xl(1 - x)) (1 - p). Thus
V(i)=pV(i,i) + (1 -p)V(i,j), andV(j) = (1 - (xI(1 -
x)) (1 - p))V(j,j ) + (xI(1 - x)) (1 - p)V(j, i). And so
V(i) > V(j)is equivalent top[(l - x)V(i,i) + x(V(i,j) +
V(j, i) - V(j,j ) ) - V(i,j)] > (1 - x)V(j,j ) + x(V(i,j)
+ V(j,i) - V(j,j)) - V(i,j). WithA = (1 -x)V(i,i),B =
(1 - x)V(j,j), and C = x(V(i,j ) + V(j, i) - V(j,j)) -
V(i,j), this becomesp(A + C) > B + C. Now, ifj = ALL
D then B = (1 - x)V(j,j) = (1 - x)P/(l - 6). I f i is a
nonneutral invader of ALL D then playing another i it
cooperates in at least one period. In consequence A = (1 -
x)V(i,i) > (1 -x)P/(l - 6).This implies that there is some
p < 1 such thatp(A + C) > B + C which means that the
nonneutral i can invade the population of ALL Ds if it
clusters at the level p or higher.
Part (ii). Consider a strategy called FIRST IMPRESSIONS
(FI). FI is defined as follows: (i)never defect first; (ii)if the
opponent defected in period 1, defect in period 2 and
Download full-text 
