Content uploaded by Johann Mitlöhner
Author content
All content in this area was uploaded by Johann Mitlöhner
Content may be subject to copyright.
Using the Iterated Prisoner's Dilemma for Explaining the Evolution of
Cooperation in Open Source Communities
Daniel Eckert Stefan Koch Johann Mitlöhner
Institute of Public Economics,
Karl-Franzens University
Graz, Austria
Department of Information Business, University of Economics and BA
Vienna, Austria
daniel.eckert@uni-graz.at stefan.koch@wu-wien.ac.at mitloehn@wu-wien.ac.at
Abstract – Software development, and especially open source
projects, typically involve repeated interactions between
participants and groups of participants. We propose to
analyse this situation by means of the standard model for the
evolution of cooperation, the iterated prisoner's dilemma. The
prisoner's dilemma is a well-known model for a two-person
game, in which each side can choose to either cooperate or
defect, and in which the payoffs are arranged in a defined
hierarchy (e.g. the highest payoff is achieved by defecting
while the other player cooperates). As a first step, the
prisoner's dilemma needs to be formulated for the open source
development model, i.e. what constitutes cooperation, playing
defect and payoffs. Then, computer simulations using a
population of stochastic reactive strategies can be applied,
using a strategy's payoff as fitness measure for determining its
frequency in the next generation. As a further extension, the
effects of misinterpretation of other's behaviour can be
included into the model. We will discuss which insights into
open source software development can be gained by applying
this model.
I. INTRODUCTION
Software projects typically consist of multiple steps with
several partners cooperating to achieve a certain goal. The
dynamics of these interactions are an important factor
influencing software development risk, especially because
of misunderstandings possible due to differences in
organizational culture or group selective behaviour. We
propose to analyse this situation by means of the standard
model for the evolution of cooperation, the iterated
prisoner's dilemma, and will discuss whether and in which
form this model can be applied to open source software
development. This form of software development very
much relies on the participants' cooperation, and it is not
yet clear how this cooperation forms and evolves. It has
been assumed that common beliefs and values play an
important part in this process [1], which are increasingly
reinforced during participation [2].
II. GAME THEORY AND COMMERCIAL
SOFTWARE DEVELOPMENT
A. Prisoner's Dilemma
Limiting the analysis to projects with two participating
individuals or teams we model all interactions as a 2-
person game with the strategies cooperate (C) and defect
(D) available to both players. The payoff matrix defines the
utilities for the row and column players depending on their
respective strategy choices (see Table I.).
TABLE I.
PAYOFF MATRIX OF GENERIC PRISONER'S DILEMMA
C D
C(R,R) (S,T)
D(T,S) (P,P)
Given this situation, the so-called prisoner's dilemma is
realized with T>R>P>S: both partners opting for Defect
results in the low payoff (P,P), while both partners chosing
Cooperate returns the higher payoff (R,R). However, the
highest payoff T, termed temptation, can be realized by
defecting on a cooperating partner, who in turn receives the
lowest payoff S. An example in terms of negative utilities
can be formulated using costs associated with information
exchange: by chosing to cooperate and expending
documentation costs d player A can document their part of
the system sufficiently so that player B can readily use
player A's modules; if player A plays Defect then this
documentation is lacking, and player B has to expend
higher costs e to derive the necessary information. The
payoffs in terms of negative utility are R = -d, P = -e, T =
0, S = -d-e. With e>d the payoffs of the prisoner's dilemma
are again realized.
B. Iterated Prisoner's Dilemma
For a fixed number of iterations which is known in
advance to the players the dominant strategy is Defect
since it maximizes minimal payoff for each player.
However, in an iterated prisoner's dilemma where the
number of iterations is not initially known to the players
cooperative outcomes are possible [3]. Software
development projects typically involve repeated
interactions of the participants during their lifecycle. Due
to changes of schedule and other factors the number of
interactions is not known initially. Therefore, the iterated
prisoner's dilemma is a plausible model of the development
process.
In an iterated game cooperative and non-cooperative
behaviour will typically be reciprocated to a certain extend.
The players are influenced in their choice of strategy by the
opponent's previous move, if they are able to perceive it.
Such reactive strategies in the iterated prisoner's dilemma
can be modelled stochastically.
A stochastic reactive strategy specifies the conditional
probabilities of C and D as a reaction to all possible
histories of the game that fall into the memory of the
player. In the simplest case, where the initial probability to
play C in the first round can be ignored due to the infinite
iteration of the game, and where the memory of the players
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191
is minimal, taking account only of the opponent's move in
the previous round, a reactive strategy E can be determined
by the pair (p,q), where p is the conditional probability to
play C after an opponent's C in the previous round, and q is
the probability to play C after an opponent's D. Some well-
known strategies have been termed Tit for Tat, which can
be represented by the pair (1,0), generous TFT by (1, 0.3),
the random strategy by (0.5, 0.5) and AlwaysD by (0,0).
Nowak and Sigmund [4] have shown that the payoff A for
strategy Ei against strategy Ej the infinitely iterated
prisoner's dilemma is
A(Ei,Ej) = [(R-T)+(P-S)] c c' + (S-P) c + (T-P) c' + P (1)
with
c = (qi+(pi-qi)qj) / (1-(pi-qi)(pj-qj)) (2)
and
c' = (qj+(pj-qj)qi) / (1-(pj-qj)(pi-qi)) (3)
for 0<p<1 and 0<q<1.
In order to model the evolution of strategies in a
population of players we abstract from individual
participants playing certain strategies and instead study the
strategy population consisting of m stochastic reactive
strategies Ei. The current state of the strategy population is
denoted by the vector x where xi denotes the prevalence or
frequency of strategy Ei in the population of players. Using
a replicator equation which relates the growth in frequency
of a strategy to its fitness Hofbauer and Sigmund [5] have
described the evolution of the reactive strategies for the
infinitely iterated prisoner's dilemma as an adaptive
dynamics, where the frequency of a reactive strategy in the
following generation is given by
xit1=xitfix/
f
(4)
where
fix=∑
j=1
n
xjAEi,E j
(5)
denotes the average payoff for Ei in a population of
strategies with frequency-vector x = (x1,...,xn) and
f=∑
i=1
n
xifix
(6)
is the average payoff in the population.
Running computer simulations on the basis of this model
Nowak and Sigmund [6] have shown that cooperation can
emerge among a population of randomly chosen reactive
strategies, as long as a stochastic version of Tit for Tat is
added to the population. In this setting it plays the role of a
police for the enforcement of reciprocity, insofar as it is
immune to exploitation and rewards cooperation. But since
it is precisely this provocability that makes it susceptible to
errors engendering an endless vendetta of retaliations, it is
finally superseded by Generous Tit for Tat (a nice strategy
that forgives every third defection).
The fact that the presence of Tit for Tat in the strategy
population is necessary to bring about a cooperative
outcome highlights the importance of a minimal social
structure that is required for the evolution of cooperation.
Even if Tit for Tat can be proven to be the strategy that can
invade a population of universal defectors in the smallest
cluster [6], its role shows that the prestructuration of the
population determines the evolution of the patterns of
interaction that constitute the final social structure. The
precise impact of the prestructuration of the population
depends on the degree of its incorporation into the
strategies: As Frank [7] has shown, if cooperators can
recognize each other with the help of some label they can
increase their payoff by interacting selectively with one
another. This mechanism can even explain the spontaneous
emergence of label-selective behaviour, as Rick Riolo
quoted by Holland [8] has shown with the help of computer
simulations.
C. Group Selective Behaviour and Misinterpretation
In interactions between partners of different origin such
as in-house development versus cooperation with other
organizations a certain level of discriminating behaviour is
often observed in interactions with the alien group. In
terms of reactive strategies the probabilities for Cooperate
are group-selective, resulting in different (p,q) values for
interactions among natives and between natives and aliens.
In an extreme form cooperation only occurs among
members of the same group. The common cultural and
normative background within an organization can serve as
the base for capturing uncertainty in social interaction.
Introducing noise in the interaction models the danger of
misinterpretation of the opponent's behaviour and allows
for some amount of misimplementation of the original
strategies [9].
III. APPLICATION TO OPEN SOURCE SOFTWARE
DEVELOPMENT
A. Introduction
Open source software development has generated
increasing interest in the last years, both from the business
and academic world, as some projects in different
application domains like Linux together with the suite of
GNU utilities, GNOME, KDE, Apache, sendmail, bind,
and several programming languages have achieved huge
success in their respective markets.
The main ideas of this development model are described
in the seminal work of Raymond [10], 'The Cathedral and
the Bazaar', in which he contrasts the traditional type of
software development of a few people planning a cathedral
in splendid isolation with the new collaborative bazaar
form of open source software development. In this, a large
number of developer-turned users come together without
monetary compensation to cooperate under a model of
rigorous peer-review and take advantage of parallel
debugging that leads to innovation and rapid advancement
in developing and evolving software products, thus forming
an example for 'egoless programming' as proposed by
Weinberg already in 1971 [11]. In order to allow for this to
happen and to minimize duplicated work, the source code
of the software needs to be accessible, and new versions
need to be released often. To this end, software licenses
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191
that grant the necessary rights to the users, like free
redistribution, inclusion of the source code, the possibility
for modifications and derived works and some others have
been developed. One model for such licenses is the Open
Source Definition, which lists a number of requirements for
specific licenses [12]. The most prominent example which
fulfils these criteria while still being even more stringent, is
the GNU General Public Licence (GPL), developed by the
GNU project and advocated by the Free Software
Foundation [13].
The main question to be asked is whether the iterated
prisoner's dilemma can be applied in this setting. Its use
has been demonstrated in commercial software
development [14], with the main focus on group selective
behaviour and misunderstandings in the
consultant/customer interaction. Naturally, this distinction
is not applicable in open source software development. On
the other hand, several basic assumptions hold true: All
open source development projects after an initial stage
performed by a single person consist of several people, as
empirical studies have shown, in fact of more people than
commercial projects [15,16]. These people need to
cooperate, and contrary to commercial projects, they need
to cooperate without external influences, e.g. management
intervention or fear of job loss. Therefore the choice to
cooperate is indeed made on a more personal level,
depending on prior experiences and motivation. Also the
assumption of an unknown but finite number of
interactions holds true: Open source development efforts
entail repeated interactions between the participants, but
even more so than in commercial projects which are at
least planned to some extent, and the number of these
interactions are unknown to all players. Therefore the
iterated prisoner's dilemma in general can be applied.
In the next sections the different parts of the model are
discussed in the context of open source software
development.
B. The Open Source Software Development Players
First we define who the players participating in open
source software development are. There are several
possible lines of argumentation, and we consider two of the
different player sets: As a first type of game we consider all
participants in an open source software development
project as players, while in the second, more widely
defined approach, we define all users of a given software as
players. Even if the software does not yet exist, a possible
user community based on the planned functionality can be
assumed.
C. The Open Source Software Development Game
Given two possible defined ranges of players, the notion
of a single game has to be defined. It is important to note
that this game is iterated, i.e. performed often. The most
valuable artifact in an open source software development
effort is the software itself. Therefore it seems appropriate
to define a game along these lines as well. The most
interesting idea to explore is to use one release of the
software as one game. Depending on the release policy, an
alternative might be to use a certain time intervall, e.g. one
month, for defining a single game. Conveniently, this
notion of defining a game can be used for both types of
player sets to be considered.
D. Cooperation in Open Source Software Development
Naturally, the two available strategies that are available
to the players have to be defined next, i.e. cooperate or
defect. In open source software development, the most
simple definition of cooperation is participation. Using this
definition, cooperate would mean nothing else than
investing effort in the project. Again this definition is
usable for both player sets defined. In the most simple case
we need not further define the extent of effort, or the type
of work done, i.e. coding, documenting, or something else.
A more refined approach usable for the participating player
set would be to define defect as investing no or lower effort
than before, i.e. decreasing participation. Then, for
participants the two strategies would be to either invest
high effort, i.e. cooperate, or invest low effort, i.e. defect.
E. Payoffs in Open Source Software Development
Depending on the action taken, a payoff needs to be
realized. This payoff is generally defined in terms of utility,
which poses a severe problem. In open source software
development, motivations of participants can differ
[17,18], hinting at different utility functions, from which a
certain abstraction is necessary. Therefore we use the
resulting software as the main payoff, measured for
example using some software metric like lines-of-code or
function points [19]. This can also serve as a proxy for
some of the other constructs having been proposed as
motivating factors, e.g. reputation. A larger software
offering more functionality can be seen as being more
successful and thus conferring more reputation.
We combine this with the notion of negative utility
caused by the effort invested. This leads to the classical
four alternatives: The best outcome is for player A to
defect while B is cooperating. In this case, both get the
same utility from software functionality F, but A does not
have to invest effort I, which instead needs to be invested
by player B in addition to B's own effort, in the worst case
with higher costs I'>I, because A would have been more
familiar with the problem, or for similar reasons. If both
players are cooperating they invest their own efforts, and
both get the resulting software. If both players were to
defect, no effort is invested and no software is produced.
This results in the payoff matrix depicted in Table II.
F. Group Selective Behaviour and Misunderstandings
The presence of group selective behaviour might be of
interest in the context of open source software
development, especially when considering the whole user
group as players: If a player knows that the other player is
sharing the same norms and beliefs, e.g. on freedom of
software, he can be more sure of the other player's
cooperation, and might himself cooperate to a larger
degree. In this case, group selective behaviour might
indeed be present in open source software development.
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191
TABLE II.
PAYOFF MATRIX IN OPEN SOURCE SOFTWARE DEVELOPMENT
C D
C(F-I,F-I) (F-I-I',F)
D(F,F-I-I') (0,0)
The sometimes elaborate joining rituals and scripts
[20,1,2] might then be interpreted as a mechanism
implemented to reduce the possibility for
misunderstandings: Anyone having joined through these
rituals and scripts shares these norms and beliefs, and can
easily be identified as such a player, who therefore is less
likely to defect. Therefore noise is reduced, which can lead
to a more cooperative outcome even if discrimatory
strategies are present.
We construct group selective strategies EJ (for Mr.
Jekyll) and EH (for Mr. Hyde) such that for each non-
discriminatory Jekyll strategy with basic p and q values pi
and qi both in inter- and intra-group interaction EiJ=((pi,qi),
(pi,qi)) there is a corresponding Hyde strategy EiH=((pi*,qi*),
(pi*,qi*)) with pi*=pi and qi*=qi in case of intra-group
interaction, and pi*=qi*=θ=0.001 in case of inter-group
interaction: i.e. EiH=((pi,qi),(θ,θ)).
Each of these two subpopulations of strategies is
characterized by a frequency vector consisting of the
frequencies of the corresponding strategies and zeros
otherwise such that xJ + xH = x.
The fitness fi(x) of a given strategy i within a population
of m strategies consists of the contributions of fitness it
achieves when played by members of the own (native)
group (fin(x)) and the alien group (fia(x)) weighted by their
respective population shares rn and ra . We denote by πij the
proportion of group is contacts that are with js (under the
idealizing assumption characterizing our model πnn = rn
and πna = ra:
fi
nx=∑
j=1
m
nn xjAEi,pj,q j
(7)
na[xj
JAEi,pj,q jxj
HAEi,,]
fi
ax=∑
j=1
n
aa xjAEi,pj,q j
(8)
an[xj
JAEi,pj,q jxj
HAEi,,]
The total fitness fi is simply
fix=rnfi
nxrafi
ax.
(9)
If we denote by xin and xia the frequencies of strategy i in
native and alien use, resp., the frequencies in time step t+1
are given by
xi
nt1=xi
ntfi
nx/
fn
(10)
and
xi
at1=xi
atfi
ax/
fa
(11)
with the average native and alien fitness values
fn=∑
i=1
n
xi
nfi
nx
(12)
and
fa=∑
i=1
n
xi
afi
ax.
(13)
The total frequency of strategy i is
xit1=rnxi
nt1raxi
at1.
(14)
On the basis of this model we investigated the dynamic
behaviour of the strategy population. Following Nowak and
Sigmund [21] we initialized the population with 100 pairs
of p- and q-values distributed uniformly in the unit square;
for each pair we added a duplicate strategy: a Mr. Hyde
lurking behind his Dr. Jekyll sibling and ready to defect in
inter-group interaction. All 200 strategies have the same
initial frequency of 0.005, allowing an equal proportion of
Jekylls and Hydes. We ran 1000 time steps since
preliminary tests showed in accordance with the results of
Nowak and Sigmund that interesting dynamics occur only
after the first several hundred time steps. We set the
proportion of aliens to 0.1. Figure I shows the result of a
typical run, where typical is defined as resulting in a total
fitness value with the smallest deviation from the average
of 20 runs with different random numbers. Shortly after
t=200 the Tit-for-Tat Jekyll strategies take over the
population, drastically improving native and alien fitness;
after t=400 there is a shift towards Generous TFT which
further increases fitness.
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191
FIGURE I.
NON-DISCRIMINATORY OUTCOME IN SPITE OF DISCRIMINATORY STRATEGIES
IV. CONCLUSION
The main question remaining is what is to be gained by
applying the iterated prisoner's dilemma in the context of
open source software development. If the model can indeed
be used, it might allow for several insights into this
phenomenon. These insights can be roughly grouped into
two different levels: The open source idea in general and
its proliferation, and the situation in specific projects.
On the general level, this model gives an explanation
why open source software development has arisen and is
prospering. Evolving cooperation and thus formation of
open source development communities can be seen as a
natural choice. Therefore it strengthens other explanatory
approaches [22,23] in the reasoning that this idea is not
strictly motivated by altruism or similar motivations, but a
sound decision. Given a certain software functionality, the
possible user base can be estimated, and starting with a
certain rate of cooperators, the evolution of cooperation
within the total population can be simulated. This gives a
positive outlook for the open source movement, as prior
results indicate that even a small rate of cooperators might
be sufficient to establish cooperation in the long run.
On the level of specific projects, the environment,
especially any steps taken to reduce noise and thus
possibilities for misunderstandings seem to gain
prominence. Indeed, the joining scripts and even rituals in
some projects might in this way find a grounding in game
theory and can thus be explained. Their presence and
success might be found to be necessary for achieving a
cooperative situation and thus form a critical success factor
for open source projects. Group-selective behaviour is
interesting to explore in hybrid or gated open source
communities [24], composed of a mixture from a leading
company's employees and voluntary developers. Between
those groups selective behaviour might arise and endanger
a cooperative, i.e. successful, outcome.
Data currently available on open source projects and
communities could be used to document the situation in
these environments, i.e. the population of strategies in
place. This could be used as a starting point for simulations
of future behaviour, in order to get an understanding of
whether and when a cooperative outcome will result.
V. REFERENCES
[1] M.S. Elliott and W. Scacchi, “Free software
development: Cooperation and conflict in a virtual
organizational culture”, in S. Koch (ed.), Free/Open
Source Software Development, Hershey, PA: Idea
Group Publishing, 2004.
[2] E.G: Coleman and B. Hill, “The social production of
ethics in debian and free software communities:
Anthropological lessons for vocational ethic”, in S.
Koch (ed.), Free/Open Source Software
Development, Hershey, PA: Idea Group Publishing,
2004.
[3] R. Axelrod, The Evolution of Cooperation, London:
Penguin Books, 1984.
[4] M. Nowak and K. Sigmund, “Game dynamical
aspects of the prisoner's dilemma”, J. Appl. Math.
Comp., vol. 30, 1989, pp. 191-213.
[5] J. Hofbauer and K. Sigmund, The Theory of
Evolution and Dynamical Systems, Cambridge:
Cambridge University Press, 1988.
[6] M. Nowak and K. Sigmund, “The evolution of
stochastic strategies in the prisoner's dilemma”, Acta
Applicandae Mathematicae, vol. 20, 1990, pp. 247-
265.
[7] R. Frank, Passions within Reason. The strategic role
of the emotions, New York: W.W. Norton & Co.,
1988.
[8] J.H. Holland, Hidden Order - How adaptation builds
complexity, Reading, Mass.: Addison-Wesley, 1995.
[9] R. Axelrod and J. Wu, “How to cope with noise in
the iterated prisoner's dilemma”, Journal of Conflict
Resolution, vol. 39, 1995, pp. 183-189.
[10] E.S. Raymond, The Cathedral and the Bazaar,
Cambridge, Massachusetts: O’Reilly & Associates,
1999.
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191
[11] G.M. Weinberg, The Psychology of Computer
Programming, Silver Anniversary Edition, New
York: Dorset House Publishing, 1998.
[12] B. Perens, “The Open Source Definition”, in
DiBona, C. et al. (eds.), Open Sources: Voices from
the Open Source Revolution, Cambridge,
Massachusetts: O’Reilly & Associates, 1999
[13] R.M. Stallman, Free Software, Free Society:
Selected Essays of Richard M. Stallman, Boston,
Massachusetts: GNU Press, 2002.
[14] W. Janko and J. Mitlöhner, “Ein spieltheoretisches
Modell der Berater-Kunden-Interaktion in IT-
Projekten”, in W. Gaul and M. Schader (eds.)
Mathematische Methoden der Wirtschafts-
wissenschaften, Heidelberg: Physica-Verlag, 1999.
[15] A. Mockus, R. Fielding and J: Herbsleb, “Two case
studies of open source software development:
Apache and Mozilla”, ACM Transactions on
Software Engineering and Methodology, vol. 11, no.
3, 2002, pp. 309-346.
[16] S. Koch, “Profiling an Open Source Project Ecology
and Its Programmers”, Electronic Markets, vol. 10,
no. 2, 2004, pp. 77-88.
[17] A. Hars and S. Ou, “Working for free? - Motivations
for participating in Open Source projects”, in
Proceedings of the 34th Hawaii International
Conference on System Sciences, Hawaii, 2001.
[18] G. Hertel, S. Niedner and S. Hermann, “Motivation
of software developers in open source projects: An
internet-based survey of contributors to the Linux
kernel”, Research Policy, vol. 32, no. 7, 2003, pp.
1159-1177.
[19] A.J. Albrecht and J.E. Gaffney, “Software Function,
Source Lines of Code, and Development Effort
Prediction: A Software Science Validation”, IEEE
Transactions on Software Engineering, vol. 9, no. 6,
1983, pp. 639-648.
[20] G. von Krogh, S. Spaeth, and K.R. Lakhani,
“Community, joining and specialization in open
source software innovation: A case study”, Research
Policy, vol. 32, no. 7, 2003, pp. 1217-1241, 2003.
[21] M. Nowak and K. Sigmund, “Tit for tat in
heterogenous populations”, Nature, vol. 355, January
1992.
[22] R.A. Ghosh, “Cooking pot markets: an economic
model for the trade in free goods and services on the
Internet”, First Monday, vol. 3, no. 3, March 1998.
[23] J. Lerner and J. Tirole, “The simple economics of
Open Source”, Working Paper 7600, National
Bureau of Economic Research, 2000.
[24] S. Sharma, V. Sugumaran and B. Rajagopalan, “A
framework for creating hybrid-OSS communities”,
Information Systems Journal, vol. 12, no. 1, 2002,
pp. 7-25.
Proceedings of the First International Conference on Open Source Systems
Genova, 11th-15th July 2005
Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191