Content uploaded by Johann Mitlöhner

Author content

All content in this area was uploaded by Johann Mitlöhner

Content may be subject to copyright.

Using the Iterated Prisoner's Dilemma for Explaining the Evolution of

Cooperation in Open Source Communities

Daniel Eckert Stefan Koch Johann Mitlöhner

Institute of Public Economics,

Karl-Franzens University

Graz, Austria

Department of Information Business, University of Economics and BA

Vienna, Austria

daniel.eckert@uni-graz.at stefan.koch@wu-wien.ac.at mitloehn@wu-wien.ac.at

Abstract – Software development, and especially open source

projects, typically involve repeated interactions between

participants and groups of participants. We propose to

analyse this situation by means of the standard model for the

evolution of cooperation, the iterated prisoner's dilemma. The

prisoner's dilemma is a well-known model for a two-person

game, in which each side can choose to either cooperate or

defect, and in which the payoffs are arranged in a defined

hierarchy (e.g. the highest payoff is achieved by defecting

while the other player cooperates). As a first step, the

prisoner's dilemma needs to be formulated for the open source

development model, i.e. what constitutes cooperation, playing

defect and payoffs. Then, computer simulations using a

population of stochastic reactive strategies can be applied,

using a strategy's payoff as fitness measure for determining its

frequency in the next generation. As a further extension, the

effects of misinterpretation of other's behaviour can be

included into the model. We will discuss which insights into

open source software development can be gained by applying

this model.

I. INTRODUCTION

Software projects typically consist of multiple steps with

several partners cooperating to achieve a certain goal. The

dynamics of these interactions are an important factor

influencing software development risk, especially because

of misunderstandings possible due to differences in

organizational culture or group selective behaviour. We

propose to analyse this situation by means of the standard

model for the evolution of cooperation, the iterated

prisoner's dilemma, and will discuss whether and in which

form this model can be applied to open source software

development. This form of software development very

much relies on the participants' cooperation, and it is not

yet clear how this cooperation forms and evolves. It has

been assumed that common beliefs and values play an

important part in this process [1], which are increasingly

reinforced during participation [2].

II. GAME THEORY AND COMMERCIAL

SOFTWARE DEVELOPMENT

A. Prisoner's Dilemma

Limiting the analysis to projects with two participating

individuals or teams we model all interactions as a 2-

person game with the strategies cooperate (C) and defect

(D) available to both players. The payoff matrix defines the

utilities for the row and column players depending on their

respective strategy choices (see Table I.).

TABLE I.

PAYOFF MATRIX OF GENERIC PRISONER'S DILEMMA

C D

C(R,R) (S,T)

D(T,S) (P,P)

Given this situation, the so-called prisoner's dilemma is

realized with T>R>P>S: both partners opting for Defect

results in the low payoff (P,P), while both partners chosing

Cooperate returns the higher payoff (R,R). However, the

highest payoff T, termed temptation, can be realized by

defecting on a cooperating partner, who in turn receives the

lowest payoff S. An example in terms of negative utilities

can be formulated using costs associated with information

exchange: by chosing to cooperate and expending

documentation costs d player A can document their part of

the system sufficiently so that player B can readily use

player A's modules; if player A plays Defect then this

documentation is lacking, and player B has to expend

higher costs e to derive the necessary information. The

payoffs in terms of negative utility are R = -d, P = -e, T =

0, S = -d-e. With e>d the payoffs of the prisoner's dilemma

are again realized.

B. Iterated Prisoner's Dilemma

For a fixed number of iterations which is known in

advance to the players the dominant strategy is Defect

since it maximizes minimal payoff for each player.

However, in an iterated prisoner's dilemma where the

number of iterations is not initially known to the players

cooperative outcomes are possible [3]. Software

development projects typically involve repeated

interactions of the participants during their lifecycle. Due

to changes of schedule and other factors the number of

interactions is not known initially. Therefore, the iterated

prisoner's dilemma is a plausible model of the development

process.

In an iterated game cooperative and non-cooperative

behaviour will typically be reciprocated to a certain extend.

The players are influenced in their choice of strategy by the

opponent's previous move, if they are able to perceive it.

Such reactive strategies in the iterated prisoner's dilemma

can be modelled stochastically.

A stochastic reactive strategy specifies the conditional

probabilities of C and D as a reaction to all possible

histories of the game that fall into the memory of the

player. In the simplest case, where the initial probability to

play C in the first round can be ignored due to the infinite

iteration of the game, and where the memory of the players

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

is minimal, taking account only of the opponent's move in

the previous round, a reactive strategy E can be determined

by the pair (p,q), where p is the conditional probability to

play C after an opponent's C in the previous round, and q is

the probability to play C after an opponent's D. Some well-

known strategies have been termed Tit for Tat, which can

be represented by the pair (1,0), generous TFT by (1, 0.3),

the random strategy by (0.5, 0.5) and AlwaysD by (0,0).

Nowak and Sigmund [4] have shown that the payoff A for

strategy Ei against strategy Ej the infinitely iterated

prisoner's dilemma is

A(Ei,Ej) = [(R-T)+(P-S)] c c' + (S-P) c + (T-P) c' + P (1)

with

c = (qi+(pi-qi)qj) / (1-(pi-qi)(pj-qj)) (2)

and

c' = (qj+(pj-qj)qi) / (1-(pj-qj)(pi-qi)) (3)

for 0<p<1 and 0<q<1.

In order to model the evolution of strategies in a

population of players we abstract from individual

participants playing certain strategies and instead study the

strategy population consisting of m stochastic reactive

strategies Ei. The current state of the strategy population is

denoted by the vector x where xi denotes the prevalence or

frequency of strategy Ei in the population of players. Using

a replicator equation which relates the growth in frequency

of a strategy to its fitness Hofbauer and Sigmund [5] have

described the evolution of the reactive strategies for the

infinitely iterated prisoner's dilemma as an adaptive

dynamics, where the frequency of a reactive strategy in the

following generation is given by

xit1=xitfix/

f

(4)

where

fix=∑

j=1

n

xjAEi,E j

(5)

denotes the average payoff for Ei in a population of

strategies with frequency-vector x = (x1,...,xn) and

f=∑

i=1

n

xifix

(6)

is the average payoff in the population.

Running computer simulations on the basis of this model

Nowak and Sigmund [6] have shown that cooperation can

emerge among a population of randomly chosen reactive

strategies, as long as a stochastic version of Tit for Tat is

added to the population. In this setting it plays the role of a

police for the enforcement of reciprocity, insofar as it is

immune to exploitation and rewards cooperation. But since

it is precisely this provocability that makes it susceptible to

errors engendering an endless vendetta of retaliations, it is

finally superseded by Generous Tit for Tat (a nice strategy

that forgives every third defection).

The fact that the presence of Tit for Tat in the strategy

population is necessary to bring about a cooperative

outcome highlights the importance of a minimal social

structure that is required for the evolution of cooperation.

Even if Tit for Tat can be proven to be the strategy that can

invade a population of universal defectors in the smallest

cluster [6], its role shows that the prestructuration of the

population determines the evolution of the patterns of

interaction that constitute the final social structure. The

precise impact of the prestructuration of the population

depends on the degree of its incorporation into the

strategies: As Frank [7] has shown, if cooperators can

recognize each other with the help of some label they can

increase their payoff by interacting selectively with one

another. This mechanism can even explain the spontaneous

emergence of label-selective behaviour, as Rick Riolo

quoted by Holland [8] has shown with the help of computer

simulations.

C. Group Selective Behaviour and Misinterpretation

In interactions between partners of different origin such

as in-house development versus cooperation with other

organizations a certain level of discriminating behaviour is

often observed in interactions with the alien group. In

terms of reactive strategies the probabilities for Cooperate

are group-selective, resulting in different (p,q) values for

interactions among natives and between natives and aliens.

In an extreme form cooperation only occurs among

members of the same group. The common cultural and

normative background within an organization can serve as

the base for capturing uncertainty in social interaction.

Introducing noise in the interaction models the danger of

misinterpretation of the opponent's behaviour and allows

for some amount of misimplementation of the original

strategies [9].

III. APPLICATION TO OPEN SOURCE SOFTWARE

DEVELOPMENT

A. Introduction

Open source software development has generated

increasing interest in the last years, both from the business

and academic world, as some projects in different

application domains like Linux together with the suite of

GNU utilities, GNOME, KDE, Apache, sendmail, bind,

and several programming languages have achieved huge

success in their respective markets.

The main ideas of this development model are described

in the seminal work of Raymond [10], 'The Cathedral and

the Bazaar', in which he contrasts the traditional type of

software development of a few people planning a cathedral

in splendid isolation with the new collaborative bazaar

form of open source software development. In this, a large

number of developer-turned users come together without

monetary compensation to cooperate under a model of

rigorous peer-review and take advantage of parallel

debugging that leads to innovation and rapid advancement

in developing and evolving software products, thus forming

an example for 'egoless programming' as proposed by

Weinberg already in 1971 [11]. In order to allow for this to

happen and to minimize duplicated work, the source code

of the software needs to be accessible, and new versions

need to be released often. To this end, software licenses

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

that grant the necessary rights to the users, like free

redistribution, inclusion of the source code, the possibility

for modifications and derived works and some others have

been developed. One model for such licenses is the Open

Source Definition, which lists a number of requirements for

specific licenses [12]. The most prominent example which

fulfils these criteria while still being even more stringent, is

the GNU General Public Licence (GPL), developed by the

GNU project and advocated by the Free Software

Foundation [13].

The main question to be asked is whether the iterated

prisoner's dilemma can be applied in this setting. Its use

has been demonstrated in commercial software

development [14], with the main focus on group selective

behaviour and misunderstandings in the

consultant/customer interaction. Naturally, this distinction

is not applicable in open source software development. On

the other hand, several basic assumptions hold true: All

open source development projects after an initial stage

performed by a single person consist of several people, as

empirical studies have shown, in fact of more people than

commercial projects [15,16]. These people need to

cooperate, and contrary to commercial projects, they need

to cooperate without external influences, e.g. management

intervention or fear of job loss. Therefore the choice to

cooperate is indeed made on a more personal level,

depending on prior experiences and motivation. Also the

assumption of an unknown but finite number of

interactions holds true: Open source development efforts

entail repeated interactions between the participants, but

even more so than in commercial projects which are at

least planned to some extent, and the number of these

interactions are unknown to all players. Therefore the

iterated prisoner's dilemma in general can be applied.

In the next sections the different parts of the model are

discussed in the context of open source software

development.

B. The Open Source Software Development Players

First we define who the players participating in open

source software development are. There are several

possible lines of argumentation, and we consider two of the

different player sets: As a first type of game we consider all

participants in an open source software development

project as players, while in the second, more widely

defined approach, we define all users of a given software as

players. Even if the software does not yet exist, a possible

user community based on the planned functionality can be

assumed.

C. The Open Source Software Development Game

Given two possible defined ranges of players, the notion

of a single game has to be defined. It is important to note

that this game is iterated, i.e. performed often. The most

valuable artifact in an open source software development

effort is the software itself. Therefore it seems appropriate

to define a game along these lines as well. The most

interesting idea to explore is to use one release of the

software as one game. Depending on the release policy, an

alternative might be to use a certain time intervall, e.g. one

month, for defining a single game. Conveniently, this

notion of defining a game can be used for both types of

player sets to be considered.

D. Cooperation in Open Source Software Development

Naturally, the two available strategies that are available

to the players have to be defined next, i.e. cooperate or

defect. In open source software development, the most

simple definition of cooperation is participation. Using this

definition, cooperate would mean nothing else than

investing effort in the project. Again this definition is

usable for both player sets defined. In the most simple case

we need not further define the extent of effort, or the type

of work done, i.e. coding, documenting, or something else.

A more refined approach usable for the participating player

set would be to define defect as investing no or lower effort

than before, i.e. decreasing participation. Then, for

participants the two strategies would be to either invest

high effort, i.e. cooperate, or invest low effort, i.e. defect.

E. Payoffs in Open Source Software Development

Depending on the action taken, a payoff needs to be

realized. This payoff is generally defined in terms of utility,

which poses a severe problem. In open source software

development, motivations of participants can differ

[17,18], hinting at different utility functions, from which a

certain abstraction is necessary. Therefore we use the

resulting software as the main payoff, measured for

example using some software metric like lines-of-code or

function points [19]. This can also serve as a proxy for

some of the other constructs having been proposed as

motivating factors, e.g. reputation. A larger software

offering more functionality can be seen as being more

successful and thus conferring more reputation.

We combine this with the notion of negative utility

caused by the effort invested. This leads to the classical

four alternatives: The best outcome is for player A to

defect while B is cooperating. In this case, both get the

same utility from software functionality F, but A does not

have to invest effort I, which instead needs to be invested

by player B in addition to B's own effort, in the worst case

with higher costs I'>I, because A would have been more

familiar with the problem, or for similar reasons. If both

players are cooperating they invest their own efforts, and

both get the resulting software. If both players were to

defect, no effort is invested and no software is produced.

This results in the payoff matrix depicted in Table II.

F. Group Selective Behaviour and Misunderstandings

The presence of group selective behaviour might be of

interest in the context of open source software

development, especially when considering the whole user

group as players: If a player knows that the other player is

sharing the same norms and beliefs, e.g. on freedom of

software, he can be more sure of the other player's

cooperation, and might himself cooperate to a larger

degree. In this case, group selective behaviour might

indeed be present in open source software development.

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

TABLE II.

PAYOFF MATRIX IN OPEN SOURCE SOFTWARE DEVELOPMENT

C D

C(F-I,F-I) (F-I-I',F)

D(F,F-I-I') (0,0)

The sometimes elaborate joining rituals and scripts

[20,1,2] might then be interpreted as a mechanism

implemented to reduce the possibility for

misunderstandings: Anyone having joined through these

rituals and scripts shares these norms and beliefs, and can

easily be identified as such a player, who therefore is less

likely to defect. Therefore noise is reduced, which can lead

to a more cooperative outcome even if discrimatory

strategies are present.

We construct group selective strategies EJ (for Mr.

Jekyll) and EH (for Mr. Hyde) such that for each non-

discriminatory Jekyll strategy with basic p and q values pi

and qi both in inter- and intra-group interaction EiJ=((pi,qi),

(pi,qi)) there is a corresponding Hyde strategy EiH=((pi*,qi*),

(pi*,qi*)) with pi*=pi and qi*=qi in case of intra-group

interaction, and pi*=qi*=θ=0.001 in case of inter-group

interaction: i.e. EiH=((pi,qi),(θ,θ)).

Each of these two subpopulations of strategies is

characterized by a frequency vector consisting of the

frequencies of the corresponding strategies and zeros

otherwise such that xJ + xH = x.

The fitness fi(x) of a given strategy i within a population

of m strategies consists of the contributions of fitness it

achieves when played by members of the own (native)

group (fin(x)) and the alien group (fia(x)) weighted by their

respective population shares rn and ra . We denote by πij the

proportion of group is contacts that are with js (under the

idealizing assumption characterizing our model πnn = rn

and πna = ra:

fi

nx=∑

j=1

m

nn xjAEi,pj,q j

(7)

na[xj

JAEi,pj,q jxj

HAEi,,]

fi

ax=∑

j=1

n

aa xjAEi,pj,q j

(8)

an[xj

JAEi,pj,q jxj

HAEi,,]

The total fitness fi is simply

fix=rnfi

nxrafi

ax.

(9)

If we denote by xin and xia the frequencies of strategy i in

native and alien use, resp., the frequencies in time step t+1

are given by

xi

nt1=xi

ntfi

nx/

fn

(10)

and

xi

at1=xi

atfi

ax/

fa

(11)

with the average native and alien fitness values

fn=∑

i=1

n

xi

nfi

nx

(12)

and

fa=∑

i=1

n

xi

afi

ax.

(13)

The total frequency of strategy i is

xit1=rnxi

nt1raxi

at1.

(14)

On the basis of this model we investigated the dynamic

behaviour of the strategy population. Following Nowak and

Sigmund [21] we initialized the population with 100 pairs

of p- and q-values distributed uniformly in the unit square;

for each pair we added a duplicate strategy: a Mr. Hyde

lurking behind his Dr. Jekyll sibling and ready to defect in

inter-group interaction. All 200 strategies have the same

initial frequency of 0.005, allowing an equal proportion of

Jekylls and Hydes. We ran 1000 time steps since

preliminary tests showed in accordance with the results of

Nowak and Sigmund that interesting dynamics occur only

after the first several hundred time steps. We set the

proportion of aliens to 0.1. Figure I shows the result of a

typical run, where typical is defined as resulting in a total

fitness value with the smallest deviation from the average

of 20 runs with different random numbers. Shortly after

t=200 the Tit-for-Tat Jekyll strategies take over the

population, drastically improving native and alien fitness;

after t=400 there is a shift towards Generous TFT which

further increases fitness.

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

FIGURE I.

NON-DISCRIMINATORY OUTCOME IN SPITE OF DISCRIMINATORY STRATEGIES

IV. CONCLUSION

The main question remaining is what is to be gained by

applying the iterated prisoner's dilemma in the context of

open source software development. If the model can indeed

be used, it might allow for several insights into this

phenomenon. These insights can be roughly grouped into

two different levels: The open source idea in general and

its proliferation, and the situation in specific projects.

On the general level, this model gives an explanation

why open source software development has arisen and is

prospering. Evolving cooperation and thus formation of

open source development communities can be seen as a

natural choice. Therefore it strengthens other explanatory

approaches [22,23] in the reasoning that this idea is not

strictly motivated by altruism or similar motivations, but a

sound decision. Given a certain software functionality, the

possible user base can be estimated, and starting with a

certain rate of cooperators, the evolution of cooperation

within the total population can be simulated. This gives a

positive outlook for the open source movement, as prior

results indicate that even a small rate of cooperators might

be sufficient to establish cooperation in the long run.

On the level of specific projects, the environment,

especially any steps taken to reduce noise and thus

possibilities for misunderstandings seem to gain

prominence. Indeed, the joining scripts and even rituals in

some projects might in this way find a grounding in game

theory and can thus be explained. Their presence and

success might be found to be necessary for achieving a

cooperative situation and thus form a critical success factor

for open source projects. Group-selective behaviour is

interesting to explore in hybrid or gated open source

communities [24], composed of a mixture from a leading

company's employees and voluntary developers. Between

those groups selective behaviour might arise and endanger

a cooperative, i.e. successful, outcome.

Data currently available on open source projects and

communities could be used to document the situation in

these environments, i.e. the population of strategies in

place. This could be used as a starting point for simulations

of future behaviour, in order to get an understanding of

whether and when a cooperative outcome will result.

V. REFERENCES

[1] M.S. Elliott and W. Scacchi, “Free software

development: Cooperation and conflict in a virtual

organizational culture”, in S. Koch (ed.), Free/Open

Source Software Development, Hershey, PA: Idea

Group Publishing, 2004.

[2] E.G: Coleman and B. Hill, “The social production of

ethics in debian and free software communities:

Anthropological lessons for vocational ethic”, in S.

Koch (ed.), Free/Open Source Software

Development, Hershey, PA: Idea Group Publishing,

2004.

[3] R. Axelrod, The Evolution of Cooperation, London:

Penguin Books, 1984.

[4] M. Nowak and K. Sigmund, “Game dynamical

aspects of the prisoner's dilemma”, J. Appl. Math.

Comp., vol. 30, 1989, pp. 191-213.

[5] J. Hofbauer and K. Sigmund, The Theory of

Evolution and Dynamical Systems, Cambridge:

Cambridge University Press, 1988.

[6] M. Nowak and K. Sigmund, “The evolution of

stochastic strategies in the prisoner's dilemma”, Acta

Applicandae Mathematicae, vol. 20, 1990, pp. 247-

265.

[7] R. Frank, Passions within Reason. The strategic role

of the emotions, New York: W.W. Norton & Co.,

1988.

[8] J.H. Holland, Hidden Order - How adaptation builds

complexity, Reading, Mass.: Addison-Wesley, 1995.

[9] R. Axelrod and J. Wu, “How to cope with noise in

the iterated prisoner's dilemma”, Journal of Conflict

Resolution, vol. 39, 1995, pp. 183-189.

[10] E.S. Raymond, The Cathedral and the Bazaar,

Cambridge, Massachusetts: O’Reilly & Associates,

1999.

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

[11] G.M. Weinberg, The Psychology of Computer

Programming, Silver Anniversary Edition, New

York: Dorset House Publishing, 1998.

[12] B. Perens, “The Open Source Definition”, in

DiBona, C. et al. (eds.), Open Sources: Voices from

the Open Source Revolution, Cambridge,

Massachusetts: O’Reilly & Associates, 1999

[13] R.M. Stallman, Free Software, Free Society:

Selected Essays of Richard M. Stallman, Boston,

Massachusetts: GNU Press, 2002.

[14] W. Janko and J. Mitlöhner, “Ein spieltheoretisches

Modell der Berater-Kunden-Interaktion in IT-

Projekten”, in W. Gaul and M. Schader (eds.)

Mathematische Methoden der Wirtschafts-

wissenschaften, Heidelberg: Physica-Verlag, 1999.

[15] A. Mockus, R. Fielding and J: Herbsleb, “Two case

studies of open source software development:

Apache and Mozilla”, ACM Transactions on

Software Engineering and Methodology, vol. 11, no.

3, 2002, pp. 309-346.

[16] S. Koch, “Profiling an Open Source Project Ecology

and Its Programmers”, Electronic Markets, vol. 10,

no. 2, 2004, pp. 77-88.

[17] A. Hars and S. Ou, “Working for free? - Motivations

for participating in Open Source projects”, in

Proceedings of the 34th Hawaii International

Conference on System Sciences, Hawaii, 2001.

[18] G. Hertel, S. Niedner and S. Hermann, “Motivation

of software developers in open source projects: An

internet-based survey of contributors to the Linux

kernel”, Research Policy, vol. 32, no. 7, 2003, pp.

1159-1177.

[19] A.J. Albrecht and J.E. Gaffney, “Software Function,

Source Lines of Code, and Development Effort

Prediction: A Software Science Validation”, IEEE

Transactions on Software Engineering, vol. 9, no. 6,

1983, pp. 639-648.

[20] G. von Krogh, S. Spaeth, and K.R. Lakhani,

“Community, joining and specialization in open

source software innovation: A case study”, Research

Policy, vol. 32, no. 7, 2003, pp. 1217-1241, 2003.

[21] M. Nowak and K. Sigmund, “Tit for tat in

heterogenous populations”, Nature, vol. 355, January

1992.

[22] R.A. Ghosh, “Cooking pot markets: an economic

model for the trade in free goods and services on the

Internet”, First Monday, vol. 3, no. 3, March 1998.

[23] J. Lerner and J. Tirole, “The simple economics of

Open Source”, Working Paper 7600, National

Bureau of Economic Research, 2000.

[24] S. Sharma, V. Sugumaran and B. Rajagopalan, “A

framework for creating hybrid-OSS communities”,

Information Systems Journal, vol. 12, no. 1, 2002,

pp. 7-25.

Proceedings of the First International Conference on Open Source Systems

Genova, 11th-15th July 2005

Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191