Incentivising Cooperation between Agents for Content Sharing

Abstract

The performance of many emerging communication paradigms depend on high levels of cooperation amongst the peers in the network. Although an individual's best strategy may be to selfishly consume resources without reciprocation, the optimal social performance requires agents in the network to behave in an altruistic manner. This paper considers a P2P data dissemination scenario, and applies an autonomic trust protocol that forms social network structures to incentive cooperation. Trust links are formed according to the simple criterion that `individuals seek to interact with others at least as cooperative as themselves' and these links are used to prioritise the choice of peers to interact with. The success of the protocol is validated through a prisoner's dilemma based simulation which uses the similarity of interest between peers to define pay-offs. While the variation in interests reduces the average payoff (per iteration) received by the most cooperative individuals, only the most `divergent' and uncooperative nodes are heavily affected and ostracized from interaction by other cooperative nodes.

Full text

Page 1

Incentivising cooperation between agents for content sharing
S.M. Allen, M.J. Chorley, G.B. Colombo, and R.M. Whitaker
School of Computer Science and Informatics, Cardiff University,
{Stuart.M.Allen, M.J.Chorley, G.Colombo, R.M.Whitaker}@cs.cardiff.ac.uk
Abstract—The performance of many emerging communica-
tion paradigms depend on high levels of cooperation amongst
the peers in the network. Although an individual’s best strategy
may be to selfishly consume resources without reciprocation,
the optimal social performance requires agents in the network
to behave in an altruistic manner. This paper considers a
P2P data dissemination scenario, and applies an autonomic
trust protocol that forms social network structures to incentive
cooperation. Trust links are formed according to the simple
criterion that ‘individuals seek to interact with others at least
as cooperative as themselves’ and these links are used to
prioritise the choice of peers to interact with. The success of
the protocol is validated through a prisoner’s dilemma based
simulation which uses the similarity of interest between peers
to define pay-offs. While the variation in interests reduces the
average payoff (per iteration) received by the most cooperative
individuals, only the most ‘divergent’ and uncooperative nodes
are heavily affected and ostracized from interaction by other
cooperative nodes.
I. INTRODUCTION
Cooperation and the ability to trust that another party will
reciprocate cooperation is of high relevance to a number of
emerging communication systems and applications. These
include, for example, contemporary distributed electronic
systems such as peer-to-peer networks; mobile ad-hoc net-
works; and more recently opportunistic and pervasive net-
works. Trust and reputation models have been developed by
authors to introduce incentive mechanisms for reciprocation
within groups by encouraging interactions between coopera-
tive individuals while limiting the opportunities for defective
behaviours.
This paper models the scenario where agents interact
for content sharing and builds on a dynamic trust protocol
introduced in [1]. We introduce agents with individual
interest profiles and we assume that each agent derives its
utility relative to these interests. Agent’s own behaviours
(cooperation level), interests and first-hand observation of
others (history of past interactions) are the main components
that drive the formation and maintenance of social groups of
similarly cooperative individuals (among which interaction
should be prioritized).
Experimental results show that the decentralized system
remains successful in incentivising cooperative over selfish
behaviours in this more realistic application scenario. That
is, nodes accepting resources from others while not recipro-
cating by pushing any of their own resources are ostracised,
thus increasing the utility attained by cooperative individu-
als. In addition, our results show how system performance
is affected by the interest preferences of different groups,
where the global utility of the system is maximised when
all nodes share the same interest profile.
II. RELATED WORK
Several authors have investigated the importance of trust
and reputation models as incentives to cooperation within
networks of agents. It is widely recognised that uncooper-
ative behaviours are only capable of partial benefits since
they produce positive utilities to single individuals whereas
the most altruistic elements of the community are heavily
penalized [2]. A fundamental principle is that of reciprocity,
which has been used in several sociological studies of social
dilemmas and agent based modeling [3] and it is also
recognised to be one of the principal incentives to acquire
positive reputation for reciprocative actions [4]. Reciprocity
is behind the well known ‘tit for tat’ strategy that can be
applied in the game theory for the IPD [5].
Cooperative and altruistic behaviours have been applied to
variations of the P2P paradigm such as Mobile Peer-to-Peer
(MP2P) which adds a spatial dimension in which mobile
entities can interact and exchange resources [6]. Social
MP2P networking (SMP2P) further extends this concept
for either single or multi-hop communication by exploiting
‘social structures’ between devices based, for example, on
previous cooperative interactions between them [7]. This has
led to a number of applications in the field of Mobile ad Hoc
Networks [8] and Opportunistic Networks [9].
A similar approach is used in [10], in which locally
induced social groups are based on self-similarity of interests
of the most frequently encountered nodes. Sharing of mem-
bership information in social communities is used for a more
effective placement of resources and their consequent re-
trieval from the mobile network. Enforcement and incentive
based techniques can also be dynamically applied to gain
control over the average cooperation level of the community.
Beyond reciprocity of the actions made by individuals, other
incentives for cooperation have been socially modeled [11],
basing on self-similarity between nodes invoking altruism.
Similarity is often measured by introducing observable
traits or tags. Tags are useful meta-data because they are
potentially very flexible in representing physical spaces and
people as well as content [12]. The use of shared tags
2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology
978-0-7695-4191-4/10 $26.00 © 2010 IEEE
DOI 10.1109/WI-IAT.2010.145
79

Page 2

in content organization (collaborative tagging) has been
widely used as a basis for suggestion and recommendation
systems [13] and can be successfully used to organize the
network in social communities [14].
III. TRUST MODEL
In [1] an abstract protocol is described for encouraging
cooperation between parties where interactions are modeled
as single instances of a Prisoners Dilemma game. Nodes
gain a positive utility when their partner cooperates, while
they pay a price (negative or neutral utility) when defected
against. Pseudocode is shown in Figure 1 with the related
parameters defined in Table I. We summarise in the follow-
ing the key aspects of the algorithm:
• Relationship formation: The basis for forming rela-
tionships is that each individual seeks to interact with
a peer that is at least as cooperative as itself. Nodes
assess the average payoff they have received from each
peer over a recent number of interactions, and invite
to form a relationship those whose average payoff per
interaction is at least that which they would expect if
it had the same level of cooperation. The invited node
will then accept the invitation provided their average
payoff per interaction is above some threshold, defined
as vaccept= vcoop.α, where α ∈ [0,1] is a scaling
factor representing how risk averse the node is. Nodes
can drop relationship links whenever the average payoff
per interaction (over some time window) falls beneath
vaccept.
• Peer selection: A node vi chooses an opponent to
invite using a roulette wheel selection. Each individual
in the social group of vi (those nodes with which vi
has formed a relationship) is assigned a weight based
on the payoff vihas received from vjover their recent
interactions. The probability of selecting an opponent
outside of this social group is weighted by the recent
payoff produced by all nodes not belonging it. Note
that this probability can become significant whenever
nodes present social groups of very limited size.
• Acceptance: A node viagrees to an invitation to play
from vjif and only if vi’s recent history of interaction
with vjhas yielded a non-negative payoff (with a small
probability of forgiving a node giving negative payoff).
In the absence of a specific protocol during the IPD the
selfish strategy is best, with the defective nodes gaining
the highest utility at the expense of the most cooperative
individuals. This is shown in Figure 1, in which selection of
pairs is conducted at random and nodes are enforced to play
a game session at each interaction (Enforced Cooperation).
However, when the PD sessions are played following the
protocol described here, this tendency is reversed, with the
most cooperative nodes receiving the highest payoffs (Social
Networking Model) (see [1] for further details).
parameter
vi
vcoop
i
vaccept
i
nv
α
description
node i
cooperation probability for vi
relationship acceptance threshold for vi
number of nodes
scaling factor between invite and accept
thresholds for an node
node memoryspan for payoff
neighbours of viat a particular point
total payoff of vifrom interacting with
vjover the last m interactions with vj
m
Ni
tpij
Table I
PARAMETERS
Algorithm 1 Simulation Pseudo-code for the model
Initialize parameters
for numiterationsdo
for vi= 1 to to nvdo
vi
removes any
tpij/m < vaccept
i
node viselect a player, say vjpossibly from its social
network
node vjdecides whether to play with vi
if vjdecides to play then
vi and vj select their PD strategy according to
vcoop
payoffs for viand vj calculated according to the
original PD payoff matrix (see [1]).
tpij,tpjiupdated
end if
if (vinvite
i
< tpij/m) ∧ (tpji/m > vaccept
viand vjform a relationship
else
if (vinvite
j
< tpji/m) ∧ (tpij/m > vaccept
viand vjform a relationship
end if
end if
end for
end for
neighbour
vj
such that
j
) then
i
) then
Individual profiles of interests are generated for each of
the network nodes. Resources are characterized by their
category of interest, represented by one of a global set of M
‘tags’. Each node vihas normalised relative interest values
for each of these tags, denoted by Fi
such that:
M
?
When nodes i and j interact, the payoff accrued by each
node from content sharing represents the similarity of their
interest distributions, defined as U?
m(for 1 ≤ m ≤ M),
m=1
Fi
m= 1
ij:
U?
ij=
M
?
m=1
Fi
mFj
m
80

Page 3

0
0.5
1
1.5
2
2.5
0 0.1 0.20.3 0.4 0.5 0.60.7 0.80.91
Average Payoff per Iteration
Cooperation
Social Networking Model
Enforced Cooperation
Figure 1.Average Payoff per Iteration against Cooperation
To enable direct comparison with the results of previous
simulations [1] that applied the same model to the original
PD scenario, the payoffs are normalised as follows. We
calculate a theoretical maximum payoff mp over all pairs
of nodes:
mp =max
vi,vj∈V,i?=jU?
ij
This value is used to normalise the payoff to the range
[0,2] so that:
Uij=2U?
ij
mp
In common with other content dissemination studies
(e.g. [10]), we quantify the similarity of the interests of node
vjto those of viusing the Kullback Leibler divergence [15]
where a divergence of 0 means nodes vi and vj have
identical interest distributions. :
Di,j=
M
?
m=1
Fi
mlogFi
m
Fj
m
We modify in this work the payoff table for the PD
game to add a fixed cost C of transferring content to the
cooperation strategy, as seen in Table II, in order to simulate
a more realistic data dissemination scenario. If an agent
chooses to cooperate it pushes one of its carried resources
to the currently connected node and, consequently, a cost
is incurred. There is no cost incurred by a defecting agent
who pushes no content. If the cost of cooperation is positive
(C > 0) then a node’s best strategy is to defect, as this
maximises the payoff received regardless to the particular
strategy chosen by its opponent. However, as long as indi-
viduals share sufficient common interests (Uij−C > 0), the
total utility is maximised when they pairwise cooperate.
To allow direct comparison with the results obtained with
the original PD payoff matrix (see [1]), the cost C of
cooperation is set as: C =mp
2.
node j
node i
Cooperate
Defect
Cooperate
Uij− C,Uij− C
Uij,−C
Table II
MODIFIED PD PAYOFF TABLE
Defect
−C,Uij
0,0
IV. EXPERIMENTAL RESULTS
This Section shows and discusses the results obtained by a
number of simulations in which the modified trust protocol,
described in Section III, has been applied to a network of
agents presenting different preferences of interests.
In this work we adopt the setting for the parameters
that showed the best performance in [1]. Cooperation levels
defining individuals behaviours are assigned within the range
[0,1] according to an uniform random distribution.
We then define a set of M interests for each node accord-
ing to Zipf’s law [16] to produce relative interest values
for each interest, Fm. This reflects a ‘global’ probability
distribution of interests derived from a real web application.
We consider a population of 100 agents partitioned in a
number of ‘interest communities’ V1,...,VN, so that all
nodes belonging to the same community share the same
profile of interests.
Fvi
m= Fvj
mfor i,j ∈ Vk, 1 ≤ k ≤ N
We have conducted experiments with one community
(coinciding with the whole network), five and ten. For each
combination of numbers of interests and number of interest
communities we have performed and averaged five different
runs using different random seeds values (used in the criteria
for selection and acceptance to play defined in Section III).
All experiments shown in this work correspond to runs
conducted for 5000 iterations.
Table III shows the distribution of interests for a sample
case corresponding to four different interests and five com-
munities. Each of the communities shares the same profile
of interests defined by a permutation of the original Zipf
distribution with four tags. Table IV shows for the same
example the values of the ‘divergence’ metric defined in
Section III.
Fi
0.48
0.12
0.12
0.12
0.12
1
Fi
0.24
0.16
0.16
0.24
0.24
2
Fi
0.16
0.24
0.48
0.16
0.48
3
Fi
0.12
0.48
0.24
0.48
0.16
4
V1
V2
V3
V4
V5
Table III
DISTRIBUTION OF INTERESTS FOR 4 INTERESTS AND 5 COMMUNITIES
Figure 2 shows payoff per iteration against cooperation
for sample scenarios with one, five, and ten communities of
interests, each of them presenting distributions using four
81

Page 4

V1
0
0.531
0.483
0.499
0.431
V2
0.531
0
0.166
0.032
0.275
V3
0.483
0.166
0
0.275
0.032
V4
0.499
0.032
0.275
0
0.351
V5
0.431
0.275
0.032
0.351
0
V1
V2
V3
V4
V5
Table IV
DIVERGENCE VALUES FOR 4 INTERESTS AND 5 COMMUNITIES
interests These experiments show that when nodes have
different interest profiles (i.e. there is more than one interest
community) the payoff received per iteration reduces con-
siderably with the number of communities in the network.
0
0.5
1
1.5
2
2.5
0 0.10.20.30.40.5 0.60.7 0.80.91
Average Payoff per Iteration
Cooperation
1 Interest Community
5 Interest Communities
10 Interest Communities
Figure 2.
Communities composed by 4 Interests
Cooperation against Payoff per Iteration with 1, 5 and 10
Figure 3 shows a sample case with four interests and
five communities by examining the average payoff received
for each node against their cooperation from the point of
view of each single community. It appears clear that for
one community only there is no real correlation between
payoff and cooperation. In fact, for community one even
its most cooperative nodes appear unable to produce high
payoff values. As shown in Table IV this community is
revealed to be the one showing the greatest divergence with
any of the others (i.e. that whose interest profile shows the
largest diversity against the rest of the population).
A. Correlation between Divergence and Social Links
Because of the way the payoff matrix is defined, see
Table II, the payoff received by nodes in a specific com-
munity becomes inevitably low when interacting with nodes
outside of the community itself. As a consequence, when
the divergence is large this reduces the ability of its nodes
to form social links with other nodes outside the same
community.
Figure 4 plots the average number of social links formed
between any pair of communities at each iteration against
0
0.2
0.4
0.6
0.8
1
1.2
1.4
00.10.2 0.30.40.5 0.60.70.80.91
Average Payoff per Iteration
Cooperation
Interest Community 1
Interest Community 2
Interest Community 3
Interest Community 4
Interest Community 5
Figure 3.
Communities
Cooperation against Payoff per Iteration - 4 Interests 5
the ‘divergence’ metric for each pair of communities for five
runs performed with different random seeds. The plot shows
a clear linear correlation between the divergence of two
communities and the number of links placed between them,
which dramatically decreases even for small divergence
values.
0.00001
0.0001
0.001
0.01
0.1
1
10
0 0.10.2 0.3 0.4 0.50.60.7 0.80.91
Average number of Links between Groups per Iteration
Divergence of Groups
Run 1
Run 2
Run 3
Run 4
Run 5
Average
Figure 4. Average Number of Social Links between Interest Communities
(per Iteration) against Communities Divergence
B. Correlation between Average Payoff and Social Links
As the divergence of a specific community is large, ac-
cording to the correlation diagram shown in Figure 4 we can
expect that each of the nodes belonging to that community
presents a limited social group size. This is shown by the
plot in Figure 5, which gives the relation between a nodes
cooperation and its number of social links for each of the
different communities partitioning the original network. The
curves show a behaviour comparable to that of Figure 3:
the most cooperative nodes belonging to all of the interest
82

Page 5

communities (except community one) increase their number
of social links more than linearly.
0
0.5
1
1.5
2
2.5
00.1 0.20.3 0.4 0.50.60.70.80.91
Average Number of Neighbours
Cooperation
Interest Community 1
Interest Community 2
Interest Community 3
Interest Community 4
Interest Community 5
Figure 5.
Cooperation
Average Number of Social Links (per Iteration) against
The average payoffs received by the nodes which fail
to maintain social groups of significant size will also be
affected, since this reduces their chances of cooperation
with other highly cooperative nodes not belonging to their
social groups of trust. Moreover, in the selection process
all of the nodes outside the social group of a node vi
are weighted with the same probability value (equal to the
average of the utility they produced to viwithin the memory-
span window m). This will inevitably include also the least
cooperative nodes in the population. Hence whenever a node
has a social group of particularly limited size the probability
of playing a session outside its social neighbourhood can
become significant and, as a consequence, the average return
in terms of payoff heavily penalised. This is confirmed by
Figure 6 showing the (positive) correlation between average
payoff (per iteration) and number of social links, with the
nodes that maximise the number of their social relationships
producing the greatest utilities.
In particular, the most ‘divergent’ community (one) is
composed of nodes not sharing interests with the majority of
the network, and also lacks a significant number of highly
cooperative nodes. This interest community fails to maintain
social links above a minimum threshold and appears unable
to produce any substantial payoff values. This results in even
its most cooperative agents being ostracised from forming
trust links and interacting with other cooperative nodes in
the population, thus confirming and explaining the poor
performance observed in Figure 3.
C. Partial adherence to trust protocol
The experiments presented in the previous sections have
shown how the most ‘divergent’ nodes in the network gain
little or no utility from the application of the trust protocol.
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0 0.51 1.52 2.5
Average Payoff per Iteration
Average Number of Neighbours
Interest Community 1
Interest Community 2
Interest Community 3
Interest Community 4
Interest Community 5
Figure 6.
Iteration)
Average Payoff against Average Number of Social Links (per
As such, there is no incentive for them to adhere to the
protocol. This is considered in Figure 7 which shows payoff
against cooperation for a scenario in which the nodes of
the first community do not form social groups to prioritize
interaction and play every PD session when asked.
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 0.1 0.20.3 0.40.50.6 0.7 0.80.91
Average Payoff per Iteration
Cooperation
Interest Community 1
Interest Community 2
Interest Community 3
Interest Community 4
Interest Community 5
Figure 7.
composed by 3,4, and 5 interests - Community one do not adhere to the
trust protocol
Cooperation against Payoff per Iteration with 5 Communities
The plot clearly shows that the performance of the most
cooperative nodes belonging to this community has wors-
ened considerably (see Figure 3 for comparison). In fact they
are now not only subjected to ostracism by the majority of
the network but also inevitably exposed to exploitation by
defective nodes of low cooperation level.
However, all of the agents belonging to other interest
communities (all sharing very similar profiles of interests)
remain unaffected and are still able to successfully prevent
the exploitation of malicious nodes including those belong-
ing to the non-adhering community (that only show a little
83

Page 6

increase of their payoff values).
V. CONCLUSION
This paper engages a trust model to apply to a data
dissemination scenario in which a network of agents pair-
wise connect and exchange tagged resources. Each agent is
characterized by a profile of interests (tags) representing its
preferences and an utility is realised according to how well
a received resource matches its own distribution of interests.
Simulation is performed through an adaptation of the IPD in
which the original payoff matrix has been modified in order
to better reflect a resource dissemination scenario.
The model is based on social networks of trust by means
of which cooperative agents prioritize and reciprocate inter-
actions, thus protecting themselves from being exploited by
uncooperative nodes. Social links are placed in the first in-
stance on the principle that ‘individuals seek to interact with
others at least as cooperative as themselves’ by observing the
payoff received during the history of previous encounters.
Note that the placement of links will then depend on the
self-similarity with other network agents in terms of both
behaviour (cooperation level) and interest preference.
The application of the protocol is successful in reversing
the common trend in which selfish individuals (who receive
but do not forward any resources to neighbouring peers) get
the most benefit by increasing their payoffs at the expense of
cooperative and altruistic behaviours. Diverse communities
having different interests worsens the overall performance of
the system. Here individual utilities are reduced since their
agents will receive lower payoffs when interacting with peers
belonging to a different ‘interest community’.
There is a strong correlation between divergence of in-
terest profiles and number of social links placed. Hence,
social groups of trust will be formed primarily between
nodes belonging to the same ‘community of interests’. As
a consequence, the most ‘divergent’ agents are prevented
from forming links outside their own interest communities
and this will affect significantly their final performances. In
fact, the lack of social links reduces the opportunity to play
with other cooperatives nodes while exposes agents to the
exploitation by defective and uncooperative behaviours. This
form of ostracism can dramatically reduce the outcomes of
the most divergent communities whenever they also present,
in average, medium or low cooperation levels. However, the
rest of the network (composed by nodes with very similar
interest profiles) remains unaffected and its nodes still show
a positive correlation between their payoff and cooperation
levels. Note that this happens even when the community
composed by the most most dissimilar nodes does not adhere
at all to the trust protocol.
VI. ACKNOWLEDGEMENTS
This research was funded by SOCIALNETS grant
217141, an EC - FP7 Future Emerging Technologies project
concerning pervasive adaptation.
REFERENCES
[1] S. M. Allen, G. Colombo, and R. M. Whitaker, “Cooperation
through self-similar social networks,” ACM Transactions on
Autonomous Adaptive Systems, vol. 5, no. 1, pp. 1–29, 2010.
[2] G. Hardin, “The tragedy of the commons,” Science, vol. 162,
pp. 1243–1248, 1968.
[3] M. Macy and R. Willer, “From factors to actors: Computa-
tional sociology and agent-based modeling,” Annual Review
of Sociology, vol. 28, pp. 1143–166, 2002.
[4] L. Mui, M. Mohtashemi, and A. Halberstadt, “Notions of
reputation in multi-agent systems,” in Proceedings of first
AAMAS ’02, 2002.
[5] R. Axelrod, The evolution of cooperation. Basic Books, New
York, 1984.
[6] A. Mondal, S. K. Madria, and M. Kitsuregawa, “Research
issues and overview of economic models in mobile-p2p
networks,” in Proceedings of DEXA 2007, 3-7 September
2007, Regensburg, Germany, 2007, pp. 809–813.
[7] D. J. Watts, P. S. Dodds, and M. E. J. Newman, “Identity
and search in social networks,” Science, 2002. [Online].
Available: http://arxiv.org/abs/cond-mat/0205383
[8] V. S. Pavan, V. Srinivasan, P. Nuggehalli, C. F. Chiasserini,
and R. R. Rao, “Cooperation in wireless ad hoc networks,”
in In Proceedings of IEEE Infocom, 2003, pp. 808–817.
[9] C. Boldrini, M. Conti, and A. Passarella, “Contentplace:
social-aware data dissemination in opportunistic networks,”
in Proceedings of MSWiM ’08. New York, NY, USA: ACM,
2008, pp. 203–210.
[10] E. Jaho and I. Stravrakakis, “Joint interest- and locality- aware
content dissemination in social networks,” in Proceedings of
WONS’09, 2009.
[11] R. Riolo, M. Cohen, and R. Axelrod, “Evolution ofcoopera-
tion without reciprocity,” Nature, vol. 414, pp. 441–443, 2001.
[12] A. Java, X. Song, T. Finin, and B. Tseng, “Why we twitter:
An analysis of a microblogging community,” Advances in Web
Mining and Web Usage Analysis, pp. 118–138, 2009.
[13] S. Golder and B. A. Huberman, “The structure of
collaborative tagging systems,” Journal of Information
Science, vol. 32, no. 2, Aug 2005. [Online]. Available:
http://arxiv.org/abs/cs.DL/0508082
[14] D. Ramage, P. Heymann, C. D. Manning, and H. Garcia-
Molina, “Clustering the tagged web,” in Proceedings of
WSDM ’09. New York, NY, USA: ACM, 2009, pp. 54–
63.
[15] S. Kullback, Information theory and statistics.
publications Mineola, MN, 1997.
Dover
[16] G. K. Zipf, The Psycho-Biology of Language.
bridge,Massachusetts, 1935.
Cam-
84