No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Emergent behaviours in multi-agent systems with Evolutionary Game Theory
Abstract
The mechanisms of emergence and evolution of collective behaviours in dynamical Multi-Agent Systems (MAS) of multiple interacting agents, with diverse behavioral strategies in co-presence, have been undergoing mathematical study via Evolutionary Game Theory (EGT). Their systematic study also resorts to agent-based modelling and simulation (ABM) techniques, thus enabling the study of aforesaid mechanisms under a variety of conditions, parameters, and alternative virtual games. This paper summarises some main research directions and challenges tackled in our group, using methods from EGT and ABM. These range from the introduction of cognitive and emotional mechanisms into agents’ implementation in an evolving MAS, to the cost-efficient interference for promoting prosocial behaviours in complex networks, to the regulation and governance of AI safety development ecology, and to the equilibrium analysis of random evolutionary multi-player games. This brief aims to sensitize the reader to EGT based issues, results and prospects, which are accruing in importance for the modeling of minds with machines and the engineering of prosocial behaviours in dynamical MAS, with impact on our understanding of the emergence and stability of collective behaviours. In all cases, important open problems in MAS research as viewed or prioritised by the group are described.
... Since Darwin, the challenge of explaining the evolution of cooperative behaviour has been actively explored across various fields, including evolutionary biology, ecology, economics and multi-agent systems [1][2][3][4][5][6]. Several mechanisms have been proposed to account for the evolution of cooperation, such as kin and group selection, direct and indirect reciprocity, structured populations, pre-commitments and incentives [3,7]. ...
... An important ongoing initiative in AI research is the design and implementation of AI for social goods [70], using AI-based solutions for effectively addressing social problems. To this end, there is an emerging body of evolutionary modelling studies that address prosocial behaviours in hybrid systems of human and AI agents in co-presence [4,5,[71][72][73][74][75][76]. As developing AI is costly, it is crucial to understand what kind of AI are most conducive for prosocial behaviours, in a cost-effective way. ...
Understanding the emergence of prosocial behaviours among self-interested individuals is an important problem in many scientific disciplines. Various mechanisms have been proposed to explain the evolution of such behaviours, primarily seeking the conditions under which a given mechanism can induce highest levels of cooperation. As these mechanisms usually involve costs that alter individual pay-offs, it is, however, possible that aiming for highest levels of cooperation might be detrimental for social welfare—the latter broadly defined as the total population pay-off, taking into account all costs involved for inducing increased prosocial behaviours. Herein, by comparing stochastic evolutionary models of two well-established mechanisms of prosocial behaviour—namely, peer and institutional incentives—we demonstrate that the objectives of maximizing cooperation and of maximizing social welfare are often misaligned. First, while peer punishment is often more effective than peer reward in promoting cooperation—especially with a higher impact-to-cost ratio—the opposite is true for social welfare. In fact, welfare typically decreases (increases) with this ratio for punishment (reward). Second, for institutional incentives, while maintaining similar levels of cooperation, rewards result in positive social welfare across a much broader range of parameters. Furthermore, both types of incentives often achieve optimal social welfare when their impact is moderate rather than maximal, indicating that careful planning is essential for costly institutional mechanisms to optimize social outcomes. These findings are consistent across varying mutation rates, selection intensities and game configurations. Overall, we argue for the need of adopting social welfare as the main optimization objective when designing and implementing evolutionary mechanisms for social and collective goods.
... As the share of human-AI interactions grows, and the capabilities of intelligent agents evolve, the presence of AI in human decision-making is increasingly impactful in shaping the trade-offs of social dilemmas and, therefore, our capacity to achieve common goals. The emergent consequences of these hybrid human-AI interaction dynamics remain largely obscure and under-researched [9,10,[21][22][23][24][25][26]. Existing theoretical models that consider hybrid dynamics (e.g. ...
... Research into the emergence and maintenance of cooperative behaviour has been a key concern for disciplines as varied as game theory, sociology, complex systems, multi-agent systems, physics and psychology for decades [1,7,24,[28][29][30][31][32][33][34]. Illustrating the conflict between individual and collective welfare decision-making, the Prisoner's dilemma is widely adopted as a suitable framework for the study of the evolution and emergence of cooperation among large populations [7,[35][36][37]. ...
As artificial intelligence (AI) systems are increasingly embedded in our lives, their presence leads to interactions that shape our behaviour, decision-making and social interactions. Existing theoretical research on the emergence and stability of cooperation, particularly in the context of social dilemmas, has primarily focused on human-to-human interactions, overlooking the unique dynamics triggered by the presence of AI. Resorting to methods from evolutionary game theory, we study how different forms of AI can influence cooperation in a population of human-like agents playing the one-shot Prisoner's dilemma game. We found that Samaritan AI agents who help everyone unconditionally, including defectors, can promote higher levels of cooperation in humans than Discriminatory AI that only helps those considered worthy/cooperative, especially in slow-moving societies where change based on payoff difference is moderate (small intensities of selection). Only in fast-moving societies (high intensities of selection), Discriminatory AIs promote higher levels of cooperation than Samaritan AIs. Furthermore, when it is possible to identify whether a co-player is a human or an AI, we found that cooperation is enhanced when human-like agents disregard AI performance. Our findings provide novel insights into the design and implementation of context-dependent AI systems for addressing social dilemmas.
... Since Darwin, the challenge of explaining the evolution of cooperative behavior has been actively explored across various fields, including Evolutionary Biology, Ecology, Economics, and Multiagent Systems (Han, 2022a, Nowak, 2006b, Paiva et al., 2018, Perc et al., 2017, Sigmund, 2010. ...
... An important ongoing initiative in Artificial Intelligence (AI) research is the design and implementation of AI for social goods (Tomašev et al., 2020), using AI-based solutions for effectively addressing social problems. To this end, there is an emerging body of evolutionary modelling studies that address prosocial behaviours in hybrid systems of human and AI agents in co-presence (Capraro et al., 2024, Fernández Domingos et al., 2022, Guo et al., 2023, Han, 2022a, Paiva et al., 2018, Santos, 2024, Shen et al., 2024. As developing AI is costly, it is crucial to understand what kind of AI are most conducive for prosocial behaviours, in a cost effective way. ...
Understanding the emergence of prosocial behaviours among self-interested individuals is an important problem in many scientific disciplines. Various mechanisms have been proposed to explain the evolution of such behaviours, primarily seeking the conditions under which a given mechanism can induce highest levels of cooperation. As these mechanisms usually involve costs that alter individual payoffs, it is however possible that aiming for highest levels of cooperation might be detrimental for social welfare -- the later broadly defined as the total population payoff, taking into account all costs involved for inducing increased prosocial behaviours. Herein, by comparatively analysing the social welfare and cooperation levels obtained from stochastic evolutionary models of two well-established mechanisms of prosocial behaviour, namely, peer and institutional incentives, we demonstrate exactly that. We show that the objectives of maximising cooperation levels and the objectives of maximising social welfare are often misaligned. We argue for the need of adopting social welfare as the main optimisation objective when designing and implementing evolutionary mechanisms for social and collective goods.
... Han, Albrecht, and Woolridge [11] investigated the mechanisms of the emergence and evolution of collective behaviours in multi-agent systems (MAS) through the lens of evolutionary game theory and agent-based modelling. Their work emphasises the importance of cognitive and emotional mechanisms in promoting prosocial behaviours within organisations. ...
... Han, Albrecht, and Woolridge [11] investigate the mechanisms of emergence and evolution of collective behaviours in multi-agent systems (MAS) through the lens of evolutionary game theory (EGT) and agent-based modelling (ABM). They explore various strategies; from incorporating cognitive and emotional mechanisms to promoting prosocial behaviours, emphasising the importance of these approaches in understanding and engineering MAS. ...
In the complex organisational landscape, managing workforce diversity effectively has become crucial due to rapid technological advancements and shifting societal values. This study explores strategic workforce management through the novel methodological framework consisting of the evolutionary game theory concept integrated with replicator dynamics and traditional game theory, addressing a notable gap in the literature and suggesting an evolutionarily stable workforce structure. Key findings indicate that targeted rewards for the most Enthusiastic employee type can reduce overall costs and enhance workforce efficiency, although managing a diverse team remains complex. The study reveals that while short-term incentives boost immediate productivity, long-term rewards facilitate favourable behavioural changes, which are crucial for sustaining organisational performance. Additionally, the role of artificial intelligence (AI) is highlighted, emphasising its potential to integrate with these theoretical models, thereby enhancing decision-making processes. The study underscores the importance of strategic leadership in navigating these dynamics, suggesting that leaders must tailor their approaches to balance short-term incentives and long-term rewards to maintain an optimal workforce structure.
... The ability of EGT, to model strategic interactions and dynamic behaviours within populations makes it suitable for cybersecurity applications. EGT has been successfully applied in many fields-such as computer science, physics, biology and economics [33,34,35,36,37,38,39,40]-as well as diverse important application domains-such as healthcare [41,42], AI safety and governance [43,44], and climate change mitigation [45,46]-creating flexible solutions in a complex dynamics context. ...
In the evolving digital landscape, it is crucial to study the dynamics of cyberattacks and defences. This study uses an Evolutionary Game Theory (EGT) framework to investigate the evolutionary dynamics of attacks and defences in cyberspace. We develop a two-population asymmetric game between attacker and defender to capture the essential factors of costs, potential benefits, and the probability of successful defences. Through mathematical analysis and numerical simulations, we find that systems with high defence intensities show stability with minimal attack frequencies, whereas low-defence environments show instability, and are vulnerable to attacks. Furthermore, we find five equilibria, where the strategy pair always defend and attack emerged as the most likely stable state as cyber domain is characterised by a continuous battle between defenders and attackers. Our theoretical findings align with real-world data from past cyber incidents, demonstrating the interdisciplinary impact, such as fraud detection, risk management and cybersecurity decision-making. Overall, our analysis suggests that adaptive cybersecurity strategies based on EGT can improve resource allocation, enhance system resilience, and reduce the overall risk of cyberattacks. By incorporating real-world data, this study demonstrates the applicability of EGT in addressing the evolving nature of cyber threats and the need for secure digital ecosystems through strategic planning and proactive defence measures.
... To address multi-agent interactions, and in addition to methods tailored to individual agents, game theory [43] formalizes interactions as games, so as to model, predict and optimize the strategic responses of rational agents [8,20,28]. Game theory has been employed to model and understand human choices in various contexts [50,52], and AI-based players have been increasingly tested to reproduce classical game scenarios and provide additional complexity to them [15,23,56,57], as well as to interact in game-like distributed technologies [30]. However, due to varying research protocols and discipline constraints, bridging the gap between theoretical game theory and empirical investigations of LLM agents in a reproducible, systematic and user-friendly setting is still a challenge [38]. ...
Letting AI agents interact in multi-agent applications adds a layer of complexity to the interpretability and prediction of AI outcomes, with profound implications for their trustworthy adoption in research and society. Game theory offers powerful models to capture and interpret strategic interaction among agents, but requires the support of reproducible, standardized and user-friendly IT frameworks to enable comparison and interpretation of results. To this end, we present FAIRGAME, a Framework for AI Agents Bias Recognition using Game Theory. We describe its implementation and usage, and we employ it to uncover biased outcomes in popular games among AI agents, depending on the employed Large Language Model (LLM) and used language, as well as on the personality trait or strategic knowledge of the agents. Overall, FAIRGAME allows users to reliably and easily simulate their desired games and scenarios and compare the results across simulation campaigns and with game-theoretic predictions, enabling the systematic discovery of biases, the anticipation of emerging behavior out of strategic interplays, and empowering further research into strategic decision-making using LLM agents.
... Undesirable Dispositions from Competition. It is plausible that evolution selected for certain conflict-prone dispostions in humans, such as vengefulness, aggression, risk-seeking, selfishness, dishonesty, deception, and spitefulness towards out-groups (Grafen, 1990;Han, 2022;Konrad & Morath, 2012;McNally & Jackson, 2013;Nowak, 2006;Rusch, 2014). Such traits could also be selected for in ML systems that are trained in more competitive multi-agent settings. ...
The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents' incentives, as well as seven key risk factors (information asymmetries, network effects, selection pressures, destabilising dynamics, commitment problems, emergent agency, and multi-agent security) that can underpin them. We highlight several important instances of each risk, as well as promising directions to help mitigate them. By anchoring our analysis in a range of real-world examples and experimental evidence, we illustrate the distinct challenges posed by multi-agent systems and their implications for the safety, governance, and ethics of advanced AI.
... Game-theoretic approaches to strategic thinking and their extension to population dynamics (i.e. Evolutionary Game Theory) [31] have deeply investigated this problem and the decision-making processes behind it. In this context, it has also been shown that agents can effectively learn in autonomy to improve their performance in dealing with others [62]. ...
The increasing complexity and unpredictability of many ICT scenarios let us envision that future systems will have to dynamically learn how to act and adapt to face evolving situations with little or no a priori knowledge, both at the level of individual components and at the collective level. In other words, such systems should become able to autonomously (i.e., self-) develop mental models of themselves and of their environment. Autonomous mental development includes: learning models of own capabilities; learning how to act purposefully towards the achievement of specific goals; and learning how to act in the presence of others, i.e., at the collective level. In this paper, we introduce a conceptual framework for autonomous mental development in ICT systems – at both the individual and collective levels – by framing its key concepts and illustrating suitable application domains. Then, we overview the many research areas that are contributing or can potentially contribute to the realization of the framework, and identify some key research challenges.
... Subsequently, we refine the policy by prioritizing state-action pairs that demonstrate high fitness or returns, drawing on the concept of replicator dynamics [20]. This evolutionary game theory concept has been successfully applied in analyzing both normative and descriptive behaviors among agents [32,9]. Unlike traditional methods, our approach incrementally modifies the policy with new batches of data without relying on gradient calculations, ensuring convergence to optimality with theoretical guarantees. ...
Reinforcement learning (RL) has been successfully applied to solve the problem of finding obstacle-free paths for autonomous agents operating in stochastic and uncertain environments. However, when the underlying stochastic dynamics of the environment experiences drastic distribution shifts, the optimal policy obtained in the trained environment may be sub-optimal or may entirely fail in helping find goal-reaching paths for the agent. Approaches like domain randomization and robust RL can provide robust policies, but typically assume minor (bounded) distribution shifts. For substantial distribution shifts, retraining (either with a warm-start policy or from scratch) is an alternative approach. In this paper, we develop a novel approach called {\em Evolutionary Robust Policy Optimization} (ERPO), an adaptive re-training algorithm inspired by evolutionary game theory (EGT). ERPO learns an optimal policy for the shifted environment iteratively using a temperature parameter that controls the trade off between exploration and adherence to the old optimal policy. The policy update itself is an instantiation of the replicator dynamics used in EGT. We show that under fairly common sparsity assumptions on rewards in such environments, ERPO converges to the optimal policy in the shifted environment. We empirically demonstrate that for path finding tasks in a number of environments, ERPO outperforms several popular RL and deep RL algorithms (PPO, A3C, DQN) in many scenarios and popular environments. This includes scenarios where the RL algorithms are allowed to train from scratch in the new environment, when they are retrained on the new environment, or when they are used in conjunction with domain randomization. ERPO shows faster policy adaptation, higher average rewards, and reduced computational costs in policy adaptation.
... Differently, we investigate how pre-commitment can be utilised in promoting cooperation in a coordination problem using technology adoption decision making scenario. In this chapter, we will summarise and synthesise our theoretical analysis and numerical simulations showcasing the strategic interaction among firms investing in a competitive technology market (Ogbo et al., 2022a,b;Han, 2022b;Han et al., 2022;Han, 2022a), using Evolutionary Game Theory (EGT) methods for finite, well-mixed populations (Nowak et al., 2004;Imhof et al., 2005;Hauert et al., 2007). ...
The adoption of new technologies by firms is a fundamental driver of technological change, enhancing competitiveness across various industries. Recent advancements in information technologies have amplified the strategic significance of technology in the competitive landscape, reshaping global markets and the workplace. Technological innovation continues at a swift pace, but its success hinges on effective adoption. Embracing new technologies sets businesses apart, fostering innovation, and attracting customers and investors. However, the decision to adopt technology poses challenges, especially regarding which technologies to choose in a dynamical market. Firms often invest in technology to gain a competitive edge, potentially neglecting broader social benefits in the process. This chapter summarises the authors' research on evolutionary dynamics of decision making regarding technology adoption. They employ methods from Evolutionary Game Theory (EGT), exploring scenarios with well-mixed populations and distributed networked environments.
... Evolutionary game theory offers a theoretical scaffold to probe the rise of cooperation, especially when kin selection is non-functional, as observed among genetically unrelated individuals [14][15][16][17][18][19][20][21][22][23]. In recent years, various theoretical models such as the prisoner's dilemma game, snowdrift game, stag hunt game and public goods game have been proposed to study the evolution of cooperation in real-world scenarios [24][25][26]. ...
The evolution and long-term sustenance of cooperation has consistently piqued scholarly interest across the disciplines of evolutionary biology and social sciences. Previous theoretical and experimental studies on collective risk social dilemma games have revealed that the risk of collective failure will affect the evolution of cooperation. In the real world, individuals usually adjust their decisions based on environmental factors such as risk intensity and cooperation level. However, it is still not well understood how such conditional behaviours affect the evolution of cooperation in repeated group interactions scenario from a theoretical perspective. Here, we construct an evolutionary game model with repeated interactions, in which defectors decide whether to cooperate in subsequent rounds of the game based on whether the risk exceeds their tolerance threshold and whether the number of cooperators exceeds the collective goal in the early rounds of the game. We find that the introduction of conditional cooperation strategy can effectively promote the emergence of cooperation, especially when the risk is low. In addition, the risk threshold significantly affects the evolutionary outcomes, with a high risk promoting the emergence of cooperation. Importantly, when the risk of failure to reach collective goals exceeds a certain threshold, the timely transition from a defective strategy to a cooperative strategy by conditional cooperators is beneficial for maintaining high-level cooperation.
... Evolutionary game theory offers a theoretical scaffold to probe the rise of cooperation, especially when kin selection is non-functional, as observed among genetically unrelated individuals [14][15][16][17][18][19][20][21][22][23]. In recent years, various theoretical models such as the prisoner's dilemma game, snowdrift game, stag hunt game and public goods game have been proposed to study the evolution of cooperation in real-world scenarios [24][25][26]. ...
The evolution and long-term sustenance of cooperation has consistently piqued scholarly interest across the disciplines of evolutionary biology and social sciences. Previous theoretical and experimental studies on collective risk social dilemma games have revealed that the risk of collective failure will affect the evolution of cooperation. In the real world individuals usually adjust their decisions based on environmental factors such as risk intensity and cooperation level. However, it is still not well understood how such conditional behaviors affect the evolution of cooperation in repeated group interactions scenario from a theoretical perspective. Here, we construct an evolutionary game model with repeated interactions, in which defectors decide whether to cooperate in subsequent rounds of the game based on whether the risk exceeds their tolerance threshold and whether the number of cooperators exceeds the collective goal in the early rounds of the game. We find that the introduction of conditional cooperation strategy can effectively promote the emergence of cooperation, especially when the risk is low. In addition, the risk threshold significantly affects the evolutionary outcomes, with a high risk promoting the emergence of cooperation. Importantly, when the risk of failure to reach collective goals exceeds a certain threshold, the timely transition from a defective strategy to a cooperative strategy by conditional cooperators is beneficial for maintaining high-level cooperation.
... Introduction. Evolutionary Game Theory has been widely used to study myriad questions in diverse disciplines like Evolutionary Biology, Ecology, Physics, Sociology and Computer Science, including the mechanisms underlying the emergence and stability of cooperation (Nowak, 2006;Perc et al., 2017;Han, 2022) and how to mitigate climate and Artificial Intelligence risks (Santos and Pacheco, 2011;Góis et al., 2019;Sun et al., 2021;Han et al., 2020). Institutional incentives, either positive (reward) and negative (punishment), are among the most important mechanisms for promoting the evolution of prosocial behaviours (Sigmund et al., 2001;Van Lange et al., 2014). ...
... A benchmark often used in studying intention recognition methods is based on the context of iterated prisoner's dilemma (IPD) game, where an agent's behavioural strategy (considered as the agent's intention) is to be inferred based on the agent's and its opponents' past actions during the course of a repeated game (Han et al., 2012(Han et al., , 2011bFujimoto and Kaneko, 2019;Nakamura and Ohtsuki, 2016;Montero-Porras et al., 2022). Game theory has proven suitable and powerful for modelling strategic behaviours in multi-agent settings (Han, 2022;Bloembergen et al., 2015;Parsons and Wooldridge, 2002;Nisan et al., 2007;Shoham and Leyton-Brown, 2008;Abate et al., 2021;Alalawi et al., 2019), which thus provides a suitable framework to study intention recognition. ...
Intention recognition entails the process of becoming aware of another agent's intention by inferring it through its actions and their effects on the environment. It allows agents to prevail when interacting with others in both cooperative and hostile environments. One of the main challenges in intention recognition is generating and collecting large amounts of data, and then being able to infer and recognise strategies. To this aim, in the context of repeated interactions, we generate diverse datasets, characterised by various noise levels and complexities. We propose an approach using different popular machine learning methods to classify strategies represented by sequences of actions in the presence of noise. Experiments have been conducted by varying the noise level and the number of generated strategies in the input data. Results show that the adopted methods are able to recognise strategies with high accuracy. Our findings and approach open up a novel research direction, consisting of combining machine learning and game theory in generating large and complex datasets and making inferences. This can allow us to explore and quantify human behaviours based on data-driven and generative models.
... As a result, this strengthens cooperation, though their beholders incur a significant emotional cost to achieve this. Our analysis provides novel insights into the design and engineering of self-organised and distributed cooperative multi-agent systems and how guilt-capable agents should be distributed to optimise cooperative outcomes, depending on the specific MAS network structure [1,8,11,16,20,28,33]. ...
Building ethical machines may involve bestowing upon them the emotional capacity to self-evaluate and repent on their actions. While reparative measures, such as apologies, are often considered as possible strategic interactions, the explicit evolution of the emotion of guilt as a behavioural phenotype is not yet well understood. Here, we study the co-evolution of social and non-social guilt of homogeneous or heterogeneous populations, including well-mixed, lattice and scale-free networks. Social guilt comes at a cost, as it requires agents to make demanding efforts to observe and understand others, while non-social guilt only requires the awareness of the agents' own state and hence incurs no social cost. Those choosing to be non-social are however more sensitive to exploitation by other agents due to their social unawareness. Resorting to methods from evolutionary game theory, we study whether such social and non-social guilt can evolve, depending on the underlying structure of the populations or systems of agents. In structured population settings, both social and non-social guilt can evolve through clustering with emotional prone strategies, allowing them to be protected from exploiters, especially in case of non-social (less costly) strategies. Overall, our findings provide important insights into the design and engineering of self-organised and distributed cooperative multi-agent systems.
... Several mechanisms have been proposed to explain the dilemmas of cooperation, including kin selection, direct and indirect reciprocity, incentives or networked structures; see surveys in (Nowak, 2006, Perc et al., 2017, Sigmund, 2010a. In contrast, there is a significant lack of studies looking at the role of cognitive and emotional mechanisms in behavioural evolution (Andras et al., 2018, Dafoe et al., 2021, Han, 2022. Given that emotions play a crucial role in humans' decision making (Marsella andGratch, 2014, Turrini et al., 2010), it is crucial to take into account these complex mechanisms to provide a more complete rendering of the evolution of cooperation, not just amongst humans, but between humans and machines. ...
Building ethical machines may involve bestowing upon them the emotional capacity to self-evaluate and repent on their actions. While reparative measures, such as apologies, are often considered as possible strategic interactions, the explicit evolution of the emotion of guilt as a behavioural phenotype is not yet well understood. Here, we study the co-evolution of social and non-social guilt of homogeneous or heterogeneous populations, including well-mixed, lattice and scale-free networks. Socially aware guilt comes at a cost, as it requires agents to make demanding efforts to observe and understand the internal state and behaviour of others, while non-social guilt only requires the awareness of the agents' own state and hence incurs no social cost. Those choosing to be non-social are however more sensitive to exploitation by other agents due to their social unawareness. Resorting to methods from evolutionary game theory, we study analytically, and through extensive numerical and agent-based simulations, whether and how such social and non-social guilt can evolve and deploy, depending on the underlying structure of the populations, or systems, of agents. The results show that, in both lattice and scale-free networks, emotional guilt prone strategies are dominant for a larger range of the guilt and social costs incurred, compared to the well-mixed population setting, leading therefore to significantly higher levels of cooperation for a wider range of the costs. In structured population settings, both social and non-social guilt can evolve and deploy through clustering with emotional prone strategies, allowing them to be protected from exploiters, especially in case of non-social (less costly) strategies. Overall, our findings provide important insights into the design and engineering of self-organised and distributed cooperative multi-agent systems.
... • DeepMind [7] • Five AI [9] • Heriot-Watt University [13] • King's College London [2] • Teesside University [8] • University of Aberdeen [3] • University of Edinburgh [1] • University of Essex [11] • University of Lancaster [4] C ...
This paper explores the dynamics of political competition, resource availability, and conflict through a simulation-based approach. Utilizing agent-based models (ABMs) within an evolutionary game theoretical framework, we investigate how individual behaviors and motivations influence collective outcomes in civil conflicts. Our study builds on the theoretical model developed by Basuchoudhary et al. (2023), which integrates factors such as resource availability, state capacity, and political entrepreneurship to explain the evolution of civil conflict. By simulating boundedly rational agents, we demonstrate how changes in resource availability can alter the nature of civil conflict, leading to different equilibrium outcomes. The findings highlight the importance of understanding individual motivations and adaptive behaviors in predicting the stability and resolution of conflicts. This research contributes to the growing body of literature on the use of agent-based models in evolutionary game theory and provides valuable insights into the complex interactions that drive civil violence.
Within the intricate dynamics of a blockchain system, participants with bounded rationality may engage in dishonest behavior, thereby disrupting the consensus of the system in the absence of incentive mechanisms. This vulnerability is particularly common in voting-based consensus mechanisms like Practical Byzantine Fault Tolerance (PBFT), yet existing analyses often overlook the rational perspectives of participants. In this paper, we introduce an evolutionary game-theoretic approach to extend the current incentive mechanisms, enhancing the system’s stability and security. In permissioned blockchain environments, we developed a three-party evolutionary game model involving proposers, validators and auditors that dynamically assesses node strategies under various economic incentives and attack scenarios on an improved mechanism basis. Using replicator dynamic equations, we identified the conditions under which nodes converge to different evolutionarily stable strategies. Through simulation experiments, we validated the incentive mechanism’s effectiveness in promoting honest participation and resisting bribery attacks. When dishonest nodes do not exceed 80%, the system can quickly evolve to the honest state and can cope with changes in the payoff for malfeasance, whereas the original mechanism could not reach the ideal state under any condition. This achievement is attributed to the design of the mechanism and the judicious adjustment of reward and punishment parameters. The research contributes valuable findings to the discourse on consensus security. It provides key insights for developing more robust and rational incentive mechanisms in future protocols.
In social dilemmas, individuals face a conflict between their own self-interest and the collective interest of the group. The provision of reward has been shown to be an effective means to drive cooperation in such situations. However, previous research has often made the idealized assumption that rewards are constant and do not change in response to changes in the game environment. In this paper, we introduce reward into the public goods game and develop a coevolutionary game model in which the strength of reward is adaptively adjusted according to the population state. Specifically, we assume that decreasing levels of cooperation lead to an increase in reward strength, while increasing levels of cooperation result in a decrease in reward strength. By investigating coevolutionary dynamics between population state and reward systems, we find that interior stable coexistence state can emerge in our model, where the levels of cooperation and reward strength remain at constant levels. In addition, we also reveal that the emergence of a full cooperation state only requires a minimal level of reward strength. Our study highlights the potential of adaptive feedback reward as a tool for achieving long-term stability and sustainability in social dilemmas.
One of the famous economic models in game theory is the duopoly Stackelberg model, in which a leader and a follower firm manufacture a single product in the market. Their goal is to obtain the maximum profit while competing with each other. The desired dynamics for a firm in a market is the convergence to its Nash equilibrium, but the dynamics of real-world markets are not always steady and can result in unpredictable market changes that exhibit chaotic behaviors. On the other hand, to approach reality more, the two firms in the market can be considered heterogeneous. The leader firm is bounded rationale, and the follower firm is adaptable. Modifying the cost function that affects the firms’ profit by adding the marginal cost term is another step toward reality. We propose a Stackelberg model with heterogeneous players and marginal costs, which exhibits chaotic behavior. This model’s equilibrium points, including the Nash equilibrium, are calculated by the backward induction method, and their stability analyses are obtained. The influence of changing each model parameter on the consequent dynamics is investigated through one-dimensional and two-dimensional bifurcation diagrams, Lyapunov exponents spectra, and Kaplan-Yorke dimension. Eventually, using a combination of state feedback and parameter adjustment methods, the chaotic solutions of the model are successfully tamed, and the model converges to its Nash equilibrium.
Social punishment, whereby cooperators punish defectors, has been suggested as an important mechanism that promotes the emergence of cooperation or maintenance of social norms in the context of the one-shot (i.e. non-repeated) interaction. However, whenever antisocial punishment, whereby defectors punish cooperators, is available, this antisocial behavior outperforms social punishment, leading to the destruction of cooperation. In this paper, we use evolutionary game theory to show that this antisocial behavior can be efficiently restrained by relying on prior commitments, wherein agents can arrange, prior to an interaction, agreements regarding posterior compensation by those who dishonor the agreements. We show that, although the commitment mechanism by itself can guarantee a notable level of cooperation, a significantly higher level is achieved when both mechanisms, those of proposing prior commitments and of punishment, are available in co-presence. Interestingly, social punishment prevails and dominates in this system as it can take advantage of the commitment mechanism to cope with antisocial behaviors. That is, establishment of a commitment system helps to pave the way for the evolution of social punishment and abundant cooperation, even in the presence of antisocial punishment.
Both conventional wisdom and empirical evidence suggest that arranging a prior commitment or agreement before an interaction takes place enhances the chance of reaching mutual cooperation. Yet it is not clear what mechanisms might underlie the participation in and compliance with such a commitment, especially when participation is costly and non-compliance can be profitable. Here, we develop a theory of participation and compliance with respect to an explicit commitment formation process and to institutional incentives where individuals, at first, decide whether or not to join a cooperative agreement to play a one-shot social dilemma game. Using a mathematical model, we determine whether and when participating in a costly commitment, and complying with it, is an evolutionarily stable strategy, resulting in high levels of cooperation. We show that, given a sufficient budget for providing incentives, rewarding of commitment compliant behaviours better promotes cooperation than punishment of non-compliant ones. Moreover, by sparing part of this budget for rewarding those willing to participate in a commitment, the overall level of cooperation can be significantly enhanced for both reward and punishment. Finally, the presence of mistakes in deciding to participate favours evolutionary stability of commitment compliance and cooperation.
Regulation of advanced technologies such as Artificial Intelligence (AI) has become increasingly important, given the associated risks and apparent ethical issues. With the great benefits promised from being able to first supply such technologies, safety precautions and societal consequences might be ignored or shortchanged in exchange for speeding up the development, therefore engendering a racing narrative among the developers. Starting from a game-theoretical model describing an idealised technology race in a fully connected world of players, here we investigate how different interaction structures among race participants can alter collective choices and requirements for regulatory actions. Our findings indicate that, when participants portray a strong diversity in terms of connections and peer-influence (e.g., when scale-free networks shape interactions among parties), the conflicts that exist in homogeneous settings are significantly reduced, thereby lessening the need for regulatory actions. Furthermore, our results suggest that technology governance and regulation may profit from the world’s patent heterogeneity and inequality among firms and nations, so as to enable the design and implementation of meticulous interventions on a minority of participants, which is capable of influencing an entire population towards an ethical and sustainable use of advanced technologies.
We present a summary of research that we have conducted employing AI to better understand human morality. This summary adumbrates theoretical fundamentals and considers how to regulate development of powerful new AI technologies. The latter research aim is benevolent AI, with fair distribution of benefits associated with the development of these and related technologies, avoiding disparities of power and wealth due to unregulated competition. Our approach avoids statistical models employed in other approaches to solve moral dilemmas, because these are “blind” to natural constraints on moral agents, and risk perpetuating mistakes. Instead, our approach employs, for instance, psychologically realistic counterfactual reasoning in group dynamics. The present paper reviews studies involving factors fundamental to human moral motivation, including egoism vs. altruism, commitment vs. defaulting, guilt vs. non-guilt, apology plus forgiveness, counterfactual collaboration, among other factors fundamental in the motivation of moral action. These being basic elements in most moral systems, our studies deliver generalizable conclusions that inform efforts to achieve greater sustainability and global benefit, regardless of cultural specificities in constituents.
Institutions and investors face the constant challenge of making accurate decisions and predictions regarding how best they should distribute their endowments. The problem of achieving an optimal outcome at a minimal cost has been extensively studied and resolved using several heuristics. However, these works usually failed to address how an external party can target different types of fair behaviour or do not take into account how limited information can shape this complex interplay. Here, we consider the Ultimatum game in a spatial setting and propose a hierarchy of interference mechanisms based on the amount of information available to an external decision-maker and desired standards of fairness. Our analysis reveals that monitoring the population at a macroscopic level requires more strict information gathering in order to obtain an optimal outcome and that local observations can mediate this requirement. Moreover, we identify the conditions which must be met for an individual to be eligible for investment in order to avoid unnecessary spending. We further explore the effects of varying mutation or behavioural exploration rates on the choice of investment strategy and total accumulated costs to the investor. Overall, our analysis provides new insights about efficient heuristics for cost-efficient promotion of fairness in societies. Finally, we discuss the differences between our findings and previous work done on cooperation dilemmas and present our suggestions for promoting fairness as an external decision-maker.
Institutions can provide incentives to enhance cooperation in a population where this behaviour is infrequent. This process is costly, and it is thus important to optimize the overall spending. This problem can be mathematically formulated as a multi-objective optimization problem where one wishes to minimize the cost of providing incentives while ensuring a minimum level of cooperation, sustained over time. In this paper, we provide a rigorous analysis of this optimization problem, in a finite population and stochastic setting, studying both pair-wise and multi-player cooperation dilemmas. We prove the regularity of the cost functions for providing incentives over time, characterize their asymptotic limits (infinite population size, weak selection and large selection) and show exactly when reward or punishment is more cost efficient. We show that these cost functions exhibit a phase transition phenomena when the intensity of selection varies. By determining the critical threshold of this phase transition, we provide exact calculations for the optimal cost of incentive, for any given intensity of selection. Numerical simulations are also provided to demonstrate analytical observations. Overall, our analysis provides for the first time a selection-dependent calculation of the optimal cost of institutional incentives (for both reward and punishment) that guarantees a minimum level of cooperation over time. It is of crucial importance for real-world applications of institutional incentives since intensity of selection is often found to be non-extreme and specific for a given population.
In this paper, we study analytically the statistics of the number of equilibria in pairwise social dilemma evolutionary games with mutation where a game’s payoff entries are random variables. Using the replicator–mutator equations, we provide explicit formulas for the probability distributions of the number of equilibria as well as other statistical quantities. This analysis is highly relevant assuming that one might know the nature of a social dilemma game at hand (e.g., cooperation vs coordination vs anti-coordination), but measuring the exact values of its payoff entries is difficult. Our delicate analysis shows clearly the influence of the mutation probability on these probability distributions, providing insights into how varying this important factor impacts the overall behavioural or biological diversity of the underlying evolutionary systems.
Graphic abstract
Upon starting a collective endeavour, it is important to understand your partners’ preferences and how strongly they commit to a common goal. Establishing a prior commitment or agreement in terms of posterior benefits and consequences from those engaging in it provides an important mechanism for securing cooperation. Resorting to methods from Evolutionary Game Theory (EGT), here we analyse how prior commitments can also be adopted as a tool for enhancing coordination when its outcomes exhibit an asymmetric payoff structure, in both pairwise and multi-party interactions. Arguably, coordination is more complex to achieve than cooperation since there might be several desirable collective outcomes in a coordination problem (compared to mutual cooperation, the only desirable collective outcome in cooperation dilemmas). Our analysis, both analytically and via numerical simulations, shows that whether prior commitment would be a viable evolutionary mechanism for enhancing coordination and the overall population social welfare strongly depends on the collective benefit and severity of competition, and more importantly, how asymmetric benefits are resolved in a commitment deal. Moreover, in multi-party interactions, prior commitments prove to be crucial when a high level of group diversity is required for optimal coordination. The results are robust for different selection intensities. Overall, our analysis provides new insights into the complexity and beauty of behavioural evolution driven by humans’ capacity for commitment, as well as for the design of self-organised and distributed multi-agent systems for ensuring coordination among autonomous agents.
The field of Artificial Intelligence (AI) is going through a period of great expectations, introducing a certain level of anxiety in research, business and also policy. This anxiety is further energised by an AI race narrative that makes people believe they might be missing out. Whether real or not, a belief in this narrative may be detrimental as some stake-holders will feel obliged to cut corners on safety precautions, or ignore societal consequences just to “win”. Starting from a baseline model that describes a broad class of technology races where winners draw a significant benefit compared to others (such as AI advances, patent race, pharmaceutical technologies), we investigate here how positive (rewards) and negative (punishments) incentives may beneficially influence the outcomes. We uncover conditions in which punishment is either capable of reducing the development speed of unsafe participants or has the capacity to reduce innovation through over-regulation. Alternatively, we show that, in several scenarios, rewarding those that follow safety measures may increase the development speed while ensuring safe choices. Moreover, in the latter regimes, rewards do not suffer from the issue of over-regulation as is the case for punishment. Overall, our findings provide valuable insights into the nature and kinds of regulatory actions most suitable to improve safety compliance in the contexts of both smooth and sudden technological shifts.
Rapid technological advancements in Artificial Intelligence (AI), as well as the growing deployment of intelligent technologies in new application domains, have generated serious anxiety and a fear of missing out among different stake-holders, fostering a racing narrative. Whether real or not, the belief in such a race for domain supremacy through AI, can make it real simply from its consequences, as put forward by the Thomas theorem. These consequences may be negative, as racing for technological supremacy creates a complex ecology of choices that could push stake-holders to underestimate or even ignore ethical and safety procedures. As a consequence, different actors are urging to consider both the normative and social impact of these technological advancements, contemplating the use of the precautionary principle in AI innovation and research. Yet, given the breadth and depth of AI and its advances, it is difficult to assess which technology needs regulation and when. As there is no easy access to data describing this alleged AI race, theoretical models are necessary to understand its potential dynamics, allowing for the identification of when procedures need to be put in place to favour outcomes beneficial for all. We show that, next to the risks of setbacks and being reprimanded for unsafe behaviour, the timescale in which domain supremacy can be achieved plays a crucial role. When this can be achieved in a short term, those who completely ignore the safety precautions are bound to win the race but at a cost to society, apparently requiring regulatory actions. Our analysis reveals that imposing regulations for all risk and timing conditions may not have the anticipated effect as only for specific conditions a dilemma arises between what is individually preferred and globally beneficial. Similar observations can be made for the long-term development case. Yet different from the short-term situation, conditions can be identified that require the promotion of risk-taking as opposed to compliance with safety regulations in order to improve social welfare. These results remain robust both when two or several actors are involved in the race and when collective rather than individual setbacks are produced by risk-taking behaviour. When defining codes of conduct and regulatory policies for applications of AI, a clear understanding of the timescale of the race is thus required, as this may induce important non-trivial effects.
This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI alignment, which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify ‘true’ moral principles for AI; rather, it is to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people’s moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified.
Understanding how to design agents that sustain cooperation in multi-agent systems has been a long-lasting goal in distributed artificial intelligence. Proposed solutions rely on identifying free-riders and avoiding cooperating or interacting with them. These mechanisms of social control are traditionally studied in games with linear and deterministic payoffs, such as the prisoner’s dilemma or the public goods game. In reality, however, agents often face dilemmas in which payoffs are uncertain and non-linear, as collective success requires a minimum number of cooperators. The collective risk dilemma (CRD) is one of these games, and it is unclear whether the known mechanisms of cooperation remain effective in this case. Here we study the emergence of cooperation in CRD through partner-based selection. First, we discuss an experiment in which groups of humans and robots play a CRD. This experiment suggests that people only prefer cooperative partners when they lose a previous game (i.e., when collective success was not previously achieved). Secondly, we develop an evolutionary game theoretical model pointing out the evolutionary advantages of preferring cooperative partners only when a previous game was lost. We show that this strategy constitutes a favorable balance between strictness (only interact with cooperators) and softness (cooperate and interact with everyone), thus suggesting a new way of designing agents that promote cooperation in CRD. We confirm these theoretical results through computer simulations considering a more complex strategy space. Third, resorting to online human–agent experiments, we observe that participants are more likely to accept playing in a group with one defector when they won in a previous CRD, when compared to participants that lost the game. These empirical results provide additional support to the human predisposition to use outcome-based partner selection strategies in human–agent interactions.
The rhetoric of the race for strategic advantage is increasingly being used with regard to the development of artificial intelligence (AI), sometimes in a military context, but also more broadly. This rhetoric also reflects real shifts in strategy, as industry research groups compete for a limited pool of talented researchers, and nation states such as China announce ambitious goals for global leadership in AI. This paper assesses the potential risks of the AI race narrative and of an actual competitive race to develop AI, such as incentivising corner-cutting on safety and governance, or increasing the risk of conflict. It explores the role of the research community in responding to these risks. And it briefly explores alternative ways in which the rush to develop powerful AI could be framed so as instead to foster collaboration and responsible progress.
Intelligent machines have reached capabilities that go beyond a level that a human being can fully comprehend without sufficiently detailed understanding of the underlying mechanisms. The choice of moves in the game Go (generated by Deep Mind?s Alpha Go Zero [1]) are an impressive example of an artificial intelligence system calculating results that even a human expert for the game can hardly retrace [2]. But this is, quite literally, a toy example. In reality, intelligent algorithms are encroaching more and more into our everyday lives, be it through algorithms that recommend products for us to buy, or whole systems such as driverless vehicles. We are delegating ever more aspects of our daily routines to machines, and this trend looks set to continue in the future. Indeed, continued economic growth is set to depend on it. The nature of human-computer interaction in the world that the digital transformation is creating will require (mutual) trust between humans and intelligent, or seemingly intelligent, machines. But what does it mean to trust an intelligent machine? How can trust be established between human societies and intelligent machines?
The problem of promoting the evolution of cooperative behaviour within populations of self-regarding individuals has been intensively investigated across diverse fields of behavioural, social and computational sciences. In most studies, cooperation is assumed to emerge from the combined actions of participating individuals within the populations, without taking into account the possibility of external interference and how it can be performed in a cost-efficient way. Here, we bridge this gap by studying a cost-efficient interference model based on evolutionary game theory, where an exogenous decision-maker aims to ensure high levels of cooperation from a population of individuals playing the one-shot Prisoner’s Dilemma, at a minimal cost. We derive analytical conditions for which an interference scheme or strategy can guarantee a given level of cooperation while at the same time minimising the total cost of investment (for rewarding cooperative behaviours), and show that the results are highly sensitive to the intensity of selection by interference. Interestingly, we show that a simple class of interference that makes investment decisions based on the population composition can lead to significantly more cost-efficient outcomes than standard institutional incentive strategies, especially in the case of weak selection.
We study the situation of an exogenous decision-maker aiming to encourage a population of autonomous , self-regarding agents to follow a desired behaviour at a minimal cost. The primary goal is therefore to reach an efficient trade-off between pushing the agents to achieve the desired configuration while minimising the total investment. To this end, we test several interference paradigms resorting to simulations of agents facing a cooperative dilemma in a spatial arrangement. We systematically analyse and compare interference strategies rewarding local or global behavioural patterns. Our results show that taking into account the neighbour-hood's local properties, such as its level of coop-erativeness, can lead to a significant improvement regarding cost efficiency while guaranteeing high levels of cooperation. As such, we argue that local interference strategies are more efficient than global ones in fostering cooperation in a population of autonomous agents.
In this paper, we study the distribution of the number of internal equilibria of a multi-player two-strategy random evolutionary game. Using techniques from the random polynomial theory, we obtain a closed formula for the probability that the game has a certain number of internal equilibria. In addition, by employing Descartes' rule of signs and combinatorial methods, we provide useful estimates for this probability. Finally, we also compare our analytical results with those obtained from samplings.
The analysis of equilibrium points in random games has been of great interest in evolutionary game theory, with important implications for understanding of complexity in a dynamical system, such as its behavioural, cultural or biological diversity. The analysis so far has focused on random games of independent payoff entries. In overcoming this restrictive assumption, here we study equilibrium points in random games with correlated entries. Namely, we analyse the mean value and the distribution of the number of (stable) internal equilibria in multi-player two-strategy evolutionary games where the payoff matrix entries are correlated random variables. Our contributions are as follows. We first obtain a closed formula for the mean number of internal equilibria, characterise its asymptotic behaviour and study the effect of the correlation. We then provide analytical formulas to compute the probability of attaining a certain number of internal equilibria, and derive an approximate formula for the computation of this probability. Last but not least, we reveal some universal estimates that are independent of the distribution of the payoff matrix entries, and provide numerical simulations to support the obtained analytical results.
Extensive cooperation among unrelated individuals is unique to humans, who often sacrifice personal benefits for the common good and work together to achieve what they are unable to execute alone. The evolutionary success of our species is indeed due, to a large degree, to our unparalleled other-regarding abilities. Yet, a comprehensive understanding of human cooperation remains a formidable challenge. Recent research in social science indicates that it is important to focus on the collective behavior that emerges as the result of the interactions among individuals, groups, and even societies. Non-equilibrium statistical physics, in particular Monte Carlo methods and the theory of collective behavior of interacting particles near phase transition points, has proven to be very valuable for understanding counterintuitive evolutionary outcomes. By studying models of human cooperation as classical spin models, a physicist can draw on familiar settings from statistical physics. However, unlike pairwise interactions among particles that typically govern solid-state physics systems, interactions among humans often involve group interactions, and they also involve a larger number of possible states even for the most simplified description of reality. The complexity of solutions therefore often surpasses that observed in physical systems. Here we review experimental and theoretical research that advances our understanding of human cooperation, focusing on spatial pattern formation, on the spatiotemporal dynamics of observed solutions, and on self-organization that may either promote or hinder socially favorable states.
Inspired by psychological and evolutionary studies, we present here theoretical models wherein agents have the potential to express guilt with the ambition to study the role of this emotion in the promotion of pro-social behaviour. To achieve this goal, analytical and numerical methods from evolutionary game theory are employed to identify the conditions for which enhanced cooperation emerges within the context of the iterated prisoners dilemma. Guilt is modelled explicitly as two features, i.e. a counter that keeps track of the number of transgressions and a threshold that dictates when allevi-ation (through for instance apology and self-punishment) is required for an emotional agent. Such an alleviation introduces an effect on the payoff of the agent experiencing guilt. We show that when the system consists of agents that resolve their guilt without considering the co-player's attitude towards guilt alleviation then cooperation does not emerge. In that case those guilt prone agents are easily dominated by agents expressing no guilt or having no incentive to alleviate the guilt they experience. When, on the other hand, the guilt prone focal agent requires that guilt only needs to be alleviated when guilt alleviation is also manifested by a defecting co-player, then cooperation may thrive. This observation remains consistent for a generalised model as is discussed in this article. In summary, our analysis provides important insights into the design of multi-agent and cogni-tive agent systems where the inclusion of guilt modelling can improve agents' cooperative behaviour and overall benefit.
This paper discusses means for promoting artificial intelligence (AI) that is designed to be safe and beneficial for society (or simply “beneficial AI”). The promotion of beneficial AI is a social challenge because it seeks to motivate AI developers to choose beneficial AI designs. Currently, the AI field is focused mainly on building AIs that are more capable, with little regard to social impacts. Two types of measures are available for encouraging the AI field to shift more toward building beneficial AI. Extrinsic measures impose constraints or incentives on AI researchers to induce them to pursue beneficial AI even if they do not want to. Intrinsic measures encourage AI researchers to want to pursue beneficial AI. Prior research focuses on extrinsic measures, but intrinsic measures are at least as important. Indeed, intrinsic factors can determine the success of extrinsic measures. Efforts to promote beneficial AI must consider intrinsic factors by studying the social psychology of AI research communities.
Before engaging in a group venture agents may require commitments from other members in the group, and based on the level of acceptance (participation) they can then decide whether it is worthwhile joining the group e ort. Here, we show in the context of Public Goods Games and using stochastic evolutionary game theory modelling, which implies imitation and mutation dynamics, that arranging prior commitments while imposing a minimal participation when interacting in groups induces agents to behave cooperatively. Our analytical and numerical results show that if the cost of arranging the commitment is su ciently small compared to the cost of cooperation, commitment arranging behavior is frequent, leading to a high level of cooperation in the population. Moreover, an optimal participation level emerges depending both on the dilemma at stake and on the cost of arranging the commitment. Namely, the harsher the common good dilemma is, and the costlier it becomes to arrange the commitment, the more participants should explicitly commit to the agreement to ensure the success of the joint venture. Furthermore, considering that commitment deals may last for more than one encounter, we show that commitment proposers can be lenient in case of short-term agreements, yet should be strict in case of long-term interactions.
We study the situation of a decision-maker who aims to encourage the players of an evolutionary game theoretic system to follow certain desired behaviours. To do so, she can interfere in the system to reward her preferred behavioural patterns. However, this action requires certain cost (e.g., resource consumption). Given this, her main goal is to maintain an efficient trade-off between achieving the desired system status and minimising the total cost spent. Our results reveal interesting observations, which suggest that further investigations in the future are required.
Agents make commitments towards others in order to influence others in a certain way, often by dismissing more profitable options. Most commitments depend on some incentive that is necessary to ensure that the action is in the agent's interest and thus, may be carried out to avoid eventual penalties. The capacity for using commitment strategies effectively is so important that natural selection may have shaped specialized capacities to make this possible. Evolutionary explanations for commitment, particularly its role in the evolution of cooperation, have been actively sought for and discussed in several fields, including Psychology and Philosophy. In this paper, using the tools of evolutionary game theory, we provide a new model showing that individuals tend to engage in commitments, which leads to the emergence of cooperation even without assuming repeated interactions. The model is characterized by two key parameters: the punishment cost of failing commitment imposed on either side of a commitment, and the cost of managing the commitment deal. Our analytical results and extensive computer simulations show that cooperation can emerge if the punishment cost is large enough compared to the management cost.
We herein analyse the density, f(n,d), and the expected number, E(n, d), of internal equilibria in a d-player n-strategy random evolutionary game where the game payoff matrix is generated from normal distributions. First, we study their asymptotic behaviour for n = 2 and varying d (i.e. general multiplayer game with two strategies), proving various novel results regarding the upper bounds and limiting behaviour, including when considering stable equilibria only. Second, we analyze the distribution of equilibrium points, proving monotone properties of f(n,d) and its scaled version. We then provide numerical illustration and conjecture a similar asymptotic behaviour of E(n, d) in the general case, i.e. random games with arbitrary n and d. Last but not least, we establish a connection between f(n,d) and two well-known classes of polynomials, the Legendre and Bernstein polynomials, making our results directly relevant to the already rich study of such polynomials. In short, our results contribute both to the evolutionary game theory and to the random polynomial theory.
Commitments have been shown to promote cooperation if, on the one hand, they can be sufficiently enforced, and on the other hand, the cost of arranging them is justified with respect to the benefits of cooperation. When either of these constraints is not met it leads to the prevalence of commitment free-riders, such as those who commit only when someone else pays to arrange the commitments. Here, we show how intention recognition may circumvent such weakness of costly commitments. We describe an evolutionary model, in the context of the one-shot Prisoner's Dilemma, showing that if players first predict the intentions of their co-player and propose a commitment only when they are not confident enough about their prediction, the chances of reaching mutual cooperation are largely enhanced. We find that an advantageous synergy between intention recognition and costly commitments depends strongly on the confidence and accuracy of intention recognition. In general, we observe an intermediate level of confidence threshold leading to the highest evolutionary advantage, showing that neither unconditional use of commitment nor intention recognition can perform optimally. Rather, our results show that arranging commitments is not always desirable, but that they may be also unavoidable depending on the strength of the dilemma.
Social institutions often use rewards and penalties to promote cooperation. Providing incentives tends to be costly, so it is important to find effective and efficient policies for the combined use of rewards and penalties. Most studies of cooperation, however, have addressed rewarding and punishing in isolation and have focused on peer-to-peer sanctioning as opposed to institutional sanctioning. Here, we demonstrate that an institutional sanctioning policy we call ‘first carrot, then stick’ is unexpectedly successful in promoting cooperation. The policy switches the incentive from rewarding to punishing when the frequency of cooperators exceeds a threshold. We find that this policy establishes and recovers full cooperation at lower cost and under a wider range of conditions than either rewards or penalties alone, in both well-mixed and spatial populations. In particular, the spatial dynamics of cooperation make it evident how punishment acts as a ‘booster stage’ that capitalizes on and amplifies the pro-social effects of rewarding. Together, our results show that the adaptive hybridization of incentives offers the ‘best of both worlds’ by combining the effectiveness of rewarding in establishing cooperation with the effectiveness of punishing in recovering it, thereby providing a surprisingly inexpensive and widely applicable method of promoting cooperation.
When starting a new collaborative endeavor, it pays to establish upfront how strongly your partner commits to the common goal and what compensation can be expected in case the collaboration is violated. Diverse examples in biological and social contexts have demonstrated the pervasiveness of making prior agreements on posterior compensations, suggesting that this behavior could have been shaped by natural selection. Here, we analyze the evolutionary relevance of such a commitment strategy and relate it to the costly punishment strategy, where no prior agreements are made. We show that when the cost of arranging a commitment deal lies within certain limits, substantial levels of cooperation can be achieved. Moreover, these levels are higher than that achieved by simple costly punishment, especially when one insists on sharing the arrangement cost. Not only do we show that good agreements make good friends, agreements based on shared costs result in even better outcomes.
Decision making about which are the scrutinized intentions of others, usually called intention reading or intention recognition, is an elementary basic decision making process required as a basis for other higher- level decision making, such as the intention-based decision making which we have set forth in previous work. We present herein a recognition method possessing several features desirable of an elementary process: (i) The method is context-dependent and incremental, enabling progressive construction of a three-layer Bayesian network model as more actions are observed, and in a context-situated manner that relies on a logic programming knowledge base concerning the context; (ii) The Bayesian network is structured from a specific knowledge base of readily specified and readily maintained Bayesian network fragments with simple structures, thereby enabling the efficient acquisition of that knowledge base (engineered either by domain experts or else automatically from a plan corpus); and, (iii) The method addresses the issue of intention change and abandonment, and can appropriately resolve the issue of the recogni- tion of multiple intentions. The several aspects of the method have been experimentally evaluated in applications and achieving definite success, using the Linux plan corpus and the so-called IPD plan corpora, which are playing sequences generated by game playing strategies needing to be recognized, in the iterated Prisoner’s Dilemma. One other application concerns variations of Elder Care in the context of Ambient Intelligence.
This paper envisions a future where autonomous agents are used to foster and support pro-social behavior in a hybrid society of humans and machines. Pro-social behavior occurs when people and agents perform costly actions that benefit others. Acts such as helping others voluntarily, donating to charity, providing informations or sharing resources, are all forms of pro-social behavior. We discuss two questions that challenge a purely utilitarian view of human decision making and contextualize its role in hybrid societies: i) What are the conditions and mechanisms that lead societies of agents and humans to be more pro-social? ii) How can we engineer autonomous entities (agents and robots) that lead to more altruistic and cooperative behaviors in a hybrid society? We propose using social simulations, game theory, population dynamics, and studies with people in virtual or real environments (with robots) where both agents and humans interact. This research will constitute the basis for establishing the foundations for the new field of Pro-social Computing, aiming at understanding, predicting and promoting pro-sociality among humans, through artificial agents and multiagent systems.
With the introduction of Artificial Intelligence (AI) and related technologies in our daily lives, fear and anxiety about their misuse as well as their inherent biases, incorporated during their creation, have led to a demand for governance and associated regulation. Yet regulating an innovation process that is not well understood may stifle this process and reduce benefits that society may gain from the generated technology, even under the best intentions. Instruments to shed light on such processes are thus needed as they can ensure that imposed policies achieve the ambitions for which they were designed. Starting from a game-theoretical model that captures the fundamental dynamics of a race for domain supremacy using AI technology, we show how socially unwanted outcomes may be produced when sanctioning is applied unconditionally to risk-taking, i.e. potentially unsafe, behaviours. We demonstrate here the potential of a regulatory approach that combines a voluntary commitment approach reminiscent of soft law, wherein technologists have the freedom of choice between independently pursuing their course of actions or establishing binding agreements to act safely, with either a peer or governmental sanctioning system of those that do not abide by what they pledged. As commitments are binding and sanctioned, they go beyond the classic view of soft law, akin more closely to actual law-enforced regulation. Overall, this work reveals how voluntary but sanctionable commitments generate socially beneficial outcomes in all scenarios envisageable in a short-term race towards domain supremacy through AI technology. These results provide an original dynamic systems perspective of the governance potential of enforceable soft law techniques or co-regulatory mechanisms, showing how they may impact the ambitions of developers in the context of the AI-based applications.
The actions of intelligent agents, such as chatbots, recommender systems, and virtual assistants are typically not fully transparent to the user. Consequently, users take the risk that such agents act in ways opposed to the users’ preferences or goals. It is often argued that people use trust as a cognitive shortcut to reduce the complexity of such interactions. Here we formalise this by using the methods of evolutionary game theory to study the viability of trust-based strategies in repeated games. These are reciprocal strategies that cooperate as long as the other player is observed to be cooperating. Unlike classic reciprocal strategies, once mutual cooperation has been observed for a threshold number of rounds they stop checking their co-player’s behaviour every round, and instead only check it with some probability. By doing so, they reduce the opportunity cost of verifying whether the action of their co-player was actually cooperative. We demonstrate that these trust-based strategies can outcompete strategies that are always conditional, such as Tit-for-Tat, when the opportunity cost is non-negligible. We argue that this cost is likely to be greater when the interaction is between people and intelligent agents, because of the reduced transparency of the agent. Consequently, we expect people to use trust-based strategies more frequently in interactions with intelligent agents. Our results provide new, important insights into the design of mechanisms for facilitating interactions between humans and intelligent agents, where trust is an essential factor.
This book offers the first systematic guide to machine ethics, bridging between computer science, social sciences and philosophy. Based on a dialogue between an AI scientist and a novelist philosopher, the book discusses important findings on which moral values machines can be taught and how. In turn, it investigates what kind of artificial intelligence (AI) people do actually want.
What are the main consequences of the integration of AI in people’s every-day life? In order to co-exist and collaborate with humans, machines need morality, but which moral values should we teach them? Moreover, how can we implement benevolent AI? These are just some of the questions carefully examined in the book, which offers a comprehensive account of ethical issues concerning AI, on the one hand, and a timely snapshot of the power and potential benefits of this technology on the other. Starting with an introduction to common-sense ethical principles, the book then guides the reader, helping them develop and understand more complex ethical concerns and placing them in a larger, technological context. The book makes these topics accessible to a non-expert audience, while also offering alternative reading pathways to inspire more specialized readers.
A leading expert on evolution and communication presents an empirically based theory of the evolutionary origins of human communication that challenges the dominant Chomskian view.
Human communication is grounded in fundamentally cooperative, even shared, intentions. In this original and provocative account of the evolutionary origins of human communication, Michael Tomasello connects the fundamentally cooperative structure of human communication (initially discovered by Paul Grice) to the especially cooperative structure of human (as opposed to other primate) social interaction. Tomasello argues that human cooperative communication rests on a psychological infrastructure of shared intentionality (joint attention, common ground), evolved originally for collaboration and culture more generally. The basic motives of the infrastructure are helping and sharing: humans communicate to request help, inform others of things helpfully, and share attitudes as a way of bonding within the cultural group. These cooperative motives each created different functional pressures for conventionalizing grammatical constructions. Requesting help in the immediate you-and-me and here-and-now, for example, required very little grammar, but informing and sharing required increasingly complex grammatical devices. Drawing on empirical research into gestural and vocal communication by great apes and human infants (much of it conducted by his own research team), Tomasello argues further that humans' cooperative communication emerged first in the natural gestures of pointing and pantomiming. Conventional communication, first gestural and then vocal, evolved only after humans already possessed these natural gestures and their shared intentionality infrastructure along with skills of cultural learning for creating and passing along jointly understood communicative conventions. Challenging the Chomskian view that linguistic knowledge is innate, Tomasello proposes instead that the most fundamental aspects of uniquely human communication are biological adaptations for cooperative social interaction in general and that the purely linguistic dimensions of human communication are cultural conventions and constructions created by and passed along within particular cultural groups.
Bradford Books imprint
Much research in artificial intelligence is concerned with the development of autonomous agents that can interact effectively with other agents. An important aspect of such agents is the ability to reason about the behaviours of other agents, by constructing models which make predictions about various properties of interest (such as actions, goals, beliefs) of the modelled agents. A variety of modelling approaches now exist which vary widely in their methodology and underlying assumptions, catering to the needs of the different sub-communities within which they were developed and reflecting the different practical uses for which they are intended. The purpose of the present article is to provide a comprehensive survey of the salient modelling methods which can be found in the literature. The article concludes with a discussion of open problems which may form the basis for fruitful future research.
We provide an elementary geometric derivation of the Kac integral formula for the expected number of real zeros of a random polynomial with independent standard normally distributed coefficients. We show that the expected number of real zeros is simply the length of the moment curve (1, t, …, tⁿ) projected onto the surface of the unit sphere, divided by π. The probability density of the real zeros is proportional to how fast this curve is traced out. We then relax Kac’s assumptions by considering a variety of random sums, series, and distributions, and we also illustrate such ideas as integral geometry and the Fuhini-Studv metric.
PLEASE DO NOT SEND REQUESTS FOR FULL TEXT: THE EDITORS DO NOT OWN THE COPYRIGHT AND CANNOT SHARE THE TEXT.
Plan recognition, activity recognition, and intent recognition together combine and unify techniques from user modeling, machine vision, intelligent user interfaces, human/computer interaction, autonomous and multi-agent systems, natural language understanding, and machine learning. Plan, Activity, and Intent Recognition explains the crucial role of these techniques in a wide variety of applications including: personal agent assistants. Computer and network security. Opponent modeling in games and simulation systems. Coordination in robots and software agents. Web e-commerce and collaborative filtering. Dialog modeling. Video surveillance. Smart homes. In this book, follow the history of this research area and witness exciting new developments in the field made possible by improved sensors, increased computational power, and new application areas.
Online vendors are increasingly using virtual sales assistants (VSA), either in the form of an animated picture or a photograph of a real person, to help customers with their product-related information needs. Currently, what is known is that the use of a VSA in an online web shop results in positive outcomes such as trust and purchase intention. What remains unknown, however, is whether or not VSA gender-product gender congruence has a positive effect on customers' attitude towards the product-related advice, the VSA, and the online vendor and on customers' online purchase intention.
To determine the hypothesized effect of VSA gender-product gender on variables such as trust and purchase intention, a 2 (VSA gender: male vs female)–×–3 (product gender: male, female, and neutral) experiment with 183 inhabitants (between the age of 18 and 30) of the Netherlands was implemented. Results of the multivariate analysis of variance (MANOVA) reveal that congruence between VSA gender and product gender (e.g. female VSA providing advice about a feminine product) positively influences customers' belief in the credibility of the product-related advice, their trust in the VSA and the online vendor, and their purchase intention. The separate main effects of VSA gender and product gender on the aforementioned dependent variables, however, are not statistically significant. Furthermore, customers' gender did not serve any moderating function in the relationship between VSA gender-product gender congruence and the dependent variables.
How does cooperation emerge among selfish individuals? When do people share resources, punish those they consider unfair, and engage in joint enterprises? These questions fascinate philosophers, biologists, and economists alike, for the "invisible hand" that should turn selfish efforts into public benefit is not always at work.The Calculus of Selfishnesslooks at social dilemmas where cooperative motivations are subverted and self-interest becomes self-defeating. Karl Sigmund, a pioneer in evolutionary game theory, uses simple and well-known game theory models to examine the foundations of collective action and the effects of reciprocity and reputation.Focusing on some of the best-known social and economic experiments, including games such as the Prisoner's Dilemma, Trust, Ultimatum, Snowdrift, and Public Good, Sigmund explores the conditions leading to cooperative strategies. His approach is based on evolutionary game dynamics, applied to deterministic and probabilistic models of economic interactions.Exploring basic strategic interactions among individuals guided by self-interest and caught in social traps,The Calculus of Selfishnessanalyzes to what extent one key facet of human nature--selfishness--can lead to cooperation.
In this paper, our goal is to achieve the emergence of cooperation in self-interested agent societies operating on highly connected scale-free networks. The novelty of this work is that agents are able to control topological features during the network formation phase. We propose a commitment-based dynamic coalition formation approach that result in a single coalition where agents mutually cooperate. Agents play an iterated Prisoner's Dilemma game with their immediate neighbors and offer commitments to their wealthiest neighbors in order to form coalitions. A commitment proposal, that includes a high breaching penalty, incentivizes opponent agents to form coalitions within which they mutually cooperate and thereby increase their payoff. We have analytically determined, and experimentally substantiated, how the value of the penalty should be set with respect to the minimum node degree and the payoff values such that convergence into optimal coalitions is possible. Using a computational model, we determine an appropriate partner selection strategy for the agents that results in a network facilitating the convergence into a single coalition and thereby maximizing average expected payoff.
We understand a sociotechnical system as a multistakeholder cyber-physical system. We introduce governance as the administration of such a system by the stakeholders themselves. In this regard, governance is a peer-to-peer notion and contrasts with traditional management, which is a top-down hierarchical notion. Traditionally, there is no computational support for governance and it is achieved through out-of-band interactions among system administrators. Not surprisingly, traditional approaches simply do not scale up to large sociotechnical systems.
We develop an approach for governance based on a computational representation of norms in organizations. Our approach is motivated by the Ocean Observatory Initiative, a thirty-year $400 million project, which supports a variety of resources dealing with monitoring and studying the world's oceans. These resources include autonomous underwater vehicles, ocean gliders, buoys, and other instrumentation as well as more traditional computational resources. Our approach has the benefit of directly reflecting stakeholder needs and assuring stakeholders of the correctness of the resulting governance decisions while yielding adaptive resource allocation in the face of changes in both stakeholder needs and physical circumstances.
Conflicts between animals of the same species usually are of ``limited
war'' type, not causing serious injury. This is often explained as due
to group or species selection for behaviour benefiting the species
rather than individuals. Game theory and computer simulation analyses
show, however, that a ``limited war'' strategy benefits individual
animals as well as the species.