Nicole Orzan’s research while affiliated with University of Groningen and other places

Communication is a widely used mechanism to promote cooperation in multi-agent systems. In the field of emergent communication, agents are typically trained in specific environments: cooperative, competitive or mixed-motive. Motivated by the idea that real-world settings are characterized by incomplete information and that humans face daily interactions under a wide spectrum of incentives, we aim to explore the role of emergent communication when simultaneously exploited across all these contexts. In this work, we pursue this line of research by focusing on social dilemmas. To do this, we developed an extended version of the Public Goods Game, which allows us to train independent reinforcement learning agents simultaneously in different scenarios where incentives are (mis)aligned to various extents. Additionally, agents experience uncertainty in terms of the alignment of their incentives with those of others. We equip agents with the ability to learn a communication policy and study the impact of emergent communication in the face of uncertainty among agents. Our findings show that in settings where all agents have the same level of uncertainty, communication can enhance the cooperation of the whole group. However, in cases of asymmetric uncertainty, the agents that do not face uncertainty learn to use communication to deceive and exploit their uncertain peers.

Addressing the question of how to achieve optimal decision-making under risk and uncertainty is crucial for enhancing the capabilities of artificial agents that collaborate with or support humans. In this work, we address this question in the context of Public Goods Games. We study learning in a novel multi-objective version of the Public Goods Game where agents have different risk preferences, by means of multi-objective reinforcement learning. We introduce a parametric non-linear utility function to model risk preferences at the level of individual agents, over the collective and individual reward components of the game. We study the interplay between such preference modelling and environmental uncertainty on the incentive alignment level in the game. We demonstrate how different combinations of individual preferences and environmental uncertainty sustain the emergence of cooperative patterns in non-cooperative environments (i.e., where competitive strategies are dominant), while others sustain competitive patterns in cooperative environments (i.e., where cooperative strategies are dominant).

Addressing the question of how to achieve optimal decision-making under risk and uncertainty is crucial for enhancing the capabilities of artificial agents that collaborate with or support humans. In this work, we address this question in the context of Public Goods Games. We study learning in a novel multi-objective version of the Public Goods Game where agents have different risk preferences, by means of multi-objective reinforcement learning. We introduce a parametric non-linear utility function to model risk preferences at the level of individual agents, over the collective and individual reward components of the game. We study the interplay between such preference modelling and environmental uncertainty on the incentive alignment level in the game. We demonstrate how different combinations of individual preferences and environmental uncertainties sustain the emergence of cooperative patterns in non-cooperative environments (i.e., where competitive strategies are dominant), while others sustain competitive patterns in cooperative environments (i.e., where cooperative strategies are dominant).

Airborne wind energy is a lightweight technology that allows power extraction from the wind using airborne devices such as kites and gliders, where the airfoil orientation can be dynamically controlled in order to maximize performance. The dynamical complexity of turbulent aerodynamics makes this optimization problem unapproachable by conventional methods such as classical control theory, which rely on accurate and tractable analytical models of the dynamical system at hand. Here we propose to attack this problem through reinforcement learning, a technique that-by repeated trial-and-error interactions with the environment-learns to associate observations with profitable actions without requiring prior knowledge of the system. We show that in a simulated environment reinforcement learning finds an efficient way to control a kite so that it can tow a vehicle for long distances. The algorithm we use is based on a small set of intuitive observations and its physically transparent interpretation allows to describe the approximately optimal strategy as a simple list of manoeuvring instructions.

Airborne Wind Energy is a lightweight technology that allows power extraction from the wind using airborne devices such as kites and gliders, where the airfoil orientation can be dynamically controlled in order to maximize performance. The dynamical complexity of turbulent aerodynamics makes this optimization problem unapproachable by conventional methods such as classical control theory, which rely on accurate and tractable analytical models of the dynamical system at hand. Here we propose to attack this problem through Reinforcement Learning, a technique that -- by repeated trial-and-error interactions with the environment -- learns to associate observations with profitable actions without requiring prior knowledge of the system. We show that in a simulated environment Reinforcement Learning finds an efficient way to control a kite so that it can tow a vehicle for long distances. The algorithm we use is based on a small set of intuitive observations and its physically transparent interpretation allows to describe the approximately optimal strategy as a simple list of manoeuvring instructions.