Mehdi Dastani

Utrecht University | UU · Department of Information and Computing Sciences

Training a task-oriented dialogue policy using deep reinforcement learning is promising but requires extensive environment exploration. The amount of wasted invalid exploration makes policy learning inefficient. In this paper, we define and argue that dead-end states are important reasons for invalid exploration. When a conversation enters a dead-e...

Improving sample efficiency of Reinforcement Learning (RL) in sparse-reward environments poses a significant challenge. In scenarios where the reward structure is complex, accurate action evaluation often relies heavily on precise information about past achieved subtasks and their order. Previous approaches have often failed or proved inefficient i...

Reward machines allow the definition of rewards for temporally extended tasks and behaviors. Specifying “informative” reward machines can be challenging. One way to address this is to generate reward machines from a high-level abstract description of the learning environment, using techniques such as AI planning. However, previous planning-based ap...

Synthetic populations are representations of actual individuals living in a specific area. They play an increasingly important role in studying and modeling individuals and are often used to build agent-based social simulations. Traditional approaches for synthesizing populations use a detailed sample of the population (which may not be available)...

Reward machines allow the definition of rewards for temporally extended tasks and behaviors. Specifying "informative" reward machines can be challenging. One way to address this is to generate reward machines from a high-level abstract description of the learning environment, using techniques such as AI planning. However, previous planning-based ap...

The development of autonomous agents operating in dynamic and stochastic environments requires theories and models of how beliefs and intentions are revised while taking their interplay into account. In this paper, we initiate the study of belief and intention revision in stochastic environments, where an agent's beliefs and intentions are specifie...

This research focuses on establishing trust in multiagent systems where human and AI agents collaborate. We propose a computational notion of actual trust, emphasising the modelling of an agent’s capacity to deliver tasks. Unlike reputation-based trust or performing a statistical analysis on past behaviour, our approach considers the specific setti...

We present Pure-Past Action Masking (PPAM), a lightweight approach to action masking for safe reinforcement learning. In PPAM, actions are disallowed (“masked”) according to specifications expressed in Pure-Past Linear Temporal Logic (PPLTL). PPAM can enforce non-Markovian constraints, i.e., constraints based on the history of the system, rather th...

Reinforcement learning (RL) has emerged as a key technique for designing dialogue policies. However, action space inflation in dialogue tasks has led to a heavy decision burden and incoherence problems for dialogue policies. In this paper, we propose a novel decomposed deep Q-network (D2Q) that exploits the natural structure of dialogue actions to...

In this paper we discuss how causal models can be used for modeling multi-agent interaction in complex organizational settings, where agents’ decisions may depend on other agents’ decisions as well as the environment. We demonstrate how to reason about the dynamics of such models using concurrent game structures where agents can change the organisa...

Synthetic populations are microscopic representations of actual citizens living in a specific area. They play an increasingly important role in studying and modeling citizens and are often used to build agent-based social simulations.Traditional approaches for synthesizing populations use a detailed sample of the population (which may not be availa...

Synthetic populations are representations of actual individuals living in a specific area. They play an increasingly important role in studying and modeling individuals and are often used to build agent-based social simulations. Traditional approaches for synthesizing populations use a detailed sample of the population (which may not be available)...

Synthetic populations are representations of actual individuals living in a specificarea. They play an increasingly important role in studying and modeling individuals and are often used to build agent-based social simulations. Traditional approaches for synthesizing populations use a detailed sample of the population (which may not be available) o...

There have been a number of attempts to develop a formal definition of causality that accords with our intuitions about what constitutes a cause. Perhaps the best known is the “modified” definition of actual causality, HPm, due to Halpern. In this paper, we argue that HPm gives counterintuitive results for some simple causal models. We propose Dyna...

Reward machines have recently been proposed as a means of encoding team tasks in cooperative multi-agent reinforcement learning. The resulting multi-agent reward machine is then decomposed into individual reward machines, one for each member of the team, allowing agents to learn in a decentralised manner while still achieving the team task. However...

The need for tools and techniques to formally analyze and trace the responsibility for unsafe outcomes to decision-making actors is urgent. Existing formal approaches assume that the unsafe outcomes for which actors can be held responsible are actually realized. This paper considers a broader notion of responsibility where unsafe outcomes are not n...

Agent-based modeling is increasingly being used in computational epidemiology to characterize important behavioral dimensions, such as the heterogeneity of the individual responses to interventions, when studying the spread of a disease. Existing agent-based simulation frameworks and platforms currently fall in one of two categories: those that can...

To build a theory of intention revision for agents operating in stochastic environments, we need a logic in which we can explicitly reason about their decision-making policies and those policies' uncertain outcomes. Towards this end, we propose PLBP, a novel probabilistic temporal logic for Markov Decision Processes that allows us to reason about p...

In multi-agent systems, norm enforcement is a mechanism for steering the behavior of individual agents in order to achieve desired system-level objectives. Due to the dynamics of multi-agent systems, however, it is hard to design norms that guarantee the achievement of the objectives in every operating context. Also, these objectives may change ove...

Training a dialogue policy using deep reinforcement learning requires a lot of exploration of the environment. The amount of wasted invalid exploration makes their learning inefficient. In this paper, we find and define an important reason for the invalid exploration: dead-ends. When a conversation enters a dead-end state, regardless of the actions...

In multi-agent systems, norm enforcement is a mechanism for steering the behavior of individual agents in order to achieve desired system-level objectives. Due to the dynamics of multi-agent systems, however, it is hard to design norms that guarantee the achievement of the objectives in every operating context. Also, these objectives may change ove...

Ensuring the trustworthiness of autonomous systems and artificial intelligence is an important interdisciplinary endeavour. In this position paper, we argue that this endeavour will benefit from technical advancements in capturing various forms of responsibility, and we present a comprehensive research agenda to achieve this. In particular, we argu...

Norms have been widely proposed as a way of coordinating and controlling the activities of agents in a multi-agent system (MAS). A norm specifies the behaviour an agent should follow in order to achieve the objective of the MAS. However, designing norms to achieve a particular system objective can be difficult, particularly when there is no direct...

Background Social distancing has been implemented by many countries to curb the COVID-19 pandemic. Understanding public support for this policy calls for effective and efficient methods of monitoring public opinion on social distancing. Twitter analysis has been suggested as a cheaper and faster-responding alternative to traditional survey methods....

To support the trustworthiness of AI systems, it is essential to have precise methods to determine what or who is to account for the behaviour, or the outcome, of AI systems. The assignment of responsibility to an AI system is closely related to the identification of individuals or elements that have caused the outcome of the AI system. In this wor...

Norms have been widely proposed as a way of coordinating and controlling the activities of agents in a multi-agent system (MAS). A norm specifies the behaviour an agent should follow in order to achieve the objective of the MAS. However, designing norms to achieve a particular system objective can be difficult , particularly when there is no direct...

Communication is an effective mechanism for coordinating the behavior of multiple agents. In the field of multi-agent reinforcement learning, agents can improve the overall learning performance and achieve their objectives by communication. Moreover, agents can communicate various types of messages, either to all agents or to specific agent groups,...

Now that within the humanities more and more data sources have been created, a new opportunity is within reach: the searching of patterns spanning across data sources from archives, museums, and other cultural heritage institutes. These institutes adopt various digitization strategies based on differences in selection procedures. This results in he...

Modelling social phenomena in large-scale agent-based simulations has long been a challenge due to the computational cost of incorporating agents whose behaviors are determined by reasoning about their internal attitudes and external factors. However, COVID-19 has brought the urgency of doing this to the fore, as, in the absence of viable pharmaceu...

Agent-based simulation is increasingly being used to model social phenomena involving large numbers of agents. However, existing agent-based simulation platforms severely limit the kinds of the social phenomena that can modeled, as they do not support large scale simulations involving agents with complex behaviors. In this paper, we present a scala...

Norms have been widely proposed as a way of coordinating and controlling the activities of agents in a multi-agent system (MAS). A norm specifies the behaviour an agent should follow in order to achieve the objective of the MAS. However, designing norms to achieve a particular system objective can be difficult, particularly when there is no direct...

Safe and reliable deployment of collaborative AI-human multiagent systems requires formal semantics and verifiable tools to reason about forward-lookingresponsibilities of agents (e.g., who can/should ensure some properties in prospect) aswell as their backward-looking responsibilities (e.g., who to blame, praise, or see accountable for an already...

To develop and effectively deploy Trustworthy Autonomous Systems (TAS), we face various social, technological, legal, and ethical challenges in which different notions of responsibility can play a key role. In this work, we elaborate on these challenges, discuss research gaps, and show how the multidimensional notion of responsibility can play a ke...

In this work, we present a dynamic Task Coordination framework (TasCore) for multiagent systems. Here task coordination refers to a twofold problem where an exogenously imposed state of affairs should be satisfied by a multiagent system (MAS). To address this problem the involved agents or agent groups need to be assigned tasks to fulfill (task all...

To achieve system-level properties of a multiagent system, the behavior of individual agents should be controlled and coordinated. One way to control agents without limiting their autonomy is to enforce norms by means of sanctions. The dynamicity and unpredictability of the agents’ interactions in uncertain environments, however, make it hard for d...

In this paper we address the interplay among intention, time, and belief in dynamic environments. The first contribution is a logic for reasoning about intention, time and belief, in which assumptions of intentions are represented by preconditions of intended actions. Intentions and beliefs are coherent as long as these assumptions are not violated...

In this paper we address the interplay among intention, time, and belief in dynamic environments. The first contribution is a logic for reasoning about intention, time and belief, in which assumptions of intentions are represented by preconditions of intended actions. Intentions and beliefs are coherent as long as these assumptions are not violated...

We present an overview of the Task Coordination (TC) problem in multiagent systems and discuss the specific elements that are required to develop a solution to this problem. Task coordination refers to a twofold problem where an exogenously imposed state of affairs should be satisfied by a multiagent system (MAS): (1) the agents need to be assigned...

In this work, we present a dynamic Task Coordination framework ( Open image in new window ) for multiagent systems. Here task coordination refers to a twofold problem where an exogenously imposed state of affairs should be satisfied by a multiagent system. To address this problem the involved agents or agent groups need to be assigned tasks to fulf...

Sociotechnical systems (STSs) are defined by the interaction between technical systems, like software and machines, and social entities, like humans and organizations. The entities within an STS are autonomous, thus weakly controllable, and the environment where the STS operates is highly dynamic. As a result, the design artifacts that represent th...

Multiagent Systems (MAS) research reached a maturity to be confidently applied to real-life complex problems. Successful application of MAS methods for behavior modeling, strategic reasoning, and decentralized governance, encouraged us to focus on applicability of MAS techniques in a class of industrial systems and to elaborate on potentials and ch...

Multiagent Systems (MAS) research reached a maturity to be confidently applied to real-life complex problems. Successful application of MAS methods for behavior modeling, strategic reasoning, and decentralized governance, encouraged us to focus on applicability of MAS techniques in a class of industrial systems and to elaborate on potentials and ch...

To fulfill the overall objectives of a multiagent system, the behavior of individual agents should be controlled and coordinated. Runtime norm enforcement is one way to do so without over-constraining the agents' autonomy. Due to the dynamicity and uncertainty of the environment, however, it is hard to specify norms that, when enforced, will fulfil...

p>A central issue in the specification and verification of autonomous agents and multiagent systems is the ascription of responsibility to individual agents and groups of agents When designing a (multi)agent system, we must specify which agents or groups of agents are responsible for bringing about a particular state of affairs Similarly, when veri...

Autonomous vehicles will most likely participate in traffic in the near future. The advent of autonomous vehicles allows us to explore innovative ideas for traffic control such as norm-based traffic control. A norm is a violable rule that describes correct behavior. Norm-based traffic controllers monitor traffic and effectuate sanctions in case veh...

The concept of a norm is found widely across fields including artificial intelligence, biology, computer security, cultural studies, economics, law, organizational behaviour and psychology. The concept is studied with different terminology and perspectives, including individual, social, legal and philosophical. If a norm is an expected behaviour in...

To guarantee the overall intended objectives of a multiagent systems, the behavior of individual agents should be controlled and coordinated. Such coordination can be achieved, without limiting the agents’ autonomy, via runtime norm enforcement. However, due to the dynamicity and uncertainty of the environment, the enforced norms can be ineffective...

Norms with sanctions have been widely employed as a mechanism for controlling and coordinating the behavior of agents without limiting their autonomy. The norms enforced in a multi-agent system can be revised in order to increase the likelihood that desirable system properties are fulfilled or that system performance is sufficiently high. In this p...

This paper builds on an existing notion of group responsibility and proposes two ways to define the degree of group responsibility: structural and functional degrees of responsibility. These notions measure the potential responsibilities of (agent) groups for avoiding a state of affairs. According to these notions, a degree of responsibility for a...

This paper builds on an existing notion of group responsibility and proposes two ways to define the degree of group responsibility: structural and functional degrees of responsibility. These notions measure the potential responsibilities of (agent) groups for avoiding a state of affairs. According to these notions, a degree of responsibility for a...

In this paper, we study normative multi-agent systems from a supervisory control theory perspective. Concretely, we show how to model three well-known types of norm enforcement mechanisms by adopting well-studied supervisory control theory techniques for discrete event systems. Doing so provides a semantics for normative multi-agent systems rooted...

This paper provides a formalization of the other-condemning anger emotion which is a social type of anger triggered by the behaviour of other agents. Other-condemning anger responds to frustration of committed goals by others, and motivates goal-congruent behavior towards the blameworthy agents. Understanding this type of anger is crucial for model...

In an organisational setting such as an online marketplace, an entity called the ‘organisation’ or ‘institution’ defines interaction protocols, monitors agent interaction, and intervenes to enforce the interaction protocols. The organisation might be a software system that thus regulates the marketplace, for example. In this article we abstract ove...

This article studies and analyzes three other-condemning moral emotions: anger, contempt, and disgust. We utilize existing psychological theories—appraisal theories of emotion and the CAD triad hypothesis—and incorporate them into a unified framework. A semiformal specification of the elicitation conditions and prototypical coping strategies for th...

This book constitutes the refereed proceedings of the 8th International Conference on Intelligent technologies for Interactive Entertainment, INTETAIN 2016, held in Utrecht, The Netherlands, in June 2016. The 19 full papers, 5 short and 6 workshop papers were selected from 49 submissions and present novel interactive techniques and their applicatio...

In this short note we address the issue of expressing norms (such as obligations and prohibitions) in temporal logic. In particular, we address the argument from [Governatori 2015] that norms cannot be expressed in Linear Time Temporal Logic (LTL).

In this paper, we introduce a specific form of graded group responsibility called “distant responsibility” and provides a formal analysis for this concept in multi-agent settings. This concept of responsibility is formalized in concurrent structures based on the power of agent groups in such structures. A group of agents is called responsible for a...

The increasing presence of autonomous (software) systems in open environments in general, and the complex interactions taking place among them in particular, require flexible control and coordination mechanisms to guarantee desirable overall system level properties without limiting the autonomy of the involved systems. In artificial intelligence, a...

Decentralized monitors can be subject to robustness and security risks. Robustness risks include attacks on the monitor’s infrastructure in order to disable parts of its functionality. Security risks include attacks that try to extract information from the monitor and thereby possibly leak sensitive information. Formal methods to analyze the design...

This paper builds on an existing notion of group responsibility and proposes two ways to define the degree of group responsibility: structural and functional degrees of responsibility. These notions measure potential responsibilities of agent groups for avoiding a state of affairs. According to these notions, a degree of responsibility for a state...

We consider the problem of whether a coalition of agents has a knowledge-based strategy to ensure some outcome under a resource bound. We extend previous work on verification of multi-agent systems where actions of agents produce and consume resources, by adding epistemic pre- and postconditions to actions. This allows us to model scenarios where a...

In this paper we concern ourselves with normative multi-agent systems, which are multi-agent systems governed by a set of norms. In these systems, the internals and architecture of the participating agents may be unknown to us, which disables us to make any strong assumption on the possible behaviour that these agents may exhibit. Thus, we cannot s...

Various agent-based programming languages and frameworks have been proposed to support the development of autonomous agents and multi-agent systems. They have provided a valuable contribution to the identification and operationalisation of agent concepts and abstractions by proposing specific programming constructs. Unfortunately, these contributio...

p>This paper1 builds on an existing notion of group responsibility and proposes two ways to define the degree of group responsibility: structural and functional degrees of responsibility. These notions measure potential responsibilities of agent groups for avoiding a state of affairs. According to these notions, a degree of responsibility for a sta...

The AGM theory of belief revision is based on propositional belief sets. In this paper we develop a logic for revision of temporal belief bases, containing expressions about temporal propositions (tomorrow it will rain), possibility (it may rain tomorrow), actions (the robot enters the room) and pre- and post-conditions of these actions. We prove t...

We propose a semi-formal specification of the elicitation conditions and prototypical coping strategies for three of the moral emotions: anger, contempt and disgust. We utilize existing psychological theories -- appraisal theories of emotion and the CAD triad hypothesis -- and incorporate them into a unified framework. Key features of the approach,...

Norms have been widely proposed as a means of coordinating and controlling the behaviour of agents in a multi-agent system. A key challenge in normative MAS is norm enforcement: how and when to restrict the agents' behaviour in order to obtain a desirable outcome? Even if a norm can be enforced theoretically, it may not be enforceable in a grounded...

This book constitutes the proceedings of the 13th International Workshop on Computational Logic in Multi-Agent Systems, CLIMA XIII, held in Montpellier, France, in August 2012. The 11 regular papers were carefully reviewed and selected from 27 submissions and presented with three invited papers. The purpose of the CLIMA workshops is to provide a fo...

Robot knowledge of the world is created from discrete and asynchronous events received from its perception components. Proper representation and maintenance of robot knowledge is crucial to enable the use of robot knowledge for planning, user-interaction, etc. This paper identifies some of the main issues related to the representation, maintenance...

The use of normative systems is widely accepted as an effective approach to control and regulate the behaviour of agents in multi-agent systems. When norms are added to a normative system, the behaviour of such a system changes. As of yet, there is no clear formal methodology to model the dynamics of a normative system under addition of various typ...

We propose a programming framework for the implementation of norm-aware multi-agent systems. The framework integrates the N-2APL norm-aware agent programming language with the 2OPL organisation programming language. Integration of N-2APL and 2OPL is achieved using a tuple space which represents both the (brute) state of the multi-agent environment...

Due to external requirements we cannot always construct a centralized organization, but have to construct one that is distributed. A distributed organization is a network of organizations which can locally observe and control the environment. In this paper we analyze how norms can be enforced through the joint effort of the individual local organiz...

In this paper, we consider the runtime monitoring of norms with imperfect monitors. A monitor is imperfect for a norm if it has insufficient observational capabilities to determine if a given execution trace of a multi-agent system complies with or violates the norm. One approach to the problem of imperfect monitors is to enhance the observational...

Various agent-based programming languages and frameworks have been proposed to support the development of multi-agent systems. They have contributed to the identification and operationalisation of multi-agent system concepts, features and abstractions by proposing specific programming constructs. Unfortunately, these contributions have not yet been...

This chapter surveys the multi-agent programming research field by presenting and discussing some typical multi-agent programming languages and frameworks. It provides an overview of the concepts and abstractions that are used to describe multi-agent systems. It is argued that the existing programming languages and frameworks are designed to suppor...

We present a weakest precondition calculus for belief updates in a high-level agent specification language. The weakest precondition calculus supports a deductive method which allows us to reason about important safety and leads-to properties of the semantics of agent specifications.

We consider the problem of updating a multi-agent system with a set of conditional norms. A norm comes into effect when its condition becomes true, and imposes either an obligation or a prohibition on an agent which remains in force until a state satisfying a deadline condition is reached. If the norm is violated, a sanction is imposed on the agent...

The BDI-oriented multi-agent programming language 2APL allows the implementation of an agent's beliefs in terms of logical facts and rules. An agent's beliefs represent information about its surrounding environment including other agents. Repeated querying of the beliefs by the 2APL interpreter causes unnecessary overhead resulting in poor runtime...

In many logic-based BDI agent programming languages, plan selection involves inferencing over some underlying knowledge representation. While context-sensitive plan selection facilitates the development of flexible, declarative programs, the overhead of evaluating repeated queries to the agent’s beliefs and goals can result in poor run time perform...