Victor Lesser

• PhD
• Professor Emeritus at University of Massachusetts Amherst

The 2021 edition of AAMAS, the International Conference on Autonomous Agents and Multiagent Systems, took place from the 3rd to 7th of May 2021 (aamas2021.soton.ac.uk). This year it was organized in the form of a virtual event and attracted over 1,000 registered participants. As every year, the conference featured an exciting programme of contribut...

The role concept provides a useful tool to design and understand complex multi-agent systems, which allows agents with a similar role to share similar behaviors. However, existing role-based methods use prior domain knowledge and predefine role structures and behaviors. In contrast, multi-agent reinforcement learning (MARL) provides flexibility and...

As artificial intelligence (AI) systems become increasingly ubiquitous, the topic of AI governance for ethical decision-making by AI has captured public imagination. Within the AI research community, this topic remains less familiar to many researchers. In this paper, we complement existing surveys, which largely focused on the psychological, socia...

As artificial intelligence (AI) systems become increasingly ubiquitous, the topic of AI governance for ethical decision-making by AI has captured public imagination. Within the AI research community, this topic remains less familiar to many researchers. In this paper, we complement existing surveys, which largely focused on the psychological, socia...

In today’s world, intelligent embedded devices and sensors are interconnected into a dynamic and global network infrastructure is referred to as the Internet-of-Things (IoT). It has been widely recognized that the performance of an IoT is highly affected by how it is organized. A large scale system may have billions of possible ways of being organi...

Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-ma...

Hierarchical crowdsourcing networks (HCNs) provide a useful mechanism for social mobilization. However, spontaneous evolution of the complex resource allocation dynamics can lead to undesirable herding behaviours in which a small group of reputable workers are overloaded while leaving other workers idle. Existing herding control mechanisms designed...

Organization is an important mechanism for improving performance in complex multiagent systems. Yet, little consideration has been given to the performance gain that organization can provide across a broad range of conditions. Intuitively, when agents are mostly idle, organization offers little benefit. In such settings, almost any organization—app...

Automating negotiations in markets where multiple buyers and sellers operate is a scientific challenge of extraordinary importance. One-to-one negotiations are classically studied as bilateral bargaining problems, while one-to-many and many-to-many negotiations are studied as auctioning problems. This paper aims at bridging together these two appro...

Technology for supporting people in their daily lives such as personal assistant agents and smart homes carry great potential for making our lives more connected, healthy, efficient and safe by executing tasks on our behalf and guiding our actions. We make two key observations:1) supportive technology is inherently social in the sense that its supp...

Networks are seen everywhere in our modern life, including the Internet, the Grid, P2P file sharing, and sensor networks. Consequently, researchers in Artificial Intelligence (and Multi-Agent Systems in particular) have been actively seeking methods for optimizing the performance of these networks. A promising yet challenging optimization direction...

In this paper we propose a novel DCOP algorithm, called DJAO, that is able to efficiently find a solution with low communication overhead; this algorithm can be used for optimal and bounded approximate solutions by appropriately setting the error bounds. Our approach builds on distributed junction trees used in Action-GDL to represent independence...

Significant research progress and understanding about the nature of coordination has been made over the years. Development of the DCOP and DEC-MDP frameworks in the past decade has been especially important. Although these advances are very important for multi-agent coordination theory, they overlook a set of coordination behaviors and phenomena th...

Coordinating multi-agent reinforcement learning provides a promising approach to scaling learning in large cooperative multi-agent systems. It allows agents to learn local decision policies based on their local observations and rewards, and, meanwhile, coordinates agents' learning processes to ensure the global learning performance. One key questio...

Coordinated multi-agent reinforcement learning (MARL) provides a promising approach to scaling learning in large cooperative multi-agent systems. Distributed constraint optimization (DCOP) techniques have been used to coordinate action selection among agents during both the learning phase and the policy execution phase (if learning is off-line) to...

Learning consistent policies in decentralized settings is often problematic. The agents have a myopic view of their neighboring states that could lead to inconsistent action choices. The fundamental question addressed in this work is how to determine and obtain the minimal overlapping context among decentralized decision makers required to make the...

Many distributed constraint optimization (DCOP) algorithms include nodes' local maximization operation that searches for the optimal variable assignment in a limited context. When the variable domain is discrete, this operation is exponential in the number of associated variables and thus computationally challenging. McAuley's recent work on effici...

The problem of finding agents’ rational strategies in bargaining with incomplete information is well known to be challenging. The literature provides a collection of results for very narrow uncertainty settings, but no generally applicable algorithm. This lack has led researchers to develop heuristic approaches in an attempt to find outcomes that,...

Trust is an important mechanism enabling agents to self-police open and dynamic multi-agent systems (ODMASs). Trusters evaluate the reputation of trustees based on their past observed performance, and use this information to guide their future interaction decisions. Existing trust models tend to concentrate trusters' interactions on a small number...

In open and dynamic multiagent systems (MASs), agents often need to rely on resources or services provided by other agents to accomplish their goals. During this process, agents are exposed to the risk of being exploited by others. These risks, if not mitigated, can cause serious breakdowns in the operation of MASs and threaten their long-term well...

It is crucial for embedded systems to adapt to the dynamics of open environments. This adaptation process becomes especially challenging in the context of multiagent systems. In this paper, we argue that multiagent meta-level control is an effective way to determine when this adaptation process should be done and how much effort should be invested...

Over the last few years, my research group has begun exploring the issues involved in learning when there are hundreds to thousands of agents. We have been using the idea of organization control as a low overhead way of coordinating the learning of such large agent collectives. In this lecture, the results of this research will be discussed and its...

An organizationally adept agent (OAA) adjusts its behavior when given annotated organizational guidelines. More importantly, it can also determine when such guidelines become ineffective and proactively adapt its behavior to better achieve organizational objectives. We present the high-level aspects of this architecture and analyze its effectivenes...

We present a novel ensemble architecture for learning problem-solving techniques from a very small number of expert solutions and demonstrate its effectiveness in a complex real-world domain. The key feature of our “Generalized Integrated Learning Architecture” (GILA) is a set of heterogeneous independent learning and reasoning (ILR) components, co...

Many message passing algorithms on graphical models include maximization operations on sums of local node function and message values from neighbors. In recent work by McAuly et al, faster maximization computation was achieved in a static environment by offline presorting of the values of local functions. However, this efficiency is only guaranteed...

To deal with the prohibitive complexity of calculating policies in Decentralized MDPs, researchers have proposed models that exploit structured agent interactions. Settings where most agent actions are independent except for few actions that affect the transitions and/or rewards of other agents can be modeled using Event-Driven Interactions with Co...

Solving a coordination problem in a decentralized environment requires a large amount of resources and thus exploiting the innate system structure and external information as much as possible is necessary for such a problem to be solved in a computationally effective manner. This work proposes new techniques for saving communication and computation...

Distributed Constraint Satisfaction (DCSP) has long been considered an important problem in multi-agent systems research. This is because many real-world problems can be represented as constraint satisfaction and these problems often present themselves in a distributed form. In this article, we present a new complete, distributed algorithm called A...

In many multi-agent applications such as distributed sensor nets, a network of agents act collaboratively under uncertainty and local interactions. Networked Distributed POMDP (ND-POMDP) provides a framework to model such cooperative multi-agent decision making. Existing work on ND-POMDPs has focused on offline techniques that require accurate mode...

In electronic commerce markets where selfish agents behave individually, agents often have to acquire multiple resources in order to accomplish a high level task with each resource acquisition requiring negotiations with multiple resource providers. Thus, it is crucial to efficiently coordinate these interrelated negotiations. This paper presents t...

A negotiation chain is formed when multiple related nego-tiations are spread over multiple agents. In order to appro-priately order and structure the negotiations occurring in the chain so as to optimize the expected utility, we present an extension to a single-agent concurrent negotiation frame-work. This work is aimed at semi-cooperative multi-ag...

Experts report on the latest artificial intelligence research concerning reasoning about reasoning itself. The capacity to think about our own thinking may lie at the heart of what it means to be both human and intelligent. Philosophers and cognitive scientists have investigated these matters for many years. Researchers in artificial intelligence h...

This article analyzes important issues regarding the design of a successful negotiation agent for ANAC and presents the design of Yushu, one of the top-scoring agents in the first Automated Negotiating Agents Competition (ANAC). Yushu uses simple heuristics to implement a conservative concession strategy based on a dynamically computed measure of c...

Distributed collaborative adaptive sensing (DCAS) of the atmosphere is a new paradigm for detecting and predicting hazardous weather using a large dense network of short-range, low-powered radars to sense the lowest few kilometers of the earths atmosphere. In DCAS, radars are controlled by a collection of Meteorological Command and Control (MC&C) a...

In this paper, we propose a genetic algorithm aided optimization scheme for designing the organization of hierarchical multiagent systems. We introduce the hierarchical genetic algorithm, in which hierarchical crossover with a repair strategy and mutation of small perturbation are used. The phenotypic hierarchical structure space is translated to t...

We consider the role of negotiation in deciding decommitment penalties. In our model, agents simultaneously negotiate over both the contract price and decommitment penalty in the contracting game and then decide whether to decommit from contracts in the decommitment game. Experimental results show that setting penalties through negotiation achieved...

In many multi-agent applications such as distributed sensor nets, a network of agents act collaboratively under uncertainty and local interactions. Networked Distributed POMDP (ND-POMDP) provides a framework to model such cooperative multi-agent decision making. Existing work on ND-POMDPs has focused on offline techniques that require accurate mode...

It is crucial for embedded systems to adapt to the dynamics of open environments. This adaptation process becomes especially challenging in the context of multiagent systems. In this paper, we argue that multiagent meta-level control is an effective way to determine when this adaptation process should be done and how much effort should be invested...

Embedded systems consisting of collaborating agents capable of interacting with their environment are becoming ubiquitous. It is crucial for these systems to be able to adapt to the dynamic and uncertain characteristics of an open environment. In this paper, we argue that multiagent meta-level control (MMLC) is an effective way to determine when th...

Due to the non-stationary environment, learning in multi-agent systems is a challenging problem. This paper first introduces a new gradient-based learning algorithm, augmenting the basic gradient ascent approach with policy prediction. We prove that this augmentation results in a stronger notion of convergence than the basic gradient ascent, that i...

We consider a multiagent resource allocation problem where individual users intend to route traffic by requesting the help of entities across a network, and a cost is incurred at each network node that depends on the amount of traffic to be routed. We propose to study contract-based network resource allocation. In our model, users and nodes in the...

Due to the non-stationary environment, learning in multi-agent systems is a challenging problem. This paper first introduces a new gradient-based learning algorithm, augmenting the basic gradient ascent approach with policy prediction. We prove that this augmentation results in a stronger notion of convergence than the basic gradient ascent, that i...

Decentralized reinforcement learning (DRL) has been applied to a number of distributed applications. However, one of the main challenges faced by DRL is its convergence. Previous work has shown that hierarchically organizational control is an effective way of coordinating DRL to improve its speed, quality, and likelihood of convergence. In this pap...

We consider the problem of allocating networked resources in dy- namicenvironment, suchascloudcomputingplatforms, whereproviders strategically price resources to maximize their utility. Resource allo- cation in these environments, where both providers and consumers are selfish agents, presents numerous challenges since the number of con- sumers and...

Embedded systems consisting of collaborating agents capable of interacting with their environment are becoming ubiquitous. It is crucial for these systems to be able to adapt to the dynamic and uncertain characteristics of an open environment. In this paper, we argue that multiagent meta-level control (MMLC) is an effective way to determine when th...

It is crucial for social systems to adapt to the dynamics of open environments. This adaptation process becomes espe-cially challenging in the context of multiagent systems. In this paper, we argue that multiagent meta-level control is an effective way to determine when this adaptation process should be done and how much effort should be invested i...

Choosing when to communicate is a fundamental problem in multi-agent systems. This problem becomes particularly challenging when communication is constrained and each agent has different partial information about the overall situation. We take a decision-theoretic approach to this problem that balances the benefits of communication against the cost...

In situations where Bayesian networks (BN) inferencing approximation is allowable, we show how to reduce the amount of sensory observations necessary and in a multi-agent context the amount of agent communication. To achieve this, we introduce Pseudo-Independence, a relaxed independence relation that quantitatively differentiates the various degree...

Resource allocation in computing clusters is traditionally centralized, which limits the cluster scale. Effective resource allocation in a network of computing clusters may enable building larger computing infrastructures. We consider this problem as a novel application for multiagent learning (MAL). We propose a MAL algorithm and apply it for opti...

Automating negotiations in markets where multiple buyers and sellers operate is a scientific challenge of extraordinary importance. One-to-one negotiations are classically studied as bilateral bargaining problems, while one-to-many and many-to-many negotiations are studied as auctioning prob- lems. This paper aims at bridging together these two app...

Variants of the decentralized MDP model focus on problems exhibiting some special structure that makes them easier to solve in practice. Our work is concerned with two main issues. First, we propose a new model, Event-Driven Interaction with Complex Rewards, that addresses problems having structured transition and reward dependence. Our model captu...

In building practical sensor networks, it is often beneficial to use only a subset of sensors to take measurements because of computational, communication, and power limitations. Thus, selecting a subset of nodes to perform measurements whose results will closely mirror the results of having all the nodes perform measurements is an important proble...

It is a challenging problem to find agents' rational strategies in bargaining with incomplete information. In this paper we perform a game theoretic anal- ysis of agents' rational strategies in finite horizon bilateral bargaining with one-sided uncertainty regarding agents' reserve prices. The negotiation setting considered in this paper has four f...

Multi-Agent Reinforcement Learning (MARL) algorithms suffer from slow convergence and even divergence, especially in large- scale systems. In this work, we develop an organization-based control framework to speed up the convergence of MARL algo- rithms in a network of agents. Our framework defines a multi-level organizational structure for automate...

Enterprises currently employ Cloud services to improve the scal-ability of their services and resource providers strategically price resources to maximize their utilities. While Nash equilibrium is the dominant concept for studying such kind of interaction, evolu-tionary game theory seems more appropriate for modeling agents' strategic interactions...

Several multiagent reinforcement learning (MARL) algorithms have been proposed to optimize agents decisions. Due to the complexity of the problem, the majority of the previously developed MARL algorithms assumed agents either had some knowledge of the underlying game (such as Nash equilibria) and/or observed other agents actions and the rewards the...

The ability to create effective multi-agent organizations is key to the development of larger, more diverse multi-agent systems. In this article we present KB-ORG: a fully automated, knowledge-based organization designer for multi-agent systems. Organization design is the process that accepts organizational goals, environmental expectations, perfor...

As the scale and scope of distributed and multi-agent systems grow, it becomes increasingly important to design and manage the participants’ interactions. The potential for bottlenecks, intractably large sets of coordination partners, and shared bounded resources can make individual and high-level goals difficult to achieve. To address these proble...

Several multiagent reinforcement learning (MARL) algorithms have been proposed to optimize agents' decisions. Only a subset of these MARL algorithms both do not require agents to know the underlying environment and can learn a stochastic policy (a policy that chooses actions according to a probability distribution). Weighted Policy Learner (WPL) is...

A database view is a dynamic virtual table composed of the result set of a query, often executed over dierent underlying databases. The view maintenance problem concerns how a view is refreshed when the data sources are updated. We study the view maintenance problem when self-interested database managers from dierent institutions are involved, each...

Multi-agent systems benet greatly from an organization de- sign that guides agents in determining when to communi- cate, how often, with whom, with what priority, and so on. However, this same organization knowledge is not utilized by general-purpose wireless network routing algorithms nor- mally used to support agent communication. We show that in...

This paper presents the design and implementation of negotiation agents that negotiate with other entities for acquiring multiple re-P sources. In our approach, agents utilize a time-dependent negotiation strategy in which the reserve price of each negotiation is-sue is dynamically determined by 1) the likehhood that negotiation " will not be succe...

Multi-Agent Reinforcement Learning (MARL) algorithms suffer from slow convergence and even divergence, especially in large-scale systems. In this work, we develop a supervision framework to speed up the convergence of MARL algorithms in a network of agents. The framework defines an organizational structure for automated supervision and a communicat...

Embedded systems consisting of collaborating agents capable of interacting with their environment are be- coming ubiquitous. It is crucial for these systems to be able to adapt to the dynamic and uncertain characteris- tics of an open environment. The question of when this adaptation process should be done and how much effort should be invested in...

Through automated negotiation we aim to improve task allocation in a distributed sensor network. In particular, we look at a type of adaptive weather-sensing radar that permits the radar to focus its scanning on certain regions of the atmosphere. Current control systems can only computationally handle the decision making for a small number of radar...

This paper designed and developed negotiation agents with the distinguishing features of 1) conducting continuous time negotiation rather than discrete time negotiation, 2) learning the response times of trading parties using Bayesian learning and, 3) deciding when to make a proposal using a multi-objective genetic algorithm (MOGA) to evolve their...

A satisficing solution to a problem is one that is “good enough” or satisfactory in a particular situation. Because of the lack of task predictability, and interdependences among tasks it is desirable to use both approximate solutions for tasks and approximate scheduling algorithms for scheduling task execution. Iterative refinement and the use of...

Sophisticated agents operating in open environments must make decisions that efficiently trade off the use of their limited resources between dynamic deliberative actions and domain actions. This is the meta-level control problem for agents operating in resource-bounded multi-agent environments. Con- trol activities involve decisions on when to inv...

The dominant existing routing strategies employed in peer- to-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in t...

To cope with large scale, agents are usually organized in a network such that an agent interacts only with its immediate neighbors in the network. Reinforcement learning techniques have been commonly used to optimize agents local policies in such a network because they require little domain knowledge and can be fully distributed. However, all of th...

A negotiation chain is formed when multiple related negotiations are spread over multiple agents. In order to appropriately order and structure the negotiations occurring in the chain so as to op- timize the expected utility, we present an extension to a single- agent concurrent negotiation framework. This work is aimed at semi-cooperative multi-ag...

Using or extending Markov decision processes (MDPs) or partially observable Markov decision pro-cesses (POMDPs) to model multiagent decision problems has become an important trend. Generally speaking, there are two types of models: centralized ones and decentralized. The centralized ones focus on finding the best joint action given any global state...

The distributed task allocation problem occurs in domains like web services, the grid, and other distributed systems. In this problem, the system consists of servers and mediators. Servers execute tasks and may dier in their capabilities, e.g. one server may take more time than the other in ex- ecuting the same task. Mediators act on behalf of user...

In complex distributed applications, such as distributed interpretation, a problem is often decomposed into a set of subproblems and each subproblem is distributed to an agent who will be responsible for solving it. The existence of interactions between subproblems means that the agents cannot simply solve the subproblems individually and then comb...

Most existing organizational design processes focus on either the qualitative or domain-independent features of candidate designs. This paper demonstrates the significance of domain-specific features through an examination of an organizationally-driven information retrieval network. The behavior of a distributed search process and the consequences...

The ability to coordinate eectively is critical for agents to accomplish their goals in a multi-agent system. A number of researchers have modeled the coordination problem for multi-agent systems using decision theory. The most gen- eral models have proven to be extremely complex to solve optimally (NEXP-complete). Some of the more restricted model...

Partially observable stochastic games (POSGs) provide a powerful framework for modeling multi-agent interactions. While elegant and expressive, this framework has been shown to be computationally intractable [1]. An exact dynamic programming algorithm for POSGs has been developed recently, but due to high computational demands, it has only been dem...

In cooperative peer-to-peer information retrieval systems, each node can be considered an intelligent agent and these agents work collectively to provide an information retrieval service. In order to effectively support multiple and concur- rent search sessions in the network, we propose two traffic engineering techniques that minimize processing a...

Distributed Constraint Satisfaction (DCSP) has long been considered an important problem in multi-agent systems research. This is because many real-world problems can be represented as constraint satisfaction and these problems often present themselves in a distributed form. In this article, we present a new complete, distributed algorithm called A...

This paper addresses the problem of negotiation in a complex organizational context. An integrative negotiation mechanism is introduced, which enables agents to dynamically select a negotiation attitude based on the degree of external directedness. Experimental work explores the question of whether it always improves the organization's social welfa...

In this paper, we present a cooperative mediation-based protocol that solves a distributed resource allocation problem while conforming to soft real-time constraints in a dynamic environment. Two central principles are used in this protocol that allow it to operate in constantly changing conditions. First, we frame the allocation problem as an opti...

Open environments are characterized by their uncertainty and non-determinism. Agents need to adapt their task processing to available resources, deadlines, the goal criteria specified by the clients as well their current problem solving context in order to survive in these environments. If there were no resource constraints, then an optimal Markov...

This paper discusses a solution to the problems posed by sensor resource allocation in an adaptive, distributed radar array. We have formulated a variant of the classic resource allocation problem, called the setting-based resource allocation problem, which reflects the challenges posed in domains in which sensors have multiple settings, each of wh...

The organization and collaborative protocols of agent societies are becoming increasingly important with the growing size of agent networks. Particularly, in a multi-agent-based content-sharing system, a flat, peer-to-peer (P2P) agent organization is not the most efficient organization for locating relevant agents for queries. This paper not only d...

Real-time control has become increasingly important as technologies are moved from the lab into real world situations. The complexity associated with these systems increases as control and autonomy are distributed, due to such issues as temporal and ordering constraints, shared resources, and the lack of a complete and consistent world view. In thi...

We describe a new approach to coordinating the scheduling and execution of a complex hierarchical task structure distributed among a set of agents. Our ap- proach decomposes allocation, scheduling, and moni- toring into manageable local pieces that are coordinated with one another. Each agent is assigned mediator re- sponsibilities for multiple tas...

Agents can benefit by cooperating to solve a common problem [2, 11]. For example, several robots may cooperate to move a heavy object, sweep a specific area in short time, etc. However, as the number of agents increases, having all agents involved in a detailed coordination/negotiation process will limit the scalability of the system. It is better...

In cooperative peer-to-peer information retrieval systems, each node can be considered as an intelligent agent and these agents work collectively to provide an information retrieval service. In order to support eectively, multiple, concurrent search sessions in the network, we propose a novel agent control mechanism whose elements include resource...

We present several discrete-time Markov queuing models to compare the performance of batch versus streaming processing of sensor data in a weather detection and monitoring system architecture. The first model assumes independent arrivals. The remaining models assume correlated arrivals and demonstrate how different scan strategies across multiple e...

Several approaches tackle the problem of reducing traffic jams. A class of these approaches deals with coordination of traffic lights in order to allow vehicles traveling in a given direction to pass an arterial without stopping at junctions. In short, classical approaches, which are mostly based on offline and centralized determination of the prio...

Mediation is the process of decomposing a task into subtasks, finding agents suitable for these subtasks and negotiating with agents to obtain commitments to execute these subtasks. This process involves several decisions to be made by a mediator including which tasks to mediate, when to interrupt the current task mediation to pursue a better task,...

The organizational design of a distributed system defines how entities act and interact to achieve local and global objectives. We describe how a system employing different types of organizational techniques has been used to address the challenges posed by a distributed sensor network environment. The high-level, multi-agent architecture of this re...