Multi-Agent Systems and Autonomous Agents - Science topic

Which architectural frameworks show the greatest potential for developing generalizable reinforcement learning agents that can transfer knowledge across domains, and what theoretical challenges must be addressed?

Dear Colleagues,

We are glad to announce that we are organizing the 2nd workshop on Cooperative Multi-Agent Systems Decision-Making and Learning at AAAI 2025. If you are interested, we sincerely invite you to share your project with us and the broader community via submitting a paper contribution or an extended abstract to our workshop.

Workshop Website: https://www.is3rlab.org/aaai25-cmasdl-workshop.github.io/

AAAI 2025 W11 Website: https://aaai.org/conference/aaai/aaai-25/workshop-list/

Paper Submission Link: https://easychair.org/conferences/?conf=aaai2025cmasdlworksh

Paper Submission Deadline: December 6, 2024

Natural agents, like humans, often make decisions based on a blend of biological, social, and cognitive motivations, as elucidated by theories like Maslow’s Hierarchy of Needs and Alderfer’s Existence-Relatedness-Growth (ERG) theory. On the other hand, the AI agent can be regarded as a self-organizing system that also presents various needs and motivations in its evolution through decision-making and learning to adapt to different scenarios and satisfy their needs. Combined with AI agent capacity to aid decision-making, it opens up new horizons in human-multi-agent collaboration. This potential is crucially essential in the context of interactions between human agents and intelligent agents, when considering to establish stable and reliable relationships in their cooperation, particularly in adversarial and rescue mission environments.

This workshop focuses on the role of decision-making and learning in human-multi-agent cooperation, viewed through the lens of cognitive modeling. AI technologies, particularly in human-robot interaction, are increasingly focused on cognitive modeling, encompassing everything from visual processing to symbolic reasoning and action recognition. These capabilities support human-agent cooperation in complex tasks.

We solicit contributions from topics including but not limited to:

§ Human-multi-agent cognitive modeling

§ Human-multi-agent trust networks

§ Trustworthy AI agents in Human-robot interaction

§ Trust based Human-MAS decision-making and learning

§ Consensus in Human-MAS collaboration

§ Intrinsically motivated AI agent modeling in Human-MAS

§ Innate-values-driven reinforcement learning

§ Multi-Objective MAS decision-making and learning

§ Adaptive learning with social rewards

§ Cognitive models in swarm intelligence and robotics

§ Game-theoretic approaches in MAS decision-making

§ Cognitive model application in intelligent social systems

We welcome contributions of both short (2-4 pages) and long papers (6-8 pages) related to our stated vision in the AAAI 2025 proceedings format. Position papers and surveys are also welcome. All contributions will be peer reviewed (single-blind).

Paper Submission Link: https://easychair.org/conferences/?conf=aaai2025cmasdlworksh

All accepted papers will be given the opportunity to be presented in the workshop. The accepted papers will be posted on the workshop’s website in advance so that interested participants will have a chance to view the papers first before coming to the workshop. These non-archival papers and their corresponding posters will remain available on this website after the workshop. The authors will retain copyright of their papers.

Virtual and Remote Attendance will be available to everyone who has registered for the workshops. The workshop will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center on Mar. 3, 2025. Authors are NOT allowed to present virtually.

All attendees have to register for the workshop. Please check more details about AAAI 2025 workshop 11 registration: https://aaai.org/conference/aaai/aaai-25/registration/.

Please contact is3rlab@gmail.com with any questions.

Qin Yang, Bradley University

Giovanni Beltrame, Polytechnique Montreal

Alberto Quattrini Li, Dartmouth College

Christopher Amato, Northeastern University

Hi there!

I'm starting to model an urban simulation and I'm having a bit of a dilemma regarding what language to use. Have some experience in Netlogo and I'm starting to make a shift towards Repast, GAMA or MESA (geo-mesa), because it is recommended for large scale simulations.

Have been reading papers about which tool to use, but I need someone working on simulations to help me out.

Still, I have questions because:

1- The user base of MESA is scarce and i feel that dealing with issues will be dificult

2- So far i have only seen and read about limited research done in MESA. Specially, dealing with road network integrations. (move an agent along a network)

3- It seems that Netlogo is good for prototype, will not handle big data projects

Thanks in advance, and any pointers to courses or moocs would be great.

In my opinion, the development of the necessary infrastructure and security stabilities is of key importance for the development of autonomous cars technology, so that the development of autonomous cars technology and the increase in the number of autonomous cars does not increase statistics on the number of road accidents.

In addition, the development of autonomous cars technology can be paralleled to the development of electromobility. For the development of electromobility and the number of used electric cars, it is also necessary to build the necessary infrastructure installed on roads, urban streets and interurban arteries of communication charging points for batteries into electricity.

In some countries there are active policies for the development of electromobility, under which the state from public finance funds pays extra to purchase an electric car and invests in projects to develop the necessary infrastructure for charging points in electricity. Other power plants are also being built as part of the development of renewable energy sources, because the development of electromobility is causing a significant increase in electricity demand. Unfortunately, this pro-ecological, active policy for the development of electromobility is carried out only in some countries.

Do you agree with my opinion on this matter?

In view of the above, I am asking you the following question:

Please reply

I invite you to the discussion

Thank you very much

Best wishes

Hello everyone, I'm struggling with a problem related to reinforcement learning. This is the scenario:

There is a four-way intersection in which some vehicles enter the intersection, and we aim to coordinate these vehicles according to their decisions to turn left, right, etc. Therefore, when we finish coordinating these vehicles, after a while, these vehicles will have left the intersection. Not to mention that we are dealing with a dynamic scenario in which vehicles enter the intersection randomly and we have to decide for them as fast as possible(decide for their priorities, velocities, etc.).

In order to diminish the costs of handling the problem in a centralized way, we assume that the vehicles themselves solve their coordinating problem in a decentralized way by acquiring the locational data of other vehicles.

I want to handle this problem with deep reinforcement learning. Actually, in my opinion, that would be nice to consider a multi-agent reinforcement learning method in which the agents act in a decentralized way and are trained centrally (CTDE). My assumption is that the agents in this scenario are the vehicles that are entering the intersection. Well, this is straightforward in the first place, but the problem is that I do not know how to model the problem. Because the system is multi-agent, I think I have to consider a separate DQN (DNN) for each vehicle, but these vehicles exist in the system for a short period of time, and after that I schedule them, they are gone. This is to say that I train the respective DQN of the vehicle when it is present at the intersection, and then I delete the DQN of that vehicle when it leaves. This is perplexing!

What do you think I can do about this problem?

I search for applications about recommender systems.

The Idea in general is to use algorithms which gives according to historical data of the classifications of soil. then make a mapping spell between the existing data of soil, weather and crops and an input entered, user to know the type of crop adapted to his soil in addition to that the amount of appropriate fertilizer.

If you have research/review articles comparative studies or support it will be very helpful .

I have come across the terms multi-agent systems, multi-agent models, and agent-based models in the literature. It seems some authors tend to use these terms interchangeably while some prefer one over the other. But do they mean the same thing or do they refer to different things? After thinking about this, I drafted a way to make this differentiation as follows. I would appreciate it if you could let me know what you think about it, whether you can reasonably agree with me or whether you have a completely different opinion.

multi-agent system - a complex, real-life system where many independent and inter-dependent agents simultaneously interact to reach a system-wide outcome within a set of pre-defined constraints.

multi-agent (or agent-based) model - a computer-based (often simplified) simulation model of a complex, real-life system where many independent and inter-dependent agents simultaneously interact to reach a system-wide outcome within a set of pre-defined constraints.

The question surrounds about applications where this two concept exists. if you have comparative studies, review articles or support it will be very helpful.

the question surrounds bout applications where this two concept exists. if you have comparative studies, review articles or support it will be very helpful.

I some papers, the mean square error is considered. In some other, the mse is normalized by dividing the error by the (total sampling instants x the total length of the reference trajectory).

Which solution does represent the factual error?

I am interested in creating a multi-layer mechanical network. Therefore I would like to find a software where you can visualise nodes and links moving around in 2D and 3D space.

There are a lot of agents in my model while they have interaction just through the environment. I’m using a Q-Learning algorithm to solve this model so that all the agents share a static Q-table in java (because here the agents are homogenous). Here, the environment is dynamic and the time step of environment changes is a lot smaller than the time step of agent state changes. So, the state of an agent won’t be changed until the environment has been updated through plenty of steps. Furthermore, the agents and environment have interaction with each other and can affect each other. In one hand, I need to know the new state of the agents at the next time step (i.e., to find the MaxQ(s(t+1),a) in Q-Learing algorithm). On the other hand, I can’t postpone updating the Q-table until the next step because it is shared between the agents. So, do you have any suggestion to handle my problem?

In most of AI research the goal is to to achieve higher than human performance on a single objective,

I believe that in many cases we oversimplify the complexity of human objectives, and therefore I think we should maybe step off improving human performance.

but rather focus on understanding human objectives first by observing humans in the form of imitation learning while still exploring.

In the attachment I added description of the approach I believe could enforce more human like behavior.

However I would like advice on how I could formulate a simple imitation learning environment to show a prove of concept.

One idea of mine was to build a gridworld simulating a traffic light scenario, while the agent is only rewarded for crossing the street, we still want it to respect the traffic rules.

Kind regards

Jasper Busschers master student AI

I'm new in reinforcement learning and I don't know the difference between value iteration and policy iteration methods!

I am also very confused about categories of methods in reinforcement learning. Some studies classified reinforcement learning methods in two groups: model-based and model-free. But, some other studies classified reinforcement learning methods as: value iteration and policy iteration.

I were wondering if anybody help me to know the relation between these classification, as well.

I'm trying to write a software for a multi agent system. My first choice was PyQt4 but it seems that it has a lot of drawbacks when it comes to multi threading. The software should control and guide a real robots to complete a task (e. g. Forming a shape with some cubes)

I am just starting working on vehicle decision making at intersections. I need a traffic simulation platform to

1)Find the transition probability among states of environment vehicle;

2)Test decision making algorithm for my ego vehicle;

3)Test my cooperative decision making algorithm that applies to a group of vehicles at intersection.

I am looking for new research direction in cooperative control of multi-agent systems. What are the latest trends in this field of study? any comment is much appreciated.

Hello,

Recently I found "Flexible Large Scale Agent Modelling Environment for the GPU (FLAMEGPU)" - http://www.flamegpu.com/

Maybe someone is using it and could explain if it is worth learning to use Flame-GPU? As I understand FLAMEGPU uses XML to build models and runs them on CUDA. However, I do not understand why it uses XML? Is it not worth using Python C# with CUDA or something similar?

I also found great looking agent-based models with visualization, could someone explain more precisely how FLAMEGPU is used? e.g. https://www.youtube.com/watch?v=7cjorOe810o&t=2s

What technical requirements are needed for it? Would a laptop with 500 GB SSD hard drive, 16 GB Ram, i7-4710 HQ (8 CPU) ~2.5Ghz and NVIDIA GeForce GTX 860M - 2 GB enough, or a high-performance computing server is needed to use FLAMEGPU? e.g. Amazon web services?

I know that in the Markov decision process (MDP), the probability of transition to a new state depends on the current state and chosen action of an agent. However, in my model, the new state of an agent also depends on the last previous action of itself and its neighbors. Can I solve the problem by a trick? The trick is to consider the last previous action of an agent and its neighbors to be a part of its current state space. I would appreciate it if you could let me know if there is any better solution.

I am searching for efficient simulation tool, which enable to simulate multiple robots in distributed environment and the underlying framework is based upon ROS. If know or have any idea please let me know, in worst case please share this question to increase the chances of getting the right answer. 

Thanks

In agent-oriented software engineering, there are different platforms like JADE, Jason, Jadex, MASON , ... and different methodologies like GAIA, INGENIAS, MaSE, PASSI and ... .  My question is that whether there is a kind of relation between them or not? In other word, is it possible for us to say for example if you want to use JADE platform, it is better to use M methodolgy?

I mean, are MDP and reinforcement learning as powerful as evolutionary game theory to model evolutionary dynamics of populations?

I’ve known that there are two main approaches for reinforcement learning in continuous state and action spaces: model-based and model-free. Does anybody know if this classification (classification of reinforcement learning approaches into model-based and model-free) is right for reinforcement learning in continuous state and action spaces as well. If not, what are the main approaches for continuous case?

I was reading an article on Euler-Lagrange systems. It is stated there that since M(q) and C(q,q') depend on q, it is not autonomous. As a result, we cannot use LaSalle's theorem. I have uploaded that page of the article and highlighted the sentence. (ren.pdf)

Then, I read Spong's book on robotics, and he had used LaSalle's theorem. I am confused. (spong.pdf)

I did some research, and found out that non-autonomous means it should not explicitly depend on the independent variable. Isn't independent variable time in these systems?

As much as I have read, most of the work on multi-agent-systems and thereby,on design of an agent, JADE (or other similar platforms, say JANUS,GAMA,etc) has been extensively used to model a single agent and the entire agent-based-framework.

My question is:

I want to use "Tile Coding" for discretization of my state space in reinforcement learning. But, I don't know how "Tile Coding" exactly works and how I can implement it, so I were wondering if you could mention me more or suggest me some source code of implementing "Tile Coding" in Matlab, R, Java, C and so on.

I mean how we can know a whether a "model-based reinforcement learning algorithm" or a "model-free reinforcement learning algorithm" is suitable for our case . Furthermore, there are a lot of algorithms to choose in each category (i.e., model-based or model-free), how we can find the most suitable algorithm. For example how we can choose between Q-learning, SARSA or TD-learning?

#### I am looking for mathematical and/or computational studies of the properties of an infinite hierarchy of cognitive agents, each agent a multi-agent system in itself, including any emergent properties of the hierarchy. Can anyone help? Clearly such studies require a precise definitions both of an agent and of a multi-agent system.

I seem to recall the existence of early research papers along these lines (presented at an early IJCAI?) but I cannot now find them. Maybe relevant to the MIAP agent architecture qv.

I am writing a computer program that implements an abstract social network of inter-communicating individuals (so a multiple agent system) and I want to be able to compute for each agent in the network of computational agents its individual POWER. I mean actual power, not e.g. power attributed by reputation or constitution. Thus does the mayor of the city of Metropolis have more or less power than the person about to detonate a bomb that will collapse a dam and flood the city? In the UK does the Prime Minister David Cameron have more or less power than Queen Elizabeth or Ian Hislop, editor of the famous satirical magazine Private Eye? By how much?

TO CLARIFY, although the suggestive examples I have given involve human beings [OK, maybe there is some slight doubt about Ian Hislop...] I am looking for (and not yet finding) an ALGORITHMIC means of calculating the "size" of some dynamic attribute reasonably called "power" for a COMPUTATIONAL agent that is a member of a dynamic network of COMPUTATIONAL agents.within a computer. All help much appreciated and duly acknowledged in any consequent publication(s)!

Hi, Can anyone please suggest any Software agent development tool for Beginners- Preferably Java based (apart from JACK and JADE) or .Net based. Also if there any books which give a good idea about the agent communication languages and their implementation.

Dear All,

I am looking to find the optimal size of the multi-agent based coalition. The goal of the coalition is to share the renewable power among the members of the coalition. Does any one know about the generic method or technique for finding the optimal size of the coalition?

I am looking into available power market simulators (preferably to be used in the context of multi-agent systems).

I am familiar with the AMES Wholesale Power Market Testbed (http://www2.econ.iastate.edu/tesfatsi/AMESMarketHome.htm) as well as with PowerWeb (http://www.pserc.cornell.edu/powerweb/) and MASCEM (from the university of Porto).

Further suggestions?

I wish to design a method for modeling Ebola Virus Disease (EVD) infection using multi-agent simulation and to apply it in practice. Can anyone suggest a proper way to do this with references?

Stability and sensitivity of Consensus in Multi Agent System.

I am trying to develop my own agent that extends Agent class of Jade. But, When I open Agent Class, it has so many errors that I can not remove! what can I do?? I copy all needed class to my package too. But, errors still remain!

I am looking at ways of solving a genetic algorithm such as NSGA-II within the communication framework of a Multi Agent System, i.e, using an Agent based scheme.

I am interested in doing some work in area of semantic web crawling/scraping and using that semantic data to do some discovery.

What are the recent advances, issues to be addressed, and scope of research in the area of stability analysis of multi-agent control systems during intermittent and/or permanent sensor faults or loss of observation?

Please shed some light on this area.

In need of help.

I am doing bifurcation analysis using XPP-AUTO for autonomous system (many examples available in web sources). But in the case of non autonomous system, particularly forced duffing oscillator (x ̈ + b x ̇ + ωx + βx 3=fsin(wt)), how to I do the bifurcation analysis. Is there any calculations in the numeric area which is found in the XPP-AUTO workspace.

In a multi agent or stochastic environment it's hard to select one action for an agent. Many papers are related for this task but aren't implementable in the real world.

I would like to use multi-agent systems or (and) fuzzy cognitive maps for modelling some aspects of immune system (human, mouse, ...), but I fight with the problem how to find some data in not time-consuming way all the time. Searching scientific databases with papers (PubMed, PlosOne, ...) is one of the possibility how to find data, but this is really very time-consuming activity, especially if you are not the expert in immunology. I have already find the following link http://bionumbers.hms.harvard.edu/, but I think that it is not the right way.

We have already designed and implemented a number of software applications based on the multi-agent paradigm. We have used Java based FIPA compliant platforms JADE and SAGE. Most of our work is done by using JADE.

Now we are seriously in need of a FIPA compliant multi-agent platform based on Microsoft dot NET technology. And we don't have the resources and time to build our own dot NET based FIPA compliant platform. Need guidance.

BDI architecture is one of the agent architectures based on Belief, Desires, and Intentions. We are in need of tools based on BDI.

I think a MAS could have a significant role into handling and linking higher level relations arising from a lower level NLP modules set output.

A known method for modeling and simulating the dynamics of multi-agent systems is the Petri nets. It provides the best results? It can be used for large systems? Know you a better way?

the DGS is an adaptation of Holland's Learning Classifier System with Tagged Urns and some additional structure. I'm interested in using it to model socio-technical systems related to information security, privacy, confidentiality, homeland defense, etc. Please share references to published papers or working papers, if possible. I will be coding in Mathematica for prototype, then in Java (or maybe Python) for production.

I would like to know if there is an automated way to transform probabilities (segregated into age cohorts) into a synthetic population of agents with the same statistical traits. This is: transforming data like “45% males, 55% females, 20% females smoke, 35% males between 15 – 30 years old have diabetes, etc" Into a population with, let’s say, 100 agents where 45 are males, 55 are females, 11 of these smoke, etc.

In the jungle of programming languages, and within the advances of real-time systems that is joining decision based systems and agent based systems, what is the best language to represent real-time intelligent systems and why?

Does anyone know of any clear and simple problem for assessing an algorithm proposed for organizational structures like hierarchies in Multi-agent Systems?

Hi,

is there someone who can help me with his knowledge in designing multi-agent system with A-UML? if there is any suggested paper with all details and exemples?

thank you

Indexed in the ISI Thomson - SCIE/CURRENT CONTENTS and in all major scientific databases, the International Journal of Advanced Robotic Systems is a peer-reviewed Open Access journal, published by InTech, which aims to present the latest research in robotics. 

Facts:

First issue: 2004

Periodicity: 4 time per year

Success story: In 2006, upon NASA's request, InTech published a special issue of ARS journal, to which NASA's scientists contributed their scientific achievements regarding the Mars Exploration Rover missions

Other facts: New design in 2011, online only, InTech's first publication, started in Vienna

At the moment the journal has an ongoing call for papers for a special issue that seeks to examine the state-of-art, to present original insights and up-to-date research results in the field of Robot Manipulators. Under the editorial guidance of Dr. Gerasimos Rigatos, the proposed special issue aims at analyzing recent advances in the design and applications of robotic manipulators with particular interest in problems of nonlinear estimation, multi-sensor fusion and control.

If you are interested in contributing to this special issue by submitting your research papers please visit InTech's invitation page:

For previous issues please visit http://www.intechopen.com/journals/show/title/international_journal_of_advanced_robotic_systems where you can read the journal online for free, like all other InTech's publications (academic journals and books).