Access to this full-text is provided by MDPI.
Content available from Energies
This content is subject to copyright.
Citation: Izmirlioglu, Y.; Pham, L.;
Son, T.C.; Pontelli, E. A Survey of
Multi-Agent Systems for Smartgrids.
Energies 2024,17, 3620. https://
doi.org/10.3390/en17153620
Academic Editor: José Matas
Received: 6 February 2024
Revised: 15 July 2024
Accepted: 18 July 2024
Published: 23 July 2024
Copyright: © 2024 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
energies
Review
A Survey of Multi-Agent Systems for Smartgrids
Yusuf Izmirlioglu 1,2 , Loc Pham 2, Tran Cao Son 2and Enrico Pontelli 2,*
1Department of Computer Science, University of Roehampton, London SW15 5PH, UK;
yusuf.izmirlioglu@roehampton.ac.uk
2Department of Computer Science, New Mexico State University, Las Cruces, NM 88003, USA;
yizmir@nmsu.edu (Y.I.); locpham@nmsu.edu (L.P.); stran@nmsu.edu (T.C.S.)
*Correspondence: epontell@nmsu.edu
Abstract: This paper provides a survey of the literature on the application of Multi-agent Systems
(MAS) technology for Smartgrids. Smartgrids represent the next generation electric network, as
communities are developing self-sufficient and environmentally friendly energy production. As a
cyber-physical system, the development of the vision of Smartgrids requires the resolution of major
technical problems; this has fed over a decade of research. Due to the stochastic, intermittent nature
of renewable energy resources and the heterogeneity of the agents involved in a Smartgrid, demand
and supply management, energy trade and control of grid elements constitute great challenges for
stable operation. In addition, in order to offer resilience against faults and attacks, Smartgrids should
also have restoration, self-recovery and security capabilities. Multi-agent systems (MAS) technology
has been a popular approach to deal with these challenges in Smartgrids, due to their ability to
support reasoning in a distributed context. This survey reviews the literature concerning the use of
MAS models in each of the relevant research areas related to Smartgrids. The survey explores how
researchers have utilized agent-based tools and methods to solve the main problems of Smartgrids.
The survey also discusses the challenges in the advancement of Smartgrid technology and identifies
the open problems for research from the view of multi-agent systems.
Keywords: Smartgrid; multi-agent systems; electrical infrastructure; renewable energy; Artificial
Intelligence; knowledge reasoning
1. Introduction
One of the major paradigms in the energy sector in the 21st century will be the utiliza-
tion of renewable, carbon-free energy resources for an efficient and sustainable society [
1
].
Climate change and the significant use of fossil resources have become significant concerns
of contemporary societies. Population growth and industrialization have increased the
demand for energy from households, businesses and factories. These factors bring the need
for greater production of electricity in power plants which causes pollution, toxic waste
and over consumption of natural resources.
Governments have started enacting laws to limit greenhouse gas (GHG) emissions
from factories and thermic electric plants, in order to protect the environment for sustainable
development. Furthermore, many countries are planning to close nuclear power plants and
urge entrepreneurs to invest in renewable energy resources. Climate change, pollution and
the scarcity of fossil fuels have increased the demand for renewable energy sources (RESs).
These alternative energy resources include wind, solar, hydro, biomass and agricultural
and industrial waste.
Several international protocols and agreements have been announced for the preserva-
tion of nature and reducing pollution. Key events and milestones are the enforcement of the
United Nations Framework Convention on Climate Change in 1994 [
2
], the Kyoto Protocol
in 2005 [
3
], the enforcement of the Paris Agreement in 2016 [
4
] and Directive 2009/28/EC [
5
].
These commitments have resulted in policies for decarbonizing the electricity sector and
Energies 2024,17, 3620. https://doi.org/10.3390/en17153620 https://www.mdpi.com/journal/energies
Energies 2024,17, 3620 2 of 61
providing incentives towards the use of renewable resources and clean technologies to
produce energy. Examples of such policies are the Energy Policy Act of 2005 [
6
,
7
] and the
American Recovery and Reinvestment Act of 2009 [
8
,
9
] in the U.S., the Renewable Energy
Law of 2005 [
10
] and the Golden Sun program [
11
] in China and the Renewable Energy
Directive [12] and Emissions Trading Scheme in the EU [13].
Conventional centralized electricity systems consist of a small number of large power
plants, high voltage transmission lines, transformers and local distribution networks. In
this architecture, power flow is unidirectional (from plants to consumers) and there is no
communication or feedback mechanism between producers and consumers [
14
]. The lack
of predictability of energy demand requires plants to overproduce electricity for safety
margins. In addition, considering the foregone heat during electric production and the
losses over long transmission lines, the centralized electric production system has been
recognized as inefficient and wasteful.
Aside from environmental concerns, another main factor that drives change in conven-
tional electric grids is efficiency and scalability. As populations and cities grow, the demand
for electric energy rises and consequently the load on central power plants, network and
transmission lines increases. In transportation services, electric and hybrid vehicles are
becoming more prevalent due to their efficiency. Existing plants, infrastructure and trans-
formers have bounded capacity; thus, growing the grid constitutes a great challenge in
the sense that this would incur a major revision to the infrastructure and significant cost.
Moreover, as mentioned above, transmitting electricity over long distances requires costly
infrastructure and causes losses; thus, a logical solution is to install local, decentralized
energy generators close to the end users.
Along with these motivations, the electricity industry is experiencing a radical trans-
formation from a few, centralized, large-scale thermic and nuclear plants to many, de-
centralized, small-scale clean producers. Thanks to the development of alternative and
affordable energy sources, such as photovoltaic solar panels, wind turbines and natural
gas microturbines, some households or businesses are currently installing new equipment
to produce and store electricity to satisfy in part or in full their needs and even sell excess
electricity to other consumers in the grid. These small and renewable generators have lower
emissions and lower operational and maintenance costs. In this respect, a Smartgrid seems
to be a promising direction to fulfill the increasing energy demand of the society and at the
same time preserve the environment and decrease pollution. The term “Smartgrid” refers
to a distributed system of electric energy generation with many intelligent producers and
consumers who can communicate, reason and decide.
This radical change in the electricity industry also brings some new challenges for
management, control, scheduling and security. Power obtained from solar panels and
wind turbines depends on weather conditions; thus, the amount of generated energy
varies both during the day and across days. In order to deal with the uncertainty, energy
consumption of electric devices can be shifted or rescheduled dynamically over time.
Hence, in a distributed grid with many producers and consumers, coordination, allocation
and exchange of energy becomes an important challenge. Aside from these engineering
aspects, there are also economic issues such as energy trade and allocation in the Smartgrid.
In order to manage the operation of a cyber-physical system such as a Smartgrid,
we need software tools and computational infrastructure to handle communication and
various functions of the grid elements. Multi-agent systems (MAS) seem to be a suitable
framework for management and control of a highly distributed, dynamic grid of hard and
soft components. A multi-agent system is a group of autonomous, independent entities
(agents) acting in an environment to achieve their own objectives or group objectives. These
agents can be software or a cyber-physical object as in a Smartgrid. MAS is a paradigm
for modeling autonomous and intelligent agents who can perceive and perform actions
in a dynamic environment. Agents are intelligent in the sense that they reason about the
state, optimize their benefit, make rational decisions and learn from past experiences. In
this respect, the MAS framework can be used to model a network of independent agents
Energies 2024,17, 3620 3 of 61
in the Smartgrid. The contemporary developments toward the future Smartgrid may
require integration of millions of devices such as Distributed Energy Resources, loads,
storage elements and sensors. The control systems will have to operate efficiently on a
large scale; they should also be robust to faults and attacks on the system. The complexity
of the control system can be highly reduced by distributing tasks among cooperative and
communicating agents. Agents can operate autonomously and they cooperate or compete
with each other depending on the context. This feature of MAS may play an important role
in the management of a sophisticated Smartgrid with a divide-and-conquer method: the
grid can be partitioned into several microgrids and intermediate layers (station, building,
home) can also be added. Compared to the control methodologies of conventional power
systems, this would be a bottom-up approach instead, where decisions are taken locally in
a decentralized manner.
Alternatives to MAS exist, such as SCADA (supervisory control and data acquisition),
expert systems and neural networks. However, these methods are either not suited or not
scalable for a large electric grid. SCADA has a top to bottom structure and it is used for
central management of the grid. MAS technology has been used by researchers in electrical
engineering, computer science and economics to address various issues in Smartgrids.
In this paper, we survey the literature about applications of MASs for Smartgrids. We
first give some definitions and explain the concept of MASs. We discuss the relevance and
benefits of MASs for Smartgrids and distributed energy management systems. Then, we
go through the literature on the following topics:
• MAS platforms and tools for energy management;
• Standards and protocols;
• Ontology for energy domains;
• Energy markets and trade;
• Control and management;
• Demand and supply management;
• Restoration and self-recovery;
• Protection and security;
• Simulation and implementation.
In each section, we report the main problems, challenges in that topic and how
researchers have developed tools to solve these problems. At the end of the section, we
discuss the remaining open problems. We have also created a separate section where we
provide a meta-level analysis and explain the general challenges.
There are existing survey papers [
15
–
31
] for the application of MAS techniques to
Smartgrid and energy systems. However, these surveys focus on one or several topics in
the list above.
In Table 1, we illustrate the differences in these surveys and the present survey in
terms of the topics that they review. In particular, the previous surveys do not mention the
role of knowledge reasoning and planning for future research. Furthermore these survey
papers do not state specific challenges, alternative methods and problems in each field
which are not yet studied or investigated in sufficient detail. One of the distinguishing
features of this survey is that we review the relevant work in each topic in detail, state
challenges and open problems and provide an upper picture of the state of the art, all from
the perspective of MASs.
For the literature search, we first identified the main problems and research topics
about Smartgrids on the list. In this process, we also benefited from policy papers and
other survey papers cited above. Then, for each of these topics, we searched for the papers
on Google Scholar. We included journal articles, conference articles, official policy papers
and standardization documents about MAS applications on Smartgrids, smart buildings,
smart homes and electric vehicles. Among them, we selected those papers which have
seminal contributions or propose original solutions to the problem related to the topic. We
excluded papers that have non-MAS solutions, study a subproblem or a specific problem,
or study conventional power systems.
Energies 2024,17, 3620 4 of 61
Table 1. Previous survey papers in the literature.
Survey Paper Reviewed Topics
Bayram et al. (2014) [15] Energy trade, EV (dis)charging, Market simulation
Coelho et al. (2017) [16] Microgrid control, Energy storage units and EV charging, Demand management,
Restoration, Security, Implementation
Gómez-Sanz et al. (2014) [17] Energy trade, Control, Simulation
Kantamneni et al. (2015) [18] MAS platforms, Energy trade, Control, Restoration
Kiran et al. (2017) [19] MAS platforms, Energy trade, Energy market simulation
Kulasekera et al. (2011) [20] Energy trade, Control, Restoration
Mahela et al. (2020) [21] Smartgrid standards, Control, Building energy management, EV charging
McArthur et al. (2007) [22] Protection, Simulation, Implementation, Technical Challenges
McArthur et al. (2007) [23] MAS design methodologies, Standards, Ontologies
Halhoul Merabet et al. (2014) [24] MAS platforms, Control, Implementation
Sukumaran Nair et al. (2018) [25] Supply management (economic dispatch, unit commitment), Consensus algorithms
Vithanage et al. (2019) [26] Control
Roche et al. (2010) [27] MAS platforms, MAS design methodologies, Energy trade, Demand management,
Simulation, Implementation, Future scope
Roche et al. (2013) [28] MAS organization and design methodologies, Standards, Ontologies, Energy trade,
Voltage control, Restoration, Future scope
Rohbogner et al. (2013) [29] Energy trade, Microgrid control, Voltage and frequency stabilization
Hasanuzzaman Shawon et al. (2019) [30] Energy trade, Control, Restoration, Security
Sujil et al. (2016) [31] Energy trade, Control, Supply management, EV charging, Restoration
2. Smartgrid: Basic Concepts
Smartgrid is regarded as the new generation electricity system, which integrates Infor-
mation Technology, distributed computation and Artificial Intelligence tools for efficient
energy generation and delivery with renewable resources [
21
]. In addition to the use of
renewable energy resources, other advantages of Smartgrid are accommodation of en-
ergy storage (batteries), monitoring and estimation of users’ consumption and adaptive
configuration of the grid, e.g., in response to unexpected events [15].
A Smartgrid is a cyber-physical system, meaning that it has physical infrastructure
(electrical and mechanical elements like lines, transformers, relays, etc.) and computational
components (communication, software, applications, protocols, etc.) [
32
]. It has the capabil-
ities of communication and real-time data collection and processing. A Smartgrid enables
a two-way flow of electricity and information to create an automated, widely distributed
energy network [
33
]. These capabilities of Smartgrids provide the benefits of monitoring
grid elements, dynamic energy pricing, analysis and scheduling of energy usage, efficient
allocation of energy in a wide network, resistance to cyber-attacks and auto-recovery [26].
A Smartgrid is responsible for managing electricity resources and loads. The Dis-
tributed Energy Resources (DER) in the Smartgrid are photo-voltaic panels, wind turbines
and diesel generators and the storage devices are energy capacitors, batteries [
34
]. Critical
loads are those higher-priority devices such as heaters, refrigerators, lights, security and
fire alarm systems that require delivery of electricity immediately for the welfare of the
household or community [
35
,
36
]. On the other hand, the energy demand of non-critical
loads can be postponed when necessary, e.g., as is the case of laundry, dishwasher machines
and electric vehicles (EVs).
A Smartgrid (see Figure 1) is an interconnected network of power plants, Virtual
Power Plants, microgrids, sensors, meters, transmission elements and auxiliary compo-
nents (stations, transformers, cables) [
21
]. It typically includes a main grid (also called
Energies 2024,17, 3620 5 of 61
utility/upstream/AC grid) and a multitude of smaller local microgrids and Virtual Power
Plants (VPPs), that are connected to the main grid [
37
] (Figure 2). The main grid is the pri-
mary source of energy; it is powered by large-scale plants (thermic, hydroelectric, nuclear)
and supplies electricity to the microgrids and VPPs in case their local electricity generation
is insufficient. It is also possible that microgrids and VPPs transfer (e.g., by selling it)
their excess energy to the main grid. The microgrids and Virtual Power Plants are con-
nected to the main grid through the Point of Common Coupling (PCC) where the energy
transfer occurs.
Figure 1. Components of a Smartgrid.
A microgrid consists of distributed small-scale renewable and non-renewable energy
resources, storage devices and critical and non-critical loads [
38
,
39
]. A typical example
of a microgrid is a geographically isolated area such as a campus or holiday resort which
produces its own energy. The microgrid operates either in the connected mode or in the
islanded mode. In the connected mode, the microgrid can transfer electricity from the main
grid whereas in the islanded mode, the microgrid can only use its own resources. The
islanded mode can occur as a result of a fault or outage at the main grid; yet there are also
self-sufficient microgrids that always operate intentionally in the island mode.
Households in the Smartgrid are often called prosumers, which means they are both
producers and consumers of electricity [
40
]. Prosumers exchange information and trade
energy with each other for optimal utilization and sharing of available energy. A Virtual
Power Plant (VPP) is a cluster of distributed energy generators which sell electricity in
the market and compete with large-scale power plants [
41
]. The intuition behind VPPs
is that a single household or distributed generator can only offer a negligible amount of
electricity in the market and thus cannot be a major player or supplier. A VPP is a solution
to this problem: it is a virtual coalition of small distributed resources and is seen as a
single entity (power plant) in the energy market. VPP aggregates energy and information
of its generators and sets its own production schedule, quota and price. In addition to
energy generators, a Virtual Power Plant involves consumers, loads, storage elements,
smart homes, smart buildings, electric vehicles and auxiliary components. The concept
of VPP is similar to a microgrid, except that a microgrid is defined based on the spatial
proximity of its grid elements and it is usually located in a well-defined geographical area.
On the other hand, a VPP can be instituted at a wide scale (or at the desired scale) without
a geographical limit. A VPP also has a more advanced management of energy generation,
load profile and operation cost in order to obtain revenue in the market.
Energies 2024,17, 3620 6 of 61
Utility Grid
Utility-scale
Renewables
Microgrid Controller
Microgrid can be
“islanded” from the
Utility Grid
Microgrid
Electric
Vehicles
Distributed
Renewables
Consumers
Storage
Generators
Communication
Network
Electricity
Network
Figure 2. Microgrid and Main Utility Grid.
3. Multi-Agent System Concepts and Definitions
This section provides some preliminary definitions concerning the concepts of agent,
MAS technology, multi-agent organizational paradigms, MAS platforms and software,
standards and ontologies for the energy domain.
3.1. Definition of an Agent
In the literature, there are multiple definitions for the concept of an agent resulting from
diverse applications and domain-specific features of MASs. One of these early definitions
was proposed by Wooldridge [
42
], “an agent is a computer system that is situated in some
environment and that is capable of autonomous action in this environment in order to meet its
design objectives”. According to Russel and Norvig [
43
], an agent is anything that can be
viewed as perceiving its environment through sensors and acting upon that environment
through effectors. Later, in the 2003 edition of their book [
44
], Russel and Norvig define an
intelligent agent as an autonomous entity which has the following properties:
• It has the ability to communicate and interact with its environment;
• It is able to perceive the (local) environment;
• It is guided by basic objectives;
• It has feedback behavior.
Ref. [
45
] defines an agent as an entity which is placed in an environment and senses
different parameters that are used to make a decision based on the goal of the entity.
Ferber [
46
] proposes another definition: “An agent can be a physical or virtual entity that can
act, perceive its environment (in a partial way) and communicate with others, is autonomous and
has skills to achieve its goals and tendencies”.
Power engineering and the Smartgrid community have mostly adopted the definition
of agents in Wooldridge and Jennings [
47
] as it fits the role of the agent in this context. An
agent denotes a hardware- or software-based system which has the following properties:
•
Autonomous: Agents exert partial control of their actions and internal state, seeking
to influence outcomes without the intervention of humans or external devices.
•
Social: Agents can communicate and negotiate with humans, external devices or other
agents to coordinate actions and satisfy their objectives.
• Reactive: Agents react in a timely fashion to changes in their environment.
• Proactive: Agents exhibit goal-oriented behaviors and take initiative to satisfy objectives.
Ref. [
48
] lists eight properties of agents: location, mobility, autonomy, perception,
communication, adaptation, reactivity, pro-activeness, rationality, socialness. A common
methodology to implement agents is the Belief–Desire–Intention (BDI) model: Beliefs
reflect the information of the agent about the environment. Desire represents the needs and
objectives of the agent. Intention represents the actions and strategy of the agent to achieve
its goals [49,50]. An overview of agent definitions and discussion can be found in [51,52].
Energies 2024,17, 3620 7 of 61
3.2. Anatomy of Agents
The internal architecture (or anatomy) of an agent defines its behavior, i.e., the actions
of the agent as a function of its perceptions and the changes in the environment [
53
]. This
function can be simple, as in reactive agents, or very complex, as in cognitive agents.
There are different types of agents, classified according to their level of autonomy, the way
perception is achieved, and how they can be modeled [28]:
•
Reflexive agents perform simple actions based on their perceptions; their behavior
is based on an action-selection module that receives percepts from the environment
and consults a database of condition–action rules, similar to if–then rules, to make
a decision. Such agents can be useful when fast response times are needed, e.g., for
protection. Their representation of the world (the environment) is minimal, but they
may support emergent behaviors. Emergence occurs when new characteristics appear
at a certain level of complexity.
•
Goal-based agents are directed by goals set a priori by the user. They have an internal
representation of their environment and can memorize previous percepts to make
more elaborate decisions. More precisely, once a percept is received by the agent,
the memory manager stores the information in the percept memory. A sequence of
percepts is then built in order to be used subsequently by the action-selection module
to select suitable actions in order to reach given goals.
•
Utility-based agents use a performance-measurement index, referred to as the utility
function, in order to evaluate their behavior. A utility-based agent chooses an action
that optimizes its utility or achieves a certain satisfactory level of utility. Such an agent
is rational and behaves efficiently, given its prior knowledge of the environment.
•
Learning agents belong to one of the previous classes and in addition can learn to
perform a given task more effectively. They are able to modify the function that codes
their behavior while interacting with their environment, to be more precise in perform-
ing a given task. An agent running a load-forecasting algorithm would be a typical
example. The learning can be supervised, unsupervised or reinforcement learning.
3.3. Multi-Agent Systems and Smartgrid
A multi-agent system (MAS) consists of a collection of agents that interact with each
other and the environment and perform actions to satisfy their objectives [
45
,
54
]. An
MAS supports a distributed computing paradigm: the idea of MAS is to decompose a
complex problem into several simpler sub-problems and use a separate agent to deal with
each sub-problem.
An MAS can have a different organizational structure (see Figure 3). An organizational
structure specifies the connection and logical position of agents, their roles and privileges,
information flow and coordination patterns [
28
]. There exist a variety of MAS organiza-
tional paradigms, such as hierarchy, holarchy, matrix, coalition, team, holon, congregation,
society, federation and their compounds. Each organization has its own topology among
agents. For instance, a hierarchy has a tree-like top to bottom structure; a coalition, team,
holon maintain a cluster of agents; a matrix has multiple managers or peers. A description
of these organizational paradigms can be found in [28,54,55].
As we will explain in subsequent sections, an MAS-based management system for a
Smartgrid may involve a variety of agents operating within an organizational architecture.
Some of these agents are associated with a device or component (e.g., wind turbine agent,
battery agent, EV agent, bus agent) and some agents are associated with a specific task
(e.g., auctioneer agent, control agent, energy-forecasting agent, validation agent). The MAS
organizational structure describes the region and grouping of agents in Smartgrids and
the communication topology between them. The MAS structures that are more commonly
employed in Smartgrid applications are centralized, decentralized, hierarchical and hy-
brid [
34
]. In a centralized architecture, a collection of simple, uniform agents is managed
by a single master controller agent, operating in a master–slave relationship. Namely, all
primary tasks and functions of the Smartgrid are handled by the controller agent. In this
Energies 2024,17, 3620 8 of 61
sense, a centralized MAS structure resembles a traditional SCADA-type supervisory system.
In contrast, in a decentralized structure, there is no hierarchy or master–slave relationship
between any agent in the MAS system. Agents operate and decide in a fully independent
manner. Typically, agents communicate and negotiate in a bilateral (peer-to-peer) fashion.
Figure 3. Examples of different MAS structures: (a) centralized, (b) decentralized, (c) hierarchical,
(d) holon, (e) coalition.
The centralized management framework for Smartgrid causes high data traffic load at
the level of the controller, and hence requires a powerful data center and wide bandwidth
for processing a large volume of data in a short period of time [
56
,
57
]. Another disadvantage
is that it is susceptible to “Single Point of Failure”; in other words, a fault or an attack to
the controller agent will impair the operation of the entire system [
58
–
60
]. Furthermore,
the grid will be at risk of cascading failures [
61
–
63
]. In addition, a centralized model is not
suitable for a Smartgrid which has a time-variant structure.
The distributed approach to MAS Smartgrid management is more reliable compared
to the centralized approach [
64
]. It is more robust in terms of operation, control and moni-
toring. In addition, it is simpler and more cost-effective in terms of implementation [
65
].
An MAS is an effective way of distributed control of microgrids as an alternative to tra-
ditional hardware-based centralized control. The key advantages of MASs for Smartgrid
applications are highlighted below [16,22,25,27].
•
Distributed Nature: The entire Smartgrid can be divided into microgrids and VPPs.
Moreover, intermediary layers and lower-level grid elements can be added in between.
An MAS includes many autonomous agents computing and operating in a parallel
and asynchronous manner. The decentralized structure of MAS and autonomous
agents makes the control of Smartgrids easier.
•
Flexibility: MAS supports the plug-and-play capability of microgrids, Distributed
Energy Resources, storage elements and other equipment and can adjust the control of
the grid accordingly. Agents have self-adaptive behavior to the Smartgrid environment
and act to accomplish their goals.
•
Fault tolerance: If one agent fails, the rest of the system can remain active and can
adapt to the new state by its rules and behaviors. Thus, MAS-based Smartgrids can be
more resilient to disturbances and faults.
•
Responsiveness: As agents sense changes in real time, collect the relevant informa-
tion and communicate with each other, MASs can quickly respond to the events in
the environment.
Energies 2024,17, 3620 9 of 61
•
Scalability: The complexity of the energy-generation and -distribution system can
be highly reduced by dividing it into layers, units and agents. Each autonomous
agent is responsible for a component of the Smartgrid and agents are modular. Thus,
the overall system can be expandable by simply adding new agents.
•
Local knowledge: Each agent only needs information from its local environment and
communicates with its neighbors for its own decision-making. Thus, the required data
and communication in MAS are more controlled and limited compared to a centralized
control system. This feature of MAS is especially beneficial for management of a large
system like a Smartgrid.
Some scholars have a critical view of whether the agent-based system is the right
technology for a Smartgrid. Ref. [
17
] argues that MASs may not be the right or optimal
choice for every problem or domain and discusses the incorrect use of agents. In particular,
the authors state that in some MAS models for Smartgrids, there are either too few (just
one agent) or too many agents, agents do not have decision capabilities, there are very few
choices for the agents, agent-to-agent relationships are client–server rather than peer-to-peer
and decentralization is achieved through distribution which are two different concepts.
Between the two extremes, fully centralized and fully decentralized, researchers
have also adopted hierarchical and hybrid MAS architectures for Smartgrid applications.
In a hierarchical MAS, there are multiple levels (also called layers) from top to bottom.
An agent at a given layer has control over agents at the lower layer. In a hierarchical
architecture, agents communicate with those agents in the upper layer and the lower
layer. This type of organizational structure emerges in Smartgrids as consecutive layers,
e.g., device/appliance, house, building, station and microgrid in an ascending fashion.
There is often a dedicated agent responsible for a layer such as a microgrid control agent,
zone agent, transformer agent and home energy management agent. Note that the decision
making across adjacent levels in a hierarchy does not always have to be in a master–slave
relation. Each agent can have some autonomy and fulfill the relevant assigned task. As
an example, each layer aggregates information from the lower layer and performs some
optimizations before communicating to the higher layer.
Some agent-based designs for Smartgrids are hybrid (or mixed). For instance, the grid
might be partitioned into different zones and each zone can have a different structure
(centralized, decentralized or hierarchical). Another possible situation might be that the
MAS has multiple layers and the relationship inside a layer might be different from others.
Namely, sibling agents at a layer can be organized in a centralized manner, whereas
at another layer they can be decentralized or they can form holons. Examples of such
organizational paradigms will be provided in the subsequent sections.
To achieve proper operation of Smartgrids, agents in the system perform their assigned
tasks, functions and optimization processes, e.g., opening/closing circuit, determining and
evaluating bids, fault and anomaly detection, charge scheduling and restoration [
66
]. Note
that the choice of decision rule or optimization method depends on the MAS structure and
the type of application. For example, a consensus type algorithm (which uses information
exchange between neighbors) is more suitable for a decentralized system whereas combined
heat and energy optimization of a home is more suitable in a centralized system.
In MAS-based Smartgrid models, researchers have used the following control methods
and decision functions: rule-based, fuzzy or probabilistic logic, expert systems, decision
models, data-driven control, (non)linear control, model predictive control, consensus algo-
rithms, convergence algorithms, game theory, historical data analysis, statistical methods
(logit, probit), statistical filtering, neural networks, machine learning, human immune
system and heuristic algorithms. The optimization methods used by the researchers are
reinforcement learning, dynamic programming, mathematical programming, Lagrangian
methods, gradient algorithms, swarm intelligence, evolutionary and genetic algorithms,
graph and tree search, exhaustive search, local search and (meta)heuristic optimization
algorithms. Refs. [
34
,
67
,
68
] give a description of these decision functions and optimization
Energies 2024,17, 3620 10 of 61
methods. The choice of the relevant method depends on the nature of the problem and
the context.
3.4. MAS Design Methodologies
In the literature, several MAS design methodologies have been presented [
69
–
71
], yet
the basic steps in these design methodologies are the same. These methodologies essentially
employ software-engineering and knowledge-engineering approaches for the specification
and design of agent-based systems. There are three fundamental stages in the design of an
MAS system: conceptualization, analysis and design [
31
]. In the design process, the output
of each stage is fed into the subsequent stage. The conceptualization stage defines the
problem to be solved and specifies the system requirements. During the analysis stage,
the problem and requirements are analyzed by the appropriate software- and knowledge-
engineering techniques and the main task is decomposed into a hierarchy of subtasks. In the
last stage (design stage), agents are created and assigned to these tasks in the hierarchy. The
design stage also specifies agent anatomy, functionality, MAS architecture, agent behaviour,
communication topology and an ontology for communication.
In Smartgrids, multi-agent system design steps and decomposition depend on the
type of task, grid elements and architecture. For instance, if a Smartgrid security system
requires each agent to monitor and assess neighbor agents, then the corresponding MAS
structure should be decentralized and interaction is peer-to-peer. On the other hand, for a
control system of a network of multiple microgrids, a hierarchical MAS would be better
suited because there exist different layers like main grid, microgrid, zone and building in
the network. In this case, energy transfer and communication take place across adjacent
layers or inside the same layer.
Namely, the main problem is divided into subproblems in a suitable hierarchy and
communication topology and agents are assigned to perform individual subtasks or func-
tions in the respective position. If necessary, aggregator or coordinator agent(s) can be
assigned at the respective layers or at the center of the MAS.
3.5. MAS Platforms and Software
There are many programming languages and software platforms available for design-
ing and implementing multi-agent systems. Some of these MAS languages and platforms
are JADE, Zeus, JADEx, JACK, EMERALD, JAS, Jason, AGLOBE, Agent Factory, SeSAm,
GAMA, Cougaar, Swarm, MASON, INGENIAS, Kit, Cormas, Repast, MaDKit, CybelePro,
JIAC, AgentScape, AnyLogic, Net-Logo, JAMES, OOA, Pangea, Pade, Gaml, Spade, Ja-
CaMo, Goal, Plasa and Sarl. An overview and comparison of these platforms can be found
in [72,73].
MAS development platforms are quite heterogeneous and have a number of plugins,
frameworks and libraries for both commercial and academic audiences. Among these
platforms, the ones that have been primarily used in the context of electric infrastructures
and Smartgrid projects are JADE, ZEUS, PADE, JIAC and Volttron [74].
JADE is an open source multi-agent (multi-host) platform [
75
] developed and dis-
tributed by Turci by Telecom Italia Lab in 1999. JADE has the full support of FIPA standards,
it has third-party plugins and a wide documentation. JADE includes Agent Communica-
tion Channel (ACC), Agent Management System (AMS) and Director Facilitator (DF). The
Director Facilitator service, similar to yellow pages, is useful for listing agents and their
abilities. JADE also partially supports Semantic Web technologies. In terms of implementa-
tion, agents in JADE can be seen as three-layered architectures: a message-handling layer
for processing messages, a behavioral level for defining when tasks are to be carried out
and a functional level for defining the actions the agent will perform [23].
Some of the applications of JADE in the power domain include:
•
Ref. [
76
] develop a PowerSmartgrid Prototype at Illinois Institute of Technology. They
use the JADE platform to model autonomous agents such as Distributed Energy
Energies 2024,17, 3620 11 of 61
Resource-generation agents and energy-storage agents. JADE also serves as a medium
to communicate and post messages between agents.
•
Refs. [
77
,
78
] use JADE to design a microgrid control infrastructure for an island in
Greece. The objective of the application is to control the operation of non-critical loads
in the microgrid.
•
Ref. [
79
] use JADE to develop a microgrid management system to control generation
and storage devices. The management system consists of a central controller, source
controller and load controller. Agents can trade energy by submitting bids to the
central market manager. The researchers tested this proposed architecture in laboratory
facilities under different microgrid configurations.
•
Ref. [
80
] propose a model of a microgrid simulation where agents are designed with
JADE. Agents interact and negotiate with each other for demand management. In
an experiment, they illustrate how agents react by adjusting their demand and how
prices dynamically evolve upon a change in power supply.
•
Ref. [
81
] propose a distributed energy- and resource-management system for multiple
microgrids using JADE. Allocation of energy is achieved by an auction mechanism
and JADE agents bid for energy in the market in real time.
•
Another example is [
82
] which presents an MAS-based energy management system
with JADE. In their setting, agents use the contract network protocol to allocate energy
among them.
Other examples of Smartgrid control with JADE are [
83
–
86
]. Some other papers that
use JADE to design agents for energy trade, scheduling and demand response are
[38,87–89]
.
ZEUS [
90
,
91
] is a multi-agent platform developed by the British Telecom intelligent
system research laboratory. ZEUS supports the design of agents in terms of their goals,
tasks and factual knowledge of the environment. ZEUS is open source, FIPA compliant
and enables agents’ communication using either the ACL or the KQML language. The user
can specify agent properties using a graphical user interface, and the platform automatically
generates the corresponding Java code. ZEUS provides a run-time environment, debugging
tools and planning- and process-scheduling tools. However, it supports only one agent
model, which limits the range of possible MAS designs.
A number of Smartgrid applications have been developed using ZEUS [
74
,
92
–
94
].
Refs. [
74
,
92
] discuss the design of an intelligent, distributed, autonomous power system
which can operate in “normal mode” as well as “outage mode”. There are four types
of agents: a control agent, a DER agent, a user agent and a database agent. The ZEUS
platform is used to model the agents and to support the communication among them. The
agents work in collaboration to detect upstream outages and react accordingly to allow the
microgrid to operate autonomously in islanded mode. During normal operating conditions,
the microgrid runs as a part of the local utility and coordinates its internal loads and
Distributed Energy Resources for optimal operation. In the event of an upstream outage,
the microgrid performs load controls based on a predefined prioritized list and activates
its internal generators to secure critical loads.
Refs. [
93
,
94
] adopt a hierarchical MAS system using ZEUS and Matlab for coordination
and control of a microgrid. Their system includes three levels: master level, substation
level and terminal level. DER and load agents are located at the terminal level. Simulations
reveal that MAS-based microgrid systems can perform stable operations: the microgrid can
switch from connected mode to islanded mode in cases of substation failure and then back
to connected mode when the failure is eliminated.
PADE [
95
] is a Python-based platform, specifically designed for power-engineering
applications. PADE is an open source platform for MAS development that makes use of
standard communication tools, enables the execution of predefined behaviors, and provides
a tool for tracking agents’ actions. PADE provides agents with communication capabili-
ties over standard protocols such as Generic Object Orientated Substation Event protocol
(GOOSE) and Manufacturing Message Specification protocol (MMS) of IEC 61850, along
with data standards such as the Common Information Model (CIM). PADE is thus appro-
Energies 2024,17, 3620 12 of 61
priate for power engineering applications such as monitoring and diagnostic, automation,
self-healing, adaptive protection and microgrid control.
JIAC [
96
] is an open source multi-agent platform which supports a distributed agent
framework, comprising agent nodes and enables the run-time environment for the agents.
The run-time environment can be monitored and controlled by Java Management Ex-
tension Standard (JMX). JIAC involves debugging capabilities and has been utilized in
a series of industrial applications. In particular, Ref. [
97
] describes an MAS-based de-
centralized Smartgrid management system, which monitors and controls the grid. They
also developed an application that optimizes the charging schedule of electric vehicles
and stations where users, vehicles and stations are represented by JIAC agents. Ref. [
98
]
provides an MAS-simulation framework, NeSSi, using JIAC for interconnected power and
telecommunication networks.
Volttron [
99
] is another MAS platform specialized for energy system applications. It
was developed by the Pacific Northwest National Laboratory (PNNL). Volttron is open
source and developed in Python, but capable of also supporting agents written in other
languages. Communication is established through a central “MessageBus” in the form of
topics and subtopics. Its control architecture is modeled as a three-level hierarchy of agent
classes: cloud agent (publishing data to/from a remote platform), control agent (interacting
with the devices), and passive agent (interacting with the sensors and recording data).
Volttron has been applied to the integration of electric vehicles and distributed energy
generators in a Smartgrid [
100
]. Volttron has been used to develop the TNP system,
in which agents perform grid operational services such as demand response, fault detection
and energy transactions [
101
]. Volttron is also employed as the base of other energy
management platforms, like the BEMOSS system [102,103].
3.6. MAS Toolkits for Energy Management
In addition to the general purpose MAS programming languages, a number of out-
of-the-box, ready to use MAS toolkits for Smartgrid and energy management domains
have been created. Notable examples of such toolkits are GridAgent, HomeBots, IDAPS,
Ideas and PowerMatcher. These toolkits are based on the principles of MAS and implement
different types of agents. An overview and evaluation of these toolkits is provided in [
41
].
The majority of these toolkits include components related to the challenges of energy
trading based on dynamic pricing schemes. Agents revise their energy demand based
on prices and market transactions. The majority of these toolkits are also associated
with the direct management of physical entities for grid control (the only exception is
Homebots). Homebots [
104
,
105
] is a framework to manage distributed equipment in
a home environment. It involves a load-management system: each load (e.g., lights,
appliances) is represented by an autonomous agent with a utility function.
GridAgent [
88
,
106
] is designed to manage a network of Distributed Energy Resources,
transformers and secondary networks. Loads represent buildings, such as private homes and
factories. Agents are plug and play and they can schedule the charging of electri
c vehic
les.
IDAPS [
35
,
74
,
107
] has been developed by the Advanced Research Institute of Virginia
Polytechnic Institute and State University for the purpose of controlling a microgrid. Its
typical coverage is a residential distribution circuit with houses and transformers. Aside
from energy management and trade, another feature of IDAPS is that it can detect power
outages in the main grid and operate the microgrid autonomously in an islanded mode.
The IDEAS project [
108
,
109
] was developed for coordinating a large collection of
houses with a focus on demand side management. Market prices are predicted daily and
the adaptive mechanism reschedules deferrable loads (e.g., laundry machine, dishwasher)
based on predicted market prices. The demand-management model also optimizes the
non-deferrable loads, such as thermal heating of the house, to minimize the overall energy
cost. IDEAS supports the concept of Virtual Power Plants for a cost-efficient way to
incorporate Distributed Energy Resources among houses. VPPs enable electricity producers
to participate in the energy market and trade energy with consumers on a daily basis.
Energies 2024,17, 3620 13 of 61
PowerMatcher [
110
,
111
] was developed as part of the Smartgrid European FP7 project.
PowerMatcher supports a coordination mechanism to balance demand and supply in a
multi-microgrid environment with Distributed Energy Resources. The grid is considered
as a collection of smaller Virtual Power Plants. In PowerMatcher cluster, the agents are
organized as a logical tree. The auctioneer agent is at the root node and local device
agents are at the leaf nodes. This system has been installed in three pilot regions in The
Netherlands, Germany and Greece.
3.7. MAS Standards and Protocols
Standards are required for the development, communication, interoperability and
integration of MAS systems. In addition, common languages and protocols are required
for agents to communicate. The IEEE standards committee has identified the challenge of
interoperable protocols and data formats and stated that open communication between
smart devices using common protocols is crucial to interoperability [23,112].
An IEEE Computer Society organization FIPA (Foundation for Intelligent Physical
Agents) has established standards for the abstract architecture of MAS (the entities and
the environment), agent management specifications, agent communication language and
message transport protocol [
113
,
114
]. FIPA standards have become the de facto standards
used by MAS developers in the engineering and computer science communities [
115
]. The
FIPA standard identifies two core services: the AMS (agent management service), which
maintains a directory of registered agents, and the DF (directory facilitator), which provides
a searchable directory of services that agents offer to other agents. Thus, agents do not
need to be hard coded with the contact details of other agents whose services they need;
instead, agents can search the DF for the currently available providers of services. Detailed
information about FIPA can be found in [23].
FIPA provides a standard communication language called ACL (Agent Communica-
tion Language) [
116
], defining the communication protocols among agents and the meaning
of messages. The standard described by ACL only defines the structure of messages and
interactions, not the actual content of messages and vocabulary used by agents. ACL has
origins in another communication protocol, the Knowledge Query and Manipulation
Language (KQML) [
117
]. KQML is a language and protocol for communication between
software agents and knowledge-based systems, proposed in the early 1990s as part of a
DARPA effort.
Special to energy generation and distribution systems, some standards have been
developed to promote coordination between devices, communication architecture and
integration of Distributed Energy Resources [
112
]. IEEE 2030-2011 is one of the most widely
adopted standards to support Smartgrid architectures [
118
]. IEEE 2030 classifies Smartgrid
communication architectures into three subgroups:
• Home area network (HAN);
• Neighborhood area network (NAN);
• Wide area network (WAN). [119].
Typically, a WAN covers a distance greater than 10 Km, a NAN covers a distance
between 100 m and 10 Km and a HAN covers a distance smaller than 100 m. In the
context of an electric grid, a WAN involves the components associated with bulk generation
and transmission (power stations, PMUs, protection and control units, several NANs),
while NANs address the needs of electric distribution (DERs, protection and control unit,
multiple HANs) and HANs support the individual consumers/prosumers (smart meters,
EVs, sensors). All the communication technologies are facilitated by both wired and
wireless media.
Another institution, the IEC (International Electro-technical Committee), has devel-
oped standards for electric substations, process automation and information transfer in
power systems. IEC 60870 [
120
,
121
] and DNP3 [
122
] describe the standards for com-
munication protocols and process automation in SCADA systems. The IEC 61850 stan-
dard [
123
,
124
] is about the design of the automation system for an electric substation and it
Energies 2024,17, 3620 14 of 61
includes communication protocols between power plants based on Ethernet. IEC 61850
is object-oriented and splits a physical device into logical devices, which can be further
divided into logical nodes, data objects and data attributes. The IEC 61850 communication
architecture consists of three levels: station level, bay level and process level. IEC 61850
specifies communication protocols for the client–server-based system (SCADA) and also
for the publisher–subscriber system [
125
]. For the client–server architecture, the ACSI
protocol is used whereas for the publisher-subscriber, GOOSE and GSSE protocols are
used for communication [
126
]. An application of multi-agent-based Smartgrid automation
architecture employing IEC 61850/61499 intelligent logical nodes is described in [127].
Other standards proposed by IEC for power systems are IEC 61970, for interfaces with
energy management systems and IEC 61968 for interfacing the main applications for electrical
distribution in a utility [
128
]. IEC 61970 describes a Common Information Model (CIM) [
129
]
which defines how application software can exchange information about the configuration and
status of an electrical network. CIM provides a structured class hierarchy in the description
of power system plants and topology. CIM is a three-layer domain model; it defines a
common vocabulary to describe the basic components used in electricity transportation and
distribution. CIM aims to facilitate power-management processes, e.g., outage management,
asset management and customer information management [130,131].
MultiSpeak [
132
] is another standard designed for interfacing software applications
of utilities and devices in the grid. It defines common data semantics (in XML format),
the message structure for data exchange and the messages required to support specific
business process steps. Although CIM and MultiSpeak are similar, the latter is more
oriented towards the distribution of energy.
As for the integration of distributed energy-generation sources into a Smartgrid, the tech-
nical requirements are provided by the IEEE 1547.1, 1547.2, 1547.13 standards [
133
,
134
].
These standards involve monitoring and controlling interconnected distributed resources,
information transmission and test procedures for equipment.
Given the multiplicity and diversity of standards, operating a Smartgrid using an
MAS-based approach requires a system compatible with these standards. Achieving a
coherent system respecting these specifications and harmonization has been the subject of
several proposals [135,136].
3.8. Ontology for Energy Domains
For the successful operation of an MAS, a knowledge base or ontology may be nec-
essary. An ontology is a formal representation of knowledge, under the form of a set of
concepts and of relationships between such concepts. In the Artificial Intelligence com-
munity, ontologies describe entities and their properties, relationships, constraints and
behavior that are not only machine-readable but also machine-understandable [
137
,
138
].
According to [
139
], the functions of ontology are communication, interoperation and acqui-
sition, reuse and sharing of knowledge.
Some ontologies have been specifically developed for Smartgrids and the energy
domain. FIPA-SL is an ontology standard [
140
] adopted by FIPA and includes three types
of elements: concepts (components, data types), agent actions and predicates. FIPA-SL
includes predicates, relations, actions suitable for the energy domain such as microgrid,
substation, DER, load, storage and their status [
112
,
141
]. This ontology can be used
by agents for exchanging information, asking questions and requesting execution of an
action [
142
]. Ref. [
143
] constructed an ontology for the multi-agent-based electricity market
and [
144
] constructed an ontology for the customer portfolio. Kofler et al. [
145
] develop an
ontology for the energy efficiency in a smart home. The ontology involves representation of
home facilities, their energy demand and supply and use cases. Information about energy
type, cost, tariff and provider are also included.
The literature has also provided ontologies more specifically focused on the manage-
ment of electricity markets. For example, Ref. [
146
] describes an ontology named Electricity
Market Ontology (ELMO). This ontology was developed primarily for the electricity market
Energies 2024,17, 3620 15 of 61
of Greece. ELMO uses a multi-layered architecture divided into extendible and reusable
modules. These modules can be used by organizations or transmission system operators.
Santos et al. [
147
] also present an Electricity Market Ontology (EMO). The EMO is an upper
ontology for the electricity market, from which other low-level ontologies can be extended.
In particular, ontologies for the EPEX [
148
] and Nord Pool spot market [
149
] are developed
as extensions of EMO.
4. Energy Markets and Trade
Energy markets serve the function of allocating electricity between agents in the
context of an energy economy (Table 2). For this, we need to design a medium and
exchange mechanisms for prosumer agents to exchange energy with each other. Intuitively,
producers, consumers and prosumers in the Smartgrid trade with each other, in order
to balance energy demand and supply. Researchers have employed different methods
and MAS structures for energy trade: auction, contract network, negotiation (bargaining)
and non-market methods.
Table 2. MAS-based approaches to energy markets and trade.
Problem Proposed Methods
Design market and trade models Auction [81,87,150–160], contract networks [89,161–163],
for microgrid negotiation (bargaining) [164,165],
non-market methods [35,166–168]
Multi-microgrids and large scale Extended models [81,157,159,160,168,169]
Demand and supply forecasting Neural networks [170,171], historical data [172],
support vector machine [151]
4.1. Microgrid Level
One method of allocating energy among agents in a microgrid is through auction.
In auctions, agents determine the quantity of energy to buy/sell and then submit their
bids. Auction models for electricity markets are typically double-sided with multiple
buyers and multiple sellers. The type of auction can be an ascending auction, sealed bid,
Vickrey, English, Dutch or others [
150
,
151
]. Auction models mostly have a centralized
structure [
151
–
154
] with an auctioneer agent, consumer agents and producer agents. The
auctioneer agent collects bids and announces the auction result. Note that an auction may
involve multiple rounds or iterations until the market clears [
81
,
153
,
154
]. Excess or less
energy can be sold or bought from the main grid, respectively.
As an alternative to the centralized auction models, Ref. [
87
] describes a hierarchical
MAS structure with market clearing agents, utility grid agents, coordination agents and aux-
iliary agents at different levels to realize the auction. Ref. [
173
] explores a two-layered
hierarchical MAS for auctions, where the first layer is responsible for the bid evaluation
and the swap operation across agents while the second layer performs the consensus pro-
cedure. Among decentralized auction models, Ref. [
151
] explores a peer-to-peer bidding
model between the generator and the load agents. Another example of decentralized
auction models is provided in [
150
], where the generator agents can act as auctioneers
and sell auctioned energy to the participating load agents; alternatively, load agents can
act as auctioneers and conduct auctioning to secure the right of deriving energy from the
participating generator agents. Auctions can also take place in hybrid MAS architecture,
such as holons [
155
]. Each holon (a group of generator and load agents) participates in the
auction as a whole and holons can change dynamically over time. In the prosumer setting,
an agent may switch its role from seller to buyer in auctions and vice versa. Consumer
agents can also decide whether some part of their demand can be dispatchable (shiftable to
later periods) [
87
,
152
]. Auctions can be performed to trade energy on a daily basis [
156
,
157
],
hourly basis [
151
], 15 min intervals [
81
] or continuously [
158
]. Some auction schemes have
a day-ahead market and a real-time market [159].
Energies 2024,17, 3620 16 of 61
There exist variations of auction and bidding mechanisms in the literature. Ref. [
153
]
explores risk-based auctions, where a buyer or seller agent updates its target bid price at
every round using its own risk model. Ref. [
156
] offers a complex bid model for non-critical
(dispatchable) loads. The bid involves parameters like the earliest start time point, the latest
finish time, the (minimum) duration that the device will be switched on and the power
rating of the device. In [
160
], the modified auction scheme involves incentive compatibility
and rationality conditions for agents.
MASs have also been used in contract networks for energy trading [
89
,
161
–
163
].
Contract networks are for next-hour or day-ahead trading and the price is fixed. Seller
agents offer alternative contracts where the energy is sold in batches and consumers select
among contracts. Contracts are offered and discussed in a decentralized structure. Namely,
the buyer and seller agents communicate and agree on the contract bilaterally. Agents can
make forward contracts as well as real-time contracts [163].
In negotiation models, producers and consumers bargain on the price of electricity,
with a negotiation deadline [
164
,
165
]. Ref. [
164
] develops a negotiation strategy between a
smart building and the main grid. Negotiation may span multiple rounds, the buyer and
the seller have adaptive ask price and bid price. The MAS structure for the building is
hierarchical with a negotiation agent, central coordinator agent, multiple local coordinator
agents and the load agent. The central coordinator agent is responsible for comfort opti-
mization; total energy demand of the building is determined by the local controller agents
and the load agent. Ref. [
165
] studies peer-to-peer negotiation between agents inside a
microgrid. At the beginning, the main grid operator agent announces the price for selling
and buying one unit of energy. Local generator and load agents set their initial buy/sell
price based on their operational costs and then the negotiation starts.
There are also non-market mechanisms for energy trade with multi-agent systems.
Consensus algorithms [
166
] have decentralized structure and agents communicate with
their neighbors to set up their energy demand. Ref. [
167
] develops a quantum-inspired
energy-allocation algorithm with multi-objective optimization. Their framework has three
levels of hierarchy, namely the coordination agent, the local controller agents and the
device controller agents. In [
168
], there is a public bulletin for bilateral energy exchange
and a mediator agent. The trading agents search each other via the posts in the bulletin.
A manual approach to energy management is provided by [
35
]. In their IDAPS system,
the human user agent enters the electricity price of different utility companies to a bulletin
board where the resource, load and storage agents can access.
4.2. Multi-Microgrids and Large Scale
Energy-trade models have been built for large and small scale grids. Refs. [
159
,
169
]
have a wholesale and retail sale market together. In these works, the MAS has two layers;
the upper layer includes aggregate agents such as generator companies, retailer agents
and deals with the wholesale market. The lower layer includes end-users and serves as
the retail electricity market. While [
159
] employs an auction to allocate energy, Ref. [
169
]
uses negotiation and bilateral contracts between the generator companies and retailers
and between retailers and end users. Refs. [
81
,
157
] propose auction mechanisms for
multiple microgrids and the main grid. Their MAS architecture is hierarchical with multiple
layers, in order to manage energy inside and across the microgrids. Each microgrid
optimizes its benefit and bids as a single unit in the market. Microgrids can also buy/sell
the less/excess energy from the main grid. Ref. [
168
] studies self organization of individual
agents to form a coalition. Agents (producer, consumer, storage) coordinate among each
other to form coalitions (associations) and enter the electricity spot market as a single
unit. This allows them to overcome capacity-related entry barriers and compete to increase
their benefit. Agents negotiate the potential value and surplus distribution during the
formation stage of the coalition. Ref. [
160
] proposes a modified auction with special
payment for energy-storage (battery) sharing among residential homes inside a community.
A residential unit determines the fraction of its storage to sell and the reservation price.
Energies 2024,17, 3620 17 of 61
The community is partitioned into blocks and each block has a shared facility controller
agent in the MAS hierarchy.
4.3. Demand and Supply Forecasting
We should note that the trade and demand response models of Smartgrid involve
energy demand and supply forecasting to estimate the prices and the energy consumption.
For example, Ref. [
170
] utilizes neural networks to predict the next-day load demand.
The output of the neural network forecaster is fed to a hierarchical MAS which performs
peak-load reduction. MAS consists of Load Management Agent (LMA), Feeder Agent (FA),
Distributed Generator Agent (DGA) and Demand Response Agent (DRA). A feeder agent
collects data from DGA and DRA, combines the DGA and DRA capacities and sends the
available load reduction and its price to LMA. LMA decides to dispatch the load reduction
based on the forecasted demand, FA data and the main grid price. Ref. [
171
] uses neural
network to forecast the load demand of a Virtual Power Plant. Ref. [
172
] implements day-
ahead and next-hour forecasting based on historical data. Based on prediction, prosumer
agents in MAS cooperate with each other to optimize their profit or cost. Ref. [
151
] tests
various forecasting algorithms such as weighted average of previous periods or support
vector machine for individual generator and load agents.
Challenges and Open Problems: Authors have designed energy-trade mechanisms
for sharing energy between prosumers in the stochastic environment of Smartgrids. Namely,
agent-based energy market and non-market models for both microgrid level and network
of microgrids have been constructed. In this manner, the produced energy can be allocated
among the prosumer agents in the Smartgrid.
The main issue in the above energy-trade models is that they do not consider network
hierarchy or geographical proximity. In particular, a Smartgrid has a hierarchical structure:
houses, buildings, microgrids and multi-microgrids. Normally, agents in the same house
share energy; thus, they should enter an auction as a single entity, rather than individual
agents. The same idea can be applied to the microgrid level, namely each microgrid enters
the market as a single unit and then distributes the net energy inside it. As such, the energy
trade and allocation becomes a hierarchical problem: trade across entities (house, microgrid)
and then trade inside the entity. A related issue which has not been addressed in the trade
models is the geographical proximity. It is better to trade energy with a neighbor agent
rather than another agent further away, in order to mitigate transmission loss and network
traffic. This might be achieved by introducing a cost proportional to the distance or block
in the auction mechanism.
Another critical factor missing in the existing market models is communication. Agents
can communicate and exchange information with each other about operation time, load
amount before the auction, negotiation or contract, in order to improve the outcome and
convergence speed. In particular, the neighbor agents should communicate with each other
to trade. The information-exchange procedure will require the design of a communication
topology and order.
Energy markets operate based on predicted renewable energy generation and pre-
dicted demand. Then, the question is what would happen if actual energy supply is
less than demand. To deal with this case, alternative ready-to-use strategies should be
developed such as secondary market to switch energy among consumers, rescheduling
loads inside the day, pricing of energy from storage elements (batteries) and diesel gen-
erators. These strategies should be investigated in more detail to enhance operation of
energy markets.
The existing energy-trade models assume that agents know or predict their energy
supply/demand and enter the auction. It would be interesting to study a problem where a
human agent can enter the auction and submit bids and quantity of energy manually.
Energies 2024,17, 3620 18 of 61
5. Smartgrid Control and Management
The aim of Smartgrid control is to manage microgrid resources and elements in a
smooth and efficient manner. The control and management system must ensure stable
delivery of electrical power to consumers while optimizing energy usage towards additional
objectives such as maximum renewable energy utilization, minimum energy cost, weather
conditions, etc. This is a major problem of the Smartgrid due to the intermittent and
stochastic nature of renewable energy resources and dynamic energy demand from loads.
The main issues in Smartgrid control are securing critical loads, regulating frequency and
voltage during an outage, reactive power compensation and minimization of power loss.
Researchers have addressed Smartgrid control at different scales, which we revie
w b
elow
(see also Table 3).
Table 3. An overview of MAS-based methods for control and management.
Problem Proposed Methods
Home/building energy management Combined heat and power optimization [36,174–180]
Microgrid control in connected mode Centralized [181,182], decentralized [38,183–185],
hierarchical [186–191], reinforcement learning [192,193]
Microgrid control in islanded mode Centralized [194–196], decentralized [185,197],
hierarchical [190,191]
Transition between modes Securing critical loads [35,74,83,85,93,198],
voltage and frequency regulation [199–208]
Network of microgrids and buildings Decentralized network of buildings [209,210],
hierarchical MAS of microgrids [157,211,212],
decentralized (consensus or peer-to-peer) [213,214]
5.1. Smart Home and Building
Some studies deal with energy management on a small scale, i.e., homes, buildings
or factories [
36
,
174
–
176
]. These home or building energy management systems (Figure 4)
are oriented towards Combined Heat and Power (CHP) optimization, i.e., manage electricity
consumption, lighting, heating and cooling together. These systems generally exhibit a
centralized control or a layered MAS structure consisting of zone, floor, home and building.
The objective function typically involves multiple elements such as minimizing the cost of
energy, reducing emissions and maximizing the occupants’ comfort.
In [
175
], the central controller agent receives sensor data (temperature, humidity, occu-
pancy), decides on the amount of heating/cooling and sends commands to the actuators.
The controller agent is hybrid and employs reinforcement learning, Bayesian learning,
dynamic programming and a fuzzy logic kernel. Ref. [
36
] investigates energy management
of a self-sufficient smart home in islanded mode. Solar panel, wind turbines and storage
devices supply energy to the building. MAS is hierarchical and consists of a central coordi-
nator agent and multiple local controller agents. Users can specify their preferences and
thermal comfort, then the central coordinator agent uses particle swarm optimization (PSO)
to determine the best allocation of resources. The system sheds non-critical loads in case of
insufficient power supply. Ref. [
177
] creates an energy management algorithm for a smart
home in islanded mode. MAS handles the flow of energy between the energy resources
and the storage units. In particular, the supervisory agent charges or discharges the battery
depending on the surplus or shortage of power. In the Building Energy Management
System (BEMS) of [
178
], fuzzy logic rules are used to monitor and control the indoor energy
flow. The controller increases or decreases the power based on the current comfort level
and the desired comfort level. Ref. [
176
] constructs a CHP system where the electricity
agent, heating agent, cooling agent measure and keep track of the electricity and heat flow.
The agents request electricity, hot water, heat based on their needs and the central system
Energies 2024,17, 3620 19 of 61
controls the flow. The electricity agent continuously receives electricity prices from the
main grid and attempts to reduce the local demand.
Figure 4. Energy management of smart home.
Machine learning algorithms have been applied to solve the home energy management.
In [
179
], the photovoltaic panel output and the main grid prices are predicted by neural
networks. Reinforcement learning is used for hour-ahead energy-consumption decisions.
Ref. [
174
] proposes multi-agent reinforcement learning for an industrial site to manage on-
site energy generation, energy storage devices and transfer from the main grid. Ref. [
180
]
considers the problem of management of energy-storage devices at homes. Each household
is an agent, obtains its electricity from the main grid and wants to minimize its electricity
bill. Households have fixed energy demand profiles within a day but energy prices change
depending on demand and supply. Agents use a weighted moving average algorithm to
predict day-ahead market prices and make their best-response strategy in a game-theoretic
framework. Authors develop a novel learning mechanism for optimum storage charging.
5.2. Microgrid Level
At the scale of a microgrid, researchers have utilized MAS to control microgrid re-
sources and energy management in the connected mode and the islanded mode. Transition
from the connected mode to the islanded mode (and the reverse direction) is also part of
the microgrid control problem. The control process should be rapid, adaptable and reliable:
microgrid assets must be managed in real-time and protected during transitions.
5.2.1. Operation of Microgrid in Normal Mode (Connected/Islanded)
Refs. [
38
,
186
–
189
] study the control problem of the microgrid in the connected mode.
Ref. [
38
] constructs a decentralized MAS model with three types of agents: producer agent,
consumer agent, observer agent. Their setting involves renewable and non-renewable
resources (diesel generator), loads and batteries. A producer agent is assigned to each
energy resource; a consumer agent to a load; a producer and a consumer agent to a battery;
and observer agents to the AC Bus and PCC breaker nodes. MAS agents act independently
and they decide to buy/sell from local resources or from the main grid, depending on
the spot market price and local generation price. Refs. [
186
–
188
] consider a microgrid
consisting of two systems (department and hostel), each containing a Photo Voltaic (PV)
panel, wind turbine, local consumer, battery and a diesel generator. MAS is hierarchical and
contains a grid agent, a control agent and an individual agent for each device mentioned
above. They propose an intuitive algorithm for the control agent as follows. If the renewable
resource supply is insufficient, the controller obtains the required energy from the battery.
If the battery is depleted, the controller sheds the non-critical loads and obtains the energy
from the diesel generator or the main grid (the one with lower price). In [
181
], MAS is
centralized and consists of station agent, control agent, load agent, generation agent and
storage agent. The objective is cost minimization; the station agent performs optimization
Energies 2024,17, 3620 20 of 61
and supervision of the microgrid. Control agents function as a circuit breaker or a switching
agent. Excess/less energy is transferred to/from the battery or the main grid, depending on
the price. Ref. [
182
] also adopts a centralized microgrid control. The Smartgrid controller
agent (SGC) is located at the point of common coupling to the main grid; it handles the
energy transfer inside the microgrid as well as energy transfer from the main grid. Based
on their prediction, the load, generator, battery agents submit their power to buy/sell and
its price to the SGC, for the next period. The SGC decides the amount of power for each
agent and informs them. Ref. [
183
] handles electricity and heat management of a microgrid
using a decentralized MAS with generator agents, load agents, storage agents and thermal
agents. Microturbine agent generates electricity and heat together. Every agent targets
maximizing social welfare and they perform a convergence algorithm where the price and
energy demand are updated iteratively. In another decentralized model [
184
], agents do
not directly communicate between one another to prevent heavy traffic, instead they post
messages on a bulletin board (stigspace). There are resource agents, load agents and a
broker agent. The broker agent examines the summary data, market, grid information to
set a cap on the total power drawn from the main grid. The resource agents revise their
generation plans to satisfy the cap while adhering to their local constraints. Ref. [
189
]
proposes a hybrid, hierarchical MAS structure with three layers for microgrid control. The
upper layer works in a centralized manner for energy management, the upper layer agent
solves a multi-objective optimization (including cost of electricity, emission, line losses) by
particle swarm optimization. Several coordinated middle layer agents switch the operation
mode of individual generators depending on the surplus or shortage of energy. The lower
layer contains generator, load and storage agents. Generator agents also perform local
control by sharing active and reactive power among themselves.
Researchers have also adopted reinforcement learning methods to deal with the energy
management of the microgrid. In [
192
], there is only photovoltaic panel (with unpredictable
output) and battery as internal energy resources. They propose a supply control mechanism,
namely to learn charging and discharging of storage devices as a function of the system
load and available resources. The goal is to minimize the purchase of the additional energy
from the main grid. Ref. [
193
] adopts a fuzzy-logic controller to manage renewable, non-
renewable resources, storage devices and loads in the microgrid. Reinforcement learning is
used to train the fuzzy logic controller.
The control problem of a microgrid operating in islanded mode has been addressed
by [
194
–
196
]. These works explore a centralized MAS structure for microgrid control. In [
194
],
there are seven types of agents: Single Smartgrid Controller (SGC), Load Agents (LAGs),
a Wind Turbine Agent (WTAG), Photo-Voltaic Agents (PVAGs), a Micro-Hydro Turbine Agent
(MHTAG), Diesel Generator Agents (DGAGs) and a Battery Agent (BAG). The SGC agent
coordinates the negotiation process between the buyers and the sellers. Namely, it sorts
the producers with respect to the energy price and fulfills the demand based on the sorted
price. If the electricity generated from the renewable resources in the microgrid is insufficient,
the shortage is compensated by the diesel generators. Refs. [
195
,
196
] have no critical loads and
the central controller agent uses fuzzy decision making. In [
195
], the user agent, the control
agent (energy control center), the database agent, and the Distributed Energy Resources (DER)
agent works collaboratively to perform the assigned tasks. The ECC agent uses fuzzy logic
in such a way that, depending on the output of solar and wind generators, the circuit breakers
switch to the load or to the battery. Ref. [
196
] also includes a diesel generator to compensate
the energy demand. The fuzzy logic controller aims to fulfill the demand using solar and wind
generators and minimizes the use of diesel generators. Ref. [
197
] proposes a distributed MAS
for controlling a self-sufficient microgrid. Energy management is achieved by an incremental
cost-based consensus algorithm. Communication between agents is specified by a graph with
weighted edges. Agents communicate with each other according to the graph structure to
implement the consensus algorithm.
There are also studies which deal with the control problem of microgrids in both
connected and islanded modes. Ref. [
185
] proposes a decentralized MAS system with four
Energies 2024,17, 3620 21 of 61
types of agents: source agent, load agent, breaker agent and switch agent. Breaker and
switch agents connect or disconnect loads and batteries from the energy resources. They
study different operation modes where either the main utility grid or the renewable energy
serves as the primary power source or they are integrated. They also provide algorithms
for choosing the energy source and delivering energy to the loads in these different modes.
In a more sophisticated framework [
190
,
191
], the fuzzy decision maker agent and an
evolutionary particle swarm optimization agent work together to minimize the energy cost.
The System Operator agent performs negotiations and handles the energy exchange with
the main grid. MAS has a physical layer, a communication layer and a control layer and the
communication structure is a graph (social network of agents).
5.2.2. Operation during Transition Mode
Another critical problem in microgrid control is how to provide sound transition
of microgrids from the connected mode to the islanded mode when a fault or outage
occurs. Typically, there is an agent such as PCC agent [
83
] or central controller agent [
85
]
or main grid agent [
93
], who is responsible for monitoring the voltage, phase, frequency
and disconnecting the microgrid upon a fault in the main grid. This problem has several
aspects: managing the critical and non-critical loads, restoring frequency and voltage to
their normal values and reducing power loss. Researchers have addressed these problems
with different methods and MAS architectures. Refs. [
85
,
93
,
198
] have proposed a simple
solution with a centralized MAS structure. In this approach, the central control agent
disconnects all non-critical loads upon an outage. When the fault or outage at the main
grid is fixed, the microgrid makes a transition to the connected mode again and the non-
critical loads are reconnected. However, this method may cause under-utilization of energy
resources, since the available energy may still feed a portion of non-critical loads in the
islanded mode. To remedy this pitfall, the IDAPS system of [
35
,
74
] has a priority list of
loads and disconnects non-critical loads according to this priority, during transition to the
islanded mode. In [
83
], non-critical loads are disconnected in islanded mode only if the
total energy demand is greater than the renewable energy supply. The diesel operator
is started if the renewable energy supply is not sufficient to feed even the critical loads.
Namely, the control algorithm maximizes the renewable energy usage and minimizes the
operation of the diesel generator.
Another issue in transition from the connected to the islanded mode is stabilizing the
frequency and voltage and restoring them to their normal values. As for frequency and
voltage control, authors have mostly used decentralized MAS structure with peer-to-peer
communication [
199
–
202
]. Load agents communicate with neighbor agents to restore them
back to the nominal values. Refs. [
202
,
203
] have used consensus algorithms, while [
199
,
200
]
have used an input–output feedback linearization algorithm. Ref. [
201
] attempts distributed
voltage control by splitting the feeder into a series of overlapping segments. An agent in
each segment senses the voltage level in its own segment and exchanges this information
between the agents in the adjacent segments. Then, the agent formulates the reactive power
compensation sufficient to restore the voltage in that segment. Ref. [
204
] achieves voltage
regulation with multi-agent systems and expert systems. The expert system decreases or
increases voltage based on a set of conditional rules and electric circuit equations. In the
model of [
205
], two types of agents (the estimator agent and the control agent) collaborate
for frequency regulation. The first agent estimates the frequency-bias coefficients and
provides the area-error signal; the second agent compensates the power imbalance between
generations based on the error signal. Ref. [
206
] constructed a centralized MAS structure
for voltage control of a microgrid. The control agent (on-load tap changer (LTAC)) uses
fuzzy logic rules like predefined if–then–else rules, to change the load or transformer tap
and at the same time to prevent excessive tap operation. This allows LTAC to keep the
voltage of feeders in the standard range. Ref. [
207
] uses the graph-partitioning method to
regulate the voltage by the master and local control agents in the hierarchy, after a fault
occurrence. Ref. [
208
] focuses on restoring the frequency of storage devices in the microgrid
Energies 2024,17, 3620 22 of 61
to the reference range. The cooperative control system modifies the output power of storage
devices so that they reach a balanced energy state and maintain load sharing.
To minimize power loss in electrical systems, reactive power compensation is necessary.
Ref. [
56
] proposes a hierarchical MAS consisting of holons and a decomposition algorithm
for reactive power control. Ref. [
215
] tackles minimization of reactive power needed by
the distributed generators. This challenge is addressed by solving a linear programming
problem iteratively, which converges to the optimal solution. The contract net protocol is
used for distributed coordination and assigning moderator, monitor and dispatch roles to
the agents. Decentralized leaderless or leader–follower consensus algorithms have also
been applied to the reactive power compensation [216,217].
5.3. Network of Microgrids and Smart Buildings
The overall electricity-distribution system of a Smartgrid consists of the main grid,
multiple microgrids and the smart buildings which interact with each other and the main
grid. In this context, the objective of microgrids is to share energy and information with
each other to optimize their cost and energy need.
Ref. [
209
] presents a method for optimal power flow and energy sharing among
smart buildings. Each building is an MAS agent which contains a smart meter and other
agents. However, in their framework, houses only have batteries and no renewable energy
resources. Hence, buildings share energy from their batteries. The objective function is
the summation of cost curves of battery-storage systems subject to the active and reactive
power flow constraints. Ref. [
210
] constructs a decentralized MAS for multiple smart homes
in a neighborhood. Smart homes are equipped with a photovoltaic panel and battery and
can trade energy with each other. In this non-cooperative game, the objective of home
agents is to minimize their electric bill. Each home agent uses a genetic algorithm to
optimize its energy consumption and battery (dis)charge policy.
Multi-agent systems for management of multiple microgrids and the main grid gen-
erally have a hierarchical structure due to the nature of the grid. Ref. [
211
] deals with the
control of energy transfer between multiple microgrid systems. Each microgrid consists
of photovoltaic arrays (PVs), batteries as a storage units, a diesel generator and critical
and non-critical loads. MAS is decentralized and there is a grid agent which monitors
the main grid condition. Each microgrid includes a microgrid agent which can negotiate
energy exchange across other microgrids. The objective is to share energy against dynamic
load and generation conditions. Ref. [
212
] also considers energy sharing with multiple
microgrids and multiple substations. Energy sharing and payoff distribution is based on
coalition game theory with transferable utilities. Power flow is managed by a hierarchi-
cal MAS which comprises four types of agents, namely the Point of Common Coupling
(PCC) agents, the Grid Facilitator (GF) agents, the Grid Management (GM) agent and the
MicroGrid (MG) agents. The PCC agent coordinates the overall power operations between
the main grid and each linked microgrid. The MG agent is responsible for the power
balance of its own microgrid at the distribution level. Ref. [
157
] studies optimal energy
exchange between multiple microgrids and the main grid. There is a day-ahead energy
market for the integrated microgrids. Each microgrid bids for the energy to trade with other
microgrids or buys from the main grid. Ref. [
213
] has a different approach for management
of microgrids: a consensus algorithm for distributed coordinated control. Agents discover
global information by communicating with their neighbors. Each microgrid is operated
at an optimum economic point with respect to its incremental cost. Ref. [
214
] proposes a
multi-layer MAS architecture with peer-to-peer communication to control a network of
microgrids. Primary, secondary and tertiary control are realized at separate layers.
Challenges and Open Problems: Smartgrid control and energy management models
are designed to handle energy usage problems of home, microgrid and multi-microgrid
levels under given conditions, i.e., fixed price, network architecture. These models can
be used to manage Smartgrid operation during normal mode (connected or island) and
transition mode (during outage or reconnection). The system objectives in these models are
Energies 2024,17, 3620 23 of 61
maximum renewable resource utilization, minimizing energy cost, reducing non-renewable
energy usage and energy purchase from the main grid. Additional objectives are reactive
power compensation and voltage, and frequency stabilization.
One main open problem in Smartgrid management is how to handle control and
energy trade together in a unified framework. Energy market models determine the
electricity price assuming that the demand and supply are fixed. On the other hand, control
models assume the price is exogenous and optimize the consumption of agents. Ideally,
both price and quantity should be determined together in the market. There are some
preliminary attempts [
157
] to integrate Smartgrid control and energy trade, but this topic
has not been addressed in sufficient detail.
The advantage of MAS for Smartgrids is its decentralized nature, which is more reliable
and efficient. However, the existing control models for Smartgrids often have a centralized
structure. Therefore, future research should strive towards developing decentralized
control and management models.
In some MAS-based control models, the agents are not really autonomous or intelli-
gent. For example, there are circuit breaker or PCC agents whose task is just to execute the
commands sent to them by the central controller agent. A better alternative is a decentral-
ized MAS where the circuit breaker agents decide to turn on/off autonomously, based on
the outage or connection status of the microgrid. Likewise, for energy cost optimization
during the normal operation, agents should communicate and find the optimal schedule of
energy usage on their own. In this manner, the need for a central controller agent would
be alleviated.
It would be interesting to study Smartgrid control problems in an adaptive, dynamic
setting. Recall that a Smartgrid has a stochastic structure due to the weather-dependent
nature of resources and changing operation time of loads. Namely, agents’ quantity and
temporal schedule of production and consumption may change over time. Existing control
models often use simple methods like shedding non-critical loads when (renewable) energy
supply is insufficient. To enhance energy usage and prevent sudden interruption of load
devices, more intelligent algorithms can be developed, e.g., scheduling of loads over time
to reduce instantenous peak energy usage. For example, oven, laundry machine operation
and EV charging can be made sequential rather than simultaneous. This necessitates agents
to be more proactive in making plans and performing actions. Agents should inform each
other about the relevant variables and make new schedules accordingly. Thus, researchers
should deal with the dynamic control problem with more adaptive, intelligent agents.
In Smartgrid management, the presence and actions of the human user(s) have not
been considered in the control mechanisms. During normal operation, the user may
switch on/off some critical or non-critical loads depending on his need or other conditions.
Likewise, during an outage, the user may intentionally switch off appliances based on
his own priority at that moment, which may differ from the built-in priority list of the
Smartgrid control. Hence, Smartgrid management should integrate users’ preferences into
control and power optimization. If necessary, the control software should take the relevant
input from the user about the operation of the appliances.
6. Demand and Supply Management
A critical paradigm in Smartgrids is demand and supply management. These are tools
to assist energy management of Smartgrids and prevent energy shortage. Demand and
supply management assume the other side is fixed and optimize their own side by proper
scheduling, cost minimization and developing other necessary mechanisms. The goal is
to attain the utmost utilization of scarce energy resources and shape the load profile to
prevent overloading. A variety of approaches have been investigated (Table 4) and are
discussed next.
Energies 2024,17, 3620 24 of 61
Table 4. MAS-based solutions to demand and supply management.
Problem Proposed Methods
Supply side management Economic dispatch [173,191,218–224],
unit commitment [141,225–228]
Demand response (residential) Direct control [229–231], indirect control [232–234],
unified [235,236]
Demand response (smartgrid) Direct control [237–240], indirect control [170,241–246],
consensus algorithm [247]
Electric vehicle charge scheduling Centralized [248,249],
hierarchical cooperative [248,250–255],
hierarchical non-cooperative [256,257]
Electric vehicle to building Building consumption optimization [258],
unidirectional [259], bidirectional [260],
VPP [261], global objective [262]
6.1. Supply Side Management
Supply side management mainly deals with allocating the energy production among
generators to minimize the total cost of production. The aggregate demand of loads in the
microgrid is assumed to be fixed. This problem is also known as economic dispatch. MAS-
based solutions to the economic dispatch problem have a mainly decentralized structure:
generator agents need to communicate only with their neighbors. These models mostly
utilize a consensus algorithm in which agents propagate their incremental cost informa-
tion [
218
–
222
]. Ref. [
223
] studies energy cost minimization with generators and storage
devices. The solution method is again a decentralized consensus protocol between agents.
Alternatives to the consensus algorithm also exist. Ref. [
224
] uses a distributed gradient
algorithm to allocate output between generators. Each agent computes the gradient of its
local cost function and acquires the gradients from its neighbors. Then, the agent updates
its local generation according to a weighted sum of the gradient values. In [
173
], generator
agents engage in auction and exchange output power to minimize the cost of production.
In the approach of [
191
], a dedicated optimization agent uses particle swarm optimization
to solve the economic dispatch problem.
Another problem in the supply side management is the unit commitment: how to
schedule generation time period of distributed energy sources over a time horizon to fulfill
the demand. An example of MAS-based solutions to the unit commitment problem is
presented in [
141
]. In this paper, generator agents submit their bids to a pool to sell their
energy. Ref. [
225
] designs a rule-based algorithm for day-ahead and hour-ahead scheduling
of Distributed Energy Resources, considering dynamic market prices. The Microgrid
Management, Monitoring, Control (MMC) agent receives the forecasted price data from
the Distribution System Operator (DSO) agent and runs the day-ahead and hour-ahead
scheduling algorithms. Ref. [
226
] proposes a heuristic unit commitment algorithm for a
central controller. Storage units are also considered as an energy resource, in addition
to renewable and non-renewable generators. Each energy resource agent has a priority
and the controller assigns output to an agent from the list of available agents according
to their priority. If total local production is less than demand, energy is purchased from
the main grid at an external market. Reinforcement learning has also been applied to the
unit commitment problem in a decentralized cooperative setting [
227
,
228
]. Distributed
generator agents learn to satisfy the demand profile with minimum cost subject to the
constraints. Ref. [
228
] formulates the unit commitment problem as a Markov decision
process and employs multi-step deep reinforcement learning to solve it. A survey of
economic dispatch and unit commitment models using multi-agent systems can be found
at [25].
Energies 2024,17, 3620 25 of 61
6.2. Demand Side Management
Demand side management (DSM) or demand response refers to the act of postponing
or rescheduling loads and devices in order to balance the aggregate energy supply and
the aggregate energy demand in the Smartgrid. Recall that the outputs of renewable
energy sources are also time-dependent due to weather conditions. During the day, there
are periods like morning and evening when residential energy consumption tends to be
high, so-called peak demand periods. In the case of peak demand, local renewable energy
generation is often insufficient; hence, transfer from the main grid or operation of expensive
resources, like diesel generators, might be necessary. Consequently, peak demand may also
impact power plant capacity and cause transmission line congestion and infrastructure
damage. For optimal utilization of renewable resources, an efficient demand response
mechanism should be designed in order to shift the peak energy usage to periods of low
demand or to the periods of high availability of renewable energy. Thus, consumers may
need to reschedule their non-critical loads accordingly.
Approaches to demand response are broadly divided into incentive-based programs
and time-based programs [
263
]. The incentive-based programs are usually more suitable
for industrial facilities and time-based programs are more suitable for residential users. In
both types, the day is divided into a number of periods and the goal is to attain relatively
smooth energy usage.
The focal problem in demand response programs is how to coordinate energy con-
sumption of independent agents in order to alleviate the peak demand problem. For this
purpose, authors have mainly used hierarchical or decentralized MAS structures. A simple
electric price tariff is typically not sufficient to prevent peak demand because the tariff
scheme just shifts peak demand to the beginning of the low price period. Several demand
response strategies have been proposed in the literature: peak shaving, valley filling, load
shifting, strategic conservation, strategic load growth and flexible load shape [
264
,
265
].
These strategies only deal with how to allocate energy usage across various time periods
inside the day, but not how to coordinate the individual agents or give incentive to them. In
order to achieve coordination among users, researchers have developed direct and indirect
load control methods. Direct load control attempts to schedule loads of each consumer to
smooth energy usage. Centralized direct load control achieves optimal demand scheduling;
however, it requires perfect and complete information. Indirect control methods try to
adjust demand using price signalling or market mechanisms.
MAS-based demand response methods have been applied to Smartgrids, including
residential load scheduling and electric vehicle charging.
6.2.1. Residential Demand Response
Residential Demand Response deals with management and scheduling of home ap-
pliances; some models may also involve heating/cooling equipment or (hybrid) electric
vehicles. In these models, shedding of loads is based on their priority and they use machine
learning or other algorithms for scheduling.
Ref. [
266
] introduces the Smart Home Device Scheduling (SHDS) problem, where
each device at home is an individual agent. The authors model SHDS as a distributed
constraint-optimization problem under real-time pricing schemes of the electric company. A
distributed local search algorithm is developed to find the locally optimal DCOP solutions.
Among direct load control approaches, Ref. [
229
] studies demand side management
for Home Energy Management System (HEMS). The MAS behind such a model is hierar-
chical and consists of an HEMS agent, a Demand Side Management agent and an agent
for each home appliance, at consecutive layers. Each appliance has a priority level and
the DSM agent uses an hourly based algorithm which turns off home appliances based
on their priority when the energy supply is less than the demand. In [
230
], the Energy
Management System (EMS) agent at each house employs reinforcement learning to sched-
ule the operating time of the devices. At the beginning of each period, the EMS agent
receives the current price level from the utility grid and the energy request from devices.
Energies 2024,17, 3620 26 of 61
Ref. [
231
] tackles the problem of combined optimization of distributed energy-generation
management and demand response for residential agents and electric vehicles (EVs) in a
microgrid. Grid agents, control agents and residential agents form the hierarchical MAS
from top to bottom layer. The grid agent compares the total on-site energy generation,
battery storage and aggregate residential demand and informs the control agent to shift the
interruptable and deferrable loads. The control agent receives the current load requirement
from each residential agent and the charge status of the EV agents. Depending on the
electric price of the main utility grid, the control agent runs a demand response algorithm
every hour which shifts loads or EV charging to off-peak periods, based on their priority.
As for indirect control approaches, imposing time-dependent price is a common
approach to shape residential demand. Ref. [
232
] considers the demand management of
N
houses each having
M
appliances. The MAS is decentralized; there is a grid operator agent
and several household agents. The grid agent implements a real-time pricing scheme which
depends on prices in the previous periods. The house agents utilize reinforcement learning
methods to schedule their appliances and minimize their energy cost under such dynamic
pricing schemes. Ref. [
233
] studies scheduling of residential loads and electric vehicle
charging together with a decentralized MAS. Every appliance is an agent and fed from the
main grid via the transformer. The transformer agent predicts the total load and the price
for the next 24 h and sends predicted and current load and price information to each load
and EV agent. Each load agent employs reinforcement learning to schedule their loads and
minimize their costs, subject to the constraint of not overloading the transformer. Ref. [
267
]
has a layered MAS with a market layer, an energy management layer and a household
layer. Generator agents offer their prices; load agents evaluate and accept them. In order to
minimize the total energy cost, device agents at houses utilize distributed, cooperative ant
colony optimization to schedule their energy consumption.
Market mechanisms have also been used for residential demand response. Ref. [
234
]
employs a market mechanism for demand management of a home. Load, storage and
generator agents send their bids to an aggregator agent at an upper layer in the hierarchy.
The aggregator compiles those bids and forwards to the central market. In the central
market, an equilibrium price is calculated and returned to the agents. Shiftable load agents
compute their dynamic threshold value, which depends on historical price information
and decide whether to start the device or not, by comparing the market price to the
threshold value.
The literature has also addressed the challenge of exploring supply side and demand
side management of smart homes together in a unified framework [
235
,
236
]. In [
236
], home,
building and the grid constitute different levels of the architecture. Both the home agent
and the building agent have their demand side and supply side management algorithms.
In [
235
], the HEMS agent is at the upper level of the MAS hierarchy and manages the SSM
agent and the DSM agent. The SSM agent manages the power flow from the electrical
supplies i.e., the main power grid (MG), electric vehicle (EV), solar panel and energy
storage. The DSM agent controls home appliances according to their priorities.
6.2.2. Demand Management of Smartgrids
Both direct and indirect load-control approaches have been proposed for the demand
management of microgrids and Smartgrids. As for direct load-control methods, Ref. [
237
]
studies optimal scheduling and shedding of loads and storage devices in the Smartgrid.
MAS consists of generation agents, load agents, energy market agents (EMAs), auxiliary
energy resource management agents (AMAs) and energy storage agents (ESAs). Generator
and load agents publish their day-ahead forecast data to the EMA which computes the
power imbalance and estimates the market clearing price. Upon receiving the forecasted
mismatch and prices, AMA performs direct load control; it computes the optimal operating
schedule of storage devices and non-critical loads by a genetic algorithm. Ref. [
238
] devel-
ops a load-shedding method for Smartgrids which is composed of subareas. The mediator
agent, which is at the highest layer, computes the total power imbalance and distributes
Energies 2024,17, 3620 27 of 61
the imbalance among the areas. The generation and load agents are at the lowest layer
and communicate with their area agents. The amount of demand to shed is transmitted
to the area agent who distributes it among its load agents. Ref. [
239
] considers demand
response of prosumers in the Smartgrid, which includes renewable and non-renewable
energy resources and storage devices. Decentralized MAS contains load-forecasting agents,
load agents, generator agents, market agents, distribution-management system agents
and demand side management (DSM) agents. The DSM agent organizes an electronic
auction platform to allocate the generated renewable energy between the agents. In cases
of energy shortage, the DSM agent can utilize two possible algorithms:
•
The shifting algorithm schedules the loads in order to bring the total load-consumption
curve close to the objective load-consumption curve, which is inversely proportional
to the market price at the main grid. Based on the demand forecast and auction result,
the DSM agent decides whether to defer non-critical loads to other time slots.
•
The load-curtailment algorithm charges the batteries or starts the diesel generators
depending on surplus or shortage of energy.
In [
240
], the Master agent coordinates the individual load/EV agents for shifting
demand to the most convenient hours. At each iteration, the load and EV agents solve an
optimization problem which involves own energy cost and a global objective to reduce the
peak demand. The Master agent and the individual agents iteratively exchange information
until the schedule converges.
A variety of indirect control approaches have been studied, such as price signaling,
auctions, market and contracts. Ref. [
241
] adopts price signal to reduce the peak demand.
Day-ahead demand is forecasted by a neural network using historical data. Load man-
agement agents (LMAs), zone agents (ZAs), load agents and generator agents form a
hierarchical MAS. Based on the forecasted load, the LMA identifies the peak demand level
and its time. The LMA solves an optimization problem to allocate the load and energy
generation over time, in order to reduce the peak demand at the minimum cost. This
allows the LMA agent to send the operation time to the load and generation agents via the
zone agent. In [
170
], a network feeder agent aims to reduce its load and sends the price of
available load reduction to the load-management agent. The network feeder agent receives
the generation capacity and the price from the distributed generator agent and receives the
demand-reduction capacity from the demand response agent. Then, the load-management
agent dispatches the load reduction between different feeders based on the forecasted load,
the price of each FA and the main grid. Ref. [
242
] proposes a signaling scheme for demand
management at the substation level. There is a physical layer and a logical (communication)
layer. The MAS hierarchy includes an agent for physical layer, an agent for logical layer
and an agent for each house. The control unit at the substation calibrates itself and sends
an on-peak message to the house agents. A house agent reacts to this signal by calculating
its own power flow factor and brings its power consumption to within allowable limits.
Auction and markets are also possible methods to allocate or exchange demand.
In [
243
], there is a decentralized market for trading demand response exchange. The
market participants (aggregator agents, retailer agents, distributor agents) engage in an
auction with multiple rounds. DR buyers are willing to pay for demand while DR sellers
have the capacity to curtail their loads to supply demand on request. The market operator
agent adjusts DR prices iteratively in a prescribed way until convergence. Ref. [
244
] adopts
a decentralized MAS where each house is an agent. In their setting, there are multiple
houses and critical and non-critical loads. They assume that outputs of photovoltaic panels
and wind generators are constant over days, so forecasting is not necessary. Households
buy electricity from the main grid at the market price for critical loads. Finally, they
engage in auctions among each other to allocate their renewable energy production to feed
the non-critical loads. The house agents use reinforcement learning to determine their
bidding function.
Another approach for demand response management is offering contracts to con-
sumer agents [
245
,
246
]. In [
246
], the Curtailment Service Provider (CSP) agent offers
Energies 2024,17, 3620 28 of 61
load-curtailment contracts or direct load-control contracts. In the first type of contract,
consumers receive a payment for the amount of load curtailed. In the second option,
the CSP can cut or reduce directly the consumer loads only with a prior notification. Con-
sumer agents’ participation in the demand response program is voluntary and depends on
the contract. Ref. [
245
] studies contract design for the demand response. Every contract
proposal includes time periods and rewards to the consumer for up/down capacity. In this
model, the aggregator agent offers contracts, then the agents engage in a cooperative game
to form coalitions.
Decentralized consensus algorithms have also been applied to the demand side man-
agement [
247
]. Agents only communicate with their immediate neighbors. Through
information exchange, they discover the total net active power for the decision making
about load shedding. When an agent decides to shed its load, it signs the corresponding
element of a vector and shares with other agents to achieve synchronization.
6.2.3. Electric Vehicles
Electric vehicles (EVs) are becoming more and more popular since their prices are
decreasing and their fuel cost is lower compared to the conventional vehicles. In addition,
electric vehicles do not emit greenhouse gases and are environmentally friendly. However,
the issue with electric vehicles is how to schedule their charging times. Electric cars
have large battery capacity and, during charging, they draw an electric power which is
almost an average house. Due to their high power requirement, uncoordinated charging
may cause peak demand which overloads the transformer and damages the electricity
infrastructure [
31
]. Hence, coordinating electric vehicle charging is a critical problem in the
near future.
Multi-agent systems have also been utilized by researchers to address the EV charging
problem. These models have centralized or hierarchical MAS structure, the majority
being the latter one. In the centralized models [
248
,
249
], a central coordinator agent
obtains information from all EVs in each period and computes the optimum charging
schedule to minimize the total cost of charging. As for optimization method, Ref. [
249
] uses
quadratic programming while [
248
] uses evolutionary algorithm and linear programming.
However, centralized scheduling is not practical since it needs perfect departure and arrival
information from every vehicle inside the day. Furthermore, the centralized model works
efficiently for a limited number of EVs and is not useful in a large fleet.
Hierarchical models have layers and agents defined by the geographical zones and
the electrical distribution system. Namely, the multi-agent system includes EV agent,
area/region agent, transformer agent, line/grid agent and distribution system operator
agent. Most hierarchical MAS models are cooperative [
250
–
254
], though non-cooperative
models also exist [256,257]. In cooperative models, EV agents share their information and
cooperate to schedule their charging periods whereas in non-cooperative models agents
decide their charging schedule on their own. Both models require tentative charging
plans of EV agents across time periods to estimate the demand and take measure against
overloading. Non-cooperative models utilize dynamic price signals to prevent overloading
and indirectly adjust agents’ charging schedules. EV agents decide their charging time
according to their state of charge and also the market price to minimize their cost. For
example, Ref. [
257
] assumes that each EV agent knows the mass charging behavior of other
EVs and they examine the Nash equilibrium of this non-cooperative game where agents
determine their charging periods based on the price control. In [
256
], MASs have two layers
of hierarchy; the upper layer is managed by the DSO agent and the lower layer is managed
by the fleet operator agent. In both layers, the market mechanism is used to allocate energy
across fleets and EVs. According to the market price at the lower layer, each EV computes
its optimal charging schedule.
In cooperative MASs, agents coordinate and arrange their schedule according to
the hierarchy. Typically, EV agents send their preliminary charging periods or preferred
charging periods during the day to the aggregator agent at the upper level. Depending
Energies 2024,17, 3620 29 of 61
on the hierarchy, the aggregator agent can be the fleet operator agent, the area agent or
the transformer agent. EV agents may also send their criticality or risk factors to the
aggregator agent which is calculated by their current state of charge, desired level of
charge, connection duration and departure time [
250
,
252
]. Then, the aggregator agent sums
the charging requests and calculates the total load in each period of the day. In cases of
overloading or imbalance of energy demand and supply, the aggregator agent revises the
charging periods of its associated EV agents. Namely, the aggregator computes a new
schedule by optimizing the total cost subject to the balance and capacity constraints. Various
algorithms have been proposed for computing optimum schedules such as exhaustive
search [
252
], reactive/proactive scheduling [
254
], mixed integer programming [
251
] and
optimization toolkits [
253
]. Some models [
252
,
253
] introduce more than two layers and
aggregation occurs at multiple levels. That is, EV loads are aggregated at the fleet/area level,
fleet loads are aggregated at MV/LV substation (transformer) level, MV/LV substation
loads are aggregated at LV/HV substation level and so on. In [
248
], EV charge demands
are aggregated at a single (transformer) level, but there are multiple transformer agents
at the same level. The transformer agents execute a negotiation algorithm to allocate
energy among each other and then distribute among their associated EV agents. In [
255
],
the EV demands are aggregated at the charging station level. Communication and control
signals are transferred between multiple layers (Distributed System Operator, Local Cluster,
EV aggregator).
An interesting application of electric vehicles is that they can be used as storage devices
which feed residential loads or transfer their energy to the grid. Ref. [
258
] investigates
combined heat and electricity consumption optimization of a building where EVs can
discharge to provide energy needs of home appliances.. In other words, EVs are not solely
a load to charge, but they can also discharge to fulfill the energy demand of buildings. In
their vehicle-to-building (V2B) model, the building energy management system (BEMS)
charges EVs with priority based on their state of charge and connection duration. Ref. [
260
]
considers a microgrid with bidirectional grid-to-vehicle and vehicle-to-grid services. The
number of EVs is fixed and they are connected to the grid for a certain time interval. A
decentralized consensus algorithm is proposed to schedule charging and discharging of
a population of EV agents in the grid. Ref. [
259
] studies unidirectional vehicle-to-grid
energy transfer. The authors argue that individual EVs have little power to contribute
to the grid, hence in their setting EV agents form a coalition or a Virtual Power Plant to
sell their energy in the electricity market. In [
261
], electric vehicles also form a VPP to
coordinate, but they have bidirectional energy transfer. They propose EV (dis)charging
algorithms for two different cases: Vehicles are managed by a central controller or they are
personally managed by their owners. In the first case, the objective of the central controller
is to minimize the purchased energy from the main grid. If the amount of renewable
energy generation is less than non-EV loads, then the shortage of energy is obtained from
electric vehicles. In the second case, the EV owner employs a mixed logit algorithm for
(dis)charging decision which accounts for individual cost and benefit of charging, time
spent, existing battery level and travel distance. In [
262
], EV agents decide their charging
or feeding strategy to optimize their own benefit and cost, but they also consider the global
objective. Each EV agent has a symmetric objective function which involves a penalty
for overloading the transmission lines based on the charging power and the predicted
non-EV demand.
Challenges and Open Problems: For supply side management, researchers have
studied economic dispatch and unit commitment and developed algorithms for these prob-
lems. As for demand management, researchers have utilized demand response strategies
(allocating demand across time periods), direct control (direct scheduling of loads), indirect
control (price signaling, market, auction, contract) for home and microgrid environment. In
addition, previous research have addressed charge scheduling of a group of electric vehicles
(cooperative, noncooperative methods) and constructed models for vehicle-to-building
energy transfer.
Energies 2024,17, 3620 30 of 61
In the literature, the demand response problem of the house and the microgrid have
been studied separately. The reason is that the household, as an entity, pays its own
electric bill. However, from an engineering perspective, demand management of the house
and the microgrid should be handled together. This can be achieved in a layered MAS
structure where agents cooperate to alleviate the peak demand. Thus, one direction for
future research is constructing agent-based systems and demand response programs for
the house and the microgrid.
We also note that the two tasks of energy trade and demand response are closely
related to each other. Therefore, we need MAS-based models which integrate energy market
and demand and supply management. On their own, demand and supply management
tools assume the other side is fixed and try to schedule only loads or generators. In a
unified framework, generators and loads should interact with each other and they should
be both scheduled dynamically. Another aspect of the problem is how to design the
MAS structure to coordinate various agents in the grid and at the same time to respect
geographical proximity.
Based on the fact that energy generation and energy usage of consumers are highly
volatile over time periods, the demand response problem is a continuous, dynamic problem.
One potential method to tackle this difficult problem is communication between agents
and adaptive scheduling. Namely, agents can post messages about the change in demand
and supply conditions.
As for Electric Vehicle battery charging, a decentralized MAS structure should be
designed due to the nature of the problem. Existing models assume that Electric Vehicles
can only be charged at home or office at a single duration. However, charging can be
interleaved in space and time. To deal with this discontinuous problem, an EV agent
should exchange information with other EV agents and also adaptively schedule charging
on its own.
7. Restoration and Self-Recovery
During normal operation of the electric grid, faults may happen, such as a component
failure, phase-to-earth contact and short circuit which affects stability and energy supply
to the rest of the system. Service restoration involves fault detection, location, isolation
of the faulted section from the network and restoring power to the de-energized areas. A
fault in energy-distribution systems might create other faults and outages in the network;
therefore, rapid restoration process is necessary for system resilience.
Automatic fault detection and prevention are not possible in the conventional grid
due to the lack of communication or information transfer [
268
]. This leads to unanticipated
power outages, thus a reliable power supply cannot be ensured to the end-users. Further-
more, the conventional large and centralized electric grid is prone to massive cascading
failures in the network when a single fault occurs in the transmission and/or distribution
lines. System restoration in the present electric grid is manual, the process is slow and
depends on the operator’s experience. Multi-agent systems provide an effective alternative
to this complex problem, thanks to their capability to communicate information and dis-
tribute tasks among agents. Thus, a major benefit of MAS for the future Smartgrid is the
automated self-healing capability.
The restoration problem in Smartgrid is a constrained optimization problem whose
objective is to provide electricity to as many loads in the rest of the network with available
sources until the fault is amended. The objective function may include priority of loads (i.e.,
critical or deferrable) and the optimization is subject to the network topology, quantity of
energy resources, transmission line capacity, voltage and frequency limits [
269
–
274
]. A new
configuration of energy supply to the loads is found by adjusting the switches and circuit
breakers. Hence, the restoration problem is essentially a combinatorial search problem to
find the optimum arrangement of switches and circuit breakers. Some restoration systems
aim to optimize a multi-objective function which also incorporates minimization of power
loss and minimization of switching operations [269,271,275–277].
Energies 2024,17, 3620 31 of 61
In the literature, various MAS and non-MAS-based automated restoration systems
have been constructed, a survey of these systems is provided in [
270
]. Non-MAS-based
restoration systems utilize centralized optimization with different techniques such as
knowledge-based systems, expert systems, heuristic search, tabu search, fuzzy logic, integer
programming, neural network, evolutionary algorithms and hybrid models. However,
centralized optimization is computationally expensive; in addition, a central restoration
scheme is susceptible to a single point of failure and not adaptive enough for a time-varying,
dynamic grid [278].
Multi-agent-based restoration systems exhibit a variety of MAS architectures (Table 5),
optimization methods and assumptions. The majority of these systems have a decentralized
or hierarchical MAS architecture and utilize communication, information sharing and
collaboration among agents for fault analysis and recovery.
Table 5. MAS-based solutions to restoration and recovery problems.
Problem Proposed Methods
Fault identification Centralized [279,280], rule-based [275,278,281,282],
and restoration proposal & negotiation [269,272,276,283,284], graph search [271,285],
consensus algorithms [286], reinforcement learning [274,287],
stochastic nogotiation [288], others [273,277,289–291]
In [
281
], fault identification and reconfiguration of the grid is rule-based. The generator
and load agents collect real-time voltage and current data; the line agent detects the fault
based on these data and commands the circuit breakers. Ref. [
282
] uses fuzzy logic for
self-healing. Prevention control agents collect system data (like voltage, current, power
level, erroneous component) and weather conditions and predict states that might lead to
failures. Response control agents analyze the state data and determine the malfunction in
the network. Then, the supervisor agent uses the state data and the evidence of faults to
populate the fuzzy logic rules. Examples of other rule-based systems are [275,278].
Some MAS models involve proposal and negotiation processes for service restoration.
That is, agent(s) in the faulty zone compute a reconfiguration plan and send the restoration
proposal to other agents. In [
283
], the bus agent of the de-energized zone communicates
with its adjacent bus agent to find the path to re-energize itself. The bus agent informs
the facilitator agent that its associated bus can be energized by the proposal of its adjacent
bus agent. Then, the facilitator agent updates the system topology accordingly. In [
284
],
every bus agent keeps track about the power flow situation in its neighborhood. In cases
of a line fault or load/supply change, the bus agents negotiate with each other and shed
some loads to prevent damage. Ref. [
272
] creates a MARS system which also employs
negotiation. Their framework includes four classes of agents: substation, feeder, branch
and equipment. Feeder agents propose restoration configurations of the available power
and negotiate among each other. The initiator feeder agent evaluates the proposals and
chooses the configuration with the highest power. In a similar setting [
276
], the initiator
feeder agent negotiates with other feeder agents using a contract net protocol. The initiator
feeder agent sends call-for-proposal messages and the respondent feeder agents reply with
their available remaining power capacity to the initiator agent. Afterwards, the initiator
decides on the zone combinations making use of expert-based rules extracted from the
past scenarios. In [
269
], the affected zone agents inform their feeder agent, who then
initiates restoration and starts negotiations with its neighboring feeder agents by sending a
call for proposal messages. After collecting data, this feeder agent finds a solution (local
optimum) and then requests neighbor feeder and zone agents to take control actions to
change the switches.
Graph-based search models have also been applied to restoration [
271
,
285
]. Ref. [
271
]
encodes the restoration problem as a maximum feasible network flow problem for a
microgrid in the connected mode or islanded mode. The electric grid is viewed as an
undirected weighted graph and an improved version of the shortest argument-path (SPA)
Energies 2024,17, 3620 32 of 61
algorithm is used to solve the network flow problem. Another graph-based modeling of
restoration is in [
285
]. The lower layer of the MAS includes DER and load agents, the upper
layer includes the switch and team agents. The team agents communicate with their peers
and implement a subgraph tree search to find the new network configuration.
As for other alternative approaches, Ref. [
289
] constructs a multi-agent system which
mimics the human immune system for self-healing. The agents in MAS and their functions
have a one-to-one analogy with the cells in the immunity system. The zone agent (antibody
B-cell) uses the Clonal selection block algorithm to find the fault and isolate it. In the self-
healing algorithm of [
290
], the fault-identification layer collects system data and notifies the
fault-diagnosis layer. The diagnosis layer searches for the fault identification and fetches a
solution from a list of fault-handling routines. Then, the corrective action layer is notified
about the solution to perform.
Reinforcement learning, especially Q-learning, has also been applied to solve the
restoration problem. The switch agent in [
274
] and the feeder agent in [
287
] learn the
optimal policy to reconfigure the network. Heuristic algorithms [
277
,
291
], consensus
algorithms [
286
] and expert systems [
273
] with decentralized MAS structure have also been
utilized for fault detection, isolation and network reconfiguration.
Note that decentralized restoration approaches may yield local optima rather than
global optimum. In order to escape from suboptimal configurations, as a partial remedy,
Ref. [
288
] offers stochastic negotiation which involve probabilistic expected utility. In their
framework, feeder agents represent a distributed energy source which has the ability to
restore a blackout region and zone agents represent a load zone which demands power be
restored. Feeder agents negotiate between each other and a feeder agent proposes candidate
target zone to deliver the power. Agents evaluate whether to accept the proposal of another
agent by using two alternative stochastic decision functions. This creates the possibility
of avoiding the trap of a local optima. Another attempt to reach a global optimum is by
dynamic team forming or reforming algorithm in [
292
]. Bus agents in the proactive and
reactive layer try to restore the physical bus by some predetermined countermeasure action
plans. If the countermeasure plans are not successful for restoration, i.e., the fault is serious,
then the bus agents delegate restoration to the social layer. The coordination agents in the
social layer form a team to restore the bus locally. The team size can be modified according
to the complexity of the fault.
Centralized MAS systems for restoration also exist. Ref. [
279
] offers a centralized
MAS structure which actuates an auto-reclosurer algorithm in order to restore power to
the unaffected parts of the grid. This algorithm distinguishes a temporary fault from a
permanent fault by performing multiple checks for the presence of the fault at several
periods. In [
280
], the microgrid operator agent (MOPA) collects data from equipment and
measurement agents such as current, voltage and impedance to perform fault analysis.
Based on these data, MOPA computes the final network configuration and the protection
configuration and then sends the configurations to the switcher and protection relay agents.
Some studies restrict attention to correcting one type of fault like phase-to-earth-
contact [
293
]. An identification method dedicated to this special type of fault is developed.
Node agents collect transient reactive power data and judge whether the feeder associated
with this node is faulty or not. The control agent receives the fault information from the
node agent, sends control commands to the node agents and starts the restoration process.
Until now, all restoration systems assume that there is only single fault in the network.
Refs. [
294
,
295
] deals with correcting multiple faults in the grid. In these approaches, each
fault is considered as a single fault and the solutions of individual faults are combined
to have the final reconfiguration. In [
295
], MAS is comprised of a management and
execution layer; the management layer is responsible for the recovery process. In [
294
],
fault detection is performed in a decentralized manner by the local agents (load, switch
agents) whereas the fault reconfiguration is performed by the global agent in a centralized
fashion. The reconfiguration algorithm of the global agent searches for the optimum
switching combination.
Energies 2024,17, 3620 33 of 61
Challenges and Open Problems: Previous works have studied fault detection, isola-
tion and reconfiguration problems and developed methods explained above: Rule-based,
fuzzy logic, proposal and negotiation, graph search, human immune system, reinforcement
learning, stochastic negotiation, team formation and other algorithms.
Nevertheless more robust and effective restoration methods are required to deal with
faults in Smartgrid. Accidents and failures can occur anytime and anywhere in the grid;
hence, fault detection requires continuous monitoring of the transmission line and electrical
elements. Ideally, a recovery plan should be developed before the faults occur. One benefit
of multi-agent systems for fault detection and recovery is local knowledge. Intelligent
and autonomous agents know the state of the neighbor agents and the local grid. Hence,
nearby agents can identify the fault and its location rapidly by sensing electrical values
and checking the statuses of each other. Moreover, small faults can be handled easily at the
local level without the need for large scale optimization. This feature of MAS systems has
not been utilized to a great extent for restoration purposes in the literature.
The existing restoration methods detect the fault and compute the optimum network
configuration in real time (online), which depends on the location and type of the fault.
Observe that the restoration problem does not have to be solved in real time; consequences
of each possible fault with a specific location and type are known beforehand; hence,
we can construct predefined logical rules and lookup tables to detect the fault, based on
abnormal electrical and spatial data. In a similar fashion, the optimal network configuration
can be precomputed for each possible fault location and type. The precomputation can
be performed by the system operator in the meta level; hence, a global optimum can be
achieved. Thus, local agents can pinpoint the location of the fault and then realize the
predefined restoration plans accordingly. How to implement these logical rules with an
agent-based system in a centralized or decentralized manner is part of future research.
Interestingly, previous research has not considered demand response or deferring
loads during fault and restoration. In our opinion, this is a critical issue because outage of
energy (at any time of the day) should also shape agents’ consumption profile and load
scheduling. In particular, non-critical load agents can shift their energy consumption to
later periods after recovery. Other agents can also reschedule their demand over time to
match available power by negotiating with each other. In cases of shortage of power or
urgency, non-renewable resources like diesel generators can be operated to feed the critical
loads or part of the affected zones. Note that for such a demand-management program,
all agents in the microgrid must be informed about the existence and location of the fault.
This can be accomplished by posting the fault event to a public bulletin which every agent
checks regularly. Thus, another open problem is how to notify agents and how to determine
and reschedule their energy-consumption profile upon a fault or outage in the system.
The case of multiple faults inside the network has not been studied in sufficient
detail by the existing research. In particular, determining whether there is a single fault or
multiple faults may require more sophisticated algorithms which include testing various
electrical conditions and checking signals and values. Note that those multiple faults are
likely to be at nearby locations; thus, neighbor agents should collaborate on detecting and
distinguishing the faults. Then, the problem is developing algorithms to coordinate the
agents for fault detection.
8. Protection and Security
A Smartgrid is a cyber-physical system which involves energy and information trans-
mission, and communication and software applications; moreover, it performs computa-
tionally intensive operations. Hence, a Smartgrid is highly susceptible to both physical
and cyber attacks. Physical threats include terrorism, natural disasters and industrial
catastrophes. Cyber threats include viruses, malicious software, false data injection, ma-
nipulation of components, denial of service, theft of private and financial data and attacks
against an agent or an application. Most agent-based systems designed for Smartgrids use
TCP/IP and the Internet for communication which increases the risk and vulnerability to
Energies 2024,17, 3620 34 of 61
cyber-attacks. Due to the multi-layered structure, the scope of cyber attacks can be at the
user, component, operating system, protocol or network level. The main responsibilities
of a security system are to detect the location and type of the attack, isolate the threat,
deploy preventive measures and coordinate among units and agents for stronger resilience,
relying on records of attacks and learning to adapt and improve the security system for
future threats. More detailed information about types of attacks and security issues in
energy-distribution networks can be found in [296,297].
Several security guidelines and requirements for Smartgrids have been developed;
an overview is provided by [
298
,
299
]. Some key guidelines are the European Network
and Information Security Agency (ENISA) Smartgrid Security report [
300
], the ENISA
guidelines for security measures [
301
] and the U.S. National Institute of Standards and
Technology report NISTIR-7628 [
302
]. The International Organization for Standardization
ISO/IEC TR 27019 report [
303
] provides security guidelines based on ISO/IEC 27002 for
process-control systems specific to the energy industry.
Note that there is always a trade-off between the extent of the security measures
and the computational overhead associated with it [
304
]. Therefore, installing security
components should be a wise decision which does not hamper the real-time operation.
In the security literature, it is well recognized that central management systems
are more vulnerable to cyber threats [
56
,
58
–
60
]. The primary concern is that the central
controller is an ideal target for a malicious attack; compromising just this agent will make
the whole system totally ineffective. On the other hand, a decentralized system is more
resilient to attacks thanks to its distributed intelligence, labor division and coordination.
For this reason, many Smartgrid-protection and security systems adopt decentralized
MAS architectures.
In the literature, multi-agent systems have been used to provide solutions to data
encryption and authentication, user privacy, threat detection, intrusion prevention and
system protection in the Smartgrid (Table 6). Researchers have utilized different measures
and policies to deal with the above security problems. For data encryption and authentica-
tion, Ref. [
304
] proposes a public key infrastructure (PKI) for the microgrid. The protocol
has a certificate-issuing process and a certificate-authentication process. The certificate
authority (CA) agent issues digital certificates to the user/device agents in the microgrid.
The validation authority (VA) agent verifies the validity of the certificate. By issuing and
validating digital certificates, an agent can authenticate the originality of the messages sent
by another agent.
Table 6. MAS-based solutions to security and protection of Smartgrids.
Problem Proposed Methods
Data encryption and authentication Public key infrastructure and digital certificate [304]
System protection and monitoring Structural measures [305], human immune system [306]
Data privacy Conditions on communication order and content [307]
Intrusion attacks Statistical anomaly detection [308–310],
machine learning [311,312],
consensus algorithms [312,313],
trust-based filtering [314–316]
False and malicious data Monitor state variables and control signals [317,318]
Ref. [
305
] investigates upper-level system protection and monitoring of Smartgrids.
Ref. [
305
] takes structural measures to prevent cyber attacks. The input and output format
of the data is specified for each agent and the tasks of agents are sorted in a queue. Their
approach uses continuous observation of the overall system. MAS is hierarchical and
involves a functional layer and a logical layer. The functional layer is responsible for
protection and task execution. Ref. [
306
] constructs an artificial immune system which
Energies 2024,17, 3620 35 of 61
mimics the human immune system. There are monitoring and forecasting agents which
can learn, predict and evaluate risks and detect dangerous deviations from the model state.
Ref. [
307
] studies protection of user data privacy. In their setting, prosumer agents
communicate the quantity of energy demand, supply and price information with the energy
management system (EMS) agent. To ensure privacy of prosumers, the authors impose
two conditions: no prosumer can enter into communication channels of another prosumer
and no prosumer can infer the consumption plan of any other prosumer from his received
information. To satisfy the first requirement, they take structural measures such that the
channels of prosumer agents are separated and agents cannot directly communicate with
one another. For the second requirement, the authors impose some constraints on the
content of the data sent by each prosumer.
There is also a great deal of work that tackles the intrusion attacks and false data
injection [
308
–
316
]. In these models, agents collect local data and exchange them with
their peers; the reported data from agents are inspected to detect attacks. Consequently,
most of these intrusion-prevention models have a decentralized MAS structure, but some
hierarchical MASs also exist [
309
]. The methods of attack detection are statistical data
analysis (anomaly or irregularity) [
308
–
310
], machine learning [
311
,
312
], consensus the-
ory [
312
,
313
] and trust-based filtering [
314
–
316
]. Among statistical data-analysis models,
in [
309
], the master agent (overseer) collects measurement of state variables from each agent
and identifies an anomaly, i.e., a compromised agent, using an algorithm which accounts for
the majority vote. Ref. [
308
] utilizes descriptive statistics to detect anomalies. First, energy
flow data are gathered from agents in the grid and the key characteristics of the data are
determined. Then, the profiling agent monitors the real-time incoming data in the grid and
detects an irregularity. Ref. [
310
] combines anomaly-based data analysis with model-based
probabilistic techniques for intrusion detection. MAS architecture is composed of three
layers: power, control and security. The security agents identify an intrusive event by
collecting data about system variables, CPU, memory usage and communication traffic.
Trust-based filtering methods assign a trust metric to each agent and data coming from
an agent are respected according to their trust value. Ref. [
315
] uses trust-based filtering for
robust estimation of state variables such as voltage magnitudes, angles, transformer ratios
and power flows. In their setting, every agent has a local estimate of the state variable and
updates the trust value of a neighbor agent based on the accuracy of the estimate reported
by that agent. The value of the state variable is calculated using the weighted estimates of
agents, where the weights correspond to the trust values. Ref. [
316
] also employs accuracy
of the reported state variable to update the trust value of an agent. The authors also
propose an additional data-retransmission scheme to combat cyber attacks: every load
and generator agent regularly transmits its set of previous observations (a digest update).
Control agents examine the history contained in the digest updates to reconstruct past
system states in order to maintain the situational awareness. Trust and data-retransmission
schemes can be combined. In [
314
], households, as prosumers, trade energy with each other
via blockchain technology and each household agent has a trust value. Agents evaluate the
trust of other agents and declare these trust values to their respective aggregator agents.
Both the evaluator and the evaluated agent are judged based on their honesty. They have
a repeated game theory setting and a defaulter agent may regain its trust by reporting
correct evaluations in the consecutive periods. Another trust-based security mechanism for
Smartgrids that uses blockchain technology is developed by [319].
Consensus theory can be also used for analyzing suspicious data flows and infection as
in [
313
]. In their ELVIRA framework, an agent iterates the state variable using the data com-
ing from its neighbors and the expression converges to the truth asymptotically. Hence, the
agent can compare this value to the collected data in order to identify a compromised agent.
Machine learning approaches have been devised to detect data anomaly and intrusion
attacks. Ref. [
312
] considers anomaly detection as a multi-class classification problem
and implements a supervised learning algorithm, named Support Vector Machine embed-
ded Layered Decision Tree. The aim is to prevent Denial of Service (DoS) attacks which
Energies 2024,17, 3620 36 of 61
target the load-shedding scheme. The status of the power system is classified into five
different modes. Every agent broadcasts its load profile and an adaptive load-rejection
strategy which considers historical records is realized. Ref. [
311
] has rule-based learning for
threat detection. An agent can learn and memorize cyber-attack alert rules from historical
observations. The agent becomes skeptical when the incoming data violate these rules.
Whenever an alert rule is triggered, the agent sends out a data inquiry to find out the real
system status.
False data injection is a type of intrusion where the adversary tries to compromise a
component or alters a state/control variable. In particular, the attacker may inject false
values or modify a control signal, which causes undesired opening of a circuit breaker.
Ref. [
317
] provides algorithms for the identification of false data injection attacks and the
identification of alert manipulation attacks. The network is divided into multiple subsys-
tems which include a Phasor Measurement Unit (PMU) agent, Detection and Identification
(DI) agent and Dynamic Control (DC) agent. To mitigate an attack on PMU agents, DI
agents perform a sensitivity analysis which checks the deviation of line current and voltage
magnitudes and monitors the circuit breaker status. Ref. [
318
] designs a Breaker Supervi-
sion Agent (BSA) to prevent malicious false tripping of circuit breakers which cause power
outages. In a decentralized MAS structure, agents measure the line current values and
breaker states to identify an out-of-service line and subsequently inform the BSA agent
about the outage. Then, the BSA agent checks the node voltages, relay currents and control
signals to identify a malicious trip command.
Challenges and Open Problems: Researchers have addressed protection and security
problems in the Smartgrid and developed various MAS-based solutions for system moni-
toring, data privacy and encryption, intrusion, attack detection and prevention, malicious
data flow, denial of service, false data injection and anomaly detection. Formal guidelines
and requirement documents for Smartgrid protection and security have also been created.
As explained in the text, decentralized security management is more robust compared
to a centralized security system. But then the imminent question is how to maintain overall
security and protection in a decentralized security system. In particular, when an agent
faces an attack, how does it notify the whole grid about the existence of the attack and
how do agents make a joint effort to remove the threat? To deal with these challenges,
emergency situation action plans and communication mechanisms which are suitable for
the MAS structures should be prepared beforehand. In a hierarchical MAS, the information
should propagate both upward and downward, while in a decentralized MAS information
should propagate towards each neighbor. To cope with false alarms, authentication of the
source and verification from multiple agents can be implemented.
To this end, we should note that the existing security and protection systems are not
truly decentralized: there is still a central coordinator or overseer agent who is susceptible
to attack. One method to handle this issue in a complex system is to partition the Smartgrid
into zones and assign the security duty to the zone agents. If the MAS system is layered,
an agent at each layer can be assigned to monitor the security in its own region. A Smartgrid
will become more resistant to attacks when multiple agents carry out the protection task.
As for information safety, additional measures other than encryption and authenti-
cation can be implemented. For example, agents should agree on certain time points to
perform periodic communication. In addition, an agent should be allowed to send only a set
of variables depending on its status and privileges in the grid. Information received outside
of agreed time points or privileges are classified as unauthorized/alien and ignored.
Encryption, authentication and security protocols for Smartgrid require further re-
search and investigation. A Smartgrid is a large-scale network; thus, whether all agents
and all zones will use the same encryption, decryption and authentication algorithm is an
issue. One potential solution might be agents inside a microgrid use the same encryption
algorithm; communication across microgrids employs a different one. Another issue to
resolve is the scope of data privacy inside the hierarchy. In particular, should the agents in
an apartment share their data with other apartments in the building?
Energies 2024,17, 3620 37 of 61
Previous research has concentrated on statistical analysis or trust score to detect
anomalies. In an autonomous, distributed system like a Smartgrid, individual agents should
be endowed with intelligence and reasoning to protect them against various malicious
attacks, in order to strengthen the whole system. An agent should have the capability
to monitor the state, possess predefined safety rules, reason with these rules and learn
new rules from experiences. For this, each agent must have a knowledge base which
involves information about his own state, nearby agents and the environment. Furthermore,
neighbor agents should periodically exchange data with each other to check whether any
of them have been compromised or not. Developing a robust, agent-based, intelligent
protection system is a promising direction for further research.
9. Simulation and Implementation
Simulation studies in the literature have tested their Smartgrid model on a virtual
environment in computer with MAS platform and simulation software (Table 7). Imple-
mentation projects have physically deployed their MAS and Smartgrid model on a real
site to perform as an energy management system. Various attempts at simulation and
implemention of Smartgrids have been made and these simulated and implemented sys-
tems vary with respect to scale, platform and functionality. We go over the simulation and
implemention projects of Smartgrids in this section.
9.1. Simulation
It is necessary to simulate a control system to assess its robustness and effectiveness
before physically deploying it to the Smartgrid. Simulation is also beneficial to prevent
physical damage and costs in cases of unintended consequences. Each of the papers
reviewed in the previous sections has performed simulation or test of their proposed
model, typically in Matlab or Simulink environment. Other simulation platforms specific
to multi-agent systems such as Symphony [
320
], Mosaik [
321
], OPAL-RT [
80
] have also
been used.
Table 7. MAS-based simulation and implementation of Smartgrids.
Problem Proposed Methods
Generic simulation platform MACSim [322], MACSimJX [323], Mac-Sim [324],
MECSYCO [325], MASGriP [326,327],
co-simulation [285,328]
Simulating special function of Smartgrid Energy markets (MASCEM [329], EMCAS [330],
AMES [331], GAPEX [332], Power TAC [333,334]),
security [335], demand response [80],
restoration [285]
Implementation Physical installation [76–78,88,154,336–340],
laboratory facilities [79,154,221,341,342],
hardware-in-the-loop [343,344]
9.1.1. MAS-Based Generic Simulation Platforms
Researchers have also constructed generic as well as special purpose MAS-based
simulation platforms for Smartgrids. Among the generic platforms, MACSim (Multi-Agent
Control for Simulink program) [
322
] is a medium where a Java or C/C++ program which
has agent-based design can transfer data with Simulink. MACSim has a client–server
architecture, where the client part is embedded in Simulink and the server is in the main
Java or C/C++ program which works in a multi-threaded fashion. MACSim has been
extended to MACSimJX [
323
] by incorporating JADE to the server. Hence, MACSimJX
allows the use of a generic MAS platform and MAS designs in this platform to be embedded
into simulation. In the MACSimJX framework, JADE can exchange data and messages
with Simulink through the MACSim interface. MACSimJX has been applied to control of
microgrids [85,87,345].
Energies 2024,17, 3620 38 of 61
Some of the simulation platforms are actually co-simulation platforms; namely, they
are composed of several simulators or software tools. For example, Mac-Sim (Multi-
Agent and Communication Simulator) [
324
] includes three components: JADE for MAS
framework, Opnet for network simulation framework and a run-time infrastructure which
handles messaging, object management, time synchronization and acts a mediator between
JADE and Opnet. Note that though the names are similar, MacSim and Mac-Sim are
different simulation platforms with different structure. Ref. [
285
] created a co-simulation
platform which has one more component, PSCAD/EMTDC, to integrate power system
simulations. Ref. [
328
] also created another co-simulation platform which involves MAS,
power and network domains, but this platform uses Powerfactory for power simulator and
OmNET++ for network simulator.
Ref. [
325
] addresses a more generic and broader view of co-simulation: multi-model
simulation. Their MECSYCO platform can integrate arbitrary tools and simulators by using
the MAS paradigm. Each tool or simulator is an individual agent of a decentralized MAS.
These tools/simulators interact and exchange messages via the communication network
OMNet++.
MASGriP [
326
,
327
] is an agent-based Smartgrid simulation platform which is, by de-
sign, integrated into another simulator MASCEM for Competitive Electricity Markets. In
this manner, agents can negotiate in the electricity market. Its MAS architecture includes
base agents as well as two types of aggregator agents, the Virtual Power Plants (VPPs)
and the Curtailment Service Providers (CSPs). A laboratory experiment of MASGriP
has been performed to test energy management and demand response of buildings [
346
].
MARTINE [
347
] is MAS-based simulation infrastructure for energy management of smart
buildings and microgrids. MARTINE has decision support, optimization, negotiation and
reinforcement learning capabilities. It has four layers: real-time simulation, building, MAS
and decision making.
9.1.2. Specialized Simulation Platforms
There are multi-agent simulators specially designed for energy trade markets such
as MASCEM [
329
], EMCAS [
330
], AMES [
331
] and GAPEX [
332
]. MASCEM (Multi-Agent
System for Competitive Electricity Markets) is FIPA-compliant and can cooperate with
other MAS societies via ontologies. MASCEM can simulate popular market models like
day-ahead pool, bilateral contracts, balancing market, forward market and hybrid markets.
Its MAS architecture contains five types of agents: main agent, management information
base agent, market operator agent, system operator agent and player agent. EMCAS
(Electricity Market Complex Adaptive System) allows dynamic and adaptive agent strate-
gies in the market. The framework has three components: agents, interaction layers
and planning periods. The types of agents in EMCAS are generation companies, demand
companies, transmission companies, distribution companies, independent system opera-
tors, consumers and regulators. In EMCAS, agents are free to establish their own objective
functions and apply their decision rules. Using exploration-based learning, agents explore
entirely new market strategies and observe the results of their actions. Hence, agents can
learn from their previous experiences and change their future behavior.
AMES (Agent-based Modeling of Electricity Systems) is oriented towards the study of
the U.S. wholesale electric market in accordance with the Federal Energy Regulatory Com-
mission (FERC) market design. Originally, AMES was designed for research and teaching
purposes rather than commercial-grade application. Its multi-agent system includes an
independent system operator agent (ISO), load-serving agents and generation companies
(GenCos), which are distributed across the buses of the transmission grid. AMES supports
stochastic reinforcement learning algorithms for generation companies; it facilitates the
augmentation of the empirical input data with the simulated input data. GAPEX (Genoa
Artificial Power Exchange) can simulate market-clearing procedures of most European
power-exchange markets. GAPEX is developed in Matlab, it supports several market
mechanisms and machine learning for agents and it has a statistical analysis module. Its
Energies 2024,17, 3620 39 of 61
agent concept is abstract and can range from a simple reactive agent to a more sophisticated
cognitive agent.
In the scope of energy market simulation, we should also mention a relevant initiative
called Power TAC (Trading Agents Competition) which is organized annually since 2012.
Power TAC provides a simulation platform with many energy producer, consumer agents
and the broker agents (competitors). The energy market, suppliers, customers and the
distribution utility is modeled by the Power TAC simulation platform [
333
,
334
]. Research
groups prepare their own software and enter the competition as broker agents. The task
of the broker agent is to design and offer tariff contracts to consumers and producers
in order to allocate the energy among the agents in the Smartgrid. The consumers of
electricity are the households, enterprises and owners of electric vehicles. A tariff specifies
flat prices, time-dependent prices, peak prices, load caps for certain periods of the day,
contract duration, signup bonuses, early withdrawal penalties, etc. Thus, the role of the
broker agent is to mediate the flow of electricity between energy producers and consumers.
The broker who achieves the most profit over a range of scenarios is the winner of the
competition. Note that Power TAC organizers can also set some social welfare objectives
such as fairness, utilization of renewable resources and emissions. Hence, the market
designers can create incentive mechanisms in addition to profit maximization, in order to
attain socially desirable outcomes [348].
In addition to energy markets, simulation tools for other functions of the Smartgrid
have also been developed. Ref. [
335
] has created a simulation system for multi-agent protec-
tion and security. The simulation framework consists of Simulink and JADE; the interfacing
agents mediate the data exchange between the two. Their multi-agent protection sys-
tem (MAPS) includes coordination agent, fault detection agent and communication agent.
Ref. [
80
] creates a microgrid simulation environment for demand response of agents and
the evolution of price. These agents interact with the real physical installations, OPAL-RT
is used to simulate resources that are not physically available. The co-simulation platform
of [285] tests a restoration and network-reconfiguration problem by a fault scenario.
9.2. Implementation
The ultimate objective is the implementation of the Smartgrid infrastructure: deploy-
ing the necessary and well-functioning physical components and software to operate the
network. Aside from simulation, another major concern is how the above proposed control
mechanisms behave in a real, physical context. Some models have been physically imple-
mented on a real Smartgrid in a campus or a pilot location; others are tested in a laboratory
environment or hardware-in-the-loop.
9.2.1. On-Site Implementation
Refs. [
77
,
78
] designed and installed a microgrid controller for an isolated Greek island
Kythnos as well as a test site in Athens. Household agents have renewable energy resources,
batteries, water pump and other loads. An Intelligent Load Controller (ILC) agent monitors
the power line and measures the voltage and current values. ILC has an algorithm whose
main objective is to control the operation of non-critical loads, in particular the water pump
depending on the available energy in the batteries. Load shedding is equally divided
among all houses in the settlement.
An open-source multi-agent system for residential homes and businesses is deployed
at trial test locations in Australia [
88
]. MAS agents are implemented on personal computers
and personal digital assistants. Prosumers actively participate in the market and trade
energy. DER agents communicate through the bulletin board, they can post and query
topics in the bulletin. Ref. [
154
] installed a restoration system on an urban electric grid in
Jiangning County, China. The grid at the pilot site is comprised of three substations, five
branches and six transformers. The system has five operating states and four subcontrols:
emergency control, restorative control, corrective control and preventive control. The entire
Energies 2024,17, 3620 40 of 61
self-healing functionality is controlled by a multi-agent system consisting of response layer,
coordination layer, organization layer.
Some implemented Smartgrid management systems include a microcontroller device
like Arduino, which receives input and gives output from/to grid elements, sensors and
actuators. Ref. [
336
] presents a deployment of an MAS-based controller on Arduino for a
rural Indian microgrid. The control system has a simple, compact design in order to be low
cost and affordable. Agents are hierarchically grouped into three classes: distributed grid
agents, micro agents and D agent (cloud or computer). Ref. [
337
] creates a controller on
Arduino for a Smartgrid test bed where three microgrids are interconnected to the main
grid. Each microgrid has its own photovoltaic panel, wind turbine, diesel generator, battery,
critical and non-critical load units. The controller agent of a microgrid handles the energy
transfer between its own elements as well as the energy transfer from/to other microgrids
and the main grid. Another Arduino-based Smartgrid management system is implemented
by [
349
] for two interconnected microgrids, one in a university department and the other
in a hostel. The grid agent and the controller agent manage the energy transfer between
the (non-)renewable resources, battery units, loads. In the experiments, the agents on the
controller receive input from Simulink (through MACSimJX interface), instead of physical
hardware. Ref. [
338
] deploys a microgrid management system to a commercial building
with 16 offices and 40 photovoltaic panels. Each zone is managed by a unique mGIM
agent and agents run on Raspberry Pi boards. Each agent is equipped with an hour-ahead
forecasting algorithm and agents can trade energy by peer-to-peer local auctions. The aim
is to implement an architecture for end-user representation and a light-weight solution that
can be deployed on a single board computer. In [
339
], a self-healing microgrid system is
implemented. A test bed is assembled which consists of a physical DC electric grid, Arduino
microcontrollers and Raspberry Pi computers. The agent software is hosted and executed
on the Raspberry Pi and its physical interconnection to the electric grid is achieved by the
Arduino microcontroller. The load control agent and the restoration agent together execute
the Prim’s minimum spanning tree algorithm to solve the service-restoration problem.
Some universities and research groups have installed Smartgrid management systems
on their campuses. Illinois Institute of Technology has initiated the Perfect Power Smartgrid
project [
76
] upon many power outages at its campus between 2004 and 2006. The goal
of the project is peak load reduction on distribution feeders by on-site distributed energy
resources and energy management systems at costs competitive with the system/capacity
upgrades. Distributed resources can reduce peak demand and decrease total energy costs.
The “Advanced Distribution Automation and Recovery System” was implemented in 2013
and it achieved %50 peak load reduction, uninterruptible power to critical facilities and
resilience to a single point of failure. Its MAS architecture is composed of a group of
teams. The agents within a team communicate with each other, while a team negotiates
with other teams by its mates. Ref. [
340
] presents a Multi-Agent Management System
(MAMS) for the microgrid deployed at UADY Engineering Faculty. MAMS regulates the
energy consumption of the microgrid and the energy transfer from the main grid in order to
minimize the total cost and at the same time maintain the microgrid energy balance. MAMS
is composed of seven types of agents, solar- and wind-generation agent, battery bank agent,
electric vehicle agent, public grid agent and critical and ordinary load agents. MAMS
performs power scheduling of the loads based on the solar and wind output prediction
from a neural network.
9.2.2. Implementation in Laboratory
Some authors implement and test their system under laboratory facilities [
79
,
154
,
221
,
341
,
342
]. Ref. [
79
] deploys a microgrid management system in a laboratory with a
diesel generator, battery banks and controllable loads. They used real, physical equipment
in their experiments. The effectiveness of their load shifting and curtailment algorithms
was tested under different microgrid configurations. Ref. [
221
] realizes the decentralized
consensus protocol for the economic dispatch problem (scheduling of energy generators) in
Energies 2024,17, 3620 41 of 61
a microgrid testbed. The controller runs on a dSpace platform with control desk software.
The setting includes three physical inverters and two resistive load units. A multi-agent
system for microgrid control is tested at the laboratory microgrid of National Technical
University, Athens [
154
]. The system handles energy trade by auction and transition
between the connected mode and the islanded mode. The setup contains a photovoltaic
panel, battery bank and loads. The control system is implemented as software on a personal
computer. Another multi-agent system for control and monitoring is implemented on a
laboratory Smartgrid test bed at Florida International University [
341
]. The framework
consists of renewable and non-renewable generation units, loads, transmission line models,
field sensors and actuators. The agent platform is implemented on a personal computer; an
OPC UA server handles the measurements and the information transfer. The area power
system operator agent and the DER agent are responsible for reducing excessive energy
demand and overloading during peak hours. In another project at the same university,
an MAS-based communication-assisted fault localization, isolation and restoration method
has been tested on their test bed [342].
9.2.3. Hardware-in-the-Loop
An alternative to the physical experimentation is hardware-in-the-loop simulation
where the controller or management system is implemented on physical hardware in the
laboratory, but some peripheral devices (generator, storage, load, sensor) run on a simu-
lator like Opal-RT and their input/output are obtained from the simulator. For example,
Ref. [
343
] performed a laboratory test of a fault-isolation and restoration system. The
MAS agents are designed according to the Human Immune System cells for feeder fault
location, isolation and self-healing. Agents run on electronic boards with microproces-
sors and a three-phase fault experiment is conducted in the Analogue Power Simulator
hardware. Ref. [
344
] designs an MAS-based microgrid control system for an emergency
demand response (EDR) program and conducts its hardware-in-the-loop simulation in
the laboratory. Agents are implemented on microcontrollers and the real-time data from
generators and loads are obtained from the Opal-RT system. The microgrid contains two
DERs, a micro-gas turbine (MGT), a battery energy-storage system (BESS) and a load. In
cases of a power shortage, the main grid operator requests emergency demand response for
the microgrids. The microgrid central coordinator agent (MGCC) decides to join the EDR
request depending on the energy status of its microgrid. The MGCC agent uses Contract
Net Protocol for participation of the generator and load agents in the demand response.
Challenges and Open Problems: Most researchers actually created their own model
in Matlab, Simulink and JADE to simulate their system. In addition, researchers have also
constructed a number of simulation platforms for testing Smartgrid operation. Some of the
existing simulation platforms for Smartgrids are designed as a generic agent-based system
while others are oriented towards simulating a specific function such as energy markets,
security, demand response and restoration. In this scope, energy market simulations have
received special attention.
For some aspects of Smartgrids such as security or restoration, more simulation studies
and experiments are required to understand the behavior and the issues in the grid. In
addition to security and restoration, some other functions like supply side management
and EV charging have not been tested with the generic MAS-based simulation platforms.
Thus, the effectiveness of these generic simulation platforms for all functions of Smartgrids
must be verified.
As for implementation, physical Smartgrid systems have been installed at pilot loca-
tions or at laboratory facilities. Nonetheless, these physically implemented systems so far
are small scale, e.g., building, business, campus. Moreover, energy trade between houses or
microgrids has not been implemented. An open problem is whether these models would
work on a large scale such as a district or town. Researchers should investigate whether
an MAS-based framework is efficient for managing Smartgrids and examine the problems
encountered in practice.
Energies 2024,17, 3620 42 of 61
Physical MAS systems deployed at trial locations focus on a single functionality of
the grid e.g., control, demand response or restoration. Furthermore, some of these imple-
mentations handle the main tasks of the Smartgrid by a microcontroller or software, rather
than autonomous agents. To construct a self-operating Smartgrid, ideally we require an
MAS-based setting that can handle all functions including control, trade, demand/supply
management, restoration and security. For this purpose, future studies should utilize
independent, intelligent agents working and deciding on their own. Hence, one of the
focuses of implementation should be designing agent-based systems and testing various
aspects of the grid for robustness.
10. Discussion and Meta-Level Analysis
At the end of this survey, what are the final conclusions? Each of the papers reviewed
in the preceding sections tackles a problem in a particular field of Smartgrids. The MAS
architectures and methods utilized in these papers are also different: they use different
types of agents, MAS layers and organization. As the reader will have noticed, there is no
universal model that can handle all areas of Smartgrids. Even the papers that address the
same problem use different configurations of grid elements, energy resources, load profiles
and operation modes. Thus, an important challenge is how to construct a single MAS
structure or integrate the above models in order to obtain a robust and well-functioning
Smartgrid. One solution might be combining different functionalities into a single layer or
agent. For example, control, demand scheduling, local recovery and security monitoring
duties can be assigned to the zone agent. A similar argument can also be made for various
optimization tasks: how to optimize market price, energy consumption, load/generator
scheduling, restoration together and which agents will be responsible?
We also note that some models utilize sophisticated methods and/or MAS structures,
especially the auction and demand response programs. This stems from the difficulty in
energy management of the Smartgrid and also from the usual tradeoff between efficiency
and simplicity. A complex framework can utilize and allocate resources close to optimum,
but tends to be less flexible and hard to implement. Thus, designing MAS systems that are
both efficient and practical for Smartgrids is a promising direction for research.
Some of the proposed systems in the literature are not truly agent-based. In these
systems, most agents are simple electronic components which perform the instructions
sent by the main software or the central controller. That is, the relationship is basically
master–slave and these agents are not really autonomous or intelligent. In our opinion,
the core ingredient of MASs is independent and self-operating agents.
As discussed in the introduction, the main paradigm of Smartgrids is energy manage-
ment, with consideration for the weather-dependent nature of renewable energy resources.
The problem of Smartgrids is essentially balancing the demand and supply, utilizing avail-
able resources to the utmost extent and scheduling/reshaping the demand. Numerous
models of energy trade, control and demand response have been proposed in the literature.
These models tend to be heuristic or ad hoc, tailored for a particular setting. Authors have
tested their models on their own in a simulation environment. However, it is unknown how
these different models proposed by authors compare to each other in terms of efficiency and
reliability. In addition, MAS-based solutions should also be compared to non-MAS-based
solutions to figure out whether MAS is a proper framework for Smartgrids. In this respect,
part of the effort in future research should focus on preparing benchmark problem instances
for Smartgrids to evaluate the proposed models.
Another challenge of Smartgrids is their geographical and spatial aspects. It is rela-
tively easy to assign a microgrid and design an MAS structure for an isolated, small region
such as a campus, village or military camp. However, for large, continuous residential
settlements in urban areas, it is difficult to implement Smartgrids in the whole territory and
partition them into virtual microgrids or Virtual Power Plants. Moreover, each VPP may
have a different size and configuration; therefore, how to construct an MAS design generic
enough for VPPs is a problem to deal with.
Energies 2024,17, 3620 43 of 61
The existing frameworks in the literature consider a microgrid with fixed size and
arrangement of resources and loads. Their MAS structure, hierarchy and types of agents
are designed for this certain microgrid size and element. However, as cities grow, we expect
new districts and regions to join the electric grid. Therefore, how to adapt the microgrid and
the corresponding MAS structure for an enlarging network is another issue. To cope with
this, the multi-agent system for the microgrid should have alternative designs and flexible
topology, layer and zone architecture. The number and type of agents can be adaptive
depending on the scale of the Smartgrid.
Due to digitization and advancement of information technologies, energy demand of
data centers, cloud computing and website hostings have been growing every year
[350–354]
.
These data and computing centers constitute concentrated, heavy energy-consumption
points, which is a challenge for conventional electric production and distribution networks.
Though there are preliminary works [
355
–
359
] in the literature, whether (or how) Smartgrid,
renewable energy and multi-agent system technology can be used to fulfill the high energy
need of these information-processing centers is an open direction for future research.
10.1. Review of Challenges and Open Research Problems
This section presents a sum-up and upper-level review of challenges and open prob-
lems mentioned in previous sections to create an entire agenda of key issues for Smartgrid
research, as listed below.
•
Since centralized control is computationally heavy and prone to single-point-of-failure,
we need truly decentralized models for Smartgrid control, energy management and
security. Local knowledge of distributed agents and local solutions would be both a
simple and an effective way to deal with the above problems.
•
Then, a related problem is how to achieve overall coordination of agents in a de-
centralized system against power outages, faults, overloading and security attacks.
For this purpose, collaborative ready-to-use strategies, emergency action plans and
communication protocols must be developed.
•
Agents in Smartgrid system should possess intelligence and reasoning capabilities to
detect abnormal events, perform action planning and collaboration.
•
There is also need for communication and information exchange mechanisms be-
tween agents in order to enhance energy trade, security, restoration, demand and
supply management.
•
Since Smartgrid control, energy trade, demand and supply management are closely
related to each other, we need a unified framework to handle these functions using
effective yet practical algorithms.
•
A prominent problem is embedding network hierarchy and geographical proximity
into energy trade and allocation, as these are critical factors for convenient energy
distribution and reducing transmission loss. In particular, energy sharing inside the
same building or layer should be studied.
•
Another issue is how to perform energy management and trade when actual supply
and demand differ from forecasted values, namely actual renewable energy production
is less than demand. In this situation, alternative strategies, secondary markets,
rescheduling of loads and non-renewable energy resources can be utilized.
•
As demand management involves all loads in the grid, a major challenge is how to
integrate demand response programs of house, building and microgrid. From home
appliances to vehicles and plants, alleviating peak demand constitutes a complex,
hierarchical problem to deal with.
•
Spatial and temporal reasoning should be utilized in electric vehicle charging, in order
to consider alternative time periods and locations. Vehicle-to-building models and
energy management of buildings need more detailed analyses.
•
In restoration and self-recovery, it is necessary to incorporate load shedding and
demand response. In addition, case-based predetermined rules and strategies are
required for rapid and efficient fault identification and recovery.
Energies 2024,17, 3620 44 of 61
•
Another challenge in Smartgrid control is how to incorporate manual actions and
preferences of human agents into the energy management system. Human actors tend
to intervene into the process (especially in control, trade and restoration) and set their
own bids, load priorities and device on/off actions.
•
As for protection and security, researchers should develop encryption, decryption
and authentication algorithms specific to Smartgrid domains which respect network
hierarchy and agent privacy. More advanced methods for information safety and
communication protocols are also promising directions for research.
• A primary problem is implementation of Smartgrid and microgrids/VPPs on a large
scale, such as city or region. Moreover, whether MAS technology is a proper choice
for Smartgrid operation and its efficiency should be investigated.
•
Designing MAS architecture, hierarchy and agents to perform all functions of Smart-
grids (energy trade, control, security, restoration and demand supply management)
constitutes a great challenge and problem for future research. Previous simulation
and implementation projects have not covered all the above functions and thus more
studies are required to explore these aspects.
10.2. Knowledge Reasoning and Planning for the Smartgrid
Aside from fixed network topology and grid configuration, the models for Smartgrid
reviewed above lack an important feature. Each paper has addressed a specific problem
which has a certain objective function, e.g., bidding, control, demand/supply response,
restoration, anomaly detection. And their solutions mostly employ a hard-coded algorithm
because there are no unknown elements or external constraints. Namely, agents execute
a predefined sequence of actions to achieve their objectives stated in the problem. Conse-
quently, these algorithms do not work for any other task or in a setting which has additional
constraints or a slightly different goal. Therefore, agents should be able to construct an
action plan for an arbitrary objective function in any setting with special configuration and
constraints. Doing so, the agent should take into account both the individual/local goals
as well as the global (system-wide) goals. As an example, consider a scenario where the
Photovoltaic Panel agent has detected a fault in its inverter component and needs to inform
the utility agent. The PV agent can communicate with only the local controller agent, EM
agent and neighbor agents, but it does not have direct access to the utility agent. The local
controller agent is under maintenance that day; thus, the PV agent has to find sequence
of communication actions based on the network topology to accomplish its goal. There is
no solution in the literature for this particular problem and we cannot write a hard-coded
algorithm for every special situation. Then, the agent needs to develop its own action plan.
As another example, suppose that the electric car has a low battery level, but its owner
decided to drive in the evening. In this situation, the EV agent needs to inform the other
agents in the building to replan their energy usage. In order to charge its batteries, the EV
agent should find a new schedule; for example, it proposes that the laundry agent and the
home battery agent postpone their energy consumption, as a modification to the existing
schedule. Therefore, we need to devise more generic methods that can achieve arbitrary
objectives and allow for unexpected situations and additional, external constraints.
In order to perform automated action planning, the agent should be endowed with a
knowledge base which includes possible actions, domain information, the agent’s belief
about the world and his beliefs about beliefs of other agents. According to the BDI concept,
the agent has Beliefs (about the physical world and beliefs of other agents), Desires (goals)
and Intentions (actions). The domain description in the knowledge base also includes
fluents (variables) that describe the state of the world, their possible values and the action
description (precondition, effect and observability of each action). Recall that agents
are autonomous and independent; thus, they have private beliefs, their own domain
description and local knowledge. In particular, the domain description of an agent involves
only the fluents and the actions which are relevant for him. For example, the domain
of the AC agent consists of the fluents
on
,
heat_mode
,
connected
and
temperature
and
Energies 2024,17, 3620 45 of 61
the actions
turn_on
,
turn_off
,
heat
,
cool
and
inform_temperature
. Then, one part of the
problem is how to represent the action description and the private beliefs of an agent. There
are action languages such as
mA0
[
360
] and
mA∗
[
361
] designed for the multi-agent setting;
however, whether they are suitable for the Smartgrid context and suitable for BDI and the
private view of agents should be investigated.
In a dynamic environment, agents communicate and execute actions; thus, another
important problem is how to update beliefs of agents upon an action occurrence. Belief
update is necessary for both the agent having a correct view of the world and also to
determine whether a sequence of actions attains a goal condition. For instance, consider
a home environment with energy-storage agents. At one moment, storage 1 is supplying
electricity to the other appliances while storage 2 is charging. Later, storage 1 becomes
depleted, but it knows that there is another storage which has been charging for 1.5 h.
The storage 1 agent informs the home energy management (HEM) agent about its storage
level, and the HEM agent communicates with the storage agent 2 to learn its status. Then,
the HEM agent sends instructions for storage 1 to switch to charge mode and storage
2 to switch to supply mode. After these ontic and communication actions, each of the
three agents should update their belief about the mode of each agent. Moreover, the home
energy management agent must verify that this action plan maintains the goal of proper
energy delivery.
The challenges in this belief-update and state-transition problem are the agents might
have initially incomplete or incorrect beliefs and they might have different levels of observ-
ability of an action. In a multi-agent context, not all agents might observe the execution of
an action and its effects. In particular, in a communication action, an agent sends message
to a certain set of agents; other agents outside of this audience are unaware of this action.
In some cases, an agent might partially observe the effect of the action. As an example,
the central controller agent knows that the heater agent periodically senses the temperature
level, but he cannot observe the sensed value of the temperature. Following this idea,
agents can be classified into three categories: Full observers who observe the action and
its effects, partial observers who only observe that the action takes place but not its effects
and oblivious agents who are completely unaware of the action occurrence.
In the uncertain and time-varying setting of the Smartgrid, pre-specified algorithms
for energy distribution and load scheduling may not work. For instance, EV agents may
arrive and leave, and operation time of the appliances depends on the human users and
the temperature. In such a dynamic medium of the Smartgrid, neighbor agents should
communicate with each other and plan/replan scheduling of their energy consumption
and charging. Developing (re)planning, decentralized scheduling for agents will be useful
for better utilization of the available energy.
The knowledge base model also allows for representation of different kinds of knowl-
edge such as commonsense rules, defaults and causal laws. Examples are “By default,
the wind turbine is on and the diesel generator is off”, “Electric vehicle is charging implies
it is connected to the grid” and “If the temperature is below 20
◦
C, the heater must be
on”. The rules of commonsense are beneficial for circumventing incomplete information
and enhance reasoning. Creating an ontology for commonsense knowledge, default rules,
causal laws and integrating into reasoning, planning is a promising work. As for other
tasks in the Smartgrid, the knowledge base of an agent should also include the necessary
variables and elements for auction, energy trade, load profile, security.
A Smartgrid has a hierarchical and hybrid organization. It consists of a house, a
building, a microgrid/VPP and main grid layers and each layer can have a different
structure. In addition, there may also be coalitions, holons and zones in some layer(s).
An agent should only consider local and relevant variables and actions because it cannot
consider the fluents of all agents in the system or update them in a reasonable manner.
Then, the problem is how to design the knowledge base of an agent appropriate for the
hybrid structure of the Smartgrid.
Energies 2024,17, 3620 46 of 61
Henceforth, knowledge reasoning and planning are important features of intelligent
agents, yet they have not been studied in the context of Smartgrids. Future research should
proceed in this direction in order to endow agents with tools to operate efficiently in an
uncertain environment.
11. Conclusions
Smartgrids represent the next generation in energy production and sharing. Humanity
is shifting from high-pollution nuclear, thermic plants towards carbon-free, clean and
renewable resources. Environmental regulations and tightening government policies about
carbon emissions are putting pressure on the energy production and transportation sectors
and this trend seems likely to continue in the coming decades. Moreover, households are
evolving into economically motivated prosumers who can also produce, store and sell
electricity in addition to consumption.
This paradigmatic shift necessitates a Smartgrid which can deal with the challenges of
decentralized production such as control, demand management, restoration, energy trade
and security. Volatility of weather-dependent generators and time-dependent electricity
consumption require accurate control, scheduling and storage of energy. In addition,
prosumers need to exchange energy among each other to balance demand and supply.
Thus, market and auction mechanisms should be designed and implemented. Microgrids
can become efficient energy ecosystems if prosumers can communicate, access the market
and have the necessary background to achieve their objectives.
Advancement of Smartgrids and related technologies need integration of Artificial
Intelligence, distributed systems, information and communication technologies. In this
survey, we focused on the use of multi-agent system technology for Smartgrids. Multi-
agent system theories have been developed by computer scientists to represent autonomous
agents interacting with each other and with the environment. Agents have a knowledge
base; they can decide and act on their own without human intervention. Moreover, agents
are goal-driven; they perform actions to satisfy or optimize their objectives. As a system of
autonomous and interacting entities, MASs seem to be a potential framework for modeling
distributed producers and consumers in the Smartgrid.
At the beginning of the survey, we provided some definitions and background infor-
mation about Smartgrids, multi-agent system concepts, MAS platforms, ontologies and
standards. Then, we reviewed the literature about how researchers have used MAS tech-
nology for energy trade, control, demand and supply management, restoration, security,
simulation and implementation of the Smartgrid. In each of these fields, we highlighted the
challenges and the remaining open problems for future research. In addition, we conducted
an upper-level analysis of existing MAS models and research. Observe that in each field,
a great deal of work has already been done; however, it is not clear whether the main
problems of the Smartgrid have been solved. In particular, integrating these different
models for full functionality is an issue waiting to be dealt with. Researchers should assess
whether the proposed solutions are practical and flexible enough, and for this purpose,
physical implementation may be necessary. Moreover, agents should possess intelligence
to perform automated planning and knowledge reasoning in order to accomplish their
objectives in a generic setting and maintain a consistent, up-to-date knowledge base and
infer new information.
In addition to the hardware, a Smartgrid also involves software and cyber compo-
nents which need to achieve a number of complex and important tasks mentioned above.
Multi-agent systems have the advantage of dividing a complicated problem into smaller
subproblems and using agents to solve the smaller problems. However, whether an agent-
based approach is efficient and yields the optimal or desired outcome in the Smartgrid
domain should be investigated in more detail and comparison with non-MAS methods
should be made. In particular, coordination of many autonomous and self-interested small
agents to accomplish various functions of the grid is a major concern. In this sense, possible
pitfalls and disadvantages of MAS technology, if any, should be identified and whether
Energies 2024,17, 3620 47 of 61
MAS is the right technology for Smartgrid management (or whichever aspect) should
be determined.
In the future, whether (or how) Smartgrids can be deployed on a large scale or an
economy-wide scale for electricity allocation is unclear. Doing so would require, in addition
to the massive changes in the physical infrastructure, control and coordination of millions
of independent prosumers and distributed generators/loads. Therefore, developing a
robust and scalable MAS framework for the energy systems is a promising direction for
medium- and long-term research. We hope this survey on the applications of multi-agent
systems for Smartgrids will be beneficial for researchers, engineers, designers, prosumers
and other stakeholders in terms of exploring further topics and solutions.
Funding: The authors have been partially supported by NSF grants 2151254, 1914635 and 1757207.
Tran Cao Son was also partially supported by NSF grant 1812628.
Conflicts of Interest: The authors declare they have no conflicts of interest.
References
1.
Wüstenhagen, R.; Menichetti, E. Strategic choices for renewable energy investment: Conceptual framework and opportunities for
further research. Energy Policy 2012,40, 1–10. [CrossRef]
2.
Cadman, T. The United Nations Framework Convention on Climate Change. In The Palgrave Handbook of Contemporary International
Political Economy; Shaw, T.M., Mahrenbach, L.C., Modi, R., Yi-Chong, X., Eds.; Palgrave Macmillan: London, UK, 2019; pp. 359–375.
[CrossRef]
3.
Freedman, M.; Jaggi, B. Global Warming, Commitment to the Kyoto Protocol and Accounting Disclosures by the Largest Global
Public Firms from Polluting Industries. Int. J. Account. 2005,40, 215–232. [CrossRef]
4. Paris Agreement; United Nations: New York, NY, USA, 2016.
5. Union, E. Directive 2009/28/EC of the European Parliament and of the Council of 23 April 2009 on the promotion of the use of
energy from renewable sources and amending and subsequently repealing Directives 2001/77/EC and 2003/30/EC. Off. J. Eur.
Union 2009,5, 2009.
6. U.S. Congress. Energy Policy act of 2005; US Congress: Washington, DC, USA, 2005; pp. 1–27.
7.
Malmedal, K.; Kroposki, B.; Sen, P.K. Energy Policy Act of 2005 and Its Impact on Renewable Energy Applications in USA. In
Proceedings of the 2007 IEEE Power Engineering Society General Meeting, Tampa, FL, USA, 24–28 June 2007; pp. 1–8. [CrossRef]
8.
Carley, S.; Nicholson-Crotty, S.; Fisher, E. Capacity, Guidance and the Implementation of the American Recovery and Reinvestment
Act. Public Adm. Rev. 2014,75, 113–125. [CrossRef]
9.
Carley, S. Energy Programs of the American Recovery and Reinvestment Act of 2009. Rev. Policy Res. 2016,33, 201–223. [CrossRef]
10.
Schuman, S.; Lin, A. China’s Renewable Energy Law and its impact on renewable power in China: Progress, challenges and
recommendations for improving implementation. Energy Policy 2012,51, 89–109. [CrossRef]
11.
Wang, Y.; Luo, G.; Kang, H. Successes and Failures of China’s Golden-Sun Program. In Proceedings of the 2017 6th International
Conference on Energy, Environment and Sustainable Development (ICEESD 2017), Zhuhai, China, 11–12 March 2017; Atlantis
Press: Dordrecht, The Netherlands, 2017; pp. 585–606. [CrossRef]
12. Howes, T. The EU’s new renewable energy directive (2009/28/EC). New Clim. Policies Eur. Union Intern. Legis. Clim. Dipl. 2010,
15, 3.
13.
Egenhofer, C. The Making of the EU Emissions Trading Scheme: Status, Prospects and Implications for Business. Eur. Manag. J.
2007,25, 453–463. [CrossRef]
14.
Howell, S.; Rezgui, Y.; Hippolyte, J.L.; Jayan, B.; Li, H. Towards the next generation of Smartgrids: Semantic and holonic
multi-agent management of Distributed Energy Resources. Renew. Sustain. Energy Rev. 2017,77, 193–214. [CrossRef]
15.
Bayram, I.S.; Shakir, M.Z.; Abdallah, M.; Qaraqe, K. A Survey on Energy Trading in Smartgrid. In Proceedings of the 2014
IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, GA, USA, 3–5 December 2014; pp. 258–262.
[CrossRef]
16.
Coelho, V.; Weiss, M.; Coelho, I.; Liu, N.; Guimarães, F. Multi-agent systems applied for energy systems integration: State-of-the-
art applications and trends in microgrids. Appl. Energy 2017,187, 820–832. [CrossRef]
17.
Gómez-Sanz, J.; Garcia-Rodriguez, S.; Cuartero-Soler, N.; Hernández-Callejo, L. Reviewing Microgrids from a Multi-Agent
Systems Perspective. Energies 2014,7, 3355–3382. [CrossRef]
18.
Kantamneni, A.; Brown, L.; Parker, G.; Weaver, W. Survey of multi-agent systems for microgrid control. Eng. Appl. Artif. Intell.
2015,45, 192–203. [CrossRef]
19.
Kiran, P.; Chandrakala, K.R.M.V.; Nambiar, T.N.P. Multi-agent-based systems on micro grid—A review. In Proceedings of
the 2017 International Conference on Intelligent Computing and Control (I2C2), Coimbatore, India, 23–24 June 2017; pp. 1–6.
[CrossRef]
Energies 2024,17, 3620 48 of 61
20.
Kulasekera, A.; Gopura, R.; Hemapala, K.T.M.U.; Perera, N. A Review on Multi-agent Systems in Microgrid Applications. In
Proceedings of the 2011 IEEE PES International Conference on Innovative Smartgrid Technologies-India (ISGT), Kollam, Kerala,
India, 1–3 December 2011; pp. 173–177. [CrossRef]
21.
Mahela, O.; Khosravy, M.; Gupta, N.; Khan, B.; Haes Alhelou, H.; Mahla, R.; Patel, N.; Siano, P. Comprehensive Overview of
Multi-Agent Systems for Controlling Smart Grids. CSEE J. Power Energy Syst. 2020,8, 115–131. [CrossRef]
22.
McArthur, S.; Davidson, E.; Catterson, V.; Dimeas, A.; Hatziargyriou, N.; Ponci, F.; Funabashi, T. Multi-Agent Systems for Power
Engineering Applications—Part I: Concepts, Approaches and Technical Challenges. Power Syst. IEEE Trans. 2007,22, 1743–1752.
[CrossRef]
23.
McArthur, S.; Davidson, E.; Catterson, V.; Dimeas, A.; Hatziargyriou, N.; Ponci, F.; Funabashi, T. Multi-Agent Systems for Power
Engineering Applications—Part II: Technologies, Standards and Tools for Building Multi-Agent Systems. Power Syst. IEEE Trans.
2007,22, 1753–1759. [CrossRef]
24.
Halhoul Merabet, G.; Essaaidi, M.; Talei, H.; Abid, M.R.; Khalil, N.; Madkour, M.; Benhaddou, D. Applications of Multi-Agent
Systems in Smartgrids: A survey. In Proceedings of the 2014 International Conference on Multimedia Computing and Systems
(ICMCS), Marrakech, Morocco, 14–16 April 2014; pp. 1088–1094. [CrossRef]
25.
Sukumaran Nair, A.; Hossen, T.; Campion, M.; Selvaraj, D.; Goveas, N.; Kaabouch, N.; Prakash, R. Multi-Agent Systems for
Resource Allocation and Scheduling in a Smart Grid. Technol. Econ. Smartgrids Sustain. Energy 2018,3, 1–15. [CrossRef]
26.
Vithanage, V.; Boralessa, K.; Hemapala, K.T.M.U.; Wijayapala, W. A review on Multi-Agent system-based energy-management
systems for micro grids. AIMS Energy 2019,7, 924–943. [CrossRef]
27.
Roche, R.; Blunier, B.; Miraoui, A.; Hilaire, V.; Koukam, A. Multi-agent systems for grid energy management: A short review.
In Proceedings of the IECON 2010-36th Annual Conference on IEEE Industrial Electronics Society, Glendale, AZ, USA, 7–10
November 2010; pp. 3341–3346. [CrossRef]
28.
Roche, R.; Lauri, F.; Blunier, B.; Miraoui, A.; Koukam, A. Multi-Agent Technology for Power System Control. Power Electron.
Renew. Distrib. Energy Syst. Sourceb. Topol. Control. Integr. 2013,59, 567–609. [CrossRef]
29.
Rohbogner, G.; Hahnel, U.J.; Benoit, P.; Fey, S. Multi-agent systems’ asset for Smartgrid applications. Comput. Sci. Inf. Syst. 2013,
10, 1799–1822. [CrossRef]
30.
Hasanuzzaman Shawon, M.; Muyeen, S.M.; Ghosh, A.; Islam, S.; Baptista, M.d. Multi-Agent Systems in ICT Enabled Smartgrid:
A Status Update on Technology Framework and Applications. IEEE Access 2019,7, 97959–97973. [CrossRef]
31.
Sujil, A.; Verma, J.; Kumar, R. Multi agent system: Concepts, platforms and applications in power systems. Artif. Intell. Rev. 2016,
49, 153–182. [CrossRef]
32. Yu, X.; Xue, Y. Smartgrids: A Cyber–Physical Systems Perspective. Proc. IEEE 2016,104, 1058–1070. [CrossRef]
33.
Vasu, S.; Jasmin, E.A. Realizing Autonomous and Intelligent Smartgrid Using Multi-Agent Based Control System. In Proceedings
of the 2020 International Conference on Power Electronics and Renewable Energy Applications (PEREA), Kannur, India, 27–28
November 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [CrossRef]
34.
Tazi, K.; Abbou, F.; Abdi, F. Multi-agent system for microgrids: Design, optimization and performance. Artif. Intell. Rev. 2020,
53, 1233–1292. [CrossRef]
35.
Khamesra, P.; Kasera, J.; Mehta, R. Multi-agent Systems Based Intelligent Control of Microgrid. Int. J. Res. Eng. Technol. 2014,
3, 127–133.
36.
Wang, Z.; Yang, R.; Wang, L. Intelligent multi-agent control for integrated building and micro-grid systems. In Proceedings of the
ISGT 2011, Anaheim, CA, USA, 17–19 January 2011; IEEE: New York, NY, USA, 2011; pp. 1–7. [CrossRef]
37.
Lede, A.M.R.; Molina, M.G.; Martinez, M.; Mercado, P.E. Microgrid architectures for distributed generation: A brief review. In
Proceedings of the 2017 IEEE PES Innovative Smartgrid Technologies Conference—Latin America (ISGT Latin America), Quito,
Ecuador, 20–22 September 2017; IEEE: New York, NY, USA, 2017; pp. 1–6. [CrossRef]
38.
Colson, C.; Nehrir, M.; Gunderson, R. Multi-agent Microgrid Power Management. IFAC Proc. Vol. 2011,44, 3678–3683. [CrossRef]
39.
Dimeas, A.; Hatziargyriou, N. Operation of a multiagent system for microgrid control. IEEE Trans. Power Syst. 2005,20, 1447–1455.
[CrossRef]
40.
Cui, T.; Wang, Y.; Nazarian, S.; Pedram, M. An electricity trade model for microgrid communities in Smartgrid. In Proceedings of
the ISGT 2014, Washington, DC, USA, 19–22 February 2014; pp. 1–5. [CrossRef]
41.
Basso, G.; Gaud, N.; Gechter, F.; Hilaire, V.; Lauri, F. A Framework for Qualifying and Evaluating Smartgrids Approaches: Focus
on Multi-Agent Technologies. Smartgrid Renew. Energy 2013,4, 333–347. [CrossRef]
42. Wooldridge, M. Intelligent agents. Multiagent Syst. A Mod. Approach Distrib. Artif. Intell. 1999,1, 27–73.
43. Russell, S.; Norvig, P. Artificial Intelligence: A Modern Approach; Prentice Hall: New Jersey, NJ, USA, 1995.
44. Russel, S.; Norvig, P. Artificial Intelligence—A Modern Approach; Person Education Inc.: New Jersey, NJ, USA, 2003.
45. Dorri, A.; Kanhere, S.S.; Jurdak, R. Multi-Agent Systems: A Survey. IEEE Access 2018,6, 28573–28593. [CrossRef]
46.
Ferber, J.; Weiss, G. Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence; Addison-Wesley: Reading, MA, USA,
1999; Volume 1.
47. Wooldridge, M.; Jennings, N.R. Intelligent agents: Theory and practice. Knowl. Eng. Rev. 1995,10, 115–152. [CrossRef]
48.
Herrera, M.; Pérez-Hernández, M.; Kumar Parlikad, A.; Izquierdo, J. Multi-Agent Systems and Complex Networks: Review and
Applications in Systems Engineering. Processes 2020,8, 312. [CrossRef]
Energies 2024,17, 3620 49 of 61
49.
Haddadi, A.; Sundermeyer, K. Belief-desire-intention agent architectures. In Foundations of Distributed Artificial Intelligence; John
Wiley & Sons, Inc.: Hoboken, NJ, USA, 1996; pp. 169–185.
50.
Rao, A.S.; Georgeff, M.P. BDI Agents: From Theory to Practice. In Proceedings of the International Conference on Multiagent
Systems, San Francisco, CA, USA, 12–14 June 1995; Volume 95, pp. 312–319.
51.
Franklin, S.; Graesser, A. Is it an Agent or just a Program?: A Taxonomy for Autonomous Agents. In Proceedings of the
International Workshop on Agent Theories, Architectures and Languages, Budapest, Hungary, 12–13 August 1996; Springer:
Berlin/Heidelberg, Germany, 1996; pp. 21–35. [CrossRef]
52.
Jennings, N.R.; Sycara, K.; Wooldridge, M. A roadmap of agent research and development. Auton. Agents Multi-Agent Syst. 1998,
1, 7–38. [CrossRef]
53. Russell, S.J. Artificial Intelligence a Modern Approach; Pearson Education, Inc.: London, UK, 2010.
54.
Abbas, H.; Shaheen, S.; Amin, M. Organization of Multi-Agent Systems: An Overview. Int. J. Intell. Inf. Syst. 2015,4, 46–57.
[CrossRef]
55. Horling, B.; Lesser, V. A Survey of Multi-agent Organizational Paradigms. Knowl. Eng. Rev. 2004,19, 281–316. [CrossRef]
56.
Ansari, J.; Kazemi, A.; Gholami, A. Holonic structure: A state-of-the-art control architecture based on multi-agent systems for
optimal reactive power dispatch in Smartgrids. IET Gener. Transm. Distrib. 2015,9, 1922–1934. [CrossRef]
57.
Hajian, M.; Golsorkhi, M.S.; Ranjbar, A.; Shafiee, Q.; Savaghebi, M. V-I droop-based distributed event- and self-triggered
secondary control of AC microgrids. IET Smartgrid 2023,6, 271–283. [CrossRef]
58.
Akoka, J.; Bullen, C.V. Centralization versus decentralization of information systems: A critical survey and an annotated
bibliography. Cent. Inf. Syst. Res. 1978,11, 112–114.
59.
Janakiraman, R.; Waldvogel, M.; Zhang, Q. Indra: A peer-to-peer approach to network intrusion detection and prevention.
In Proceedings of the WET ICE 2003, Twelfth IEEE International Workshops on Enabling Technologies: Infrastructure for
Collaborative Enterprises, Linz, Austria, 9–11 June 2003; IEEE: New York, NY, USA, 2003; pp. 226–231. [CrossRef]
60.
Li, W.; Meng, W.; Liu, Z.; Au, M.H. Towards Blockchain-Based Software-Defined Networking: Security Challenges and Solutions.
Ieice Trans. Inf. Syst. 2020,E103.D, 196–203. [CrossRef]
61.
Chen, X.; Dinh, H.; Wang, B. Cascading Failures in Smartgrid—Benefits of Distributed Generation. In Proceedings of the 2010
First IEEE International Conference on Smartgrid Communications, Gaithersburg, MD, USA, 4–6 October 2010; IEEE: New York,
NY, USA, 2010; pp. 73–78. [CrossRef]
62.
Hines, P.D.; Rezaei, P. Cascading Failures in Power Systems; John Wiley & Sons: Hoboken, NJ, USA, 2016; pp. 215–234. [CrossRef]
63.
Huang, Z.; Wang, C.; Stojmenovic, M.; Nayak, A. Balancing System Survivability and Cost of Smartgrid Via Modeling Cascading
Failures. IEEE Trans. Emerg. Top. Comput. 2013,1, 45–56. [CrossRef]
64.
Erol-Kantarci, M.; Mouftah, H.T. Energy-Efficient Information and Communication Infrastructures in the Smartgrid: A Survey on
Interactions and Open Issues. IEEE Commun. Surv. Tutor. 2015,17, 179–197. [CrossRef]
65.
Pandey, S.K.; Mohanty, S.R.; Kishor, N. A literature survey on load–frequency control for conventional and distribution generation
power systems. Renew. Sustain. Energy Rev. 2013,25, 318–334. [CrossRef]
66.
Khan, M.W.; Wang, J. The research on multi-agent system for microgrid control and optimization. Renew. Sustain. Energy Rev.
2017,80, 1399–1411. [CrossRef]
67.
Naderi, S.; Blondin, M.J. A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent
Systems. IEEE Access 2023,11, 139728–139744. [CrossRef]
68.
Cerquides, J.; Farinelli, A.; Meseguer, P.; Ramchurn, S.D. A Tutorial on Optimization for Multi-Agent Systems. Comput. J. 2013,
57, 799–824. [CrossRef]
69.
Brazier, F.M.T.; Dunin-Keplicz, B.M.; Jennings, N.R.; Treur, J. Desire: Modeling Multi-Agent Systems in a Compositional Formal
Framework. Int. J. Coop. Inf. Syst. 1997,6, 67–94. [CrossRef]
70.
Iglesias, C.A.; Garijo, M.; González, J.C.; Velasco, J.R. Analysis and design of multiagent systems using MAS-CommonKADS. In
Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 1998; pp. 313–327. [CrossRef]
71.
Lidula, N.; Rajapakse, A. Microgrids research: A review of experimental microgrids and test systems. Renew. Sustain. Energy Rev.
2011,15, 186–202. [CrossRef]
72.
Cardoso, R.C.; Ferrando, A. A Review of Agent-Based Programming for Multi-Agent Systems. Computers 2021,10, 16. [CrossRef]
73. Kravari, K.; Bassiliades, N. A Survey of Agent Platforms. J. Artif. Soc. Soc. Simul. 2015,18, 11. [CrossRef]
74.
Pipattanasomporn, M.; Feroze, H.; Rahman, S. Multi-agent systems in a distributed Smartgrid: Design and implementation. In
Proceedings of the 2009 IEEE/PES Power Systems Conference and Exposition, Seattle, WA, USA, 15–18 March 2009; IEEE: New
York, NY, USA, 2009; pp. 1–8. [CrossRef]
75.
Bellifemine, F.L.; Caire, G.; Greenwood, D. Developing Multi-Agent Systems with JADE; John Wiley & Sons: Hoboken, NJ, USA,
2007. [CrossRef]
76.
Flueck, A.J.; Nguyen, C.P. Integrating Renewable and Distributed resources—IIT Perfect Power Smartgrid Prototype. In
Proceedings of the IEEE PES General Meeting, Minneapolis, MN, USA, 25–29 July 2010; IEEE: New York, NY, USA, 2010; pp. 1–4.
[CrossRef]
77.
Dimeas, A.L.; Hatziargyriou, N.D. Design of an MAS for an Island System. In Proceedings of the 2007 International Conference
on Intelligent Systems Applications to Power Systems, Kaohsiung, Taiwan, 5–8 November 2007; IEEE: New York, NY, USA, 2007;
pp. 1–3. [CrossRef]
Energies 2024,17, 3620 50 of 61
78.
Dimeas, A.L.; Hatziargyriou, N.D. Control Agents for Real Microgrids. In Proceedings of the 2009 15th International Conference
on Intelligent System Applications to Power Systems, Curitiba, Brazil, 8–12 November 2009; IEEE: New York, NY, USA, 2009;
pp. 1–5. [CrossRef]
79.
Oyarzabal, J.; Jimeno, J.; Ruela, J.; Engler, A.; Hardt, C. Agent-based micro grid management system. In Proceedings of the 2005
International Conference on Future Power Systems, Amsterdam, The Netherlands, 18 November 2005; IEEE: New York, NY,
USA, 2005. [CrossRef]
80.
Gomes, L.; Pinto, T.; Faria, P.; Vale, Z. Distributed intelligent management of microgrids using a multi-agent simulation platform.
In Proceedings of the 2014 IEEE Symposium on Intelligent Agents (IA), Orlando, FL, USA, 9–12 December 2014; IEEE: New York,
NY, USA, 2014; pp. 1–7. [CrossRef]
81.
Kumar Nunna, H.S.V.S.; Doolla, S. Multiagent-Based Distributed-Energy-Resource Management for Intelligent Microgrids. IEEE
Trans. Ind. Electron. 2013,60, 1678–1687. [CrossRef]
82.
Wu, K.; Zhou, H. A multi-agent-based energy-coordination control system for grid-connected large-scale wind–photovoltaic
energy storage power-generation units. Sol. Energy 2014,107, 245–259. [CrossRef]
83.
Aung, N.; Khambadkone, A.; Srinivasan, D.; Logenthiran, T. Agent-based Intelligent Control for Real-time Operation of a
Microgrid. In Proceedings of the 2010 Joint International Conference on Power Electronics, Drives and Energy Systems & 2010
Power India, New Delhi, India, 20–23 December 2010; IEEE: New York, NY, USA, 2010; pp. 1–6. [CrossRef]
84.
Kouluri, M.K.; Pandey, R.K. Intelligent agent-based micro grid control. In Proceedings of the 2011 2nd International Conference
on Intelligent Agent & Multi-Agent Systems, Chennai, India, 7–9 September 2011; IEEE: New York, NY, USA, 2011; pp. 62–66.
[CrossRef]
85.
Leng, D.; Polmai, S. Control of a microgrid based on distributed cooperative control of multi-agent system. Unpublished
manuscript, 2014.
86.
Rivera, S.; Farid, A.M.; Youcef-Toumi, K. A multi-agent system transient stability platform for resilient self-healing operation of
multiple microgrids. In Proceedings of the ISGT 2014, Washington, DC, USA, 19–22 February 2014; IEEE: New York, NY, USA,
2014; pp. 1–5. [CrossRef]
87.
Foo. Eddy, Y.S.; Gooi, H.B.; Chen, S.X. Multi-Agent System for Distributed Management of Microgrids. IEEE Trans. Power Syst.
2015,30, 24–34. [CrossRef]
88.
James, G.; Cohen, D.; Dodier, R.; Platt, G.; Palmer, D. A deployed multi-agent framework for distributed energy applications. In
Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, Hakodate, Japan, 8–12
May 2006; ACM: New York, NY, USA, 2006; pp. 676–678. [CrossRef]
89.
Logenthiran, T.; Srinivasan, D.; Wong, D. Multi-agent coordination for DER in MicroGrid. In Proceedings of the 2008 IEEE
International Conference on Sustainable Energy Technologies, Singapore, 24–27 November 2008; IEEE: New York, NY, USA, 2008;
pp. 77–82. [CrossRef]
90.
Hyacinth, S.; Nwana, D.T.; Ndumu, L.C.L.; Collis, J.C. Zeus: A toolkit for building distributed multiagent systems. Appl. Artif.
Intell. 1999,13, 129–185. [CrossRef]
91. Singh, A.; Juneja, D.; Sharma, A. Agent Development Toolkits. arXiv 2011, arXiv:1111.5930.
92.
Feroze, H. Multi-Agent Systems in Microgrids: Design and Implementation. Ph.D. Thesis, Virginia Institute of Technology,
Blacksburg, VA, USA, 2009.
93.
Li, T.; Xiao, Z.; Huang, M.; Yu, J.; Hu, J. Control system simulation of microgrid based on IP and Multi-Agent. In Proceedings of
the 2010 International Conference on Information, Networking and Automation (ICINA), Kunming, China, 17–19 October 2010;
IEEE: New York, NY, USA, 2010; Volume 1, pp. V1-235–V1-239. [CrossRef]
94.
Xiao, Z.; Li, T.; Huang, M.; Shi, J.; Yang, J.; Yu, J.; Wu, W. Hierarchical MAS Based Control Strategy for Microgrid. Energies 2010,
3, 1622–1638. [CrossRef]
95.
Melo, L.S.; Sampaio, R.F.; Leão, R.P.S.; Barroso, G.C.; Bezerra, J.R. Python-based multi-agent platform for application on power
grids. Int. Trans. Electr. Energy Syst. 2019,29, e12012. [CrossRef]
96.
Lützenberger, M.; Küster, T.; Konnerth, T.; Thiele, A.; Masuch, N.; Heßler, A.; Keiser, J.; Burkhardt, M.; Kaiser, S.; Albayrak, S.
JIAC V: A MAS framework for industrial applications. In Proceedings of the 2013 International Conference on Autonomous
Agents and Multi-Agent Systems, Saint Paul, MN, USA, 6–10 May 2013; pp. 1189–1190.
97.
Yilmaz, C.; Albayrak, S.; Lützenberger, M. Smartgrid architectures and the multi-agent system paradigm. In Proceedings of the
ENERGY 2014, The Fourth International Conference on Smart Grids, Green Communications and IT Energy-Aware Technologies,
Chamonix, France, 20–24 April 2014; IARIA XPS Press: Wilmington, NC, USA, 2014; pp. 90–95.
98.
Grunewald, D.; Lützenberger, M.; Chinnow, J.; Bye, R.; Bsufka, K.; Albayrak, S. Agent-based network security simulation. In
Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Taipei, Taiwan, 2–6
May 2011; Volume 2, pp. 1325–1326.
99.
Akyol, B.; Haack, J.; Carpenter, B.; Ciraci, S.; Vlachopoulou, M.; Tews, C. Volttron: An agent execution platform for the electric
power system. In Proceedings of the Third International Workshop on Agent Technologies for Energy Systems, Valencia, Spain,
5 June 2012; Editorial Universitat Politecnica de Valencia: Valencia, Spain, 2012.
100.
Haack, J.; Akyol, B.; Tenney, N.; Carpenter, B.; Pratt, R.; Carroll, T. VOLTTRON™: An agent platform for integrating electric
vehicles and Smartgrid. In Proceedings of the 2013 International Conference on Connected Vehicles and Expo (ICCVE), Las
Vegas, NV, USA, 2–6 December 2013; IEEE: New York, NY, USA, 2013; pp. 81–86. [CrossRef]
Energies 2024,17, 3620 51 of 61
101.
Katipamula, S.; Lutes, R.G.; Ngo, H.; Underhill, R.M. Transactional Network Platform: Applications; Pacific Northwest National
Lab.(PNNL): Richland, WA, USA, 2013. [CrossRef]
102.
Khamphanchai, W.; Saha, A.; Rathinavel, K.; Kuzlu, M.; Pipattanasomporn, M.; Rahman, S.; Akyol, B.; Haack, J. Conceptual
architecture of building energy management open source software (BEMOSS). In Proceedings of the IEEE PES Innovative
Smartgrid Technologies, Europe, Istanbul, Turkey, 12–15 October 2014; IEEE: New York, NY, USA, 2014; pp. 1–6. [CrossRef]
103.
Khamphanchai, W.; Pipattanasomporn, M.; Kuzlu, M.; Rahman, S. An agent-based open source platform for building energy
management. In Proceedings of the 2015 IEEE Innovative Smartgrid Technologies—Asia (ISGT ASIA), Bangkok, Thailand, 3–6
November 2015; IEEE: New York, NY, USA, 2015; pp. 1–6. [CrossRef]
104.
Akkermans, H.; Ygge, F.; Gustavsson, R. HOMEBOTS: Intelligent Decentralized Services for Energy Management. In Proceedings
of the Fourth International Symposium on the Management of Industrial and Corporate Knowledge (ISMICK’ 96), Rotterdam,
The Netherlands, 21–22 October 1996; Erasmus University: Rotterdam, The Netherlands, 1996.
105.
Ygge, F.; Gustavsson, R.; Akkermans, J. HOMEBOTS: Intelligent Agents for Decentralized Load Management. In Proceedings of
the Conference on Distribution Automation and Demand Side Management DA/DSM’96, Vienna, Austria, 8–10 October 1996;
pp. 597–611.
106.
Cohen, D.A. GridAgents™: Intelligent agent applications for integration of Distributed Energy Resources within distribution
systems. In Proceedings of the 2008 IEEE Power and Energy Society General Meeting—Conversion and Delivery of Electrical
Energy in the 21st Century, Pittsburgh, PA, USA, 20–24 July 2008; IEEE: New York, NY, USA, 2008; pp. 1–5. [CrossRef]
107.
Rahman, S.; Pipattanasomporn, M.; Teklu, Y. Intelligent Distributed Autonomous Power Systems (IDAPS). In Proceedings of
the 2007 IEEE Power Engineering Society General Meeting, Tampa, FL, USA, 24–28 June 2007; IEEE: New York, NY, USA, 2007;
pp. 1–8. [CrossRef]
108. Crosbie, T.; Dawood, N. IDEAS Project Final Report; Technical Report; Teesside University: Teesside, UK, 2016. [CrossRef]
109.
Short, M.; Dawood, M.; Crosbie, T.; Dawood, N.; Ala-Juusela, M. Visualization tools for energy awareness and management in
energy positive neighborhoods. In Proceedings of the 14th International Conference on Construction Applications of Virtual
Reality, Sharjah, UAE, 16–18 November 2014; Teesside University: Teesside, UK, 2014; pp. 275–284
110.
Kok, K.; Scheepers, M.; Kamphuis, R. Intelligence in Electricity Networks for Embedding Renewables and Distributed Generation;
Springer: Berlin/Heidelberg, Germany, 2010; pp. 179–209. [CrossRef]
111.
Kok, K. The PowerMatcher: Smart Coordination for the Smart Electricity Grid. Ph.D. Thesis, Vrije Universiteit, Amsterdam, The
Netherlands, 2013.
112.
Catterson, V.; Davidson, E.; McArthur, S. Issues in Integrating Existing Multi-Agent Systems for Power Engineering Applications.
In Proceedings of the 13th International Conference on, Intelligent Systems Application to Power Systems, Arlington, VA, USA,
6–10 November 2005; IEEE: New York, NY, USA, 2005. [CrossRef]
113. Burg, B. Foundation for Intelligent Physical Agents; FIPA: New York, NY, USA, 2002.
114. Lizan, F.J.M. Intelligent Buildings: Foundation for Intelligent Physical Agents. Int. J. Eng. Res. Appl. 2017,7, 21–25. [CrossRef]
115.
Teixeira, B.; Pinto, T.; Silva, F.; Santos, G.; Praça, I.; Vale, Z. Multi-Agent Decision Support Tool to Enable Interoperability among
Heterogeneous Energy Systems. Appl. Sci. 2018,8, 328. [CrossRef]
116.
Soon, G.K.; On, C.K.; Anthony, P.; Hamdan, A.R. A Review on Agent Communication Language; Springer: Berlin/Heidelberg,
Germany, 2019; pp. 481–491. [CrossRef]
117.
Finin, T.; Fritzson, R.; McKay, D.; McEntire, R. KQML as an agent communication language. In Proceedings of the Third
International Conference on Information and Knowledge Management, Gaithersburg, MD, USA, 29 November–2 December 1994;
ACM Press: New York, NY, USA, 1994; pp. 456–463. [CrossRef]
118.
Basso, T.; DeBlasio, R. IEEE Smartgrid Series of Standards IEEE 2030 (Interoperability) and IEEE 1547 (Interconnection) Status; Technical
Report; National Renewable Energy Lab. (NREL): Golden, CO, USA, 2012.
119.
Ho, Q.D.; Gao, Y.; Rajalingham, G.; Le-Ngoc, T.; Ho, Q.D.; Gao, Y.; Rajalingham, G.; Le-Ngoc, T. Smartgrid Communications Network
(SGCN); Springer: Berlin/Heidelberg, Germany, 2014; pp. 15–30. [CrossRef]
120. Liang, Y.; Campbell, R.H. Understanding and simulating the IEC 61850 standard. Unpublished manuscript, 2008.
121.
Skoko, V.; Atlagic, B.; Isakov, N. Comparative realization of IEC 60870-5 industrial protocol standards. In Proceedings of the 2014
22nd Telecommunications Forum Telfor (TELFOR), Belgrade, Serbia, 25–27 November 2014; IEEE: New York, NY, USA, 2014;
pp. 987–990. [CrossRef]
122. Curtis, K. A DNP3 Protocol Primer; DNP Users Group: Calgary, AB, Canada, 2005.
123.
Brunner, C. IEC 61850 for power system communication. In Proceedings of the 2008 IEEE/PES Transmission and Distribution
Conference and Exposition, Chicago, IL, USA, 21–24 April 2008; IEEE: New York, NY, USA, 2008; pp. 1–6. [CrossRef]
124.
Mackiewicz, R. Overview of IEC 61850 and Benefits. In Proceedings of the 2006 IEEE PES Power Systems Conference and
Exposition, Atlanta, GA, USA, 29 October–1 November 2006; IEEE: New York, NY, USA, 2006; pp. 623–630. [CrossRef]
125. Pal, A.; Dash, R. A Paradigm Shift in Substation Engineering: IEC 61850 Approach. Procedia Technol. 2015,21, 8–14. [CrossRef]
126.
Sidhu, T.S.; Yin, Y. Modeling and Simulation for Performance Evaluation of IEC61850-Based Substation Communication Systems.
IEEE Trans. Power Deliv. 2007,22, 1482–1489. [CrossRef]
127.
Zhabelova, G.; Vyatkin, V. Multiagent Smartgrid Automation Architecture Based on IEC 61850/61499 Intelligent Logical Nodes.
IEEE Trans. Ind. Electron. 2012,59, 2351–2362. [CrossRef]
128.
McMorran, A.W. An introduction to IEC 61970-301 & 61968-11: The common information model. Univ. Strathclyde 2007,93, 180.
Energies 2024,17, 3620 52 of 61
129. Britton, J.; deVos, A. CIM-based standards and CIM evolution. IEEE Trans. Power Syst. 2005,20, 758–764. [CrossRef]
130.
Hippolyte, J.L.; Howell, S.; Yuce, B.; Mourshed, M.; Sleiman, H.A.; Vinyals, M.; Vanhee, L. Ontology-based demand side flexibility
management in Smartgrids using a multi-agent system. In Proceedings of the 2016 IEEE International Smart Cities Conference
(ISC2), Trento, Italy, 12–15 September 2016; IEEE: New York, NY, USA, 2016; pp. 1–7. [CrossRef]
131.
Santos, G.; Silva, F.; Teixeira, B.; Vale, Z.; Pinto, T. Power Systems Simulation Using Ontologies to Enable the Interoperability of
Multi-Agent Systems. In Proceedings of the 2018 Power Systems Computation Conference (PSCC), Dublin, Ireland, 11–15 June
2018; IEEE: New York, NY, USA, 2018; pp. 1–7. [CrossRef]
132.
McNaughton, G.A.; McNaughton, W.P. MultiSpeak® 3.0 Users Guide; National Rural Electric Cooperative Association: Arlington,
VA, USA, 2006.
133.
Mozina, C.J. Impact of Smartgrid and green power generation on distribution systems. In Proceedings of the 2012 IEEE
PES Innovative Smartgrid Technologies (ISGT), Washington, DC, USA, 16–20 January 2012; IEEE: New York, NY, USA, 2013;
Volume 49, pp. 1079–1090. [CrossRef]
134. IEEE Standard 1547 for Interconnecting Distributed Resources with Electric Power Systems; IEEE: New York, NY, USA, 2003.
135.
Apostolov, A. Multi-agent systems and IEC 61850. In Proceedings of the 2006 IEEE Power Engineering Society General Meeting,
Montreal, QC, Canada, 18–22 June 2006; IEEE: New York, NY, USA, 2006; 6p. [CrossRef]
136.
Saleem, A.; Honeth, N.; Nordström, L. A case study of multi-agent interoperability in IEC 61850 environments. In Proceedings of
the 2010 IEEE PES Innovative Smartgrid Technologies Conference Europe (ISGT Europe), Gothenburg, Sweden, 11–13 October
2010; IEEE: New York, NY, USA, 2010; pp. 1–8. [CrossRef]
137.
Samirmi, F.; Tang, W.; Wu, H. Power transformer condition monitoring and fault diagnosis with multi-agent system based on
ontology reasoning. In Proceedings of the 2013 IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC),
Kowloon, Hong Kong, 8–11 December 2013; IEEE: New York, NY, USA, 2013; pp. 1–6. [CrossRef]
138.
Van Dam, K.; Lukszo, Z. Modeling Energy and Transport Infrastructures as a Multi-Agent System using a Generic Ontology. In
Proceedings of the 2006 IEEE international conference on systems, Man and Cybernetics, Taipei, Taiwan, 8–11 October 2006; IEEE:
New York, NY, USA, 2006; Volume 1, pp. 890–895. [CrossRef]
139.
Wei, S.; Xiangnan, W.; Houji, C.; Guowei, P. Multi-agent architecture of energy-management system based on IEC 61970 CIM. In
Proceedings of the 2007 International Power Engineering Conference (IPEC 2007), Singapore, 3–6 December 2007; IEEE: New
York, NY, USA, 2007; pp. 1366–1370.
140.
Ma, Z.; Schultz, M.J.; Christensen, K.; Værbak, M.; Demazeau, Y.; Jørgensen, B.N. The Application of Ontologies in Multi-Agent
Systems in the Energy Sector: A Scoping Review. Energies 2019,12, 3200. [CrossRef]
141.
Logenthiran, T.; Srinivasan, D.; Khambadkone, A. Multi-agent system for energy resource scheduling of integrated microgrids in
a distributed system. Electr. Power Syst. Res. 2011,81, 138–148. [CrossRef]
142.
Santos, G.; Pinto, T.; Vale, Z.; Morais, H.; Praça, I. Upper Ontology for Multi-Agent Energy Systems’ Applications. Adv. Intell.
Syst. Comput. 2013,217, 617–624. [CrossRef]
143.
Poveda, G.; Schumann, R. An Ontology-Driven Approach for Modeling a Multi-agent-Based Electricity Market. In Proceedings
of the Multiagent System Technologies: 14th German Conference, MATES 2016, Klagenfurt, Austria, 27–30 September 2016;
Proceedings 14; Springer: Berlin/Heidelberg, Germany, 2016; Volume 9872, pp. 27–40. [CrossRef]
144.
Tilipakis, N.; Douligeris, C.; Neris, A. Ontology-based tools for the management of customers’ portfolios in a deregulated
electricity market environment. In Metadata and Semantics; Springer: Boston, MA, USA, 2009; pp. 269–278. [CrossRef]
145.
Kofler, M.; Reinisch, C.; Kastner, W. A semantic representation of energy-related information in future smart homes. Energy Build.
2012,47, 169–179. [CrossRef]
146.
Alexopoulos, P.; Kafentzis, K.; Zoumas, C. ELMO: An Interoperability Ontology for the Electricity Market. In Proceedings of the
International Conference on e-Business, Wuhan, China, 23–24 May 2009; SciTePress: Setubal, Portugal, 2009; pp. 15–20.
147.
Santos, G.; Pinto, T.; Vale, Z.; Praça, I.; Morais, H. Enabling communications in heterogeneous multi-agent systems: Electricity
markets ontology. ADCAIJ Adv. Distrib. Comput. Artif. Intell. J. 2016,5, 15–42. [CrossRef]
148.
Santos, G.; Pinto, T.; Praça, I.; Vale, Z. EPEX ontology: Enhancing agent-based electricity market simulation. In Proceedings of
the 2017 19th International Conference on Intelligent System Application to Power Systems (ISAP), San Antonio, TX, USA, 17–20
September 2017; IEEE: New York, NY, USA, 2017; pp. 1–6. [CrossRef]
149.
Santos, G.; Pinto, T.; Praça, I.; Vale, Z. Nord pool ontology to enhance electricity markets simulation in MASCEM. In Proceedings
of the Progress in Artificial Intelligence: 18th EPIA Conference on Artificial Intelligence, EPIA 2017, Porto, Portugal, 5–8 September
2017; Proceedings 18; Springer: Berlin/Heidelberg, Germany, 2017; pp. 283–294. [CrossRef]
150.
Duan, R.; Deconinck, G. Future electricity market interoperability of a multi-agent model of the Smartgrid. In Proceedings of the
2010 International Conference on Networking, Sensing and Control (ICNSC), Chicago, IL, USA, 10–12 April 2010; IEEE: New
York, NY, USA, 2010; pp. 625–630. [CrossRef]
151.
Gomes, L.; Vale, Z.A.; Corchado, J.M. Multi-agent microgrid management system for single-board computers: A case study on
peer-to-peer energy trading. IEEE Access 2020,8, 64169–64183. [CrossRef]
152.
Lopez, M.; Martín, S.; Aguado, J.; De La Torre, S. Market-oriented operation in microgrids using multi-agent systems. In
Proceedings of the 2011 International Conference on Power Engineering, Energy and Electrical Drives, Malaga, Spain, 11–13 May
2011; IEEE: New York, NY, USA, 2011; pp. 1–6. [CrossRef]
Energies 2024,17, 3620 53 of 61
153.
Ramachandran, B.; Srivastava, S.K.; Edrington, C.S.; Cartes, D.A. An Intelligent Auction Scheme for Smartgrid Market Using a
Hybrid Immune Algorithm. IEEE Trans. Ind. Electron. 2011,58, 4603–4612. [CrossRef]
154.
Dimeas, A.; Hatziargyriou, N. A Multi-Agent System for Microgrids. In Proceedings of the 2004 Hellenic Conference on Artificial
Intelligence, Samos, Greece, 5–8 May 2004; Springer: Berlin/Heidelberg, Germany, 2004; Volume 3025, pp. 447–455. [CrossRef]
155.
Akbari, A.; Mozayani, N. A Holonic Multi Agent System For Operating Smartgrid Market. In Proceedings of the 4rd Conference
on Emerging Trends in Energy Conservation, Tehran, Iran, 18 February 2015; Civilica: Tehran, Iran, 2015.
156.
Babar, M.; Nguyen, P.; Cuk, V.; Kamphuis, R.; Kling, W. Complex bid model and strategy for dispatchable loads in real time
market-based demand response. In Proceedings of the IEEE PES Innovative Smartgrid Technologies, Europe, Istanbul, Turkey,
12–15 October 2014; IEEE: New York, NY, USA, 2014; Volume 2015. [CrossRef]
157.
Logenthiran, T.; Srinivasan, D. Multi-Agent System for the Operation of an Integrated Microgrid. J. Renew. Sustain. Energy 2011,
4, 013116. [CrossRef]
158.
Vytelingum, P.; Ramchurn, S.D.; Voice, T.C.; Rogers, A.; Jennings, N.R. Trading agents for the smart electricity grid. In Proceedings
of the Adaptive Agents and Multi-Agent Systems, Toronto, ON, Canada, 10–14 May 2010; Volume 1, pp. 897–904.
159.
Shafie-khah, M.; Catalão, J. A Stochastic Multi-Layer Agent-Based Model to Study Electricity Market Participants Behavior. IEEE
Trans. Power Syst. 2014,30, 867–881. [CrossRef]
160.
Tushar, W.; Chai, B.; Yuen, C.; Huang, S.; Smith, D.; Poor, H.V.; Yang, Z. Energy Storage Sharing in Smartgrid: A Modified
Auction Based Approach. IEEE Trans. Smartgrid 2016,7, 1462–1475. [CrossRef]
161.
Amato, A.; Di Martino, B.; Scialdone, M.; Venticinque, S. Multi-agent Negotiation of Decentralized Energy Production in Smart
Micro-grid. In Proceedings of the Intelligent Distributed Computing VIII, Madrid, Spain, 3–5 September 2015; Springer: Berlin,
Germany, 2015; Volume 570, pp. 155–160. [CrossRef]
162.
Mezquita, Y.; Gazafroudi, A.S.; Corchado, J.M.; Shafie-Khah, M.; Laaksonen, H.; Kamišali´c, A. Multi-Agent Architecture for
Peer-to-Peer Electricity Trading-based on Blockchain Technology. In Proceedings of the 2019 XXVII International Conference
on Information, Communication and Automation Technologies (ICAT), Sarajevo, Bosnia and Herzegovina, 20–23 October 2019;
IEEE: New York, NY, USA, 2019; pp. 1–6. [CrossRef]
163.
Morstyn, T.; Teytelboym, A.; Mcculloch, M.D. Bilateral Contract Networks for Peer-to-Peer Energy Trading. IEEE Trans. Smartgrid
2019,10, 2026–2035. [CrossRef]
164.
Wang, Z.; Wang, L. Adaptive Negotiation Agent for Facilitating Bi-Directional Energy Trading between Smart Building and
Utility Grid. IEEE Trans. Smartgrid 2013,4, 702–710. [CrossRef]
165.
Nagata, T.; Ueda, Y.; Utatani, M. A multi-agent approach to Smartgrid energy management. In Proceedings of the 2012 10th
International Power & Energy Conference (IPEC), Ho Chi Minh City, Vietnam, 12–14 December 2012; IEEE: New York, NY, USA,
2012; pp. 327–331. [CrossRef]
166.
Ali, S.; Ahmed, S.; Marwat, S.N.K. A practical approach to consensus-based control of multi-agent systems. In Proceedings of the
2018 International Symposium on Recent Advances in Electrical Engineering (RAEE), Islamabad, Pakistan, 17–18 October 2018;
IEEE: New York, NY, USA, 2018; pp. 1–5. [CrossRef]
167.
Badawy, R.; Yassine, A.; Heßler, A.; Hirsch, B.; Albayrak, S. A novel multi-agent system utilizing quantum-inspired evolution for
demand side management in the future Smartgrid. Integr. Comput.-Aided Eng. 2013,20, 127–141. [CrossRef]
168.
Beer, S.; Sonnenschein, M.; Appelrath, H.J. Towards a self-organization mechanism for agent associations in electricity spot
markets. In Proceedings of the Informatik 2011, Berlin, Germany, 4–7 October 2011; GI-Jahrestagung 2011; p. 266.
169.
Lopes, F.; Ilco, C.; Sousa, J. Bilateral negotiation in energy markets: Strategies for promoting demand response. In Proceedings of
the 2013 10th international conference on the European Energy Market (EEM), Stockholm, Sweden, 27–31 May 2013; IEEE: New
York, NY, USA, 2013; pp. 1–6. [CrossRef]
170.
Amini, M.H.; Nabi, B.; Haghifam, M.R. Load management using multi-agent systems in smart distribution network. In
Proceedings of the 2013 IEEE Power & Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; IEEE: New
York, NY, USA, 2013; pp. 1–5. [CrossRef]
171.
Hernández, L.; Baladron, C.; Aguiar, J.M.; Carro, B.; Sanchez-Esguevillas, A.; Lloret, J.; Chinarro, D.; Gomez-Sanz, J.J.; Cook, D. A
Multi-Agent-System Architecture for Smartgrid Management and Forecasting of Energy Demand in Virtual Power Plants. IEEE
Commun. Mag. 2013,51, 106–113. [CrossRef]
172. Klaimi, J.; Rahim-Amoud, R.; Merghem-Boulahia, L.; Jrad, A. A novel loss-based energy management approach for Smartgrids
using Multi-Agent Systems and Intelligent Storage Systems. Sustain. Cities Soc. 2018,39, 344–357. [CrossRef]
173.
Binetti, G.; Davoudi, A.; Naso, D.; Turchiano, B.; Lewis, F.L. A distributed auction-based algorithm for the nonconvex economic
dispatch problem. IEEE Trans. Ind. Inf. 2013,10, 1124–1132. [CrossRef]
174.
Roesch, M.; Linder, C.; Zimmermann, R.; Rudolf, A.; Hohmann, A.; Reinhart, G. Smartgrid for Industry Using Multi-Agent
Reinforcement Learning. Appl. Sci. 2020,10, 6900. [CrossRef]
175.
Simões, M.G.; Bhattarai, S. Multi agent-based energy management control for commercial buildings. In Proceedings of the 2011
IEEE Industry Applications Society Annual Meeting, Orlando, FL, USA, 9–13 October 2011; IEEE: New York, NY, USA, 2011;
pp. 1–6. [CrossRef]
176.
Zhao, P.; Suryanarayanan, S.; Simoes, M.G. An Energy Management System for Building Structures Using a Multi-Agent
Decision-Making Control Methodology. IEEE Trans. Ind. Appl. 2013,49, 322–330. [CrossRef]
Energies 2024,17, 3620 54 of 61
177.
Chahinaze, A.; Faquir, S.; Yahyaouy, A. Intelligent Optimization And Management System For Renewable Energy Systems Using
Multi-Agent. IAES Int. J. Artif. Intell. 2019,8, 352. [CrossRef]
178.
Hurtado, L.; Nguyen, P.; Kling, W. Agent-based control for building energy management in the Smartgrid framework. In
Proceedings of the IEEE PES Innovative Smartgrid Technologies Europe, Istanbul, Turkey, 12–15 October 2015; IEEE: New York,
NY, USA, 2014; Volume 2015, pp. 1–6. [CrossRef]
179.
Xu, X.; Jia, Y.; Xu, Y.; Xu, Z.; Chai, S.; Lai, C.S. A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home
Energy Management. IEEE Trans. Smartgrid 2020,11, 3201–3211. [CrossRef]
180.
Vytelingum, P.; Voice, T.; Ramchurn, S.; Rogers, A.; Jennings, N. Agent-Based Micro-Storage Management for the Smartgrid. In
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, Toronto, ON, Canada, 10–14
May 2010; Volume 1, pp. 39–46. [CrossRef]
181.
Khan, M.W.; Wang, J.; Xiong, L.; Ma, M. Modeling and optimal management of distributed microgrid using multi-agent systems.
Sustain. Cities Soc. 2018,41, 154–169. [CrossRef]
182.
Nagata, T.; Ueda, Y.; Utatani, M. A multi-agent approach to Smartgrid operations. In Proceedings of the 2012 IEEE International
Conference on Power System Technology (POWERCON), Auckland, New Zealand, 30 October–2 November 2012; IEEE: New
York, NY, USA, 2012; pp. 1–5. [CrossRef]
183.
Alishavandi, A.M.; Moghaddas-Tafreshi, S.M. Interactive decentralized operation with effective presence of renewable energies
using multi-agent systems. Int. J. Electr. Power Energy Syst. 2019,112, 36–48. [CrossRef]
184.
Li, J.; James, G.; Poulton, G. Set-Points Based Optimal Multi-Agent Coordination for Controlling Distributed Energy Loads. In
Proceedings of the 2009 Third IEEE International Conference on Self-Adaptive and Self-Organizing Systems, San Francisco, CA,
USA, 14–18 September 2009; IEEE: New York, NY, USA, 2009; pp. 265–271. [CrossRef]
185.
Kuo, M.T.; Lu, S.D. Design and implementation of real-time intelligent control and structure based on multi-agent systems in
microgrids. Energies 2013,6, 6045–6059. [CrossRef]
186.
Leo, R.; Milton, R.; Mahadevan, S. Multi agent systems-based distributed control and automation of micro-grid using MACSimJX.
In Proceedings of the 2016 10th International Conference on Intelligent Systems and Control (ISCO), Coimbatore, India, 7–8
January 2016; IEEE: New York, NY, USA, 2016; pp. 1–6. [CrossRef]
187.
Leo, R.; Morais, A.A.; Rathnakumar, R.; Ponnivalavan, S.; Thavam, L.D. Micro-grid Grid Outage Management Using Multi-agent
Systems. In Proceedings of the 2017 Second International Conference on Recent Trends and Challenges in Computational Models
(ICRTCCM), Tindivanam, India, 3–4 February 2017; IEEE: New York, NY, USA, 2017; pp. 363–368. [CrossRef]
188.
Raju, L.; Morais, A.A. Multi-agent systems-based advanced energy management of smart micro-grid. In Multi Agent Systems-
Strategies and Applications; IntechOpen: London, UK, 2020. [CrossRef]
189.
Dou, C.X.; Liu, B. Multi-Agent Based Hierarchical Hybrid Control for Smart Microgrid. IEEE Trans. Smartgrid 2013,4, 771–778.
[CrossRef]
190.
Radhakrishnan, B.M.; Srinivasan, D. A multi-agent-based distributed energy management scheme for smart grid applications.
Energy 2016,103, 192–204. [CrossRef]
191.
Radhakrishnan, B.M.; Srinivasan, D.; Mehta, R. Fuzzy-based multi-agent system for distributed energy management in Smartgrids.
Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 2016,24, 781–803. [CrossRef]
192.
Carvalho, M.; Perez, C.; Granados, A. An adaptive multi-agent-based approach to Smartgrids control and optimization. Energy
Syst. 2012,3, 61–76. [CrossRef]
193.
Menon, R.B.; Menon, S.B.; Srinivasan, D.; Jain, L. Online reinforcement learning in multi-agent systems for distributed energy
systems. In Proceedings of the 2014 IEEE Innovative Smartgrid Technologies-Asia (ISGT ASIA), Kuala Lumpur, Malaysia, 20–23
May 2014; IEEE: New York, NY, USA, 2014; pp. 791–796. [CrossRef]
194.
Gupta, R.; Jha, D.K.; Yadav, V.K.; Kumar, S. A multi-agent framework for operation of a Smartgrid. Energy Power Eng. 2013,
5, 1330. [CrossRef]
195.
Manickavasagam, K. Intelligent energy control center for distributed generators using multi-agent system. IEEE Trans. Power
Syst. 2014,30, 2442–2449. [CrossRef]
196.
Menon, R.B.; Menon, S.B.; Srinivasan, D.; Jain, L. Fuzzy logic decision-making in multi-agent systems for Smartgrids. In
Proceedings of the 2013 IEEE Computational Intelligence Applications in Smartgrid (CIASG), Singapore, 16–19 April 2013; IEEE:
New York, NY, USA, 2013; pp. 44–50. [CrossRef]
197.
Ullah, M.H.; Alseyat, A.; Park, J.D. Multi-agent system-based distributed energy management in Smartgrid under uncertainty.
In Proceedings of the 2019 IEEE Energy Conversion Congress and Exposition (ECCE), Baltimore, MD, USA, 29 September–3
October 2019; IEEE: New York, NY, USA, 2019; pp. 3462–3468. [CrossRef]
198.
Manickavasagam, K.; Nithya, M.; Priya, K.; Shruthi, J.; Krishnan, S.; Misra, S.; Manikandan, S. Control of distributed generator
and Smartgrid using multi-agent system. In Proceedings of the 2011 1st International Conference on Electrical Energy Systems,
Chennai, Tamilnadu, India, 3–5 January 2011; IEEE: New York, NY, USA, 2011; pp. 212–217. [CrossRef]
199.
Bidram, A.; Lewis, F.L.; Davoudi, A.; Qu, Z. Frequency control of electric power microgrids using distributed cooperative control
of multi-agent systems. In Proceedings of the 2013 IEEE International Conference on Cyber Technology in Automation, Control
and Intelligent Systems, Nanjing, China, 26–29 May 2013; IEEE: New York, NY, USA, 2013; pp. 223–228. [CrossRef]
200.
Bidram, A.; Davoudi, A.; Lewis, F.L.; Qu, Z. Secondary control of microgrids based on distributed cooperative control of
multi-agent systems. IET Gener. Transm. Distrib. 2013,7, 822–831. [CrossRef]
Energies 2024,17, 3620 55 of 61
201.
Manditereza, P.T.; Bansal, R.C. Multi-agent-based distributed voltage control algorithm for smart grid applications. Electr. Power
Components Syst. 2016,44, 2352–2363. [CrossRef]
202.
Shayanfar, H.A.; Malek, S. Photovoltaic Microgrids Control by the Cooperative Control of Multi-Agent Systems. In Proceedings
of the 2015 30th International Power System Conference (PSC), Tehran, Iran, 23–25 November 2015; IEEE: New York, NY, USA,
2015; pp. 287–293. [CrossRef]
203.
Wu, X.; Jiang, P.; Lu, J. Multiagent-Based Distributed Load Shedding for Islanded Microgrids. Energies 2014,7, 6050–6062.
[CrossRef]
204.
Fishov, A.G.; Klavsuts, I.L.; Klavsuts, D.A. Multi-Agent Regulation of Voltage in Smartgrid System with the Use of Distributed
Generation and Customers. Appl. Mech. Mater. 2015,698, 761–767. [CrossRef]
205.
Singh, V.P.; Kishor, N.; Samuel, P. Distributed Multi-Agent System-Based Load Frequency Control for Multi-Area Power System
in Smartgrid. IEEE Trans. Ind. Electron. 2017,64, 5151–5160. [CrossRef]
206.
Sajadi, A.; Farag, H.; Biczel, P.; El-Saadany, E. Voltage regulation based on fuzzy multi-agent control scheme in smart grids.
In Proceedings of the 2012 IEEE Energytech, Cleveland, OH, USA, 29–31 May 2012; IEEE: New York, NY, USA, 2012; pp. 1–5.
[CrossRef]
207.
Shahbazi, H.; Karbalaei, F. Decentralized Voltage Control of Power Systems Using Multi-agent Systems. J. Mod. Power Syst. Clean
Energy 2020,8, 249–259. [CrossRef]
208.
Morstyn, T.; Hredzak, B.; Agelidis, V.G. Distributed Cooperative Control of Microgrid Storage. IEEE Trans. Power Syst. 2014,
30, 2780–2789. [CrossRef]
209.
Kim, B.; Lavrova, O. Optimal Power Flow and Energy-sharing Among Multi-agent Smart Buildings in the Smartgrid. In
Proceedings of the 2013 IEEE Energytech, Cleveland, OH, USA, 21–23 May 2013; IEEE: New York, NY, USA, 2013; pp. 1–5.
[CrossRef]
210.
Celik, B.; Roche, R.; Bouquain, D.; Miraoui, A. Coordinated Neighborhood Energy Sharing Using Game Theory and Multi-agent
Systems. In Proceedings of the 2017 IEEE Manchester PowerTech, Manchester, UK, 18–22 June 2017; IEEE: New York, NY, USA,
2017; pp. 1–6. [CrossRef]
211.
Abd El-Rahim, A.M.; Abd-El-Geliel, M.; Helal, A. Micro Grid Energy Management Using Multi-agent Systems. In Proceedings of
the 2016 Eighteenth International Middle East Power Systems Conference (MEPCON), Cairo, Egypt, 27–29 December 2016; IEEE:
New York, NY, USA, 2016; pp. 772–779. [CrossRef]
212.
Mangiatordi, F.; Pallotti, E.; Panzieri, D.; Capodiferro, L. Multi Agent System for Cooperative Energy Management in Microgrids.
In Proceedings of the 2016 IEEE 16th International Conference on Environment and Electrical Engineering (EEEIC), Florence,
Italy, 7–10 June 2016; IEEE: New York, NY, USA, 2016; pp. 1–5. [CrossRef]
213.
Lee, S.J.; Choi, J.Y.; Lee, H.J.; Won, D.J. Distributed Coordination Control Strategy for a Multi-Microgrid Based on a Consensus
Algorithm. Energies 2017,10, 1017. [CrossRef]
214.
Wang, Y.; Nguyen, T.L.; Xu, Y.; Tran, Q.T.; Caire, R. Peer-to-peer control for networked microgrids: Multi-layer and multi-agent
architecture design. IEEE Trans. Smartgrid 2020,11, 4688–4699. [CrossRef]
215.
Baran, M.E.; El-Markabi, I.M. A Multiagent-Based Dispatching Scheme for Distributed Generators for Voltage Support on
Distribution Feeders. IEEE Trans. Power Syst. 2007,22, 52–59. [CrossRef]
216.
Fan, Y.; Zhang, C.; Song, C. Sampling-based Self-triggered Coordination Control for Multi-agent Systems with Application to
Distributed Generators. Int. J. Syst. Sci. 2018,49, 3048–3062. [CrossRef]
217.
Khazaei, J.; Nguyen, D.H. Multi-Agent Consensus Design for Heterogeneous Energy Storage Devices with Droop Control in
Smartgrids. IEEE Trans. Smartgrid 2017,10, 1395–1404. [CrossRef]
218.
Yu, W.; Li, C.; Yu, X.; Wen, G.; Lü, J. Distributed consensus strategy for economic power dispatch in a smart grid. In Proceedings
of the 2015 10th Asian Control Conference (ASCC), Sabah, Malaysia, 31 May–3 June 2015; IEEE: New York, NY, USA, 2015;
Volume 11, pp. 4688–4699. [CrossRef]
219.
Wang, R.; Li, Q.; Zhang, B.; Wang, L. Distributed consensus-based algorithm for economic dispatch in a microgrid. IEEE Trans.
Smartgrid 2018,10, 3630–3640. [CrossRef]
220.
Yang, S.; Tan, S.; Xu, J.X. Consensus-based approach for economic dispatch problem in a smart grid. IEEE Trans. Power Syst. 2013,
28, 4416–4426. [CrossRef]
221.
Li, C.; Savaghebi, M.; Guerrero, J.M.; Coelho, E.A.; Vasquez, J.C. Operation Cost Minimization of Droop-Controlled AC
Microgrids Using Multiagent-Based Distributed Control. Energies 2016,9, 717. [CrossRef]
222.
Yu, W.; Li, C.; Yu, X.; Wen, G.; Lü, J. Economic power dispatch in Smartgrids: A framework for distributed optimization and
consensus dynamics. Sci. China Inf. Sci. 2018,61, 1–16. [CrossRef]
223.
Zhao, T.; Ding, Z. Distributed Agent Consensus-Based Optimal Resource Management for Microgrids. IEEE Trans. Sustain.
Energy 2017,9, 443–452. [CrossRef]
224.
Zhang, W.; Liu, W.; Wang, X.; Liu, L.; Ferrese, F. Online Optimal Generation Control Based on Constrained Distributed Gradient
Algorithm. IEEE Trans. Power Syst. 2014,30, 35–45. [CrossRef]
225.
Amicarelli, E.; Tran, Q.T.; Bacha, S. Multi-agent System for Day-ahead Energy Management of Microgrid. In Proceedings of
the 2016 18th European Conference on Power Electronics and Applications (EPE’16 ECCE Europe), Karlsruhe, Germany, 5–9
September 2016; IEEE: New York, NY, USA, 2016; pp. 1–10. [CrossRef]
Energies 2024,17, 3620 56 of 61
226.
Hajjar, S.; Bratcu, A.I.; Hably, A. A Day-ahead Centralized Unit Commitment Algorithm for A Multi-agent Smartgrid. In
Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS 2015)—4th International
Workshop on Smart Energy Networks & Multi-Agent Systems (SEN MAS 2015), Lodz, Poland, 13–16 September 2015; PTI 2015;
pp. 265–271.
227.
Farzaneh, G.; Mohsen, A.; Vali, D. A Multi-agent reinforcement learning algorithm with fuzzy approximation for Distributed
Stochastic Unit Commitment. J. Intell. Fuzzy Syst. 2019,37, 6613–6628. [CrossRef]
228.
Qin, J.; Yu, N.; Gao, Y. Solving Unit Commitment Problems with Multi-step Deep Reinforcement Learning. In Proceedings of the
2021 IEEE International Conference on Communications, Control and Computing Technologies for Smartgrids (SmartgridComm),
Aachen, Germany, 25–28 October 2021; IEEE: New York, NY, USA, 2021; pp. 140–145. [CrossRef]
229.
Li, W.; Logenthiran, T.; Woo, W.L.; Phan, V.T.; Srinivasan, D. Implementation of Demand Side Management of a Smart Home
Using Multi-agent System. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC,
Canada, 24–29 July 2016; IEEE: New York, NY, USA, 2016; pp. 2028–2035. [CrossRef]
230.
Aladdin, S.; El-Tantawy, S.; Fouda, M.M.; Eldien, A.S.T. MARLA-SG: Multi-Agent Reinforcement Learning Algorithm for Efficient
Demand Response in Smartgrid. IEEE Access 2020,8, 210626–210639. [CrossRef]
231.
Fazal, R.; Solanki, J.; Solanki, S.K. Demand Response Using Multi-agent System. In Proceedings of the 2012 North American
Power Symposium (NAPS), Champaign, IL, USA, 9–11 September 2012; IEEE: New York, NY, USA, 2012; pp. 1–6. [CrossRef]
232.
Lee, J.; Wang, W.; Niyato, D. Demand Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning for Smartgrids. In
Proceedings of the 2020 IEEE International Conference on Communications, Control and Computing Technologies for Smartgrids
(SmartgridComm), Online, 11–13 November 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [CrossRef]
233.
Dusparic, I.; Harris, C.; Marinescu, A.; Cahill, V.; Clarke, S. Multi-agent Residential Demand Response Based on Load Forecasting.
In Proceedings of the 2013 1st IEEE Conference on Technologies for Sustainability (SusTech), Portland, OR, USA, 1–2 August
2013; IEEE: New York, NY, USA, 2013; pp. 90–96. [CrossRef]
234.
Mets, K.; Strobbe, M.; Verschueren, T.; Roelens, T.; De Turck, F.; Develder, C. Distributed Multi-agent Algorithm for Residential
Energy Management in Smartgrids. In Proceedings of the 2012 IEEE Network Operations and Management Symposium, Maui,
Hawaii, USA, 16–20 April 2012; IEEE: New York, NY, USA, 2012; pp. 435–443. [CrossRef]
235.
Li, W.; Logenthiran, T.; Woo, W.L. Intelligent Multi-agent System for Smart Home Energy Management. In Proceedings of the
2015 IEEE Innovative Smartgrid Technologies-Asia (ISGT ASIA), Bangkok, Thailand, 3–6 November 2015; IEEE: New York, NY,
USA, 2015; pp. 1–6. [CrossRef]
236.
Li, W.; Logenthiran, T.; Phan, V.T.; Woo, W.L. Proposed Optimised Smartgrid System using Multi-Agent System. In Proceedings
of the 2018 IEEE Innovative Smartgrid Technologies-Asia (ISGT Asia), Singapore, 22–25 May 2018; IEEE: New York, NY, USA,
2018; pp. 528–533. [CrossRef]
237.
Nunna, H.K.; Srinivasan, D. A Multi-agent System for Energy Management in Smart Microgrids with Distributed Energy Storage
and Demand Response. In Proceedings of the 2016 IEEE International Conference on Power Electronics, Drives and Energy
Systems (PEDES), Trivandrum, India, 14–17 December 2016; IEEE: New York, NY, USA, 2016; pp. 1–5. [CrossRef]
238.
Santos, A.Q.; Monaro, R.M.; Coury, D.V.; Oleskovicz, M. A New Real-time Multi-agent System for Under Frequency Load
Shedding in a Smartgrid Context. Electr. Power Syst. Res. 2019,174, 105851. [CrossRef]
239.
Logenthiran, T.; Srinivasan, D.; Shun, T.Z. Multi-agent System for Demand Side Management in Smartgrid. In Proceedings of the
2011 IEEE Ninth International Conference on Power Electronics and Drive Systems, Singapore, 5–8 December 2011; IEEE: New
York, NY, USA, 2011; pp. 424–429. [CrossRef]
240.
Mocci, S.; Natale, N.; Pilo, F.; Ruggeri, S. Demand Side Integration in LV Smartgrids with Multi-agent Control System. Electr.
Power Syst. Res. 2015,125, 23–33. [CrossRef]
241.
Biabani, M.; Golkar, M.A.; Sajadi, A. Operation of a Multi-Agent System for Load management in smart power distribution
system. In Proceedings of the 2012 11th International Conference on Environment and Electrical Engineering, Venice, Italy, 18–25
May 2012; IEEE: New York, NY, USA, 2012; pp. 525–530. [CrossRef]
242.
Kremers, E.; de Durana, J.G.; Barambones, O. Multi-agent Modeling for the Simulation of a Simple Smart Microgrid. Energy
Convers. Manag. 2013,75, 643–650. [CrossRef]
243.
Nguyen, D.T.; Negnevitsky, M.; de Groot, M. Walrasian Market Clearing for Demand Response Exchange. IEEE Trans. Power Syst.
2011,27, 535–544. [CrossRef]
244.
Najafi, S.; Talari, S.; Gazafroudi, A.S.; Shafie-khah, M.; Corchado, J.M.; Catalão, J.P. Decentralized control ofDR using a multi-agent
method. In Sustainable Interdependent Networks: From Theory to Application; Springer International Publishing: Berlin/Heidelberg,
Germany, 2018; pp. 233–249. [CrossRef]
245.
Haring, T.; Mathieu, J.L.; Andersson, G. Decentralized Contract Design for Demand Response. In Proceedings of the 2013 10th
International Conference on the European Energy Market (EEM), Stockholm, Sweden, 27–31 May 2013; IEEE: New York, NY,
USA, 2013; pp. 1–8. [CrossRef]
246.
Oliveira, P.; Gomes, L.; Pinto, T.; Faria, P.; Vale, Z.; Morais, H. Load Control Timescales Simulation in a Multi-agent Smartgrid
Platform. In Proceedings of the IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe 2013), Lyngby, Denmark,
6–9 October 2013; IEEE: New York, NY, USA, 2013; pp. 1–5. [CrossRef]
247.
Xu, Y.; Liu, W.; Gong, J. Stable Multi-Agent-Based Load Shedding Algorithm for Power Systems. IEEE Trans. Power Syst. 2011,
26, 2006–2014. [CrossRef]
Energies 2024,17, 3620 57 of 61
248.
Logenthiran, T.; Srinivasan, D. Multi-Agent System for Managing a Power Distribution System with Plug-in Hybrid Electrical
Vehicles in Smartgrid. In Proceedings of the Innovative Smart Grid Technologies Conference ISGT 2011, Kerala, India, 1–3
December 2011; IEEE: New York, NY, USA, 2011; pp. 346–351. [CrossRef]
249.
Vandael, S.; Boucké, N.; Holvoet, T.; Deconinck, G. Decentralized Demand Side Management of Plug-in Hybrid Vehicles in a
Smartgrid. In Proceedings of the First International Workshop on Agent Technologies for Energy Systems (ATES 2010), Toronto,
ON, Canada, 10–11 May 2010; 2010; pp. 67–74.
250.
Blanc-Rouchossé, J.B.; Blavette, A.; Ahmed, H.B.; Camilleri, G.; Gleizes, M.P. Multi-Agent System for Smart-Grid Control with
Commitment Mismatch and Congestion. In Proceedings of the 2019 IEEE PES Innovative Smartgrid Technologies Europe
(ISGT-Europe), Bucharest, Romania, 29 September–2 October 2019; IEEE: New York, NY, USA, 2019; pp. 1–5. [CrossRef]
251.
Nizami, M.; Hossain, M.; Rafique, S.; Mahmud, K.; Irshad, U.B.; Town, G. A Multi-agent System Based Residential Electric
Vehicle Management System for Grid-support Service. In Proceedings of the 2019 IEEE International Conference on Environment
and Electrical Engineering and 2019 IEEE Industrial and Commercial Power Systems Europe (EEEIC/I&CPS Europe), Genova,
Italy, 11–14 June 2019; IEEE: New York, NY, USA, 2019; pp. 1–6. [CrossRef]
252.
Papadopoulos, P.; Jenkins, N.; Cipcigan, L.M.; Grau, I.; Zabala, E. Coordination of the Charging of Electric Vehicles Using a
Multi-Agent System. IEEE Trans. Smartgrid 2013,4, 1802–1809. [CrossRef]
253.
Unda, I.G.; Papadopoulos, P.; Skarvelis-Kazakos, S.; Cipcigan, L.M.; Jenkins, N.; Zabala, E. Management of electric vehicle battery
charging in distribution networks with multi-agent systems. Electr. Power Syst. Res. 2014,110, 172–179. [CrossRef]
254.
Vandael, S.; De Craemer, K.; Boucké, N.; Holvoet, T.; Deconinck, G. Decentralized Coordination of Plug-in Hybrid Vehicles for
Imbalance Reduction in a Smartgrid. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent
Systems (AAMAS 2011), Taipei, Taiwan, 2–6 May 2011; Volume 2, pp. 803–810.
255.
Saner, C.B.; Trivedi, A.; Srinivasan, D. A Cooperative Hierarchical Multi-Agent System for EV Charging Scheduling in Presence
of Multiple Charging Stations. IEEE Trans. Smartgrid 2022,13, 2218–2233. [CrossRef]
256.
Hu, J.; Saleem, A.; You, S.; Nordström, L.; Lind, M.; Ostergaard, J. A Multi-agent System for Distribution Grid Congestion
Management with Electric Vehicles. Eng. Appl. Artif. Intell. 2015,38, 45–58. [CrossRef]
257.
Karfopoulos, E.L.; Hatziargyriou, N.D. A Multi-Agent System for Controlled Charging of a Large Population of Electric Vehicles.
IEEE Trans. Power Syst. 2012,28, 1196–1204. [CrossRef]
258.
Hurtado, L.; Syed, A.; Nguyen, P.; Kling, W. Multi-agent Based Electric Vehicle Charging Method for Smart Grid-smart Building
Energy Management. In Proceedings of the 2015 IEEE Eindhoven PowerTech, Eindhoven, The Netherlands, 29 June–2 July 2015;
IEEE: New York, NY, USA, 2015; pp. 1–6. [CrossRef]
259.
Kamboj, S.; Kempton, W.; Decker, K.S. Deploying Power Grid-integrated Electric Vehicles as a Multi-agent System. In Proceedings
of the 10th International Conference on Autonomous Agents and Multiagent Systems, Taipei, Taiwan, 2–6 May 2011; pp. 13–20.
260.
Moyalan, J.; Sawant, M.; Bhagyashree, U.; Sheikh, A.; Wagh, S.; Singh, N. Electric Vehicle—Power Grid Incorporation Using
Distributed Resource Allocation Approach. In Proceedings of the 2019 18th European Control Conference (ECC), Naples, Italy,
25–28 June 2019; IEEE: New York, NY, USA, 2019; pp. 3034–3039. [CrossRef]
261.
Egbue, O.; Uko, C. Multi-Agent Approach to Modeling and Simulation of Microgrid Operation with Vehicle-to-Grid System.
Electr. J. 2020,33, 106714. [CrossRef]
262.
Mocci, S.; Natale, N.; Pilo, F.; Ruggeri, S. Multi-Agent Control System for the Exploitation of Vehicle to Grid in Active LV
Networks. In Proceedings of the CIRED Workshop 2016, Helsinki, Finland, 14–15 June 2016. [CrossRef]
263.
Zhang, L.; Zhao, J.; Han, X.; Niu, L. Day-ahead Generation Scheduling with Demand Response. In Proceedings of the 2005
IEEE/PES Transmission & Distribution Conference & Exposition: Asia and Pacific, Dalian, China, 15–18 August 2005; IEEE: New
York, NY, USA, 2005; pp. 1–4. [CrossRef]
264. Gellings, C.W. The Concept of Demand side Management for Electric Utilities. Proc. IEEE 1985,73, 1468–1470. [CrossRef]
265.
Luo, T.; Ault, G.; Galloway, S. Demand Side Management in a highly decentralized energy future. In Proceedings of the 45th
International Universities Power Engineering Conference UPEC2010, Cardiff, UK, 31 August–3 September 2010; IEEE: New York,
NY, USA, 2010; pp. 1–6.
266.
Fioretto, F.; Yeoh, W.; Pontelli, E. A Multiagent System Approach to Scheduling Devices in Smart Homes. In Proceedings of the
16th Conference on Autonomous Agents and MultiAgent Systems, Sao Paulo, Brazil, 8–12 May 2017; pp. 981–989.
267.
Dethlefs, T.; Preisler, T.; Renz, W. Multi-agent-based Distributed Optimization for Demand-side-management Applications. In
Proceedings of the 2014 Federated Conference on Computer Science and Information Systems, Warsaw, Poland, 7–10 September
2014; IEEE: New York, NY, USA, 2014; pp. 1489–1496. [CrossRef]
268.
Zhou, J.; He, L.; Li, C.; Cao, Y.; Liu, X.; Geng, Y. What’s the Difference between Traditional Power Grid and Smartgrid?—From
Dispatching Perspective. In Proceedings of the 2013 IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC),
Kowloon, Hong Kong, 8–11 December 2013; IEEE: New York, NY, USA, 2013; pp. 1–6. [CrossRef]
269.
Al-Hinai, A.; Alhelou, H.H. A Multi-agent System for Distribution Network Restoration in Future Smartgrids. Energy Rep. 2021,
7, 8083–8090. [CrossRef]
270.
Kamdar, R.; Paliwal, P.; Kumar, Y. LabVIEW-based Multi-Agent Approach towards Restoration in Smartgrid. Mater. Today Proc.
2018,5, 4684–4691. [CrossRef]
271.
Li, W.; Li, Y.; Chen, C.; Tan, Y.; Cao, Y.; Zhang, M.; Peng, Y.; Chen, S. A Full Decentralized Multi-Agent Service Restoration for
Distribution Network with DGs. IEEE Trans. Smartgrid 2019,11, 1100–1111. [CrossRef]
Energies 2024,17, 3620 58 of 61
272.
Sampaio, R.F.; Melo, L.S.; Leão, R.P.; Barroso, G.C.; Bezerra, J.R. Automatic Restoration System for Power Distribution Networks
based on Multi-Agent Systems. IET Gener. Transm. Distrib. 2017,11, 475–484. [CrossRef]
273.
Hafez, A.A.; Omran, W.A.; Hegazy, Y.G. A Decentralized Technique for Autonomous Service Restoration in Active Radial
Distribution Networks. IEEE Trans. Smartgrid 2016,9, 1911–1919. [CrossRef]
274.
Ye, D.; Zhang, M.; Sutanto, D. A Hybrid Multiagent Framework with Q-Learning for Power Grid Systems Restoration. IEEE
Trans. Power Syst. 2011,26, 2434–2441. [CrossRef]
275.
Sharma, A.; Srinivasan, D.; Trivedi, A. A Decentralized Multiagent System Approach for Service Restoration Using DG Islanding.
IEEE Trans. Smartgrid 2015,6, 2784–2793. [CrossRef]
276.
Zidan, A.; El-Saadany, E.F. A Cooperative Multiagent Framework for Self-Healing Mechanisms in Distribution Systems. IEEE
Trans. Smartgrid 2012,3, 1525–1539. [CrossRef]
277.
Shirazi, E.; Jadid, S. A Multiagent Design for Self-Healing in Electric Power Distribution Systems. Electr. Power Syst. Res. 2019,
171, 230–239. [CrossRef]
278.
Nagata, T.; Sasaki, H. A Multi-agent Approach to Power System Restoration. IEEE Trans. Power Syst. 2002,17, 457–462. [CrossRef]
279.
Belkacemi, R.; Bababola, A. Experimental Implementation of Multi-agent System Algorithm for Distributed Restoration of a
Smartgrid System. In Proceedings of the IEEE SOUTHEASTCON 2014, Lexington, KY, USA, 13–16 March 2014; IEEE: New York,
NY, USA, 2014; pp. 1–4. [CrossRef]
280.
Abedini, R.; Pinto, T.; Morais, H.; Vale, Z. Multi-agent approach for power system in a Smartgrid protection context. In
Proceedings of the 2013 IEEE Grenoble Conference, Grenoble, France, 16–20 June 2013; IEEE: New York, NY, USA, 2013; pp. 1–6.
[CrossRef]
281.
Azeroual, M.; Boujoudar, Y.; Lamhamdi, T.; EL Moussaoui, H.; EL Markhi, H. Fault Location Technique Using Distributed
Multi Agent-Systems in Smartgrids. In Proceedings of the WITS 2020 6th International Conference on Wireless Technologies,
Embedded, and Intelligent Systems, Fez, Morocco, 14–16 October 2020; Springer: Berlin/Heidelberg, Germany, 2022; pp. 607–613.
[CrossRef]
282.
Ghosn, S.B.; Ranganathan, P.; Salem, S.; Tang, J.; Loegering, D.; Nygard, K.E. Agent-Oriented Designs for a Self Healing Smartgrid.
In Proceedings of the 2010 First IEEE International Conference on Smartgrid Communications, Gaithersburg, MD, USA, 4–6
October 2010; IEEE: New York, NY, USA, 2010; pp. 461–466. [CrossRef]
283.
Khamphanchai, W.; Pisanupoj, S.; Ongsakul, W.; Pipattanasomporn, M. A Multi-agent Based Power System Restoration Approach
in Distributed Smartgrid. In Proceedings of the 2011 International Conference & Utility Exhibition on Power and Energy Systems:
Issues and Prospects for Asia (ICUE), Pattaya, Thailand, 28–30 September 2011; IEEE: New York, NY, USA, 2011; pp. 1–7.
[CrossRef]
284.
Prostejovsky, A.; Lepuschitz, W.; Strasser, T.; Merdan, M. Autonomous Service-Restoration in Smart Distriubtion Grids using
Multi-Agent Systems. In Proceedings of the 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering
(CCECE), Montreal, QC, Canada, April 29–May 2 2012; IEEE: New York, NY, USA, 2012; pp. 1–5. [CrossRef]
285.
Shum, C.; Lau, W.H.; Wong, T.; Mao, T.; Chung, S.; Tse, C.; Tsang, K.F.; Lai, L.L. Modeling and Simulating Communications of
Multiagent Systems in Smart Grid. In Proceedings of the 2016 IEEE International Conference on Smart Grid Communications
(SmartgridComm), Sydney, Australia, 6–9 November 2016; IEEE: New York, NY, USA, 2016; pp. 405–410. [CrossRef]
286.
Xu, Y.; Liu, W. Novel Multiagent Based Load Restoration Algorithm for Microgrids. IEEE Trans. Smartgrid 2011,2, 152–161.
[CrossRef]
287.
Ghorbani, M.J.; Choudhry, M.A.; Feliachi, A. A Multiagent Design for Power Distribution Systems Automation. IEEE Trans.
Smartgrid 2015,7, 329–339. [CrossRef]
288.
Soo, V.W.; Peng, Y.B. A Stochastic Negotiation Approach to Power Restoration Problems in a Smartgrid. In Proceedings of
the International Conference on Principles and Practice of Multi-Agent Systems, Wollongong, Australia, 16–18 November 2011;
Springer: Berlin/Heidelberg, Germany, 2011; pp. 436–447. [CrossRef]
289.
Belkacemi, R.; Babalola, A.; Ariyo, F.; Feliachi, A. Restoration of Smartgrid Distribution System Using Two-way Communication
Capability. In Proceedings of the 2013 North American Power Symposium (NAPS), Manhattan, KS, USA, 22–24 September 2013;
IEEE: New York, NY, USA, 2013; pp. 1–4. [CrossRef]
290.
Gupta, R.; Jha, D.K.; Yadav, V.K.; Kumar, S. A Multi-agent Based Self-healing Smartgrid. In Proceedings of the 2013 IEEE PES
Asia-Pacific Power and Energy Engineering Conference (APPEEC), Kowloon, Hong Kong, 8–11 December 2013; IEEE: New York,
NY, USA, 2013; pp. 1–5. [CrossRef]
291. Solanki, J.M.; Khushalani, S.; Schulz, N.N. A Multi-Agent Solution to Distribution Systems Restoration. IEEE Trans. Power Syst.
2007,22, 1026–1034. [CrossRef]
292.
Ren, F.; Zhang, M.; Soetanto, D.; Su, X. Conceptual Design of A Multi-Agent System for Interconnected Power Systems Restoration.
IEEE Trans. Power Syst. 2012,27, 732–740. [CrossRef]
293.
Pang, Q.; Gao, H.; Minjiang, X. Multi-agent-based fault location algorithm for smart distribution grid. In Proceedings of the
Conference on Developments in Power System Protection (DPSP 2010), Manchester, UK, 29 March–1 April 2010; IET: New York,
NY, UK, 2010; pp. 1–5. [CrossRef]
294.
Chouhan, S.; Wan, H.; Lai, H.; Feliachi, A.; Choudhry, M. Intelligent Reconfiguration of Smart Distribution Network using
Multi-Agent Technology. In Proceedings of the 2009 IEEE Power & Energy Society General Meeting, Calgary, AB, Canada, 26–30
July 2009; IEEE: New York, NY, USA, 2009; pp. 1–6. [CrossRef]
Energies 2024,17, 3620 59 of 61
295.
Merdan, M.; Lepuschitz, W.; Strasser, T.; Andren, F. Multi-agent System for Self-optimizing Power Distribution Grids. In
Proceedings of the 5th International Conference on Automation, Robotics and Applications, Wellington, New Zealand, 6–8
December 2011; IEEE: New York, NY, USA, 2011; pp. 312–317. [CrossRef]
296.
Sanjab, A.; Saad, W.; Guvenc, I.; Sarwat, A.; Biswas, S. Smartgrid security: Threats, challenges and solutions. arXiv 2016,
arXiv:1606.06992. [CrossRef].
297.
Zhang, D.; Feng, G.; Shi, Y.; Srinivasan, D. Physical Safety and Cyber Security Analysis of Multi-Agent Systems: A Survey of
Recent Advances. IEEE/CAA J. Autom. Sin. 2021,8, 319–333. [CrossRef]
298.
Langer, L.; Skopik, F.; Smith, P.; Kammerstetter, M. From Old to New: Assessing Cybersecurity Risks for an Evolving Smart Grid.
Comput. Secur. 2016,62, 165–176. [CrossRef]
299.
Leszczyna, R. A Review of Standards with Cybersecurity Requirements for Smartgrid. Comput. Secur. 2018,77, 262–276.
[CrossRef]
300.
ENISA. Smartgrid Security—Recommendations for Europe and Member States; European Network and Information Security Agency
(ENISA): Athens, Greece, 2012.
301.
ENISA. Appropriate Security Measures for Smartgrids—Guidelines to Assess the Sophistication of Security Measures Implementation;
European Network and Information Security Agency (ENISA): Athens, Greece, 2012.
302.
NIST. NISTIR 7628, Revision 1: Guidelines for Smartgrid Cyber Security; U.S. National Institute of Standards and Technology (NIST):
Gaithersburg, MD, USA, 2011. [CrossRef]
303.
ISO/IEC TR 27019:2017; Information Security Management Guidelines Based on ISO/IEC 27002 for Process Control Systems
Specific to the Energy Utility Industry. International Organization for Standardization (ISO): Geneva, Switzerland, 2017.
304.
Dehalwar, V.; Baghel, R.; Kolhe, M. Multi-agent Based Public Key Infrastructure for Smartgrid. In Proceedings of the 2012 7th
International Conference on Computer Science & Education (ICCSE), Melbourne, Australia, 14–17 July 2012; IEEE: New York, NY,
USA, 2012; pp. 415–418. [CrossRef]
305.
Halinka, A.; Rzepka, P.; Szablicki, M. Agent Model of Multi-agent System for Area Power System Protection. In Proceedings
of the 2015 Modern Electric Power Systems (MEPS), Wroclaw, Poland, 6–9 July 2015; IEEE: New York, NY, USA, 2015; pp. 1–4.
[CrossRef]
306.
Bakhtadze, N.N.; Yadykin, I.B.; Lototsky, V.A.; Maximov, E.M.; Sakrutina, E.A. Multi-agent Approach to Design of Multimodal
Intelligent Immune System for Smartgrid. IFAC Proc. Vol. 2013,46, 1164–1169. [CrossRef]
307.
Bytschkow, D.; Quilbeuf, J.; Igna, G.; Ruess, H. Distributed MILS Architectural Approach for Secure Smartgrids. In Proceedings of
the International Workshop on Smartgrid Security, Munich, Germany, 26 February 2014; Springer: Berlin/Heidelberg, Germany,
2014; pp. 16–29. [CrossRef]
308.
Kisielewicz, T.; Stanek, S.; Zytniewski, M. A Multi-Agent Adaptive Architecture for Smart-Grid-Intrusion Detection and
Prevention. Energies 2022,15, 4726. [CrossRef]
309.
Singh, V.K.; Ozen, A.; Govindarasu, M. A Hierarchical Multi-Agent Based Anomaly Detection for Wide-Area Protection in
Smartgrid. In Proceedings of the 2018 Resilience Week (RWS), Denver, CO, USA, 21–23 August 2018; IEEE: New York, NY, USA,
2018; pp. 63–69. [CrossRef]
310.
Wei, D.; Lu, Y.; Jafari, M.; Skare, P.; Rohde, K. An Integrated Security System of Protecting Smartgrid Against Cyber Attacks. In
Proceedings of the 2010 Innovative Smartgrid Technologies (ISGT), Gaithersburg, MD, USA, 19–21 January 2010; IEEE: New York,
NY, USA, 2010; pp. 1–7. [CrossRef]
311. Wang, P.; Govindarasu, M. Multi Intelligent Agent Based Cyber Attack Resilient System Protection and Emergency Control. In
Proceedings of the 2016 IEEE Power & Energy Society Innovative Smartgrid Technologies Conference (ISGT), Minneapolis, MN,
USA, 6–9 September 2016; IEEE: New York, NY, USA, 2016; pp. 1–5. [CrossRef]
312.
Wang, P.; Govindarasu, M. Multi-Agent Based Attack-Resilient System Integrity Protection for Smartgrid. IEEE Trans. Smartgrid
2020,11, 3447–3456. [CrossRef]
313.
Atif, Y.; Jiang, Y.; Lindström, B.; Ding, J.; Jeusfeld, M.; Andler, S.; Nero, E.; Brax, C.; Haglund, D. Multi-Agent Systems for Power
Grid Monitoring: Technical Report for Package 4.1 of ELVIRA Project; University of Skövde: Skövde, Sweden, 2018.
314.
Khalid, R.; Samuel, O.; Javaid, N.; Aldegheishem, A.; Shafiq, M.; Alrajeh, N. A Secure Trust Method for Multi-Agent System in
Smartgrids Using Blockchain. IEEE Access 2021,9, 59848–59859. [CrossRef]
315.
Matei, I.; Baras, J.S.; Srinivasan, V. Trust-based Multi-agent Filtering for Increased Smartgrid Security. In Proceedings of the 2012
20th Mediterranean Conference on Control & Automation (MED), Barcelona, Spain, 3–6 July 2012; IEEE: New York, NY, USA,
2012; pp. 716–721. [CrossRef]
316.
Ross, K.J.; Hopkinson, K.M.; Pachter, M. Using a Distributed Agent-Based Communication Enabled Special Protection System to
Enhance Smartgrid Security. IEEE Trans. Smartgrid 2013,4, 1216–1224. [CrossRef]
317.
Naidji, I.; Smida, M.B.; Khalgui, M.; Bachir, A. Multi Agent System-based Approach for Enhancing Cyber-physical Security in
Smartgrids. In Proceedings of the 33rd Annual European Simulation and Modeling Conference, Palma de Mallorca, Spain, 28–30
October 2019; pp. 177–182.
318.
Rahman, M.S. Distributed Multi-Agent Approach for Enhancing Stability and Security of Emerging Smartgrids. Ph.D. Thesis,
University of New South Wales, Sydney, Australia, 2014. [CrossRef]
319.
Zulfiqar, M.; Kamran, M.; Rasheed, M. A Blockchain-enabled Trust Aware Energy Trading Framework Using Games Theory and
Multi-agent System in Smat Grid. Energy 2022,255, 124450. [CrossRef]
Energies 2024,17, 3620 60 of 61
320.
Genç, Z.; Oey, M.; van Antwerpen, H.; Brazier, F. Dynamic Data-Driven Experiments in the Smartgrid Domain with a Multi-agent
Platform. In Proceedings of the Multi-Agent Based Simulation XVI: International Workshop, MABS 2015, Istanbul, Turkey, 5 May
2015; Revised Selected Papers 16; Springer: Berlin/Heidelberg, Germany, 2016; pp. 121–131. [CrossRef]
321.
Schütte, S.; Nieße, A.; Rohjans, S.; Rohlfs, H. Opc Ua Compliant Coupling of Multi-agent Systems and Smartgrid Simulations.
In Proceedings of the IECON 2013, 39th Annual Conference of the IEEE Industrial Electronics Society, Vienna, Austria, 10–13
November 2013; IEEE: New York, NY, USA, 2013; pp. 7576–7581. [CrossRef]
322.
Mendham, P.; Clarke, T. MACSIM: A Simulink Enabled Environment for Multi-agent System Simulation. IFAC Proc. Vol. 2005,
38, 325–329. [CrossRef]
323.
Robinson, C.R.; Mendham, P.; Clarke, T. MACSIMJX: A Tool for Enabling Agent Modeling with Simulink Using JADE. JoPha J.
Phys. Agents 2010,4, 1–7. [CrossRef]
324.
Perkonigg, F.; Brujic, D.; Ristic, M. MAC-Sim: A Multi-agent and Communication Network Simulation Platform for Smartgrid
Applications Based on Established Technologies. In Proceedings of the 2013 IEEE International Conference on Smart Grid
Communications (SmartgridComm), Vancouver, BC, Canada, 21–24 October 2013; IEEE: New York, NY, USA, 2013; pp. 570–575.
[CrossRef]
325.
Vaubourg, J.; Presse, Y.; Camus, B.; Bourjot, C.; Ciarletta, L.; Chevrier, V.; Tavella, J.P.; Morais, H. Multi-agent Multi-Model
Simulation of Smartgrids in the MS4SG Project. In Proceedings of the Advances in Practical Applications of Agents, Multi-Agent
Systems and Sustainability: The PAAMS Collection: 13th International Conference, PAAMS 2015, Salamanca, Spain, 3–4 June
2015; Proceedings 13; Springer: Berlin/Heidelberg, Germany, 2015; Volume 9086, pp. 240–251. [CrossRef]
326.
Oliveira, P.; Pinto, T.; Morais, H.; Vale, Z. MASGriP — A Multi-Agent Smartgrid Simulation Platform. In Proceedings of the 2012
IEEE Power and Energy Society General Meeting, San Diego, CA, USA, 22–26 July 2012; IEEE: New York, NY, USA, 2012; pp. 1–8.
[CrossRef]
327.
Oliveira, P.; Vale, Z.; Morais, H.; Pinto, T.; Praça, I. A Multi-agent Based Approach for Intelligent Smartgrid Management. IFAC
Proc. Vol. 2012,45, 109–114. [CrossRef]
328.
Ahmad, I.; Kazmi, J.H.; Shahzad, M.; Palensky, P.; Gawlik, W. Co-simulation Framework Based on Power System, Ai and
Communication Tools for Evaluating Smartgrid Applications. In Proceedings of the 2015 IEEE Innovative Smartgrid Technologies-
Asia (ISGT ASIA), Bangkok, Thailand, 3–6 November 2015; IEEE: New York, NY, USA, 2015; pp. 1–6. [CrossRef]
329.
Santos, G.; Pinto, T.; Praça, I.; Vale, Z. MASCEM: Optimizing the Performance of a Multi-agent System. Energy 2016,111, 513–524.
[CrossRef]
330. Koritarov, V.S. Real-world Market Representation with Agents. IEEE Power Energy Mag. 2004,2, 39–46. [CrossRef]
331.
Li, H.; Tesfatsion, L. Development of Open Source Software for Power Market Research: The AMES Test Bed. J. Energy Mark.
2009,2, 111. [CrossRef]
332.
Cincotti, S.; Gallo, G. The Genoa Artificial Power-Exchange. In Proceedings of the Agents and Artificial Intelligence: 4th Interna-
tional Conference, ICAART 2012, Vilamoura, Portugal, 6–8 February 2012; Revised Selected Papers 4; Springer: Berlin/Heidelberg,
Germany, 2013; pp. 348–363. [CrossRef]
333.
Collins, J.; Ketter, W.; Sadeh, N. Pushing the Limits of Rational Agents: The Trading Agent Competition for Supply Chain
Management. AI Mag. 2010,31, 63–63. [CrossRef]
334.
Ketter, W.; Collins, J.; Reddy, P. Power TAC: A Competitive Economic Simulation of the Smartgrid. Energy Econ. 2013,39, 262–270.
[CrossRef]
335.
Peidaee, P.; Kalam, A.; Moghaddam, M.H. Developing a Simulation Framework for Integrating Multi-agent Protection System
Into Smartgrids. In Proceedings of the 2017 Australasian Universities Power Engineering Conference (AUPEC), Melbourne,
Australia, 19–22 November 2017; IEEE: New York, NY, USA, 2017; pp. 1–6. [CrossRef]
336.
Purusothaman, S.D.; Rajesh, R.; Bajaj, K.K.; Vijayaraghavan, V. Implementation of Arduino-based Multi-agent System for Rural
Indian Microgrids. In Proceedings of the 2013 IEEE Innovative Smartgrid Technologies-Asia (ISGT Asia), Bangalore, India, 10–13
November 2013; IEEE: New York, NY, USA, 2013; pp. 1–5. [CrossRef]
337.
Raju, L.; Morais, A.A.; Balaji, V.; Keerthivasan, S. Multi Agent Systems and Arduino Based Smart Micro-grid Test Bed. In
Proceedings of the AIP Conference, Tamil Nadu, India, 14–15 March 2019; AIP Publishing: Melville, NY, USA, 2019; Volume 2161.
[CrossRef]
338.
Gomes, L.; Vale, Z.; Corchado, J.M. Microgrid Management System Based on a Multi-agent Approach: An Office Building Pilot.
Measurement 2020,154, 107427. [CrossRef]
339.
Eriksson, M.; Armendariz, M.; Vasilenko, O.O.; Saleem, A.; Nordström, L. Multiagent-Based Distribution Automation Solution
for Self-Healing Grids. IEEE Trans. Ind. Electron. 2014,62, 2620–2628. [CrossRef]
340.
Ricalde, L.J.; Ordoñez, E.; Gamez, M.; Sanchez, E.N. Design of a Smartgrid management system with renewable energy generation.
In Proceedings of the 2011 IEEE Symposium on Computational Intelligence Applications in Smartgrid (CIASG), Paris, France,
11–15 April 2011; IEEE: New York, NY, USA, 2011; pp. 1–4. [CrossRef]
341.
Cintuglu, M.H.; Martin, H.; Mohammed, O.A. An Intelligent Multi Agent Framework for Active Distribution Networks Based on
IEC 61850 and FIPA Standards. In Proceedings of the 2015 18th International Conference on Intelligent System Application to
Power Systems (ISAP), Porto, Portugal, 11–16 September 2015; IEEE: New York, NY, USA, 2015; pp. 1–6. [CrossRef]
342.
Habib, H.F.; Youssef, T.; Cintuglu, M.H.; Mohammed, O.A. Multi-Agent-Based Technique for Fault Location, Isolation, and
Service Restoration. IEEE Trans. Ind. Appl. 2017,53, 1841–1851. [CrossRef]
Energies 2024,17, 3620 61 of 61
343.
Belkacemi, R.; Feliachi, A.; Choudhry, M.; Saymansky, J.E. Multi-Agent systems hardware development and deployment for
smart grid control applications. In Proceedings of the 2011 IEEE Power and Energy Society General Meeting, Detroit, MI, USA,
24–29 July 2011; IEEE: New York, NY, USA, 2011; pp. 1–8. [CrossRef]
344.
Chung, I.Y.; Cheol-HeeYoo, S.J.O. Distributed Intelligent Microgrid Control Using Multi-Agent Systems. Engineering 2013,5, 1–6.
[CrossRef]
345.
Azeroual, M.; Lamhamdi, T.; El Moussaoui, H.; El Markhi, H. Simulation Tools for a Smartgrid and Energy Management for
Microgrid with Wind Power Using Multi-agent System. Wind Eng. 2020,44, 661–672. [CrossRef]
346.
Morais, H.; Vale, Z.; Pinto, T.; Gomes, L.; Fernandes, F.; Oliveira, P.; Ramos, C. Multi-agent Based Smartgrid Management and
Simulation: Situation Awareness and Learning in a Test Bed with Simulated and Real Installations and Players. In Proceedings of
the 2013 IEEE Power & Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; IEEE: New York, NY, USA,
2013; pp. 1–5. [CrossRef]
347.
Pinto, T.; Gomes, L.; Faria, P.; Sousa, F.; Vale, Z. MARTINE: Multi-Agent-based Real-Time INfrastructure for Energy. In
Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, Auckland, New Zeland, 9–13
May 2020; pp. 2114–2116.
348.
Dash, R.K.; Jennings, N.R.; Parkes, D.C. Computational-mechanism Design: A Call to Arms. IEEE Intell. Syst. 2003,18, 40–47.
[CrossRef]
349.
Raju, L.; Milton, R.; Mahadevan, S. Application of Multi Agent Systems in Automation of Distributed Energy Management in
Micro-grid using MACSimJX. Intell. Autom. Soft Comput. 2017,24, 1–9. [CrossRef]
350.
Kantarci, B.; Mouftah, H.T. Energy-Efficiency in Cloud Data Centers. In Communication Infrastructures for Cloud Computing; IGI
Global: Hershey, PA, USA, 2014; pp. 241–263. [CrossRef]
351. Peng, X.; Qin, X. Energy Efficient Data Centers Powered by On-site Renewable Energy and UPS Devices. In Proceedings of the
2020 11th International Green and Sustainable Computing Workshops (IGSC), Pullman, WA, USA, 19–22 October 2020; IEEE:
New York, NY, USA, 2020. [CrossRef]
352.
Satish, S.; Meduri, S.S. Integrating Renewable Energy Sources into Cloud Computing Data Centers: Challenges and Solutions.
Int. J. Res. Publ. Rev. 2024,5, 1598–1608. [CrossRef]
353.
Wang, H.; Huo, D. Green Cloud Computing: Site Selection of Data Centers. In Security, Trust and Regulatory Aspects of Cloud
Computing in Business Environments; IGI Global: Hershey, PA, USA, 2014; pp. 202–214. [CrossRef]
354.
Zhang, Y.; Wang, Y.; Wang, X. GreenWare: Greening Cloud-Scale Data Centers to Maximize the Use of Renewable Energy. In
Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; pp. 143–164. [CrossRef]
355.
Berezovskaya, Y.; Yang, C.W.; Vyatkin, V. Towards Multi-Agent Control in Energy-Efficient Data Centres. In Proceedings of the
IECON 2020, The 46th Annual Conference of the IEEE Industrial Electronics Society, Singapore, 18–21 October 2020; IEEE: New
York, NY, USA, 2020. [CrossRef]
356.
Chen, Q. Research on Cloud Computing Resource Management Model Based on Multi-Agent System. In Proceedings of the 2016
12th International Conference on Computational Intelligence and Security, Wuxi, China, 16–19 December 2016; IEEE: New York,
NY, USA, 2016. [CrossRef]
357.
Farahnakian, F.; Pahikkala, T.; Liljeberg, P.; Plosila, J. Hierarchical Agent-Based Architecture for Resource Management in Cloud
Data Centers. In Proceedings of the 2014 IEEE 7th International Conference on Cloud Computing, Anchorage, AK, USA, 27
June–2 July 2014; IEEE: New York, NY, USA, 2014. [CrossRef]
358.
Soltane, M.; Okba, K.; Makhlouf, D.; Eom, S.B. Smart Configuration and Auto Allocation of Resource in Cloud Data Centers. Int.
J. Bus. Anal. 2018,5, 1–23. [CrossRef]
359.
Yu, H.; Xia, Y. An Energy Saving Control Strategy Based on Multi-Agent Q-Learning Algorithm for Data Center. J. Phys. Conf. Ser.
2023,2517, 012018. [CrossRef]
360.
Baral, C.; Gelfond, G.; Pontelli, E.; Son, T.C. An Action Language for Multi-Agent Domains; Technical Report; New Mexico State
University: Las Cruces, NM, USA, 2011.
361.
Baral, C.; Gelfond, G.; Pontelli, E.; Son, T.C. An action language for multi-agent domains. Artif. Intell. 2022,302, 103601.
[CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
Available via license: CC BY 4.0
Content may be subject to copyright.
