No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Engineering Applications of Artificial Intelligence Survey paper Multi-agent systems in Peer-to-Peer energy trading: A comprehensive survey
Abstract
Energy networks around the world have experienced a significant increase in the amount of distributed generation. This decentralization of energy markets has led to a surge of interest in Peer-to-Peer energy trading between individuals. There are many challenges associated with Peer-to-Peer energy trading, e.g. behaviour modelling, decision making and optimization. Multi-agent systems is a prominent subfield of Artificial Intelligence research that is very effective for these types of problems. This research aims to provide a detailed survey of recent advances in the application of multi-agent systems in Peer-to-Peer energy trading. The challenges encountered in implementing these systems are discussed, e.g. agent learning, privacy, and computational power. The primary advantages of multi-agent systems for Peer-to-Peer energy trading reported in the literature are: improved efficiency of renewable utilization, reduced transaction costs, and scalability. Current challenges and future directions of research are also outlined.
This research introduces a methodology based on reinforcement learning to address the cost reduction challenges faced by prosumers in a smart home-based community. The proposed method tackles the unpredictability of distributed energy resources (DERs) and the heterogeneity of prosumers by introducing Hierarchical Home Energy Management with a Noise-Adaptive Proximal Policy Optimization (H2EN-PPO) framework. This multi-agent problem is addressed using deep reinforcement learning using decentralized critic and decentralized value neural networks. The highest-tier node focuses on resolving the internal energy pricing predicament, while the lower nodes manage the scheduling of household appliances. To emulate human behavior accurately, distributed actual power transition data obtained from various domestic devices is incorporated, accounting for aspects such as environmental circumstances and and dynamic pricing in real-time. Real-world datasets are employed in simulations to evaluate the proposed approach. To maintain grid stability, a suitable reward function is implemented to prevent overloading of the transformer connected to a particular community. This reward function is also designed to address the variation in human behavior when it comes to electric vehicle (EV) charging, taking into account factors like range anxiety and time anxiety. Additionally, this function is capable of optimizing the utilization of the photovoltaic (PV) generation. The findings indicate that the proposed approach substantially minimizes prosumers’ daily costs, outperforming existing methods. The H2EN-PPO framework offers advantages in terms of cost reduction and efficient resource utilization in community energy trading scenarios. The contributions of this research lie in addressing the challenges of DER uncertainty and prosumer heterogeneity, as well as incorporating realistic human behavior models into the optimization framework.
In the past decade, the global distribution of energy resources has expanded significantly. The increasing number of prosumers creates the prospect for a more decentralized and accessible energy market, where the peer-to-peer energy trading paradigm emerges. This paper proposes a methodology to optimize the participation in peer-to-peer markets based on the double-auction trading mechanism. This novel methodology is based on two reinforcement learning algorithms, used separately, to optimize the amount of energy to be transacted and the price to pay/charge for the purchase/sale of energy. The proposed methodology uses a competitive approach, and that is why all agents seek the best result for themselves, which in this case means reducing as much as possible the costs related to the purchase of energy, or if we are talking about sellers, maximizing profits. The proposed methodology was integrated into an agent-based ecosystem where there is a direct connection with agents, thus allowing application to real contexts in a more efficient way. To test the methodology, a case study was carried out in an energy community of 50 players, where each of the proposed models were used in 20 different players, and 10 were left without training. The players with training managed, over the course of a week, to save 44.65 EUR when compared to a week of peer-to-peer without training, a positive result, while the players who were left without training increased costs by 17.07 EUR.
This paper proposes a new peer-to-peer (P2P) energy trading method between energy sellers and consumers in a community based on multi-agent reinforcement learning (MARL). Each user of the community is treated as a smart agent who can choose the amount and the price of the electric energy to sell/buy. There are two aspects we need to examine: the profits for the individual user and the utility for the community. For a single user, we consider that they want to realise both a comfortable living environment to enhance happiness and satisfaction by adjusting usage loads and certain economic benefits by selling the surplus electric energy. Taking the whole community into account, we care about the balance between energy sellers and consumers so that the surplus electric energy can be locally absorbed and consumed within the community. To this end, MARL is applied to solve the problem, where the decision making of each user in the community not only focuses on their own interests but also takes into account the entire community’s welfare. The experimental results prove that our method is profitable both both the sellers and buyers in the community.
Peer-to-peer (P2P) energy trading system has the ability to completely revolutionize the current household energy system by sharing energy among residents. As the number of customers employing distributed energy resources (DERs) such as solar rooftops increase, innovation in the double auction market (DA) system is becoming more significant. In this paper, a novel model-based asynchronous advantage actor-centralized-critic with communication (MB-A3C3) approach is carried out. The model is conducted on a large scale real-world hourly 2012–2013 dataset of 300 households in Sydney having rooftop solar systems installed in New South Wales (NSW), Australia. Results reveal that the MB-A3C3 approach outperforms other reinforcement learning methods (MADDPG and A3C3), producing lower community energy bills for 300 households. In closing the gap between the real-world and theoretical problems, the algorithms herein aid in reducing customers’ electricity bills.
Electricity is still a highly demanding resource in developing countries, especifically in rural areas. Local communities in developing countries are now focusing on generating their own electricity using solar and trading the energy which is in access to the supplier. Energy trading at the peer-to-peer level is a novel paradigm. It allows customers to contribute energy to the grid and power generation and supply companies. In this paper, we investigate the energy trading enabling technologies, their applications, challenges, and benefits to the local communities from various aspects. We analyse how blockchain and IoT technologies can help energy trading at the customer level from recent research proposals. We also present our analysis of recent research on the implementation of energy trading technologies and future directions.
Energy data measured on-site from buildings can help describe the consumption behavior of end-users and can be used to examine and prove certain theorems and models, that require a large volume of data to be gathered. However, the direct extraction of this data can often be a lengthily and costly process. As a result, a dataset of a residential community was constructed based on real data, where sample consumption and photovoltaic generation profiles were attributed to 50 residential households and a public building (municipal library), a total of 51 buildings. In addition, the overall power consumption of these houses was desegregated into the consumption of 10 commonly used appliances using real energy profiles. First, several consumption and photovoltaic generation profiles, as well as a vast collection of appliance profiles, were gathered. These profiles were obtained from household readings in different locations, while the public building's profile was based on the consumption and photovoltaic production profiles of the research building GECAD. The profiles went through the process of normalization and new profiles were generated to complete the number of end-users needed. Moreover, these profiles were given a maximum consumption and production level at random before being accepted by one of the end-users. Therefore, fourteen of these households and the public building were randomly attributed with renewable solar energy. Finally, if possible, the tool created allocated, at random in previously determined intervals, the appliances' load profiles into each end-user's available consumption areas.
In recent years, local energy markets (LEMs) have been introduced to empower end-customers within energy communities at the distribution level of the power system, in order to be able to trade their energy locally in a competitive and fair environment. However, there is still some challenge with regard to the most efficient approach in organising the LEMs for the electricity exchange between consumers and prosumers while ensuring that they are responsible for their electricity-related choices, and concerning which LEM model is suitable for which prosumer or consumer type. This paper presents a hierarchical model for the organisation of agent-based local energy markets. According to the proposed model, prosumers and consumers are enabled to transact electricity within the local energy community and with the grid in a coordinated manner to ensure technical and economic benefits for the LEM’s agents. The model is implemented in a software tool called Grid Singularity Exchange (GSyE), and it is verified in a real German energy community case study. The simulation results demonstrate that trading electricity within the LEM offers economic and technical benefits compared to transacting with the up-stream grid. This can further lead to the decarbonization of the power system sector. Furthermore, we propose two models for LEMs consisting of multi-layer and single-layer hierarchical agent-based structures. According to our study, the multi-layer hierarchical model is more profitable for household prosumers as compared to trading within the single-layer hierarchical LEM. However, the single-layer LEM is more be beneficial for industrial prosumers.
This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.
In future distribution grids, prosumers (i.e., energy consumers with storage and/or production capabilities) will trade energy with each other and with the main grid. To ensure an efficient and safe operation of energy trading, in this paper, we formulate a peer-to-peer energy market of prosumers as a generalized aggregative game, in which a network operator is responsible to enforce the operational constraints of the system. We design a distributed market-clearing mechanism with conver- gence guarantee to an economically-efficient, strategically-stable, and operationally-safe configuration (i.e., a variational general- ized Nash equilibrium). Numerical studies on the IEEE 37-bus testcase show the scalability of the proposed approach and sug- gest that active participation in the market is beneficial for both prosumers and the network operator.
Peer-to-peer (P2P) energy trading is a promising energy trading mechanism due to the deployment of distributed energy resources in recent years. Trading energy between prosumers and consumers in the local energy market is undergoing massive research and development, paying significant attention to the business model of the energy market. In this paper, an extensive review was conducted on the current research in P2P energy trading to understand the business layer of the energy market concerning business model dimensions: bidding strategies and the market-clearing approach. Different types of game theoretical-based and auction-based market-clearing mechanisms are investigated, including a detailed classification of auctions. This study considers the possibility of employing the P2P technique in developing countries and reviewing existing business models and trading policies. The business layer of the P2P structure plays a vital role in developing an effective trading mechanism based on interactive energy markets.
Peer-to-peer (P2P) energy trading has recently emerged as a promising paradigm for integrating renewable and distributed energy resources into local energy grids with the presence of active prosumers. However, prosumers often have different preferences on energy trading price and amount. Therefore, in decentralized P2P energy markets, a negotiation between prosumers is needed to obtain a commonly satisfactory set of preferences, i.e., a market-clearing solution. To achieve that, this paper proposes a novel approach in which a decentralized inverse optimization problem is solved by prosumers to cooperatively learn to set their objective function parameters, given their preferential intervals of energy prices and amounts. As such, prosumers’ parameters can be determined in specific intervals computed analytically from the lower and upper bounds of their preferential intervals, if a certain learning condition is satisfied. Next, the structural robustness of prosumer’s cooperative learning against the malicious and Byzantine models of cyberattacks is studied with the weighted-mean-subsequence-reduced (WMSR) resilient consensus algorithm. A novel sufficient robustness condition is then derived. Finally, case studies are conducted on the IEEE European Low Voltage Test Feeder system to validate the effectiveness of the proposed theoretical results.
Careful consideration of grid developments illustrates the fundamental changes in its structure which its developments have taken place gradually for a long time. One of the most important developments is the expansion of the communication infrastructure that brings many advantages in the cyber layer of the system. The actual execution of the peer-to-peer (P2P) energy trading is one core advantage which also may lead to the systematic risks such as cyber-attacks. Consequently, it is necessary to form a useful way to cover such challenges. This paper focuses on the online detection of false data injection attack (FDIA), which tries to disrupt the trend of optimal peer-to-peer energy trading in the stochastic condition. Moreover, this article proposes an effective modified Intelligent Priority Selection based Reinforcement Learning (IPS-RL) method to detect and stop the malicious attacks in the shortest time for effective energy trading based on the peer to peer structure. The presented method is compared with other methods such as support vector machine (SVM), reinforcement learning (RL), particle swarm optimization (PSO)-RL, and genetic algorithm (GA)-RL to validate the functionality of the method. The proposed method is implemented and examined on three interconnected microgrids in the form of peer-to-peer structure wherein each microgrid has various agents such as photovoltaic (PV), wind turbine, fuel cell, tidal system, storage unit, etc. Eventually, the unscented transformation (UT) is applied for uncertainty analysis and making the near-reality simulations.
Owing to energy liberalization and increasing penetration of renewables, renewable energy trading among suppliers and users has gained much attention and created a new market. This article investigates a double-auction scheme operated by an aggregator with limited supervision for energy trading. To ensure beneficial bidding for renewable generators and end users (EUs) considered as agents, a multiagent Q-learning (MAQL) based bidding strategy is developed to maximize their cumulative reward. Each agent first provides their information about renewable supply or demand to an aggregator who will then return the information about aggregate supply and demand. Without knowing the business model of the aggregator, the agents use Q-tables to estimate the expected cumulative reward and determine their bidding prices accordingly. Finally, the aggregator coordinates energy trading between agents who will then update their Q-tables on the basis of the amount of power bought or sold at the prices they bid. The proposed approach can avoid some unnecessary or unrealistic assumptions generally made by model-based approaches, such as the assumption on the knowledge of others’ bidding profiles or the assumption of an oligopoly; it can consider the influence of bidding strategies on the market, which cannot be properly addressed by a conventional proportional allocation mechanism. A numerical analysis using real-world data and considering a profit maximization model for the aggregator shows that the proposed approach outperformed comparable methods in terms of profits of renewable generators and energy satisfaction level of EUs: iterative double auction by approximately 29%, heuristic bidding by 39.9%, random bidding by 38.1%, NSGA-II-based multiobjective approach by 62%, and MOEA/D-based multiobjective approach by 83.1% on average.
Current Peer-to-Peer (P2P) energy market models raise serious concerns regarding the confidentiality and integrity of energy consumption, trading and billing data. While Distributed Ledger Technology (DLT) systems (e.g., blockchain) have been proposed to enhance security, an attacker could damage other parts of the model, such as its infrastructure: an adversarial attacker could target the communication between entities by, e.g., eavesdropping or modifying data. The main goal of this paper is to propose a model for a decentralised P2P marketplace for trading energy, which addresses the problem of developing security and privacy-aware environments. Additionally, a Multi-Agent System (MAS) architecture is presented with a focus on security and sustainability. In order to propose a solution to DLT’s scalability issues (i.e., through transaction confirmation delays), off-chain state channels are considered for the energy negotiation and resolution processes. Additionally, a STRIDE (spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege) security analysis is conducted within the context of the proposed model to identify potential vulnerabilities.
As a promising solution to address the “energy trilemma” confronting human society, peer-to-peer (P2P) energy trading has emerged and rapidly developed in recent years. When carrying out P2P energy trading, customers with distributed energy resources (DERs) are able to directly trade and share energy with each other. This paper summarizes and analyzes the global development of P2P energy trading based on a comprehensive review of related academic papers, research projects, and industrial practice. Key aspects in P2P energy trading are identified and discussed, including market design, trading platforms, physical infrastructure and information and communication technology (ICT) infrastructure, social science perspectives, and policy. For each key aspect, existing research and practice are critically reviewed and insights for future development are presented. Comprehensive concluding remarks are provided at the end, summarizing the major findings and perspectives of this paper. P2P energy trading is a growing field with great potential and opportunities for both academia and industry across the world.
Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.
It is conventional wisdom in machine learning and data mining that logical models such as rule sets are more interpretable than other models, and that among such rule-based models, simpler models are more interpretable than more complex ones. In this position paper, we question this latter assumption by focusing on one particular aspect of interpretability, namely the plausibility of models. Roughly speaking, we equate the plausibility of a model with the likeliness that a user accepts it as an explanation for a prediction. In particular, we argue that—all other things being equal—longer explanations may be more convincing than shorter ones, and that the predominant bias for shorter models, which is typically necessary for learning powerful discriminative models, may not be suitable when it comes to user acceptance of the learned models. To that end, we first recapitulate evidence for and against this postulate, and then report the results of an evaluation in a crowdsourcing study based on about 3000 judgments. The results do not reveal a strong preference for simple rules, whereas we can observe a weak preference for longer rules in some domains. We then relate these results to well-known cognitive biases such as the conjunction fallacy, the representative heuristic, or the recognition heuristic, and investigate their relation to rule length and plausibility.
Distributed Energy Resources (DERs) are being integrated into the power market by customers rather than large scale energy suppliers, thereby slowly transforming the centralized, unidirectional market to a decentralized, bidirectional market and transitioning customers into prosumers. Various system architectures are used in the real field to coordinate the energy distribution in the micro/ mini-grids integrated with DERs, all of which have their strengths, weaknesses and challenges. Peer-to-peer (P2P) is an emerging architecture in the field of electrical energy trading and Distributed Generation (DG) management that can be applied in local energy markets. This paper focuses on P2P energy trading, with an in-depth discussion on its various operating algorithms, their principles, characteristics, features and scope through state of art review on P2P. Furthermore, the energy system of Nepal is used as a case study in this paper, and the micro/mini-grids of Nepal and their associated challenges, constraints and opportunities for improvement are discussed. Finally, an energy trading model is proposed to address the problems occurring in the specific case of Nepalese energy market.
Increase in the deployment of Distributed Energy Resources (DERs) has triggered a new trend to redesign electricity markets as consumer-centric markets relying on Peer-to-Peer (P2P) approaches. In the P2P markets, players can directly negotiate under bilateral energy trading to match demand and supply. The trading scheme should be designed adequately to incentivise players to participate in the trading process actively. This paper proposes a decentralised P2P energy trading scheme for electricity markets with high penetration of DERs. A novel algorithm using primal-dual gradient method is described to clear the market in a fully decentralised manner without interaction of any central entity. Also, to incorporate technical constraints in the energy trading, line flow constraints are modelled in the bilateral energy trading to avoid overloaded or congested lines in the system. This market structure respects market players' preferences by allowing bilateral energy trading with product differentiation. The performance of the proposed method is evaluated using simulation studies, and it is found that market players can trade energy to maximise their welfare without violating line flow constraints. Also, compared with other similar methods for P2P trading, the proposed approach needs lower data exchange and has a faster convergence rate.
In a community Microgrid where peer to peer (P2P) energy trading is carried out, many prosumers have conflicting interests. It is difficult for individual prosumers to capture the conflicting interests in the decision-making process (e.g. control of individual batteries). It is also difficult to motivate prosumers to cooperate for achieving the goals of P2P trading. This work proposed a P2P trading mechanism and modeled the decision-making process using the game theoretic approach and Shapley value. The game theoretic approach delivered distributed energy management solutions for individuals in the trading process considering both optimality and fairness among prosumers. The trading mechanism using Shapley value was compared with previous algorithms for P2P energy trading, e.g. bill sharing (BS), mid-market rate (MMR) and supply demand ratio (SDR). Simulation results illustrated the effectiveness of the proposed method. Optimality and fairness for P2P energy trading were significantly improved.
Peer-to-peer trading in energy networks is expected to be exclusively conducted by the prosumers of the network with negligible influence from the grid. This raises the critical question: how can enough prosumers be encouraged to participate in peer-to-peer trading so as to make its operation sustainable and beneficial to the overall electricity network? To this end, this paper proposes how a motivational psychology framework can be used effectively to design peer-to-peer energy trading to increase user participation. To do so, first, the state-of-the-art of peer-to-peer energy trading literature is discussed by following a systematic classification, and gaps in existing studies are identified. Second, a motivation psychology framework is introduced, which consists of a number of motivational models that a prosumer needs to satisfy before being convinced to participate in energy trading. Third, a game-theoretic peer-to-peer energy trading scheme is developed, its relevant properties are studied, and it is shown that the coalition among different prosumers is a stable coalition. Fourth, through numerical case studies, it is shown that the proposed model can reduce carbon emissions by 18.38% and 9.82% in a single day in Summer and Winter respectively compared to a feed-in-tariff scheme. The proposed scheme is also shown to reduce the cost of energy up to 118 ¢ and 87 ¢ per day in Summer and Winter respectively. Finally, how the outcomes of the scheme satisfy all the motivational psychology models is discussed, which subsequently shows its potential to attract users to participate in energy trading.
With increasing numbers of prosumers employed with multi-energy systems (MES) towards higher energy utilization efficiency, an advanced energy management scheme is becoming increasingly important. The incorporation of MES into the existential energy market holds promise for future power systems. The continuous double auction (CDA) market, in a decentralized manner, makes it ideal for enabling peer-to-peer (P2P) energy trading due to its high transparency and efficiency. However, the CDA market is difficult to model when considering the highly stochastic and dynamic behaviors of market participants. For this reason, we formulate this task as a Decentralized Partially Observed Markov Decision Process and propose a novel multi-agent reinforcement learning method that allows each prosumer agent to stabilize the training performance with mean-field approximation and also to maintain the scalability and privacy with market public information. Case studies constructed on a real-world scenario of 100 prosumers show that our method captures the economic benefits of the P2P energy trading paradigm without violating the prosumers' privacy, and outperforms the state-of-the-art methods in terms of policy performance, scalability, and computational performance.
Transactive energy trading, control, and management have gained much importance, receiving significant interest from both industry and academia. As a result, new strategies for its development and implementation are emerging. Prosumage, which combines household loads with solar PV and battery storage systems, has been presented as an effective technique for facilitating the integration of renewable energy sources while reducing distribution grid stress. Therefore, this paper proposes a grid-connected prosumer-centric residential smart community that uses cooperative game theory to exchange, regulate, and plan surplus energy generated by distributed energy resources (DERs) with the other neighbouring prosumers with energy deficits. The energy community considered in this work comprises ten identical prosumers (residential load, solar PV, and battery storage system), an energy community manager, and an energy retailer, all of whom are linked as a group to a distribution grid. The proposed market model seeks to examine the economic benefits of such renewable sources participating in centralized transactive energy trading. A centralized approach is adopted in the transactive energy trading among the ten proactive prosumers and the power grid through an energy community manager that administers trading activities inside the energy community. An optimization model is proposed in the centralized transactive energy trading market to optimize the financial benefits of rooftop solar PV-battery systems. The proposed model will not only optimally exchange, plan, and regulate the community domestic load, but it will also minimize total energy costs, increase system operation efficiency, and reduce system operating stress and carbon emissions.
This paper proposes a three-stage multi-energy sharing strategy for a gas-electricity integrated energy system (IES). It aims to solve the multi-energy imbalance problem among energy hubs (EHs) based on the peer-to-peer (P2P) trading mode. First, considering the characteristics of multi-energy coupling and conversion, the quantity of shareable energy is determined for EHs that participate in the P2P trading. Furthermore, EHs conduct multi-bilateral negotiations based on the Raiffa-Kalai-Smorodinsky bargaining solution (RBS) to determine the optimal energy trading price. Finally, the buyer agent and the seller agent will design the optimal energy sharing trading strategy for all EHs. Moreover, the results show that the pricing mechanism improves the fairness of satisfaction obtained by the EHs from the utility distribution, and the social welfare of the system is improved, which proves that the three-stage multi-energy sharing trading strategy is effective.
With the popularity of distributed power generation units, traditional consumers become prosumers. Meanwhile, the emerging information and communication technology brings new opportunities to the energy trading service by providing various platforms. Thus, the energy trading problem of prosumers quickly ascends to the spotlight. Nevertheless, the personalized small-scale prosumers bring challenges of coordinated operation, supply-demand balance and carbon emission control. In this paper, based on an edge-cloud platform, a distributed carbon-aware energy trading mechanism is proposed for coordinating the prosumers within the virtual power plant. Moreover, the personalized characteristics including generation cost and the carbon emissions are handled without revealing the private information under the distributed formulation. In this way, the coordination effects of the prosumers are fully explored, and a multi-agent network is employed to achieve the supply-demand balance and the carbon emission awareness. Moreover, to deal with the denial of service attacks in the cyber layer, a passivity-based neurodynamic approach is proposed, which is resilient to denial of service attacks. A set of realistic case studies on energy trading problem under denial of service attacks are given, demonstrating the effectiveness of the proposed mechanism in coordinated operation, supply-demand balancing and carbon-awareness under denial of service attacks.
Peer-to-peer trading generates a consumer-centric market and encourages a bottom-up hierarchy, allowing consumers to choose the source of energy and conduct preference-based trading. Although peer-to-peer trading has promising benefits, its application in the real-world is limited and often not encouraged. This paper presents a systematic and comprehensive literature review of the business models of 50 peer-to-peer distributed renewable energy trading projects around the world. The aim of this study is to explore the strengths, limitations and challenges of peer-to-peer distributed renewable energy trading considering its prevailing business arrangements to identify the critical requirements for its uptake in the real-world market. The findings reveal that peer-to-peer distributed renewable energy trading has characteristics of both coordinated and decentralised markets. Peer-to-peer trading is very effective when combined with value-added services such as energy storage, demand-side management, energy optimisation and information services. The key strength of peer-to-peer trading is the use of distributed ledger technology (DLT) to provide a secure and transparent trading environment. The main limitations to the implementation of peer-to-peer trading in the real world are the lack of regulations on trading between peers, the lack of public awareness, the complexity of the technology and the lack of government motivation. This study will enable distributed energy resources (DERs) owners, utility companies and network operators to understand how to stay relevant and competitive in peer-to-peer trading markets.
In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an accurate and efficient energy trading model, and (iii) the need to satisfy distribution network constraints, it is challenging for prosumers to obtain optimal energy trading decisions that minimize their individual energy costs. To address the challenge, we first formulate the above problem as a Markov decision process and propose a multi-agent deep deterministic policy gradient algorithm to learn optimal energy trading decisions. To satisfy the distribution network constraints, we propose distribution network tariffs which we incorporate in the algorithm as incentives to incentivize energy trading decisions that help to satisfy the constraints and penalize the decisions that violate them. The proposed algorithm is model-free and allows the agents to learn the optimal energy trading decisions without having prior information about other agents in the network. Simulation results based on real-world datasets show the effectiveness and robustness of the proposed algorithm.
Peer-to-peer (P2P) energy sharing and trading show advantages over demand-side management, power-to-X conversions, and energy storages, with reduction in power loss, decrease in energy quality, and good potential for renewable penetration. In this chapter, agent-based P2P energy trading with dynamic internal pricing has been presented, in terms of various energy trading forms, underlying mechanisms, and mathematical models for dynamic internal pricing. Applications and prospects of blockchain and machine learning technologies in P2P energy trading have been reviewed, to show the recent progresses and advances. In order to promote the end-users' participation, decentralized electricity market design and techno-economic incentives are reviewed, so as to improve the social acceptance and widespread popularity. Last but not least, in order to enable the participation motivations of each building owner, challenges and outlooks are presented, including the development of individual energy trading price models and time-of-use trading management strategies, initiatives of internal pricing with considerations on diversity of surplus renewable energy and demand shortage of each building, energy trading priority or preference of each building owner, the internal price-driven energy trading behavior in the electricity market, and cost-benefit allocation schemes with fair distributions.
The smart grid technology has increased the penetration of distributed energy resources by the development and expansion of communication infrastructures and thus arisen the need to redesign electricity markets. With the emergence of prosumers as smart agents capable of both producing and consuming energy, the modern electricity markets should be prosumer-centric. In the novel prosumer-centric approach, the prosumers can trade energy with each other in a peer-to-peer (P2P) fashion. In these new markets, the retailers can be thought of as profit-based companies that either generate electrical energy themselves or purchase it at variable prices from the wholesale market or other prosumers in the local market. The retailers then sell the generated or purchased energy to the local consumers, wholesale markets, or other prosumers. Smart grid technology allows the prosumers (the end-users) to purchase their needed power from any producer in the grid and sell their excess energy to any consumer or retailer. This article designs a competitive market consisting of prosumers and retailers such that all prosumers (as buyers and sellers) and retailers conduct peer-to-peer energy trading. A novel decentralized approach called primal–dual sub-gradient algorithm (PDSGA) is used to clear the designed market without third-party involvement or disclosure of players’ private information. The feasibility of the proposed market and the effectiveness of the novel decentralized scheme for market-clearing are demonstrated through numerical implementation.
Multi-Agent systems technology can offer valuable tools to develop applications in domains involving transactions of assets. However, they usually do not have a proper support for reliable and decentralized recording of the transactions that are common in this kind of system. This support can be provided by the Blockchain technology. Furthermore, it is important to have means to represent in the system concepts that are intangible, such as asset and ownership. This paper presents a model of integration between Multi-Agent Systems and Blockchain where an artificial institution connects the intangible concepts related to transactions of assets to the concrete elements composing the system. An application example illustrates an implementation following the proposed integration model, showing its advantages and limitations. In the example, agents contract each other to provide services upon the payment of a dealt value through a system based on cryptocurrencies and blockchain. It highlights the essential contributions of the proposed approach to systems where agents transact assets: regulation of the system, representation of the notion of asset that does not depend on agents, and reliable recording of transactions based on Blockchain.
A key aspect of multi-energy microgrids (MEMGs) is the capability to efficiently convert and store energy in order to reduce the costs and environmental impact. Peer-to-peer (P2P) energy trading is a novel paradigm for decentralised energy market designs. In this paper, we investigate the external P2P energy trading problem and internal energy conversion problem within interconnected residential, commercial and industrial MEMGs. These two problems are complex decision-making problems with enormous high-dimensional data and uncertainty, so a multi-agent deep reinforcement learning approach combining the multi-agent actor-critic algorithm with the twin delayed deep deterministic policy gradient algorithm is proposed. The proposed approach can handle the high-dimensional continuous action space and aligns with the nature of P2P energy trading with multiple MEMGs. Simulation results based on three real-world MG datasets show that the proposed approach significantly reduces each MG’s average hourly operation cost. The impact of carbon tax pricing is also considered.
The increasing penetration of small-scale distributed energy resources (DER) has the potential to support cost-efficient energy balancing in emerging electricity systems, but is also fundamentally affecting the conventional operation paradigm of the latter. In this context, innovative market mechanisms need to be devised to better coordinate and provide incentives for DER to utilize their flexibility. Peer-to-Peer (P2P) energy trading has emerged as an alternative approach to facilitate direct trading between consumers and prosumers interacting in an energy collective and fosters more efficient local demand–supply balancing. While previous research has primarily focused on the technical and economic benefits of P2P trading, little effort has been made towards the incorporation of prosumers’ heterogeneous characteristics in the P2P trading problem. Here, we address this research gap by classifying the participating prosumers into multiple clusters with regard to their portfolio of DER, and analyzing their trading decisions in a simulated P2P trading platform. The latter employs the mid-market rate (MMR) local pricing mechanism to enable energy trading among prosumers and penalizes the contribution to the system demand peak of each prosumer. We formulate the P2P trading problem as a multi-agent coordination problem and propose a novel multi-agent deep reinforcement learning (MADRL) method to address it. The proposed method is founded on the combination of the multi-agent deep deterministic policy gradient (MADDPG) algorithm and the technique of parameter sharing (PS), which not only enables accelerating the training speed by sharing experiences and learned policies between all agents in each cluster, but also sustains the policies’ diversity between multiple clusters. To address the non-stationarity and computational complexity of MADRL as well as persevering the privacy of prosumers, the P2P trading platform acts as a trusted third party which augments the market collective trading information to help training of prosumer agents. Experiments with a large-scale real-world data-set involving 300 residential households demonstrate that the proposed MADRL method exhibits a strong generalization capability in the test data-set and outperforms the state-of-the-art MADRL methods with regard to the system operation cost, demand peak as well as computational time.
The inclusion of distributed generation from renewable resources in the main grid generates active end-user participation not only as consumers but also as producers of electrical energy. However, this inclusion also presents difficulties in establishing an adequate operation for network power sources, especially when the end-users in a retail market are able to concurrently assume the role of buyer and seller. High variability in electricity demand and the changing nature of renewable energy generation require that the distribution system operator (DSO) includes a negotiation algorithm that allows interconnected users to indirectly participate in energy trades to and from each other and to and from the main grid in order to ensure fair energy trade competition. The aforementioned algorithm increases the participation benefits in a liberalized market since producers have the advantage of selling their power source at the best price while consumers choose with whom to buy electric energy from at the lowest price. In this work, a negotiation algorithm for the interconnected distributed energy resources is proposed as a methodology in order to self-management the electrical exchange between producers and consumers. Finally, case studies are presented to validate the self-management methodology proposed in four different scenarios, and as a result, the simulations show that fair competition and the adequate operation of interconnected power resources for the electrical transactions between produced energy and consumed energy is possible.
This paper presents a novel Hierarchical and Decentralized Energy Management System which facilitates the Peer to Peer energy trading between prosumers in the community by coordinating the operation of distributed Home Battery Storage Systems and shiftable home appliances in a decentralized way to achieve a further reduction in the household energy costs for each house, compared to being operated individually (i.e. not being a part of the community). The hierarchical system consists of three levels: Selection level, Peer to Peer Management level, and Home Management level. First, the daily energy cost of each household is optimized individually using the lower layer. Then, the results are further improved through a peer to peer energy sharing algorithm in which house pairs are selected if they can achieve better reductions in operating cost through a joint optimization process. Finally, the optimal settings for the selected couples are obtained. An effective sensitivity analysis for the proposed management system is also introduced to study the effect of the size and the efficiency of the Home Battery Storage, the size of the PV generation, and the average annual household consumption on the economic performance of the householders in the community. The results obtained are based on real historic data for several prosumers in a real community system. The results show that the proposed energy management system guarantees a further reduction in the annual household energy costs for each house (up to 8.96%) when being operated as a part of a community, compared to being operated individually.
The current electricity networks were not initially designed for the high integration of variable generation technologies. They suffer significant losses due to the combustion of fossil fuels, the long-distance transmission, and distribution of the power to the network. Recently, prosumers, both consumers and producers, emerge with the increasing affordability to invest in domestic solar systems. Prosumers may trade within their communities to better manage their demand and supply as well as providing social and economic benefits. In this paper, we explore the use of Blockchain technologies and auction mechanisms to facilitate autonomous peer-to-peer energy trading within microgrids. We design two frameworks that utilize the smart contract functionality in Ethereum and employ the continuous double auction and uniform-price double-sided auction mechanisms, respectively. We validate our design by conducting A/B tests to compare the performance of different frameworks on a real-world dataset. The key characteristics of the two frameworks and several cost analyses are presented for comparison. Our results demonstrate that a P2P trading platform that integrates the blockchain technologies and agent-based systems is promising to complement the current centralized energy grid. We also identify a number of limitations, alternative solutions, and directions for future work.
Industrial investments into distributed energy resource technologies are increasing and playing a pivotal role in the global transactive energy, as part of a wider drive to provide a clean and stable source of energy. The management of prosumers, that consume and as well generate energy, with heterogeneous energy sources is critical for sustainable and efficient energy trading procedures. This paper is proposing a blockchain-assisted adaptive model, namely SynergyChain, for improving scalability and decentralization of the prosumer grouping mechanism in the context of Peer-to-Peer (P2P) energy trading. Smart contracts are used for storing transaction information and for the creation of the prosumer groups. SynergyChain integrates a reinforcement learning module to further improve the overall system performance and profitability by creating a self-adaptive grouping technique. The proposed SynergyChain is developed using Python and Solidity and has been tested using Ethereum test nets. The comprehensive analysis using the Hourly Energy Consumption data-set shows a 39.7% improvement in the performance and scalability of the system as compared to the centralized systems. The evaluation results confirm that SynergyChain can reduce request completion time along with an 18.3% improvement in the overall profitability of the system as compared to its counterparts.
Nowadays expert systems have been used in different fields. They must be able to operate as quickly and efficiently as possible. So, they need optimization mechanism in their different parts and optimization is a critical part of almost all expert systems. Because of difficulties in real world problems, traditional optimization techniques commonly cannot solve them. Therefore, stochastic algorithms are used to do the optimization in expert systems. Particle swarm optimization (PSO) is one the most famous stochastic optimization algorithms. But this algorithm has some difficulties like losing diversity, premature convergence, trapping in local optimums and imbalance between exploration and exploitation. To overcome these drawbacks, inspired by holonic organization in multi agent systems, a new hierarchical multi group structure for PSO is presented in this paper. Considering the particles in PSO as simple agents, PSO is a kind of multi agent system. Existence of different facilities and organizations in multi agent systems and their great impact on performance encouraged us to use them. So, inspired by holonic multi agent systems, a new structure for PSO is presented. This work has been done for the first time in the literature. Meanwhile, to promote exploration and exploitation ability of proposed structure and create a suitable balance between them, different tasks are assigned to different groups of this structure. So, a holonic PSO with different task allocations (HPSO-DTA) is created. It provides the opportunity to employ all aspects for empowering PSO including parameter settings, neighborhood topologies and learning strategies to enhance the ability of it unlike other versions of PSO that use only one of these aspects to improve their solutions. This structure provides a lot of advantages for PSO. It is a new topological structure that improves the performance of PSO. It provides several leaders with efficient information to guide the particles in the search space. Also, it helps to control suitable information flow between groups and particles in order to preserve diversity and prevent from trapping in local optimums. Meanwhile, with assigning different tasks to different groups of proposed structure, an appropriate balance between exploration and exploitation is created to enhance the performance of the algorithm. In each group, based on its assigned task, particles use different parameters settings, different dynamic neighborhood topologies and different learning strategies which are proposed in this paper to enhance the performance of algorithm. A set of thirty four benchmark functions are used to evaluate the performance of proposed structure. Proposed algorithm is compared with a set of well-known PSO algorithms that their efficiency have been proved. Experimental results and comparative analysis demonstrate good performance of HPSO-DTA compared to other algorithms. Its solution accuracy, convergence speed and robustness is completely appropriate especially in more complicated benchmarks.
This paper presents an efficient peer-to-peer energy sharing framework for numerous community prosumers to reduce energy costs and to promote renewable energy utilization. Specifically, for day-ahead and real-time energy management of prosumers, an inter-community energy sharing strategy and an intra-community energy sharing strategy are proposed, respectively. In the former strategy, prosumers can share energy with any community peers, and community aggregators represent their own prosumers to coordinate energy sharing. A two-phase model is designed. In the first phase the optimal energy sharing profiles of prosumers are derived to minimize the global energy costs, and in the second phase, equilibrium-based energy sharing prices are induced considering the individual interests of prosumers. In the latter strategy, prosumers share energy only with its community peers for time-saving to handle real-time uncertainties collaboratively to reduce real-time costs. The framework efficiency is verified by the simulation cases on a typical distribution network.
The area of building energy management has received a significant amount of interest in recent years. This area is concerned with combining advancements in sensor technologies, communications and advanced control algorithms to optimize energy utilization. Reinforcement learning is one of the most prominent machine learning algorithms used for control problems and has had many successful applications in the area of building energy management. This research gives a comprehensive review of the literature relating to the application of reinforcement learning to developing autonomous building energy management systems. Energy savings of greater than 20% are reported in the literature for more complex building energy management problems when implementing reinforcement learning. The main direction for future research and challenges in reinforcement learning are also outlined.
The prevalence of distributed energy resources encourages the concept of an electricity ‘Prosumer (Producer and Consumer)’. This paper proposes a distributed electricity trading system to facilitate the peer-to-peer electricity sharing amongst prosumers. The proposed system includes two layers. In the first layer, a Multi-Agent System (MAS) is designed to support the prosumer network, and an agent coalition mechanism is proposed to ena-ble the prosumers to form coalitions and negotiate electricity trading. In the second layer, a Blockchain (BC) based transaction settlement mechanism is proposed to enable the trusted and secure settlement of electricity trading transactions formed in the first layer. Simulations are conducted based on the Java Agent Development Environment (JADE) to validate the proposed electricity trading process.
Decentralized energy storage systems (ESS) are a promising means to more effectively match the supply and demand of fluctuating renewable energies. In most countries, however, ESS market share is small and whether or not the technology will attain a critical market share is subject to homeowners' investment decisions. For policy and industry alike, it is of particular interest to identify factors that drive ESS adoption. Empirically addressing this question, we hypothesized that the factors autarky and autonomy aspirations crucially determine ESS adoption decisions. In two studies (Ntotal = 489), sketching future decentralized energy scenarios, we found evidence for the importance of both factors for homeowners' evaluations of the technology. However, only autarky significantly affected homeowners' willingness to pay extra for ESS, in that homeowners invested more in the technology when autarky was higher (Study 1) or autarky benefits were emphasized (Study 2). In accordance with concepts aspiring to optimize energy flow on the low-voltage grid level (e.g. Smart Neighborhoods), we additionally examined the influence of autarky and autonomy aspirations on homeowners' willingness to exchange self-generated energy within a local energy network. Results showed that emphasis on autarky increased the subjective value of self-generated energy, decreasing the likelihood of peer-to-peer energy trading.