Table 1 - uploaded by Johannes Hiry
Content may be subject to copyright.
Source publication
With the proliferation of electric vehicles, the electrical distribution grids are more prone to overloads. In this paper, we study an intelligent pricing and power control mechanism based on contextual bandits to provide incentives for distributing charging load and preventing network failure. The presented work combines the microscopic mobility s...
Context in source publication
Citations
... Reference (Papadopoulos et al. 2013) introduces an example of this, where control strategies for EV charging are implemented and analysed using a multi-agent system (MAS), which models relevant system components as agents, such as the DSO or the EVs. The agent-based tool Simona allows detailed modelling of grid system participants, including the modelling of technical grid equipment, regulation, monitoring and control algorithms (TU Dortmund University 2023; Römer et al. 2019). Moreover, the power distribution system simulation and analysis tool GridLAB-D (Pacific Northwest National Laboratory 2023) also provides capabilities for modeling an agent-based system with a focus on power system modeling. ...
The planned massive increase of producers and consumers such as electric vehicles, heat pumps and photovoltaic systems in distribution grids will lead to new challenges in the electrical power system. These can include grid congestions at the low voltage level but also at higher voltage levels. Control strategies can enable the efficient use of flexibilities and therefore help mitigate upcoming problems. However, they need to be evaluated carefully before their application in the energy system to avoid any unwanted effects and to choose the most fitting strategy for each application. In this publication, a Python-based modular simulation tool for developing and analysing control strategies for prosumers, which uses pandapower (Thurner et al. 2018), is presented. It is intended for sequential simulations and enables detailed operational analyses, which include evaluating the influence on grid situations, the necessary behavior of energy system components, required measurements and communications. This publication also gives an overview of control strategies, existing simulation tools, how the modular simulation tool fits in and illustrates its functionalities in an application example, which further highlights its versatility and efficiency. Time series simulations with the tool allow analyses regarding the effect of control strategies on power flow results. Moreover, the simulation tool also facilitates evaluating the behavior of energy system components (e.g. distribution substations), necessary communications and measurements as well as any faults that might occur.
... The presented general DBFS approach calculates the power flow within the agent-based discrete-event simulation environment, SIMONA. Its general concept, capabilities based on an initial version, has already been published in several publications, e.g., [2], [11], [12]. However, since the latest available publication, further developments have been added for several research projects. ...
In the energy transition context, the use of steady-state time series is a promising approach to account for temporal interdependencies and flexibilities in modern distribution power system analysis, planning, and operation processes. This paper proposes a distributed backward–forward sweep power flow algorithm executed in a discrete-event, agent-based simulation framework. The algorithm shows fast convergence, allows for concurrent execution, and scales up to large-scale multi-voltage level grids with arbitrary topology. An agent-based simulation model integrates the developed algorithm to generate detailed grid utilization, asset, and system participant time series.
We demonstrate the capabilities of our approach by performing several simulations, leveraging the proposed algorithm, on nine different benchmark grid models. The selected models comprise grids at a single voltage level, medium voltage level, and combined multi-voltage levels. The evaluation of the numerical results validates the approach and demonstrates its capabilities.
... The starting point of the proposed tool is the Eclipse SUMO (Simulation of Urban Mobility) (López et al., 2018), a traffic micro-simulator which proved its applicability for traffic simulations and simulations dealing with electromobility in many works (Ascher and Hackenberg, 2015, Burmeister et al., 2015, Yan et al., 2018, Römer et al., 2019, Khan et al., 2019. Eclipse SUMO is also an accepted tool to determine the impact of certain measures or use-cases not only on traffic parameters but also the environment, see for example (Přibyl et al., 2020). ...
When designing new electric vehicles for urban transport, both vehicle producers and operators need to establish and verify requirements that the vehicle has to fulfil regarding, e.g., energy storage capacity, driving range, or battery wear. These requirements are typically verified using simulation tools that concentrate mostly on electrical quantities, but disregard the influence of road infrastructure, surrounding traffic and detours. At the same time, an expansion of the electric vehicle fleet may have a negative impact on the existing power grid. Sufficient dimensioning of the grid elements and their verification are therefore necessary to keep the electric transport service reliable. This paper demonstrates a tool suitable for verification of battery-assisted trolleybus fleet and power infrastructure parameters, based on extensions to the SUMO traffic simulator. The tool carries out a joint simulation of electric and traffic-related quantities. It is demonstrated on seven use-cases inspired by real-life problems.
... Römer et al. [58] implemented a contextual bandit process to control charging demands of electric vehicles by adjusting the price and recommending stations to users. Considering station load, charging price, or income as features which affect driver behavior, they analyzed the effect of bandit algorithms on maximum loads at stations and average rewards of drivers. ...
With Mobility-as-a-Service platforms moving toward vertical service expansion, we propose a destination recommender system for Mobility-on-Demand (MOD) services that explicitly considers dynamic vehicle routing constraints as a form of a "physical internet search engine". It incorporates a routing algorithm to build vehicle routes and an upper confidence bound based algorithm for a generalized linear contextual bandit algorithm to identify alternatives which are acceptable to passengers. As a contextual bandit algorithm, the added context from the routing subproblem makes it unclear how effective learning is under such circumstances. We propose a new simulation experimental framework to evaluate the impact of adding the routing constraints to the destination recommender algorithm. The proposed algorithm is first tested on a 7 by 7 grid network and performs better than benchmarks that include random alternatives, selecting the highest rating, or selecting the destination with the smallest vehicle routing cost increase. The RecoMOD algorithm also reduces average increases in vehicle travel costs compared to using random or highest rating recommendation. Its application to Manhattan dataset with ratings for 1,012 destinations reveals that a higher customer arrival rate and faster vehicle speeds lead to better acceptance rates. While these two results sound contradictory, they provide important managerial insights for MOD operators.
... environmental factors and agent profiles, it is referred as contextual MAB [8]. Due to its simple and general structure, contextual MAB has been successfully applied in many fields, such as recommendation systems [9], clinical trials [10], web advertisements [11], electric vehicle charging control [12], and etc. ...
Residential loads have great potential to enhance the efficiency and reliability of electricity systems via demand response (DR) programs. One major challenge in residential DR is how to learn and handle unknown and uncertain customer behaviors. In this paper, we consider the residential DR problem where the load service entity (LSE) aims to select an optimal subset of customers to optimize some DR performance, such as maximizing the expected load reduction with a financial budget or minimizing the expected squared deviation from a target reduction level. To learn the uncertain customer behaviors influenced by various time-varying environmental factors, we formulate the residential DR as a contextual multi-armed bandit (MAB) problem, and develop an online learning and selection (OLS) algorithm based on Thompson sampling to solve it. This algorithm takes the contextual information into consideration and is applicable to complicated DR settings. Numerical simulations are performed to demonstrate the learning effectiveness of the proposed algorithm.
... Applications exist in areas of mobility. Researchers have studied the demand management of electric vehicle charging stations by changing charging prices and recommending alternative stations when one is congested [43]. Zhou et al. [40] developed a recommender system for sequential departure time and path choice with on-time arrival reliability. ...
While public transit network design has a wide literature, the study of line planning and route generation under uncertainty is not so well covered. Such uncertainty is present in planning for emerging transit technologies or operating models in which demand data is largely unavailable to make predictions on. In such circumstances, we propose a sequential route generation process in which an operator periodically expands the route set and receives ridership feedback. Using this sensor loop, we propose a reinforcement learning-based route generation methodology to support line planning for emerging technologies. The method makes use of contextual bandit problems to explore different routes to invest in while optimizing the operating cost or demand served. Two experiments are conducted. They (1) prove that the algorithm is better than random choice; and (2) show good performance with a gap of 3.7% relative to a heuristic solution to an oracle policy.
... In previous works, SUMO has been combined in other traffic related ressource allocation problems using reinforcement learning. For example, [5] uses SUMO to train price policies of the smart electrical distribution grid for prevention of overloads due to electric vehicles. In [4], the authors use SUMO to compare various selfish routing regimes and propose usage of reinforcemnt learning for self-organization. ...
Occupied truck parking lots regularly cause hazardous situations. Estimation of current parking lot state could be utilized to provide drivers parking recommendations. In this work, we highlight based on a simulation scenario, how sparse observations, as obtained by a mobile application could be utilized to estimate parking lot occupancy. Our simulated results reveal that a detection of a filled parking lot could be possible with an error of less than half an hour, if the required data would be available.
Mobility service route design requires demand information to operate in a service region. Transit planners and operators can access various data sources including household travel survey data and mobile device location logs. However, when implementing a mobility system with emerging technologies, estimating demand becomes harder because of limited data resulting in uncertainty. This study proposes an artificial intelligence-driven algorithm that combines sequential transit network design with optimal learning to address the operation under limited data. An operator gradually expands its route system to avoid risks from inconsistency between designed routes and actual travel demand. At the same time, observed information is archived to update the knowledge that the operator currently uses. Three learning policies are compared within the algorithm: multi-armed bandit, knowledge gradient, and knowledge gradient with correlated beliefs. For validation, a new route system is designed on an artificial network based on public use microdata areas in New York City. Prior knowledge is reproduced from the regional household travel survey data. The results suggest that exploration considering correlations can achieve better performance compared to greedy choices and other independent belief-based techniques in general. In future work, the problem may incorporate more complexities such as demand elasticity to travel time, no limitations to the number of transfers, and costs for expansion.
Mobility service route design requires potential demand information to well accommodate travel demand within the service region. Transit planners and operators can access various data sources including household travel survey data and mobile device location logs. However, when implementing a mobility system with emerging technologies, estimating demand level becomes harder because of more uncertainties with user behaviors. Therefore, this study proposes an artificial intelligence-driven algorithm that combines sequential transit network design with optimal learning. An operator gradually expands its route system to avoid risks from inconsistency between designed routes and actual travel demand. At the same time, observed information is archived to update the knowledge that the operator currently uses. Three learning policies are compared within the algorithm: multi-armed bandit, knowledge gradient, and knowledge gradient with correlated beliefs. For validation, a new route system is designed on an artificial network based on public use microdata areas in New York City. Prior knowledge is reproduced from the regional household travel survey data. The results suggest that exploration considering correlations can achieve better performance compared to greedy choices in general. In future work, the problem may incorporate more complexities such as demand elasticity to travel time, no limitations to the number of transfers, and costs for expansion.