Damien Ernst

Damien Ernst
University of Liège | ulg · Department of Electrical Engineering and Computer Science - Montefiore Institute

Prof. Dr. Ir.

About

223
Publications
58,729
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
10,942
Citations
Citations since 2016
51 Research Items
7616 Citations
201620172018201920202021202202004006008001,0001,2001,400
201620172018201920202021202202004006008001,0001,2001,400
201620172018201920202021202202004006008001,0001,2001,400
201620172018201920202021202202004006008001,0001,2001,400
Introduction
My name is Damien Ernst. I work as Professor at the University of Liège (Belgium). I am affiliated with the Montefiore Research Unit . I do research in control theory with a particular emphasis on power system control problems and reinforcement learning. More info about me on my website: http://www.damien-ernst.be

Publications

Publications (223)
Chapter
This chapter analyses the role that complementarity may play in renewable power generation asset siting decisions and its impact on power system design and economics. To this end, a two-stage approach is employed. In the first stage, a siting method is used to select a pre-specified number of sites optimizing a pre-defined siting criterion. In the...
Article
Full-text available
This paper studies the problem of siting renewable power generation assets using large amounts of climatological data while accounting for their spatiotemporal complementarity. The problem is cast as a combinatorial optimisation problem selecting a pre-specified number of sites so as to minimise the number of simultaneous low electricity production...
Article
This paper provides a detailed account of the impact of different offshore wind siting strategies on the design of the European power system. To this end, a two-stage method is proposed. In the first stage, a highly-granular siting problem identifies a suitable set of sites where offshore wind plants could be deployed according to a pre-specified c...
Preprint
Full-text available
In power systems, large-scale optimisation problems are extensively used to plan for capacity expansion at the supra-national level. However, their cost-optimal solutions are often not exploitable by decision-makers who are preferably looking for features of solutions that can accommodate their different requirements. This paper proposes a generic...
Article
Full-text available
Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network l...
Preprint
Full-text available
Training recurrent neural networks is known to be difficult when time dependencies become long. Consequently, training standard gated cells such as gated recurrent units and long-short term memory on benchmarks where long-term memory is required remains an arduous task. In this work, we propose a general way to initialize any recurrent network conn...
Article
Gym-ANM is a Python package that facilitates the design of reinforcement learning (RL) environments that model active network management (ANM) tasks in electricity networks. Here, we describe how to implement new environments and how to write code to interact with pre-existing ones. We also provide an overview of ANM6-Easy, an environment designed...
Article
Full-text available
This paper studies the economics of carbon-neutral synthetic fuel production from renewable electricity in remote areas where high-quality renewable resources are abundant. To this end, a graph-based optimisation modelling framework directly applicable to the strategic planning of remote renewable energy supply chains is proposed. More precisely, a...
Preprint
Full-text available
The accurate representation of variable renewable generation (RES, e.g., wind, solar PV) assets in capacity expansion planning (CEP) studies is paramount to capture spatial and temporal correlations that may exist between sites and impact both power system design and operation. However, it typically has a high computational cost. This paper propose...
Article
This paper presents a simulation-based methodology for assessing the impact of employing different distribution system operator’s remuneration strategies on the economic sustainability of electrical distribution systems. The proposed methodology accounts for the uncertainties posed by the integration of distributed electricity generation resources,...
Preprint
Full-text available
This paper studies the economics of carbon-neutral synthetic fuel production from renewable electricity in remote areas where high-quality renewable resources are abundant. To this end, a graph-based optimisation modelling framework directly applicable to the strategic planning of remote renewable energy supply chains is proposed. More precisely, a...
Article
In Wallonia, Belgium's southern region, the distribution component of the overall electricity retail tariff is essentially volumetric, i.e. based on the users' energy consumption (in €/kWh). Residential prosumers, moreover, are connected to the grid via a net-metering system. In this paper, we rely on a sophisticated multi-agent tariff simulator –...
Preprint
Full-text available
Spatiotemporal complementarity between variable renewable energy sources (RES) has received a great deal of attention in recent years. However, its value for power systems is still not properly understood. This research gap is tackled in the current work by evaluating the benefits of siting RES assets according to resource complementarity criteria....
Conference Paper
Full-text available
In this paper, we propose an extension to the policy gradient algorithms by allowing starting states to be sampled from a probability distribution that may differ from the one used to specify the reinforcement learning task. In particular, we suggest that, between policy updates, starting states should be sampled from a probability density function...
Conference Paper
Full-text available
When an agent has limited information on its environment, the suboptimality of an RL algorithm can be decomposed into the sum of two terms: a term related to an asymptotic bias (suboptimality with unlimited data) and a term due to overfitting (additional suboptimality due to limited data). In the context of reinforcement learning with partial obser...
Preprint
Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network l...
Article
This paper proposes an optimisation-based framework to tackle long-term centralised planning problems of multi-sector, integrated energy systems including electricity, hydrogen, natural gas, synthetic methane and carbon dioxide. The model selects and sizes the set of power generation, energy conversion and storage as well as carbon capture technolo...
Article
This paper proposes a framework to assess the complementarity between geographically dispersed variable renewable energy resources over arbitrary time scales. More precisely, the framework relies on the concept of critical time windows, which offers an accurate, time-domain description of low-probability power production events impacting power syst...
Chapter
Koopman operator theory offers numerous techniques for analysis and control of complex systems. In particular, in this chapter we will argue that for the problem of convergence to an equilibrium, the Koopman operator can be used to take advantage of the geometric properties of controlled systems, thus making the optimal solutions more transparent,...
Article
Full-text available
Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that...
Article
In this paper, we propose an extension to the policy gradient algorithms by allowing starting states to be sampled from a probability distribution that may differ from the one used to specify the reinforcement learning task. In particular, we suggest that, between policy updates, starting states should be sampled from a probability density function...
Article
Current global environmental challenges require vigorous and diverse actions in the energy sector. One solution that has recently attracted interest consists in harnessing high-quality variable renewable energy resources in remote locations, while using transmission links to transport the power to end users. In this context, a comparison of western...
Preprint
In this paper, we propose a new deep neural network architecture, called NMD net, that has been specifically designed to learn adaptive behaviours. This architecture exploits a biological mechanism called neuromodulation that sustains adaptation in biological organisms. This architecture has been introduced in a deep-reinforcement learning architec...
Preprint
Full-text available
This paper proposes a systematic framework to assess the complementarity of renewable resources over arbitrary geographical scopes and temporal scales which is particularly well-suited to exploit very large data sets of climatological data. The concept of critical time windows is introduced, and a spatio-temporal criticality indicator is proposed,...
Preprint
Full-text available
Current global environmental challenges require firm, yet diverse resolutions in the energy sector. One promising solution consists in harnessing high-quality variable renewable energy (VRE) resources in remote locations, while using transmission links to wheel the power towards end users. In this context, a comparison of western Europe and Greenla...
Chapter
The Global Grid advocates the connection of all regional power systems into one electricity transmission system spanning the whole globe. Power systems are currently forming larger and larger interconnections. Environmental awareness and increased electricity consumption leads more investments toward renewable energy sources, which are abundant in...
Article
Full-text available
This paper stands in the context of reinforcement learning with partial observability and limited data. In this setting, we focus on the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data), and theoretically show that while potentially increasing the asymptotic bias, a...
Article
In applications, such as biomedicine and systems/synthetic biology, technical limitations in actuation complicate implementation of time-varying control signals. In order to alleviate some of these limitations, it may be desirable to derive simple control policies, such as step functions with fixed magnitude and length (or temporal pulses). In this...
Article
In many applications, and in systems/synthetic biology, in particular, it is desirable to compute control policies that force the trajectory of a bistable system from one equilibrium (the initial point) to another equilibrium (the target point), or in other words to solve the switching problem. It was recently shown that, for monotone bistable syst...
Conference Paper
In this paper, we review past (including very recent) research considerations in using reinforcement learning (RL) to solve electric power system decision and control problems. The RL considerations are reviewed in terms of specific electric power system problems, type of control and RL method used. We also provide observations about past considera...
Article
In this paper, we review past (including very recent) research considerations in using reinforcement learning (RL) to solve electric power system decision and control problems. The RL considerations are reviewed in terms of specific electric power system problems, type of control and RL method used. We also provide observations about past considera...
Chapter
In this work, we propose a simple yet effective solution to the problem of connectome inference in calcium imaging data. The proposed algorithm consists of two steps. First, processing the raw signals to detect neural peak activities. Second, inferring the degree of association between neurons from partial correlation statistics. This paper summari...
Article
This paper addresses the problem of an aggregator controlling residential heat pumps to offer a direct control flexibility service. The service consists of a power modulation, upward or downward, that is activated at a given time period over a fixed number of periods. The service modulation is relative to an optimized baseline that minimizes the en...
Article
Full-text available
This paper addresses the problem of efficiently operating the storage devices in an electricity microgrid featuring photovoltaic (PV) panels with both short-and long-term storage capacities. The problem of optimally activating the storage devices is formulated as a sequential decision making problem under uncertainty where, at every time-step, the...
Conference Paper
This paper extends the Global Capacity ANnouncement procedure proposed in [5] along two directions. First, two new stopping criteria are considered. Second, annual losses are evaluated using representative days to approximate the injection duration curve. The extensions are validated on an updated model of a real-life system. The emphasis is on the...
Conference Paper
Full-text available
In this paper, the penetration of grid-connected photovoltaic systems is studied, experimentally tested and compared to simulation results. In particular, how the inverse current flow and unbalance situations affect the voltage in the low-voltage grid. Thus, a test platform has been developed for obtaining experimental results with grid-tied commerc...
Article
In this paper, the overvoltage problems that might arise from the integration of photovoltaic panels into low-voltage distribution networks is addressed. A distributed scheme is proposed that adjusts the reactive and active power output of inverters to prevent or alleviate such problems. The proposed scheme is model-free and makes use of limited co...
Article
Full-text available
In the Bayesian Reinforcement Learning (BRL) setting, agents try to maximise the collected rewards while interacting with their environment while using some prior knowledge that is accessed beforehand. Many BRL algorithms have already been proposed, but even though a few toy examples exist in the literature, there are still no extensive or rigorous...
Conference Paper
This paper aims to design an algorithm dedicated to operational planning for microgrids in the challenging case where the scenarios of production and consumption are not known in advance. Using expert knowledge obtained from solving a family of linear programs, we build a learning set for training a decision-making agent. The empirical performances...
Conference Paper
Full-text available
This paper addresses the problem of computing optimal structured treatment interruption strategies for HIV infected patients. We show that reinforcement learning may be useful to extract such strategies directly from clinical data, without the need of an accurate mathematical model of HIV infection dynamics. To support our claims, we report simulat...
Conference Paper
This paper presents a general process set in the GREDOR (French acronym for " Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables ") project to address the challenges in distribution systems posed by the integration of renewable generation, changing load patterns, and the changes in the electricity market sector. A use case de...
Conference Paper
This article deals with the problem of automatically establishing a correspondence between two databases populated independently over the years by a distribution company , for instance a SCADA system and a geographical information system. This problem is abstracted as a graph matching problem, well known in the combinatorial op-timisation community...
Article
Full-text available
With modern video games frequently featuring sophisticated and realistic environments, the need for smart and comprehensive agents that understand the various aspects of complex environments is pressing. Since video game AI is often specifically designed for each game, video game AI tools currently focus on allowing video game developers to quickly...
Chapter
The Global Grid advocates the connection of all regional power systems into one electricity transmission system spanning the whole globe. Power systems are currently forming larger and larger interconnections. Environmental awareness and increased electricity consumption leads more investments toward renewable energy sources, which are abundant in...
Conference Paper
This paper proposes a methodology to estimate the maximum revenue that can be generated by a company that operates a high-capacity storage device to buy or sell electricity on the day-ahead electricity market. The methodology exploits the Dynamic Programming (DP) principle and is specified for hydrogen-based storage devices that use electrolysis to...
Article
Full-text available
Immunological failure is identified from the estimation of certain parameters of a mathematical model of the HIV infection dynamics. This identification is supported by clinical research results from an original clinical trial. Standard clinical data were collected from infected patients starting Highly Active Anti-Retroviral Therapy (HAART), just...
Chapter
A general approach to real-time transient stability control is described, yielding various complementary techniques: pure preventive, open-loop emergency, and closed-loop emergency controls. Recent progress in terms of a global transient stability-constrained optimal power flow is presented, yielding in a scalable nonlinear programming formulation...
Conference Paper
Full-text available
In this paper, a distributed model-free control scheme to mitigate overvoltage problems caused by high photovoltaic generation in low-voltage feeders is proposed. The distributed controllers are implemented on the photovoltaic inverters and modulate the active and reactive power injected into the network. In particular, they direct photovoltaic uni...
Article
Full-text available
In this work, we propose a simple, but yet efficient method for the problem of connectome inference in calcium imaging data. The proposed algorithm consists of two steps. First, processing the raw signals to detect neural peak activities. Second, inferring the degree of association between neurons from partial correlation statistics. This paper sum...
Conference Paper
Full-text available
In this paper, we consider the problem of optimal exogenous control of gene regulatory networks. Our approach consists in adapting and further developing an established reinforcement learning algorithm called the fitted Q iteration. This algorithm infers the control law directly from the measurements of the system's response to external control inp...
Article
Full-text available
In order to operate an electrical distribution network in a secure and cost-efficient way, it becomes necessary, due to the rise of renewable energy based distributed generation, to develop Active Network Management (ANM) strategies. These strategies rely on short-term policies that control the power injected by generators and/or taken off by loads...
Article
Direct policy search (DPS) and look-ahead tree (LT) policies are two widely used classes of techniques to produce high performance policies for sequential decision-making problems. To make DPS approaches work well, one crucial issue is to select an appropriate space of parameterized policies with respect to the targeted problem. A fundamental issue...