Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
On Data-Driven Modeling and Control in Modern
Power Grids Stability: Survey and Perspective
Xun Gonga,b, Xiaozhe Wanga,∗, Bo Caob
aDepartment of Electrical and Computer Engineering, McGill University, 3480 Rue
University, Montreal, H3A 0E9, Quebec, Canada
bHuawei Montreal Research Centre, 7101 Park Ave, Montreal, H3N 1X9, Quebec, Canada
Abstract
Modern power grids are fast evolving with the increasing volatile renewable
generation, distributed energy resources (DERs) and time-varying operating
conditions. The DERs include rooftop photovoltaic (PV), small wind turbines,
energy storages, flexible loads, electric vehicles (EVs), etc. The grid control is
confronted with low inertia, uncertainty and nonlinearity that challenge the op-
eration security, efficacy and efficiency. The ongoing digitization of power grids
provides opportunities to address the challenges with data-driven and control.
This paper provides a comprehensive review of emerging data-driven dynami-
cal modeling and control methods and their various applications in power grid.
Future trends are also discussed based on advances in data-driven control.
Keywords: power grid dynamics and control, data-driven modeling, Koopman
operator, data-driven control, physics-informed machine learning, system
identification and control
1. Introduction
Renewable energy is replacing fuel-type generation for sustainable power
grids, aiming to reduce greenhouse gas emissions [1]. The modern power grids
are composed of diverse energy resources interconnected through power net-
works, which include centralized energy resources (e.g., synchronous genera-
tors, solar and wind farms), and distributed energy resources (DERs) (e.g.,
distributed renewable generation, energy storage systems, electric vehicles, and
thermostatically controlled loads). Particularly, many modernized energy re-
sources are power converter-interfaced and “low-inertia” in nature. The uncer-
∗Corresponding author
1This work was supported by the Fonds de Recherche du Quebec-Nature et technologies
under Grant FRQ-NT PR-298827 and by the Natural Sciences and Engineering Research
Council of Canada (NSERC) under Alliance Grants ALLRP ALLRP 571554-21.
Preprint submitted to Applied Energy August 8, 2023
arXiv:2308.03591v1 [eess.SY] 7 Aug 2023
tainty and nonlinearity of them endanger grid operation security, efficacy and
efficiency [1, 2].
The uncertainty and nonlinearity of modern power grids originate from both
the energy resources and their systemwide interactions. First, the rapid growth
of renewable energy and flexible loads introduces uncertainty and nonlinearity
due to the nonlinear stochastic nature of the renewable sources (e.g., wind and
solar) and the human behaviors[2]. Second, the interaction dynamics of energy
resources through the power networks could suffer from nonlinearity when en-
countering large disturbances. Besides, the increasing number of diverse energy
resources and their hierarchical multi-timescale operation increases the system
complexity. All these factors pose challenges to acquiring accurate system dy-
namic models and maintaining stable and secure operation of power systems.
Fortunately, the ongoing digitization (e.g., the fast-deploying information,
communication and computing techniques) throughout power grids provides
opportunities to address the operational challenges by data-driven control. In
this paper, we consider the hierarchical control framework as it is the mature
and scalable means to aggregate and manage massive energy resources in mod-
ernized power grids [3–5]. From the bottom to top of the hierarchical structure
are distributed energy resources, microgrids/virtual power plants/centralized
energy resource, and system operators. The hierarchical control includes pri-
mary control at the individual DER level, and secondary and tertiary control
at the systemwide level. Examples of the control functions for different control
types are summarized in Table 1.
With the data from the advanced sensing infrastructure (e.g., sensors or
transducers, phasor measurement units, smart meters), the energy resources can
be coordinately controlled to realize different operation objectives at different
levels and time scales in a model-free data-driven fashion. To lift the operational
challenges, effort has been made to apply data-driven control methods for differ-
ent use cases in modern power grids and microgrids, such as voltage control [6–
10], frequency control [8–14], wide-area damping control [15–17], cyber-resilient
control [12, 14, 18, 19], as well as demand response [20–23]. From the high-level
Table 1
Examples of control functions for different control types
Type of Control Functions
Voltage/frequency stability perseverance,
Primary (Component) plug-and-play of DERs,
inertia and local damping control.
Voltage restoration/regulation,
Secondary (Area) wide-area damping control,
frequency regulation.
Energy management,
Tertiary (Grid) optimal operation,
tie-line power flow control.
2
perspective, the data-driven control methods in these studies are based on sur-
rogate models which can be classified into: (1) linear system data-driven control
methods; (2) nonlinear system data-driven control methods. This paper pro-
vides a comprehensive review on the recent advances of both categories of the
methods, with the concentration on the second category that can be further clas-
sified into: (2.A) pure machine learning-based methods; (2.B) physics-informed
machine learning methods; (2.C) Koopman-based methods. Besides, the paper
will discuss the existing applications and trends of these methods in modern
power grids.
Note that emerging data-driven methods such as iterative feedback tuning,
model-free adaptive control, and learning-based control are hot topics and re-
viewed by the researchers in control and robotics communities [24–27]. For ex-
ample, the learning-based model-predictive control [26] was investigated for best
closed-loop performance through either improved prediction model or proper
parameterization of controller (costs and constraints). The mathematical for-
mulation of learning-based control and reinforcement learning (RL) were also
discussed based on different safety levels [27]. However, these review papers did
not particularly focus on power system applications. The papers [2, 28] summa-
rized data-driven control in modern power grids, whereas they mainly focused
only on the category (2.A) mentioned above, i.e., universal machine learning
methods such as RL. The review paper [29] was the first comprehensive sum-
mary of current microgrid control framework with applications of RL to address
emerging microgrid challenges (i.e., uncertainty and extreme weather). It illus-
trated the fusion of RL in three ways: (i) model identification and parameter
tuning; (ii) supplementary signal generation; (iii) controller substitution. How-
ever, the focus of [29] was still pure RL that belongs to the category (2.A); the
data-driven yet interpretable modeling (with control inputs incorporated) was
not illustrated. Besides, emerging data-driven control frameworks other than
RL were not investigated. The review paper [30] presented a comprehensive
investigation of physics-informed neural network (NN) for power system appli-
cations, which fell in the category (2.B) whereas the review concerned only a
specific learning machine (i.e., NN) and the incorporation of control was not
discussed in detail.
Motivated by the limitations of previous review papers, the goal of this paper
is to provide a comprehensive method review of data-driven approaches with a
broader scope, aiming to address increasing time-varying uncertainty and non-
linearity in modern power grids. It is important to note that data-driven control
methods are not intended to replace existing model-based control frameworks
[29], but rather to complement them as supplementary or enhancement solutions
driven by data. By offering a broader technology map and comparative vision
of different data-driven control approaches, this paper aims to inspire new ideas
and feasible solutions in power systems. Compared to previous review works
[2, 28, 30], this paper makes the following key contributions:
(1) It includes a broader range of methods, encompassing linear identifica-
tion and control techniques based on input-output models, state space repre-
sentations, transfer function identification, as well as nonlinear methods such
3
as reinforcement learning, supervised learning-based and Koopman-based ap-
proaches.
(2) It provides a comprehensive comparison of data-driven approaches from
various perspectives, including modeling structure, identification and control
methods, data requirements, adaptiveness, interpretability, scalability, and train-
ing efficiency. Particularly, this paper introduces a notable advancement by
including and reviewing Koopman-based methods for the first time. These
methods are categorized as (2.C) and are particularly promising in addressing
challenges such as nonlinearity and uncertainty while leveraging established lin-
ear identification and control techniques in conjunction with emerging machine
learning methodologies.
(3) The paper organizes different categories of data-driven methods system-
atically to generalize the data-driven control frameworks, enabling researchers
and practitioners to navigate through the diverse landscape of data-driven ap-
proaches in a more systematic manner, fostering novel ideas and feasible data-
driven solutions in power systems.
The rest of this paper is organized as follows. Section 2 presents the pre-
liminaries and a brief history of state-of-the-art data-driven control methods in
power grid applications. Sections 3-4 present critical reviews for both categories
of data-driven methods. Specifically, Section 3 elaborates on the linear system
data-driven control methods. Section 4 details nonlinear system data-driven
control methods. Section 5 provides a grid application overview and discusses
the future trend. Section 6 concludes the paper.
2. Preliminaries and an Overview on Data-Driven Control for Power
System Applications
This section presents an overview of data-driven identification and control
methods and their applications in power systems. First of all, the following
terminologies adopted in the manuscript are synonyms of or highly related to
data-driven control, thus they need to be clarified to avoid conceptual ambiguity.
1) Data-driven control: the term “data-driven” means that the identifica-
tion and/or the design of the controller are based entirely on experimental data
collected from the plant or simulators but not any explicit information from
first principle mathematical models of the controlled process [25]. Data-driven
control often refers to the closed-loop control starting and ending up with data
[25]. In this paper, we focus on the data-driven control that is based on
data-driven modeling, which is defined below.
2) Data-driven modeling: the modeling method based on identification
or learning from data while disregarding explicit knowledge of the system’s
physical behavior [31]. The modeling and identification/learning can be done
with or without control inputs incorporated.
3) Model-free control: the “model” in model-free control and model-based
control refers to the first principle physical model of the system of interest. The
term “model-free” refers to an alternative technique to control systems without
4
traditional first principle physical models. This can be done by using a simplified
representation of the system in a data-driven fashion. In other words, it is a
synonym for data-driven control [25].
4) Adaptive control: adaptive control is the control method used by a
controller which must adapt to a controlled system with parameters which vary,
or are initially uncertain [32]. The adaptive identification and control itself can
be conducted based on either physical parametric models or data-driven mod-
els (model-free representations). Data-driven control may realize adaptiveness
through online identification/learning that quickly adjusts the model parame-
ters to achieve fast adaption to time-varying uncertainty.
According to the above-mentioned context: (i) Model-free control and data-
driven control are mainly the same concepts that are interpreted from different
perspectives of a method. Although certain physics-inspired information (such
as structure design or constraints) may be incorporated in data-driven mod-
eling and control, we assume model-free control and data-driven control are
interchangeable terms in the manuscript. (ii) Data-driven control can be adap-
tive or not depending on whether online identification is realized. The two
assumptions apply to the rest of the paper without further explanation.
Generally, the data-driven control methodologies can be classified by the
assumed data-driven model, i.e., linear system control and nonlinear system
control. In power system applications, the linear transfer function model and
identification were first investigated since early-nineties to model the dynamical
modes of power grids. An example is the transfer function of impulse response
between small-signal inputs and outputs with least-squares identification pre-
sented in 1993 [33]. The identified transfer function can be applied for different
purposes such as: (i) tuning, design, and testing of power system control systems
such as power system stabilizer (PSS), static var compensator (SVC) and many
others; (ii) validation of power system small-signal models for grid planning and
operation. Different transfer functions can be identified to deal with different
potential operating conditions, whereby robust controllers can be designed ac-
cordingly. In the meantime, the authors in [34] proposed a reduced-order linear
state space model based on a minimal realization approach for modal analysis,
which shows the effectiveness of linear methods identified by transient (some-
times termed as ring-down [35]) data for conventional power systems with large
rotating masses. Similar work on multiple-input multiple-output (MIMO) state
space identification based on pulses can be found in [36], which was effectively
applied to the modal analysis of bulk power systems. The subspace methods
including ORT, N4SID, MOESP, CVA) were used for the linear state space
identification with different probing tests [36–38]. Afterwards, many variants
of transfer function identification methods [8, 13, 39–48] and state space iden-
tification methods [6, 15, 22, 49–52] were developed for data-driven control in
the context of power grid applications such as damping control, voltage control
and microgrid control (primary, secondary and tertiary) and aggregated load
control. Generally, both transfer function and linear state space methods can
represent systems well using a locally linearized model around a fixed operating
point under the assumption that the system is an LTI (linear time-invariant).
5
However, modern grids tend to be low-inertia with the increasing penetration
of volatile inverter-interfaced renewables, leading to wide-range dynamics with
higher levels of time-varying uncertainty and nonlinearity. This compromises
the LTI assumption. Also, the saturation of control signals and nonlinearity
of control input channels can also undermine the LTI assumption, leading to a
decrease in identification accuracy. The details of different linear identification
and control methods will be discussed in Section 3.
Nonlinear identification methods are suited to address the above-mentioned
challenges. Traditionally, nonlinear identification methods based on Hibert-
Huang Transform (empirical mode decomposition + Hilbert analysis) are used
to adaptively model power system nonlinear dynamics [35]. Supervised learn-
ing (i.e., a main branch of machine learning) based nonlinear modeling started
emerging in the mid-nineties because of the more powerful nonlinear fitting ca-
pability of universal learning machines such as NNs [53–56]. For example, in
1996, Innocent Kamwa et al [54] proposed a MIMO supervised-learning recur-
rent neural network (RNN) that is equivalent to a general differential equation,
which can be trained offline with the output-layer parameters then identified on-
line in a recursive fashion to adaptively describe time-varying nonlinear power
system dynamics. However, supervised learning is more often used for nonlinear
dynamic modeling without external control. To incorporate nonlinear control
input channels in the modeling, adequate training data are necessary, which
could be generated by actively probing test signals then collecting the response
data for a long period of time. Such requirement is not practical as it is often
not allowed to subjectively inject large signals into safety-critical systems like
power grids, while probing low-level test signals may not sufficiently excite the
system to obtain informative data.
Reinforcement learning (RL), as another branch of machine learning that in-
herently bridges control and mainly rely on offline simulators, becomes popular
nowadays for the applications of data-driven control of power grids [2, 28, 29].
By interacting with the simulation environment to pursue Bellman optimality,
the RL can yield optimized control policy that can be deployed. The control op-
timality is theoretically sound while without performance guarantee due to the
existence of simulator uncertainty/modeling error and optimization error [57].
Combining RL with deep learning (i.e., deep reinforcement learning (DRL)) be-
comes popular to enhance mathematical optimality and generalization capacity
of learning, and shows the effectiveness in power grid applications such as op-
timal voltage control [58], frequency regulation [59], EV charging scheduling
[60], and battery management [61]. To improve the efficiency of deep learning
, DRL may be developed under parallel computing frameworks which has been
shown effective in autonomous voltage control and emergency control [62–64].
The details of pure machine learning-based identification and control methods,
including supervised and reinforcement learning, will be provided in Section 4.1.
Although pure machine learning methods stated in the last two paragraphs
are of full capacity to fit any nonlinearity, their practical applications in power
systems face pressing challenges such as physical consistency, interpretability,
generalization, safety, etc. Physics-informed rules and laws (physics-informed
6
loss function, constraints, initialization, architecture design, hybrid and en-
semble learning, etc.) become a growing consensus to mitigate the challenges
[65]. The fusion of RL frameworks and physics-inspired information is another
promising solution as it leverages the intrinsic connection between machine
learning and control theories. The fusion can be conducted through cater-
ing the physics-informed rules and laws of supervised/unsupervised learning
for RL in environment surrogate model, value function or policy design [27, 65–
67]. However, further improvements are necessary to enhance structural inter-
pretability and avoid physically inconsistent solutions, which could compromise
data-driven control performance or even destabilize power grids. Additionally,
machine learning based methods still face practical challenges in power grid ap-
plications, including data availability, training efficiency, scalability, and adap-
tiveness. [29]. The discussion about physics-informed machine learning-based
methods will be given in Section 4.2.
The unsolved issues further motivate recent research works on adaptive
Koopman operator control with online nonlinear identification [68, 69], aiming
to adaptively map nonlinear control to linear control that works for both small
and large signals. Specifically, the methods adopt small-signal linear space aug-
mented with more physically interpretable nonlinear bases and are adaptively
identified to address time-varying uncertainty. Although these methods are on-
line and can be applied without warm-up training, Koopman state space is still
determined empirically. Koopman generators that can estimate optimal and
physical-consistent Koopman operators (based on physical states and control
inputs) could be exploited with physics-informed learning and adequate offline
data. The details of Koopman-based methods will be provided in Section 4.3.
In the following, we will concentrate on elaborating existing data-driven con-
trol methodologies in linear systems (Section 3) and nonlinear systems (Section
4) used in power grids. The overview of grid applications with these methods
is provided in Section 5.
3. Linear System Identification and Control
In existing data-driven control methodologies for linear systems, identifica-
tion and control are typically regarded as two tasks that are done in a sequential
manner. Specifically, when the data of inputs and outputs (i.e., udand yd) are
available, the identification of a model of interest y=ˆ
y(x,u;θ) for the un-
known parameter θand state xcan be written in a general form of minimizing
a loss function Lid as below:
ˆ
θ=arg min
θ,xLid(yd,ˆ
y(x,ud;θ)) (1a)
x∈ X ,ˆ
y∈ Y (1b)
where Xand Yare the state constraint set and the output constraint set,
respectively. The control thereafter can be applied to the identified model with
ˆ
θ, which is equivalent to the optimization in a general form of
min
u,xLc(yr,ˆ
y(x,u;ˆ
θ),x) (2a)
7
u∈ U,x∈ X ,ˆ
y∈ Y (2b)
where yris the reference output desired by control; Uis the input constraint
set; Lcis the loss function representing the control objectives in general.
Alternatively, the identification and control can be done simultaneously and
directly with respect to the input uand output ywhile without the state x.
That is
min
θ,u,y{Lid(yd,ud,f(y,u;θ)) + αLc(yr,f(y , u;θ))}(3a)
u∈ U,ˆ
y∈ Y (3b)
where αis the coefficient to weight the identification and control objectives in
(3a). In short, the identification-control tasks in sequential and simultaneous
fashions correspond to the indirect and direct data-driven control in the control
community. Interested readers can refer to [70] for more theoretical details.
In the following, we will discuss the indirect and direct linear system data-
driven control methods in existing and potential power system applications,
respectively.
3.1. Sequential Linear Identification and Control
For sequential identification and control, the identification plays an impor-
tant role to realize effective modeling and thus control. In what follows, we
focus on linear identification techniques that can be categorized as state space
model-based and input-output model-based. The state space model is generally
more suitable for multi-variable system modeling as it deals with the individual
input/output variables in a vector space. Besides, intermediate states properly
selected can help define the inherent input-output relationship when compared
to the input-output “black-box” modeling based on transfer functions.
3.1.1. Linear System Identification Based on State Space Model
We consider a special form of the model representation ˆ
y(x,u;θ) in (1a)
xk+1 =Axk+Buk+δk(4a)
yk=Cxk+Duk+ek(4b)
which is termed as the state-space model and has been widely used to represent
power systems dynamics operating in ambient conditions. A,Band Care
the parameters corresponding to θ.xkare the states of power systems (e.g.,
phasor angle, rotor angle, frequency, voltage), ykare the observation outputs of
power systems (e.g., frequency, voltage, power), and ukare the control inputs
(e.g., the reference power of generators). δkand ekare the process noise and
the observation noise, respectively. The linear state space form is conducive
to estimation, filtering, prediction and control. When system parameters are
unknown, mature linear identification and control techniques can be used to
apply on the system (4a)-(4b).
Given the system model in (4a)-(4b), classical linear identification can be
applied to identify the parameters A,B, and C. Examples of classical lin-
ear system identification methods are subspace methods including CVA [71],
8
N4SID[49, 51, 71–76], MOESP [72, 77], OKID [78]. Generally, the data ma-
trices with Hankel structure play an important role in these subspace methods
because the signals (the input data, output data and the noises) in these al-
gorithms are organized in the form of Hankel matrices [79, 80]. Specifically,
let UP,YP,UFand YFrepresent the “past” input and output data, as well
as the “future” input and output data, respectively. [UP,UF]Tconstitutes the
Hankel matrix of the input u, and [YP,YF]Tconstitutes the Hankel matrix of
the output y. The Hankel matrices with the depth of Land trajectory length
Tare as follows:
UP
−
UF
=
u0u1... uT−L
u1u2... uT−L+1
.
.
..
.
.....
.
.
uL−1uL. . . uT−1
(5a)
YP
−
YF
=
y0y1... yT−L
y1y2... yT−L+1
.
.
..
.
.....
.
.
yL−1yL. . . yT−1
(5b)
The data matrices can be arranged according to (4a)-(4b) as:
YP=OL/2XP+HL/2UP(6a)
YF=OL/2XF+HL/2UF(6b)
where OL/2= [C,CA,...,C A L
2−1]Tis the extended observability matrix;
HL/2is the lower Toeplitz matrix defined as:
HL/2=
D0... ... 0
CB D ... ... 0
CAB CB D ... 0
.
.
..
.
.....
.
.
CA L
2−2B C A L
2−3B. . . CB D
(7)
and Xrepresents the sequence of the state vector with the length of T−L+ 1.
Then different subspace identification methods can be applied based on (6a)-
(6b), which typically involve two steps: (a) identification of Okand Hk; (b)
estimation of system parameter matrices (i.e., A,B,C) from the identified Ok
and Hk[72]. Detailed procedures and comprehensive comparisons of the sub-
space methods can be found in [79]. Besides, recursive stochastic subspace
methods are also applied in power grid damping mode estimation and control
to realize online adaptiveness [81–83].
Among the above-mentioned subspace methods, CVA [71, 84], N4SID [49,
51, 71–75], and MOESP [85] may have the bias problem in nature since they
work under the open-loop assumption (i.e., control inputs uis not correlated to
the process noise δand the observation noise e). In contrast, the OKID (ob-
server Kalman filter identification) and the variants are free of the bias problem
even in the closed-loop condition that the control inputs ucorrelate δand e. We
9
refer readers to [79] for a detailed review of subspace methods, where the general
procedure (including pre-estimation, regression or projection, model reduction
and parameter estimation) is summarized. The above-mentioned classical iden-
tification is usually conducted with ambient data as the linearization is based
on the assumption of small signals in ambient conditions. The efficacy of the
identification may degrade during transients.
Once the state-space model is identified, mature linear control techniques
can be applied. For example, different state-feedback control methods, such as
linear quadratic regulator (LQR) and root locus-based control design, can be
applied [49–51] for wide-area damping control. Model predictive control [85],
root locus control design[75], residual-based control [82], and PID control [74]
were also designed for damping control based on the identified models. The
control parameters can be designed with heuristic optimization such as particle
swarm optimization [74].
3.1.2. Linear System Identification Based on Input-Output Models
We consider another special form of the model representation, i.e., the trans-
fer function-based model, which directly describes the time series of input uand
output yas a “black-box” without the intermediate state xin (1a) and (2a). An
example is the autoregressive moving average with external inputs (ARMAX)
model:
A(q)yk=B(q)uk−h+C(q)ζk(8)
A(q) = 1 + a1q−1+a2q−2+· · · +anaq−na,B(q) = b0+b1q−1+b2q−2+· · · +
bn−1q−n+1 and C(q) = 1 + c1q−1+c2q−2+· · · +cncq−ncare the ARMAX
polynomials; qis the delay operator; the parameters na,nb, and ncare the
orders of the ARMAX model. his the time delay of the system and 1 ≤h <
nb. The term ζkdenotes the white-noise disturbance. Other types of transfer
function-based models, such as output-error models and Box-Jenkin models, are
illustrated in [86]. These models can be identified with prediction error methods,
which aim to find the system parameters that minimize the prediction error
[87, 88]. Fig. 1 shows a schematic structure of the prediction error method for a
general input-output model, which tunes the model parameter θto minimize the
loss function of prediction error L. Generally, the cost function can be defined
as L=fc(Qcov(ep,k )), where ep,k =yk−ˆykrepresents the prediction error; fc
is a scalar monotonically increasing function; Qcov =1
Nsample ΣNsample
k=1 (ep,keT
p,k)
denotes the sample covariance matrix.
For power system applications, the authors in [41] proposed an ARMAX-
based damping control. The ARMAX was used to capture dominant inter-area
modes of power systems identified in a least-squares fashion. Likewise, the au-
thors in [44] assumed that the LVDC (low voltage direct current) system was lin-
ear and can be interpreted as a linear difference system equivalent to (8), which
was identified with singular value decomposition. Another recently developed
transfer function-based identification method was proposed in [89] based on the
frequency response (FR) of multivariate continuous-time systems and convex
optimization. This model-free method was applied to control battery energy
10
Figure 1: The schematic structure of prediction error methods.
storage systems in islanded microgrids to reduce voltage and frequency fluctu-
ations [8]. The transfer function was identified in the spectral-analysis method
[46], which applied the discrete Fourier transform to the auto-correlations of
inputs and output data. The authors in [45] modeled the power system with
continuous-time transfer functions too, which were identified online with ring-
down data using a two-stage least-squares algorithm to realize the highest re-
gression accuracy indices in both time domain and frequency domain.
Once the transfer function model is identified, classical control techniques,
such as output-feedback control with lead-lag compensator [45] and networked
predictive control [90], are applied for wide-area damping control. Generally,
there is a trade-off between identification performance and robustness; a ro-
bust controller design is often favorable. For example, H∞robust control has
been applied in microgrid primary control [8]. A configuration information-free
controller design for linear difference systems was also applied for active load sta-
bilization in [44] based on a Lyapunov stability-oriented quadratic function and
linear matrix inequality techniques. Similar LMI-based control can be found
in [13], where some other goals such as power sharing and frequency/voltage
restoration were incorporated in the objective function jointly with the stability
requirements. In [48], an ARMAX model was identified whereby the estimate
of inertia was obtained and sent to the local-area model predictive controllers
to realize frequency regulation. Similar to 1), the input-output model is linear.
Therefore, the identification is suitable for small signals in ambient conditions
and usually conducted with ambient data. The efficacy of the identification may
degrade during transients.
Another formulation of multi-input multi-output (MIMO) model is proposed
in [91] based on a novel dynamic linearization technique. Specifically, the system
of interest is adaptively linearized as:
∆yk+1 =Gk∆uk(9)
with Gk=
G11,k G12,k ... G1p,k
G21,k G22,k ... G2p,k
.
.
..
.
..
.
..
.
.
Gp1,k Gp2,k ... Gpp,k
,∥Gk∥ ≤ b(10)
11
where bis a positive constant; Gkis the matrix of pseudo-partial derivative,
which can be identified with time-varying parameter estimation algorithms such
as modified projection [91], least squares with a forgetting factor [92], and the
leakage recursive least-squares method [93]. Such a model has been used in the
identification for microgrid primary control realizing a robust local voltage con-
trol that is not sensitive to the variations of the system parameter, structure,
and control delay [47]. The model has also been used for power system stabi-
lizers in [94] with improved dynamic performance for damping low-frequency
oscillations under various operating conditions. Because the linearization is
adaptive and the model can be identified in a rolling-based fashion, it can also
handle nonlinearity and can be conducted for both ambient and transient data.
However, the effectiveness based on transient data relies on the window length
of data and the incurred inertia, which may hinder a timely model update.
3.1.3. Linear Identification Based on Ornstein-Uhlenbeck Process
The aforementioned identification techniques estimate the system param-
eters with minimized error in a mathematical sense while the mathematical
solution is not unique. Therefore, the identified models may not reflect the real
system state variables and the estimated parameters are not guaranteed to be
the parameters of the true physical system, making the control design challeng-
ing. Leveraging the statistical properties of multivariate Ornstein-Uhlenbeck
(OU) process, the data-driven identification methods developed in [6, 15–17]
can estimate the system state matrix of the true physical model and extract
modal properties and essential parameters. Assuming that the aggregated load
powers experience Gaussian stochastic perturbations, the power system dynamic
model in ambient conditions can be represented as
˙x=Ax +BPξ(11)
where the states xcan be the deviations of generator rotor angle, generator rotor
speed in wide-area damping control [15–17], or be the deviations of bus voltage
magnitude and phase angle in wide-area voltage control [6] or load identifica-
tions [95]. ξare mutually independent standard Gaussian random variables,
representing the stochastic load variations; BPis the noise intensity matrix.
Thus, the state xin (11) follows a multivariate OU process. The parameter A
corresponds to the Jacobian matrix of the true physical model containing signif-
icant information like all oscillation modes, mode shapes, participation factors,
inertia and damping constants, dynamic load time constants, network topology
parameters, etc.
The system parameter Acan be analytically obtained by applying the regres-
sion theorem of OU process, which reflects the dynamical evolving of state co-
variances and relates the statistical properties of state variables with A. Specif-
ically,
d
dt G(τ) = AG(τ),with G(τ) =<[x(t+τ)−¯
x][x(t)−¯
x]T>(12)
Hence, Acan be obtained uniquely and analytically by solving (12) according
to [96]:
A=1
τlog[G(τ)G(0)−1] (13)
12
Equation (13) tactfully provides a way to identify the true physical model A
from the statistical properties of measurement data. Alternatively, Acan be
obtained based on the Lyapunov function that the stationary covariance matrix
G(0) of multivariate OU process satisfies [15]:
AG(0) +G(0)AT=−BBT(14)
if Bis known. It was shown that the true system parameters were approxi-
mately obtained by using both identification methods (according to (13) and
(14) respectively); then the identified model can be directly utilized in control
design. Note that the OU-based methods are conducted with ambient data to
identify the small-signal model of interest. Different from 3.1.1) and 3.1.2), this
type of method can help obtain a unique identification solution that corresponds
to the true physical small-signal model. Even though the model structure has
certain physical interpretation, the methods are in nature “data-driven”.
Once the system of interest is identified by the methods stated above, con-
ventional linear optimal or robust control methods can be applied. For exam-
ple, modal linear quadratic regulators [17] and pole placement [15, 16] have
been applied in wide-area damping control. Because the OU-based methods
can identify the true system model, the wide-area damping controls designed
based on the identified model can achieve full decoupling of modes and damp
all critical modes simultaneously without affecting the others. Optimal control
has also been applied in wide-area voltage control [6].
3.1.4. Loewner Method
Most of linear models and identification are based on a representation that
can describe the system dynamics in the whole frequency range (e.g., the state
space representation or transfer function input-output models). For a selective-
frequency range, Loewner interpolation was commonly used in black-box mod-
eling of large MIMO microwave structures that show the capability for rational
interpolation of frequency data [97, 98]. In 2015, a tangential interpolation
framework based on Loewner matrix pencil was first used in power systems for
modeling frequency-dependent network equivalents that can describe electro-
magnetic transients (EMT) [39, 99]. In 2019, the Loewner interpolation method
was used for the reduced-order model and identification for electro-mechanical
models of multi-machine power systems [40]. The method can realize more
accurate identification in a particular frequency range than other techniques.
However, the Loewner methods are only suitable for linear time-invariant sys-
tems (i.e., assuming small signals at specific operating conditions), which may
not hold for modern low-inertia power grids with increasing volatile renewables.
Likewise, the identification based on Loewner method for a selective-frequency
range was compared with the conventional eigensystem realization algorithm
for a New-England 10-machine test system in 2020. The results show the per-
formance enhancement in terms of the modal shape and the absolute error of
identified frequency responses [43]. After applying Loewner methods, the iden-
tified transfer functions for different frequency ranges corresponding to different
13
potential operating conditions then can be applied for robust controller design.
3.2. Direct Linear Identification and Control
Although the linear identification and control tasks are usually conducted
sequentially as aforementioned, effort has been made to realize the two tasks
simultaneously [70] to benefit the global optimality of joint identification and
control. For example, data-enabled predictive control (DeePC) with quadratic
regularization is investigated for local converter control [100]. It identifies the
system parameters and designs control signals together by solving a single opti-
mization problem of DeePC. In summary, the DeePC for the state space model
(4a)-(4b) can be written in a generic form[100, 101]:
min
g,σy,u,yΣN−1
k=0 ∥u∥2
R+∥y−yr∥2
Q+λy∥σy∥2
2(15a)
subject to [UP,YP,UF,YF]Tg= [uini ,yini +σy,u,y]T(15b)
u∈ U,ˆ
y∈ Y (15c)
where the operator ∥a∥2
Xdenotes the quadratic form aTXa.λyis the regular-
ization parameter. σyis an auxiliary slack variable to guarantee the feasibility
of solving the optimization problem. Nis the prediction horizon. uini and yini
are the most recent input and output trajectories of length Tini.
The DeePC can be seen as a special form of (3a)-(3b), with gcorresponding
to the parameter θ. The DeePC solves the convex optimization problem (15a)-
(15c) in a receding horizon manner with a finite number of data samples to
predict future trajectories. See details in [101]. The DeePC has been applied in
modern power grids, such as local converter control [100], decentralized damping
control [102], and frequency regulation [103]. The DeePC can also be used to
guide the training phase of RL with improved sample efficiency, It was applied
to load frequency control and the control performance was improved [104]. An-
other example of direct data-driven control is a multivariable linear parameter
varying (LPV) controller applied in islanded microgrid secondary-primary con-
trol, resulting in damped oscillations in frequency and power dynamics [105].
Different from DeePC, the synthesis process is based on the frequency domain
rather than the time domain.
It has been shown in the IEEE task force paper [35] that linearized models
around the operating points can yield an approximation behaving at a reason-
able level of accuracy using transient data and ambient data, respectively. Thus,
the linear identification and control methods stated above, either indirect or di-
rect, are still useful and credible in applications like electromechanical modes
identification, dynamic load modeling, and wide-area damping control. Also,
linear methods are advantageous in two respects: (i) their parameters can be
identified optimally with the mature theory foundations of linear systems; (ii)
it is easy to realize adaptiveness due to the parameter identification efficiency
in linear models.
Nonetheless, linear system identification and control may still face challenges
in power system applications: (i) Modern power grids, with the integration
14
of increasing renewable energy sources, exhibit low inertia and fast-changing
operating conditions subject to large disturbances, making the linear models
inaccurate. (ii) The saturation of control signals and the diverse interactions
between controllers may introduce nonlinearity and uncertainty, compromising
identification accuracy and can even result in ineffective modeling [33].
4. Nonlinear System Identification and Control
To address the aforementioned complexity, uncertainty and nonlinearity of
modern power grids, nonlinear data-driven control methods gain increasing at-
tention. They are categorized as: (A) pure machine learning-based methods;
(B) physics-informed machine learning; and (C) Koopman-based methods. Gen-
erally, the category (A) uses “black-box” machine learning to model nonlinear
system dynamics and control. The category (B) intends to integrate physical in-
formation into the machine learning training or model structure design to realize
more reliable modeling. The category (C) considers the Koopman operators to
map nonlinear dynamics to linear state spaces, whereby mature linear control
methods can be readily applied. The three categories are elaborated below.
4.1. Pure Machine Learning-Based Methods
Generally, machine learning includes reinforcement, supervised and unsuper-
vised learning. Reinforcement learning, as a branch of machine learning that can
bridge control, becomes popular nowadays for the applications of data-driven
control of power grids [2, 28, 29]. The learning process generally depends on
offline simulators to interact with rather than probing data to real systems to
generate training data [2]. However, the RL training is often time-consuming,
and the control performance rests on the simulator accuracy and the optimiza-
tion error associated with highly nonlinear models.
Supervised learning can also be used for data-driven modeling for control.
However, incorporating control input data in the supervised learning-based
model, which is then used for control, is not popular because of data issues
associated with control input channels. It is not feasible to probe large testing
signals due to the practical operation requirements (especially for safety-critical
systems like power grids). Even with low-level test signals, the closed control
loops in power grids could trap the state-and-control pairs and make them ill-
distributed [106], thus lacking rich information for universal learning machines
such as NNs to generalize the impact from and to control input channels. In
a sense, supervised learning is more often seen in power grid modeling without
external control inputs rather than grid control applications. Details of the RL
and supervised learning-based methods are provided in what follows.
Reinforcement learning (RL). RL is a branch of machine learning re-
garding how to learn control policies by interacting with the environment to
explore and exploit inherent model structure according to Bellman optimality
[57, 107]. In RL, the system states are assumed to evolve according to the
Markov decision process (MDP), i.e., the probability of the system occurring
15
Figure 2: The schematic framework of RL.
in the current state is determined only by the previous state [57]. Specifically,
an MDP consists of a set of states s∈ S, actions a∈ A, rewards r∈ R, the
transition probability P r(s′, s, a) = P rob(sk+1 =s′|sk=s, ak=a), and the
policy function π(s, a) = P rob(ak+1 =a|sk+1 =s). Thus, the state transition
under the policy πis
P r(s′, s, π) = Σa∈A {π(s, a)P r(s′, s, a)}(16)
The above MDP notation generalizes a Markov process to incorporate ac-
tions and rewards for RL, which can be used to describe any nonlinear system
decision-making/control of interest. Fig. 2 shows the generic scheme of RL
agent that learns the system through interaction with the environment (i.e.,
offline simulators), and then the learned agents are deployed to the real envi-
ronment for online operation. Note that the RL agent can be either based on
universal surrogate models (e.g., actor-critic with deep NNs, approximate dy-
namic programming, deep Q-learning, etc.), or uses purely model-free paradigms
(e.g., Q-learning, SARSA, policy gradient, etc) [57, 107].
In practical applications of power grids, Q-learning is among the most popu-
lar RL techniques [108–111]. However, Q-learning is not inherently suitable for
continuous action spaces [107]. In addition, the action and reward functions for
RL are often unknown or of high uncertainty. Therefore, actor-critic methods
gained attention [14, 59] and are favorable because of the iterative exploration
and exploitation of the “actor” and “critic”. That being said, the action and
reward functions are approximated with two separate learning machines (e.g.,
various NNs, and support vector machines), which are trained with abundant
data. In a broader sense, such “actor-critic” can be seen as a direct data-driven
control paradigm equivalent to (3a)-(3b) that identifies the system and gener-
ate control actions directly. However, the universal learning machines under the
general RL framework often suffer from training efficacy and efficiency prob-
lems due to the unconstrained or loosely-constrained learning space: (a) the
16
Figure 3: The schematic framework of supervised learning based modeling for control with a
nonlinear learning machine and a linear model.
trained models may not well describe the underlying physical process well due
to modeling error and training error; (b) the computational cost of training is
high. The former issue can be mitigated by physics-informed techniques and
Koopman-based methods that will be discussed in Sections 4.2-4.3. While the
latter can be naturally lifted by parallel computing when applicable. For ex-
ample, a wide variety of scenarios in power grids can be created in simulation
to train DRL. For example, the authors in [62] parallelized training tasks based
on environment study cases (e.g., fault scenarios in power grids). Furthermore,
they proposed a two-layer parallel scheme [64] that supports task parallelism
based on environment and learning parallelism based on a parallel augment
random search algorithm, which was successfully applied in derivative-free DRL
for emergency voltage control. It can realize well-structured and more effec-
tive parameter exploration (larger learning rates and fewer hyper-parameters to
tune) than conventional action-space exploration in DRL, improving the com-
putational scalability and accelerating the training (e.g., training time reduced
by 136 times on the IEEE 300-bus system). Likewise, a parallel framework that
employs multiple workers simultaneously interacting with power grid simula-
tors was adopted for autonomous voltage control in [63], which shows improved
training efficiency and stability.
As RL is a branch of mature machine learning techniques which have been
well illustrated in many other works in the field of power systems, such as decen-
tralized resilient secondary control [14, 108], microgrid frequency regulation[59,
108], microgrid power dispatch [112], reactive power control [58], battery man-
agement [60, 61, 113], multi-area AGC [114, 115], and wide-area damping control
[116], they will not be detailed in this paper. Interested readers are referred to
[2, 28, 29] for comprehensive reviews of RL in power system/microgrid applica-
tions.
Supervised Learning-Based Method . Universal machine learning mod-
els, such as neural networks (NNs), can be used in power grids to model nonlin-
ear dynamics [117]. For example, the authors in [7] used a linear dynamic model
ARMAX to describe the microgrid voltage dynamics first and then trained NNs
17
Figure 4: The schematic framework of supervised learning-based modeling for control with
nonlinear learning machines only.
to compensate for the residual nonlinear dynamics on top of the linear model,
whereby optimal control is applied to realize stabler and faster microgrid volt-
age control. Fig. 3 shows the schematic framework of supervised learning-based
modeling for control with pure ML, where the learning machine is trained of-
fline based on a user-defined cost function L. Note that the linear model (e.g.,
ARMAX) may keep updating by online identification to realize fast adaptive-
ness. The ML model parameters may also be finely tuned online around the
offline-trained model to realize a certain level of adaptiveness. This framework
can be seen as a special form of the sequential data-driven control (1a)-(2b). It
has been shown that the well-trained machine learning model can capture power
system nonlinearity with supervised learning given abundant data [7, 118]. Be-
sides the applications in power grids, the NN-based control is also used to model
and control residual nonlinear dynamics of drone landing [119], where spectral
normalization is used to stabilize NN training and to improve generalization
capacity. Conventional loss functions for machine learning include root-mean-
squared error, mean absolute error, cross-entropy, KL-divergence, etc, which can
be used for different training algorithms such as Levenberg-Marquardt (LM) al-
gorithm, stochastic gradient descent (with momentum) (SGD or SGDM), and
adaptive moment estimation (ADAM) [57, 107].
The learning machines can also be used to model the uncertainty of systems
dynamics and then be directly used to regulate control signals as shown in Fig.
4. The offline-trained ML models may also be finely tuned with new online
data to realize certain level of adaptiveness. Such an idea of offline pre-training
and online-tuning matches the concept of transfer learning in machine learning
community, which is a learning paradigm aiming to transfer the knowledge
generalized from the data of existing systems to the learning of new similar
systems by offline training and the learning machine parameters are fine-tuned
when new data come [120]. This paradigm has been used in [10, 118, 121], where
online-tuned NNs are used for identification and control to realize accurate
power sharing and enhanced stability, respectively. The online tuning of NNs in
[10, 118] can also be seen as a special form of the direct data-driven control (3a)-
(3b), as the NNs are trained to model to obtain optimized parameters and to
generate control signals simultaneously. NN-based control has also been used in
18
a standalone DC microgrid where the NNs are trained offline to directly output
desired control signals for individual converters of distributed energy resources
to enhance microgrid voltage stability [122]. The optimal Koopman operators
and the identification need further investigation.
Pure machine learning methods can be conducted with both ambient and
transient data. However, the data should be informative enough to guaran-
tee the generalization of learning machines. Besides, pure machine learning
methods resolve the control policies as “black-boxes” with no explicit physical
interpretation. Therefore, they are often analytically and computationally less
tractable and less practical for real-time and safety-critical scenarios in power
grids. Some physics-informed learning methods have been proposed to address
the issues, which are presented as follows.
4.2. Physics-Informed Machine Learning Methods
The physics-informed machine learning can inform the learning machine of
certain physics either through the training, the model structure design, or both.
Thus, they can enhance the interpretability or explainability with more physics-
consistent solutions. In what follows, we discuss physics-informed methods from
the two perspectives. Interested readers are also referred to [65] for more detailed
information regarding physics-informed NNs that have been used in power grids.
4.2.1. Physics-Informed Training
In the training phase of machine learning, different regularization and relax-
ation techniques[70, 123] can be incorporated to avoid over-fitting and increase
physics interpretability. For example, different norms can be added in the reg-
ularization terms such that the system model complexity can be constrained
under certain mathematical and physical interpretations (e.g., stability or other
specific physical constraints). In general, the identification loss function with
the regularization terms can be written as
Lid =L(yd,ˆyd) + γLP(θ) + λLP hy(x,u,ˆy)(17)
where Lis the conventional loss function that measures the distance between
the predicted output and the target output. γand λare coefficients to weigh
the regularization terms. LPis the parametric regularization term (e.g., L1,
L2, Tikhonov) to constrain the model complexity by regularizing the model
parameters θ.LP hy is the physics-informed regularization term to add extra
physics-driven constraints. Alternatively, the physics-informed information can
be added as the inequality and equality constraints CPhy IEQ and CP hyEQ as
follows:
L=L(yd,ˆyd) + γLP(θ) (18a)
CP hyIE Q(x, u, ˆy)≤ζ(18b)
CP hyEQ (x, u, ˆy) = η(18c)
where ζand ηare arbitrary values to constrain the physics information.
The authors in [123] apply an adapted deep RL method with the training
that is based on (18) in wide-area damping control. The framework is called
bounded exploratory control-based DDPG (deep deterministic policy gradient),
19
consisting of NNs and polytopic controllers designed with the help of linear
matrix inequality (LMI)-based mixed H2/H∞optimization. With the help of
H2/H∞-oriented stability physical constraints, the method is demonstrated ef-
fective for both low-stressed and high-stressed networks. Universal learning
machines such as NN can be informed of physical ingredients in training. The
authors in [124] use NN to realize a fast dynamic state and parameter estimation
tool to assess power system stability. The cost function for NN training is in the
form of (17), consisting of a physics-informed regularization term based on the
swing equation with respect to the NN output. Similarly, the physics-informed
NNs are used in [125] to capture the dynamics of power systems, with a group of
state constraints and physics-informed differential and algebraic equation con-
straints incorporated in the NN architecture and loss functions equivalent to the
form of (18). The authors in [126] apply NN for high impedance fault detection
in power grids. To obtain more reasonable NN parameters, an elliptical regu-
larization term is added in the form of (17) to incorporate the physics-regulated
elliptical characteristics of voltages and currents into the cost function for train-
ing. In summary, these physics-informed learning-based NNs show enhanced
modeling efficacy and thus potentially beneficial for data-driven control on top
of these NNs.
4.2.2. Physics-Informed Model Structure Design
The physics-informed training discussed in Section B intends to incorporate
physical system information in the cost function L, while the physical informa-
tion can also be integrated into model structure design, i.e., “ML model” in
Fig. 3-4. In fact, proper modeling with the physics-informed structure design
can nail down the learning space within reasonable areas, which could make the
modeling more reliable and the learning faster. Pure machine learning models
as shown in Fig. 3 and Fig. 4 can be incorporated with certain levels of physics
interpretations in the structure. For example, the authors in [30] proposed a
physics-informed NN tailored by automatic differentiation [127] of the original
NN, whereby power system physical laws (e.g., underlying swing equations) are
incorporated with the bounded space of admissible solutions to the NN parame-
ters [30]. It has been shown that the physics-informed neural nets can effectively
describe the rotor angle and frequency for uncertain power inputs and identify
system inertia and damping parameters. Such design also provides opportuni-
ties to lower the requirement on the sizes of NNs and the training data set. The
physics-informed NN [128] that incorporates the regularization of power flow
equations is proposed for power system state estimation under partial observ-
ability, achieving more accurate voltage-phasors estimation than conventional
weighted least squares-based estimation. The authors in [129] proposed a hy-
brid learning structure by incorporating the physical AC power flow model into
the deep learning with autoencoders. The authors in [130] use physics-informed
NNs to accurately estimate the optimal power flow (OPF) with rigorous per-
formance guarantee. The physical interpretation is further enhanced by adding
the disparities of AC-OPF Karush–Kuhn–Tucker (KKT) condition to the NN
training loss function. Additionally, the authors in [131] use physics-informed
20
NN with an implicit Runge-Kutta integration [132] at the local converter level,
whereby deep NNs and converter physical models are seamlessly coupled and
reliable parameter estimation of power converters are achieved.
4.3. Koopman-Based Methods
As aforementioned in Section 2, data availability, training efficiency, inter-
pretability, adaptiveness, and scalability issues further motivate recent research
works on adaptive Koopman operator control [68, 69], aiming to adaptively map
nonlinear control to linear control that works for both small and large signals.
Koopman operator theory [133] shows that a nonlinear dynamical system can
be transformed into an infinite-dimensional linear system under a Koopman
embedding mapping. The Koopman-enabled linear model structure is a way
to interpret nonlinear dynamics, and is valid for global nonlinearity with the
infinite-dimensional representation as opposed to traditional locally linearized
small-signal models. Practically, one can consider finite-dimensional Koopman
invariant subspaces where dominant dynamics can be described. In Koopman-
based data-driven control, the physics interpretation on power grids of interest
can be incorporated through the model structure design. Particularly, given
a nonlinear dynamical system with external control xk+1 =F(xk,uk), where
x∈ M and u∈ U with Mand Ubeing the manifolds of state and con-
trol input, we consider the Koopman embedding mapping Φfrom the two
manifolds to a new Hilbert space Φ:M × U → H, which lies within the
span of the eigenfunctions φj. That is, Φ(x, u)=PNφ
j=1 φj(x, u)vj, where
Φ(x,u) = [Φ1(x, u),Φ2(x, u),...,Φi(x, u),...,Φp(x, u)]Tis a set of Koop-
man observables, vjare the vector-valued coefficients called Koopman modes.
The Koopman operator K, acting on the span of φj, advances the embeddings
Φ(x, u)linearly in the Hilbert space Has [133]:
Φ(xk+1,uk+1 ) = KΦ(xk,uk)
=K
Nφ
X
j=1
φj(x,u)vj=
Nφ
X
j=1
(ρjφj(xk,uk)vj)(19)
where ρjare the eigenvalues satisfying Kφj(x, u) = ρjφj(x, u). In (19), one
can assume that Φi(x, u) = gi(x) + li(u) where gi(x) is a nonlinear observ-
able function and li(u) is linear with li(0) = 0 [134]. In addition, we assume
Φi(xk+1,0) = KΦi(xk,uk) for all k. Then, gi(xk+1) + li(0) = Kgi(xk) +
Kli(uk)⇒gi(xk+1) = Kgi(xk) + Kli(uk). This assumption means that the
Koopman operator is only attempting to propagate the observable functions at
the current state xkand inputs ukto the future observable functions on the
state xk+1 but not on future inputs uk+1 [133]. Let us define z:= g(x) =
[g1(x), g2(x), . . . , gi(x),...gp(x)]T. Then we have an approximation of (19) in
a form of extended dynamic mode decomposition with control (EDMDc) [134]
as below
(P rocess model)zk+1 =Azk+Buk+δk(20a)
(Observation model)yk=Czk+ek(20b)
where ykare the outputs of the Koopman state space model. The Koopman ob-
servables in (20a)-(20b) can integrate power system physical domain knowledge
21
[11, 68, 107, 135, 136]. For example, the authors in [11] selected the Koopman
observables with osillotary terms (sinθand cos θ) to address sinusoidal-driven
interaction dynamics that emerge when subject to large perturbations and low
inertia. These trigonometric terms are physics-informed ingredients because
the general solution for differential power flow equations contains trigonomet-
ric patterns. Likewise, the authors in [68, 135] also include the functions sin θ
and cos θinto the Koopman embedding to describe underlying power flow in-
teraction dynamics. Besides the interpretability of dynamics, another advan-
tage of Koopman-based methods is the linearity enabled by Koopman oper-
ators and therefore requires no offline learning, enabling online adaptiveness.
Learning-based methods using deep auto-encoder, deep NNs, automated dic-
tionary learning [137–141] are also employed to help determine Koopman state
spaces, whereas the optimal discovery of Koopman embedding mapping remains
an open question due to complex nonlinear systems of high dimensions and un-
certainty. Given the Koopman model state space, the model parameters can
then be estimated online by least-squares regression [11, 134, 136], iterative
learning [68], enhanced OKID [69], etc.
Sparse identification of nonlinear dynamics (SINDy) is another type of method
for the data-driven discovery of dynamical systems characterized by sparse dom-
inant features. We classify SINDy as a type of Koopman-based method as it
has been proved in [142] that SINDy is a special case of EDMD. As dynami-
cal systems can be often described by a few governing equations, SINDy can
be extended to discover leading Koopman eigenfunctions whereby realizing the
control on the discovered spans [143, 144].
˙
X=ΞΘT(X,U) (21)
where Xand Uare the state data matrix and control input data matrix, respec-
tively. Ξis the coefficient matrix to be identified with sparsity. Θis a library
of candidate nonlinear functions that compose the potential feature space to be
identified. The model identified with SINDy is similar to (20a) in the sense that
both describe the nonlinear dynamical evolving in a linear fashion on the span
of a few nonlinear features, which enables the use of mature linear control with
well-characterized stability. The proper selection of the feature candidates can
inform the model of physics. For example, the authors in [145] use SINDy to
locate forced oscillation sources. Based on the physical formulation of stochas-
tic power system dynamic model [145], the power system dynamics can be seen
sparse in the feature space of zero-degree polynomial bias, linear functions and
trigonometric functions.
The Koopman-based methods have been investigated in many previous works
in power systems, such as power system nonlinearity modeling, stability assess-
ment, and forced oscillation location [135, 141, 145]. Although the external
control inputs are yet to be incorporated in these applications, it is natural
that the power system control can be conducted based on the Koopman state
space with mature linear control algorithms (e.g., optimal control, predictive
control, or robust control). That being said, the nonlinear power system con-
trol can be converted to linear control in the lifted Koopman state space. For
22
example, the LQR and model predictive control techniques have been applied
on the Koopman model (20a)-(20b) to realize microgrid voltage and frequency
secondary control, transient frequency control [11, 68]. The authors in [146]
proposed deep Koopman model predictive control for improving transient sta-
bility and frequency regulation. Particularly, a deep NN was used to obtain
more effective observable functions for Koopman state space with the training
according to a Koopman-oriented cost function. Likewise, deep NNs were used
in [147] to learn a Koopman operator model in the presence of price spoofing
in the context of market-based frequency regulation. The learned Koopman
model was used to realize a robust data-driven control against price proofing
with Lyapunov-based LMI constraints to guarantee asymptotic stability, which
enhanced cyber security.
In summary, emerging Koopman-based methods are promising due to: (i)
Dealing with nonlinear problems in a linear and dynamics-interpretable fash-
ion, leading to faster and optimal system identification with tractable identifi-
cation process and well-characterized stability [68]. (ii) Generating fixed states
(i.e., Koopman observables) from measured states that have explicit physical
meaning, allowing for easy extension of conventional model predictive control
frameworks to the Koopman state space [11, 135]. (iii) Separating the determi-
nation of the Koopman state space from parameter estimation, enabling online
identification without warm-up training and facilitating fast adaptive control.
5. Overview and Future Trends
In summary, the data-driven control methods discussed in Sections 3-4 and
their applications in power grids are presented in Table 2. Figure 5 provides
an overview of the methodology categories for data-driven identification and
control. These methods offer various advantages and disadvantages, and are
inherently suitable for different grid applications.
5.1. Overview of Existing and Potential Power Grid Applications
The data-driven methods are generic and can be tailored for various grid
applications. According to the existing work listed in Table 2, we will dis-
cuss selective application scenarios of data-driven control. They are oscillation
damping control, microgrid control, and aggregated load/EV control.
Oscillation damping control: The first representative application in power
grids is the damping control of low-frequency oscillations in either decentralized
or centralized manners [15, 42]. As the small signal assumption holds in this ap-
plication scenario, linear models are generally expressive enough to describe the
oscillation modes. Diverse local devices, global network topology and param-
eters, as well as time-varying operating conditions nowadays make power grid
oscillatory dynamics suffer from higher levels of uncertainty, compromising the
performance of conventional model-based controllers. Fortunately, such uncer-
tainty can be compensated by data-driven controllers in either a decentralized or
23
Table 2
Summary on Data-Driven Control Methods and Applications in Power Grids
centralized fashion. Examples of decentralized adaptive linear data-driven meth-
ods are with linear input-output transfer function model [8]), ARMA [42, 148]
and DeePC [102, 149]. With the development of wide-area measurement infras-
tructure, centralized wide-area damping control can be applied to enhance the
modeling and thus damping of systemwide interactive oscillation. For example,
state space model with subspace identification [49, 51, 85], ARMAX [41], as well
as Ornsein-Uhlenbeck process [6, 15, 17] with online parameter estimation are
used for damping control with wide-area phasor measurement units. Machine
learning such as RL is also used in wide-area damping control to adaptively learn
how to address nonlinearity and uncertainty [116]. However, the performance
heavily depends on its offline simulator and the adaptation is slow. Consider-
ing the simulator fidelity, offline training efficiency and optimality issues, the
application purely with RL is not as practical as adaptive linear methods.
24
Figure 5: Methodology category for data-driven identification and control.
Microgrid control: Microgrid control plays a vital role in frequency and
voltage regulation/restoration, as well as systemwide power and energy man-
agement. Roughly speaking, the dynamics in microgrids span from hundreds
of milliseconds to minutes, and the data-driven controller can sample measure-
ments in the range of tens of milliseconds to seconds. Different from traditional
power grids, microgrids are characterized by low inertia, coupled states, and
wide-frequency dynamics response and are susceptible to high uncertainty and
nonlinearity when confronting large disturbances. To address these challenges,
nonlinear learning-based methods, such as RL [14, 58, 59, 108], NNs [7, 10, 122]
and Koopman-based methods [68, 69] have been employed.
Load/EV control for demand response: Another suitable application
scenario of data-driven control in power grids are load and EV control for de-
mand response. The root causes of complicated power consumption dynamics
are the combinations of device physical dynamics (e.g., thermal dynamics for
thermostatic loads), end-user behavioral dynamics (population dynamics of ag-
gregated load and EVs) as well as the electricity market dynamics. Usually,
the end-user information may not be accessed or accurately sensed, introducing
uncertainty. The timescale of the aggregation dynamics typically ranges from
minutes to hours or even days. The measurements could be sampled by the
data-driven controller at the order of minutes from smart meters. From the
technical control perspective, a reduced-order state space model is well-suited
to efficiently describe the dynamic state evolving of a large population of load or
EVs in a linear fashion. This allows for the predictive controller design to realize]
full responsiveness to control requests from system operators, while considering
the constraints imposed by the normal end use of individual loads/EVs [22].
Online identification on top of the reduced order model is favorable to adapt to
the changes associated with weather, financial or social factors that may affect
end users. From the commercial operation perspective (e.g., economic dispatch,
25
power tracking quality, grid service provision), nonlinear methods such as RL
[21, 61, 113, 150] can directly address the nonlinearity of electricity market dy-
namics and end-user behaviors. Usually, stable operation over the timescale of
hours is the prerequisite to investigate optimal scheduling and operation. There-
fore, the application of machine learning is practical in this context, as training
efficiency is usually not the primary concern.
5.2. Practicality Overview of Data-Driven Control Methods
The data-driven identification and control can be conducted based on three
types of measurement data (ambient, transient, and probing data) [41]. Among
them, the ambient data and transient data are suitable for online applications
when subject to ambient conditions and large disturbances, respectively. The
probing data means actively injecting a probing signal into the system. The
signal needs to consistently excite the system to obtain informative datasets,
and the probing can deteriorate the electricity quality of power grids.
The data availability, granularity and quantity highly depend on the grid
application scenarios and the methods adopted. Use the three selective grid
applications discussed in Section 5.1 as examples. They usually adopt different
identification and control methods and thus have different data requirements.
Linear models are well-suited for oscillation damping control, while nonlinear
methods are gaining popularity for microgrid control and demand response.
Linear models typically require a relatively small volume of data. For example,
in wide-area damping control, accurate identification of linear models can be
achieved with online data consisting of a window length of 180-300 seconds of
ambient data [15, 41] or 10-second ring-down data [41] sampled at a frequency
of 30Hz from wide-area synchrophasor measurements. On the other hand, the
nonlinear methods usually need to train a model offline with large volumes of
data (e.g., tens of thousands of examples or more) to obtain a deterministic
warm-up reference, which then can be fine-tuned online with a small piece of
data to adapt to varying uncertainty. Koopman-based methods have the poten-
tial to use a small volume of data that are comparable to linear models without
the need of warm-up training, thus having the potential to realize faster and
more accurate adaptation for grid nonlinear dynamics provided that the Koop-
man state space (approximating the Koopman embedding mapping) is properly
predetermined.
The quality requirements of data are mainly determined by the preprocessing
techniques and the data-driven identification and control methods employed.
Even within the same method category, different algorithms for preprocessing,
identification, and control may have varying data requirements based on the
application scenario. Thus, it is challenging to establish exact requirements that
are universally applicable. Data preprocessing can involve proper filtering and
interpolation to improve data quality. Stochastic optimal identification (e.g.,
OU process regression theorem, Kalman filtering) and robust control design
(e.g., H∞, LMI) can also be incorporated to reduce the requirements on data
quality. Table 3 provides a brief overview of commonly used models and their
corresponding data requirements for three selected grid application scenarios.
26
Table 3
Summary on Dominant Modeling Methods and Data Requirement
for Selective Grid Applications
5.3. Online Identification for Different Categories of Data-Driven Control
Online adaptiveness is a desirable feature in power systems due to the time-
varying nonlinearity and complexity they exhibit. By utilizing a small amount
of online data, online identification effectively addresses uncertainty and miti-
gates modeling issues. Online identification can be integrated into all categories
of data-driven control methods, though it may be inherently easier in some
categories compared to others.
For linear data-driven identification and control. Generally, most of
the linear data-driven identification and control methods discussed in Section
3 are more computationally efficient than nonlinear methods, thus they can
realize online identification and control by using rolling windows, so long as the
identification at each time step can be completed within a time interval specified
for the grid application of interest. For example, the rolling window-based online
identification has been used to online identify OU process models for wide-area
voltage control [6] and dynamic load modeling [95]. In some cases, recursive
formulations can further improve the algorithms’ efficiency. For example, the
recursive or stochastic subspace methods have been used to online identify state
space model for adaptive damping control [82, 83] and electromechanical mode
estimation [81]. The recursive least-squares method has been used to online
identify linear ARMAX models for adaptive control of a converter in a grid-tied
microgrid [148] and of power system stabilizer [42]. The recursive estimations of
some matrix quantities [96] have been used to online identify OU process linear
models for the estimation of dynamic system state matrix of multi-machine
power grids.
For machine learning-based nonlinear identification. The idea of pre-
training and online tuning was explored for power system analysis in nineties
[54]. In 1999, adaptive neuro-identifier and -controller were proposed in [121] for
power system stabilizer in a multi-machine power system. These approaches in-
volved obtaining a pre-trained machine learning model through offline warm-up
training, which served as a deterministic baseline for subsequent online tuning.
This baseline helped reduce output oscillation [121]. Nonetheless, the reliability
of adaptive machine learning-based methods is limited due to the lack of phys-
ical interpretability and the biased information from online data. Deserved to
be mentioned, such an offline-training online-tuning paradigm aligns with the
concept of transferable features and learning in the machine learning commu-
nity, which aims to transfer the knowledge learned from existing system data to
new similar systems through offline training, with fine-tuning of learning ma-
27
chine parameters when new data is available [120]. The development of transfer
learning techniques nowadays [151] may provide additional opportunities for
researchers in power sectors to borrow up-to-date ideas from machine learning
community to realize more time-efficient and reliable adaptive learning for non-
linear data-driven control, in terms of trade-offing present and past data in a
rolling and more intelligent fashion.
For Koopman-based methods. To our best knowledge, Koopman-based
models provide an opportunity to realize online nonlinear identification. They
are well suited for online identification by nature due to the linearity after
Koopman embedding mapping. They are particularly favorable when obtaining
a “warm-up” model is challenging due to limited on-field data volume and run-
time requirements. For example, Koopman-based microgrid secondary voltage
and frequency control [68, 69] were successfully implemented without warm-up
training, even when the microgrid configuration and control parameters were
unknown. However, the determination of the Koopman state space still relies
on empirical methods. An alternative approach is to pre-learn Koopman gen-
erators using physics-informed machine learning and offline data in power grid
control. This allows for estimating optimal and physically consistent Koopman
operators based on physical states and control inputs. Subsequently, the pa-
rameters of the Koopman state space model can be identified online in a linear
identification manner.
5.4. Future Trend
Modern power systems have increasingly high uncertainty and nonlinear-
ity due to the increasing penetration of inverter-based resources, leading to
more complex system dynamics within wider frequency bands than conven-
tional power grids. As a result, modeling efficacy tends to be more problematic.
Conventional machine learning-based methods may not be able to realize satis-
factory control performance efficiently and safely due to increasingly high com-
plexity of power grids and the “black-box” nature of machine learning. Emerging
data-driven control methods could better address uncertainty and nonlinearity,
while being interpretable or having certain physics-informed performance guar-
antees to enable their applications in real-world power systems. In what follows,
we will briefly discuss the trend of each method category.
5.4.1. Linear Data-Driven Identification and Control
Linear data-driven identification and control are still trending in certain
power grid applications when the introduced model uncertainty is reasonably
bounded (i.e., the small signal assumption roughly holds). A typical example
is oscillation damping control as discussed in Section 5.1. However, with the
increasingly high penetration of volatile renewables and power electronics de-
vices, the nonlinearity makes the effectiveness of linear models compromised.
Nevertheless, adaptive linear control with online identification has the potential
to address this issue to some extent by compensating for time-varying model
uncertainty owing to the high learning efficiency and fewer data requirements
28
of linear models, while needing further investigation to consolidate in practical
applications.
Data-enabled predictive control (DeePC), a direct linear control method, is
also emerging in control community and deserves attention for power grid appli-
cations. It is an efficient direct data-driven control method based on behavioral
systems theory to learn a non-parametric system model that synthesizes the op-
timality of both identification and control[101]. The direct equivalence between
DeePC and subspace predictive control has been demonstrated [152]. Adding
quadratic regularization terms with DeePC leads to more robust identification
against data noise [153]. Although DeePC is still in the early stage, the lin-
ear nature is of potential to combine with online learning to adaptively address
time-varying uncertainty and adopt Koopman operator-based structures to ad-
dress nonlinearity. Also, direct optimal identification and control with a small
piece of data subject to disturbances and noises may be further developed.
5.4.2. Machine Learning-Based Methods
It is vital in real-world applications to ensure safe operation after apply-
ing any designed controllers. Generally speaking, most of the existing machine
learning methods are based on universal approximators but the learning space
for parametric training is too broad. Therefore, training of these universal
learning machines is often mathematically (sub)optimal for training data while
not being reasonable/generalizable for unforeseen cases in real-world applica-
tions. This tends to cause over-fitting and compromise the modeling reliability
for safety-critical power grids. Among different machine learning-based meth-
ods discussed in Section 4, RL-based methods are the most popular nowadays
and seem trending as they are where machine learning meets the feedback con-
trol. However, the time-consuming training (e.g., tens of thousands of iterations
to converge [2]) compromises the fast deployment and adaption to time-varying
operating conditions and grid topologies. From the cyberinfrastructure perspec-
tive, their practical applications rely on data platforms with sufficient comput-
ing resources, data repositories, communication and appropriate technologies to
improve training efficiency. From the power grid operation perspective, most
of RLs are “black-box” without enough physical interpretability and are not
tractable to facilitate the incorporation of physics-inspired information. The
safety and scalability of the power grid applications are problematic too.
To further enhance the RL reliability for increasingly complex power grids,
safe exploration methods in RL can be considered to bound the parametric
learning within a safety region [154]. Another possible way is to add safety
constraints at the learned control policy in RL by either incorporating con-
straints [155] or adding a heuristic safety layer to adjust the control actions [2].
In addition to directly adding the safety constraints/layers in “black-box” RL
framework, the power grid modeling and control can be enhanced in a more
interpretable framework by incorporating physics-inspired information in RL.
For example, the Markov decision processes can be constrained with Bayesian
to include physical priors through value or policy functions [27, 67]. Addition-
ally, model predictive control can be combined with RL, whereby the model
29
information can be better incorporated into RL frameworks when applicable.
For example, the authors in [156] leverage model predictive control (MPC) and
an RL paradigm (dynamic programming), whereby more information can be
processed effectively with enhanced reliability, and fast convergence in learning
can be achieved. In addition, the standard nonlinear model predictive con-
trol (NMPC) scheme can be used as a function approximator in RL, through
which the RL can benefit from the rich theory of NMPC with enhanced in-
terpretability on closed-loop performance [66]. Although these methods have
yet to be fully investigated for power system applications, they are potential
to data-driven model reliability and thus help augment grid operation safety.
Some other technical trends in RL research community for more reliable learn-
ing also worth attention, including but not limited to the interpretable RL for
enhanced consistency in safety-critical applications [157], transfer RL[158] and
meta RL for improving adaptiveness to new situations [159], federated RL to
preserve data-privacy [160], Bayesian RL to incorporate uncertainty and prior
knowledge in learning [67], inverse RL to extract proper reward functions that
the RL agents seek to maximize [161], integral RL for continuous-time dynamics
modeling [162], etc.
Supervised learning-based surrogate models may also be designed for mod-
ern power grid control systems with physical information incorporated by un-
supervised learning. The optimization objectives in such a learning paradigm
are [163]: (a) to minimize the training error of supervised learning, and mean-
while (b) to minimize a certain physics-informed error of unsupervised learning
to regularize the supervised learning in (a). By incorporating regularization
through unsupervised learning rather than directly adding regularization terms
in objective functions, the generalization of learning machines can be improved
due to the powerful unsupervised learning capacity and physics-informed shrink
of parameter searching space. Similar ideas can be found in semi-supervised
learning for power flow analysis in [164], where auto-encoders (i.e., a decoder
+ an encoder) are used to incorporate supervised and unsupervised learning.
The encoder is where one can include physical knowledge of power grids such
as the power mismatches [165], system topology information, etc [164]. Besides,
not only the constraints of systemwide power flow but also that of local DERs
can be included during the model design and training phases such that more
interpretable machine learning-based methods can be realized.
5.4.3. Koopman-Based Methods and Others
The above physics-inspired modeling techniques are often trained with ade-
quate offline datasets while rarely used for online modeling and real-time control.
Thus, they may lack fast adaptiveness to time-varying operating conditions of
modern power grids. In addition, the identification using the closed-loop con-
trol data is also challenging due to the time-varying uncertainty from control
input channels. In view of the authors, a dynamics-interpretable Koopman-
inspired structure (e.g., Koopman operators and their variants such as EDMD
and SINDy) with the help of emerging physics-informed techniques are promis-
ing for online learning regarding adaptiveness and scalability. Specifically, the
30
Koopman observables Φ(x,u) in (19) or the basis library Θin (21) define the
Koopman-based model structure, which is the prerequisite for achieving certain
physics interpretability on system dynamics and thus improving data-driven
model efficacy and transparency for data-driven dynamical control.
Although promising, Koopman-based methods are still in the early stages of
development. One of the main challenges limiting the applications of Koopman-
based methods into power grids is the difficulty in determining the Koopman
basis (i.e., Koopman observables or eigenfunctions) that formulates a proper
Koopman state space structure. By and large, the Koopman basis is empir-
ically determined based on power grid domain knowledge or learned directly
from data with either universal training or self-dictation based on the dynamic
relationship of the Koopman-based space and the original space [137–141]. For
example, in high-inertia power grids dominated by multiple machines, the sys-
tem dynamics are mainly driven by electromechanics, and the candidates of
Koopman basis functions can be relatively straightforward (the combination of
the trigonometric functions [11, 135]) by considering the differential equations of
the electromechanical physical models and the network power flow. Although
no guarantee for optimal design, the dominant dynamical characteristics can
be well-captured. However, in future power grids with greater modernization,
the determination of Koopman state space merely based on observation may
not always work due to complex dynamics resulting from low inertia, a large
number of diverse distributed energy resources, varying operating conditions as
well as fast-evolving network topologies. The advancements of machine learning
techniques now and future will offer tremendous opportunities to learn optimal
Koopman embedding mapping. For example, a potential solution is to apply
Koopman-based modeling in the RL frameworks, in which the Koopman-based
methods can help RL enhance environment/value/policy function modeling and
the RL framework in turn can help Koopman-based methods explore the prob-
lem inherent structure by pursuing Bellman optimality [57]. Besides, physics-
inspired domain information in power grids has yet to be incorporated in the
deep learning-based Koopman basis discovery while deserving more attention.
Given appropriate Koopman structures, the development of Koopman-based
online estimation and control are also trending because of their structural ad-
vantages inherited from linear systems. First, as Koopman operator-based space
is linear, mature linear identification techniques are applicable to enhance iden-
tification, which could be used for power grid applications in the future. For
example, the linear matrix inequality (LMI) can be used to relax the minimiza-
tion of the loss function with regularization terms (i.e., (17) or (18)) to convex
problems [166, 167], which can help realize modular, optimal and fast identifica-
tion. Besides, the mechanism with recursive measurement data may be designed
to improve Koopman-based identification efficiency aiming at faster adaption to
time-varying uncertainty and nonlinearity. Second, online ensemble learning to
combine individual models is another way to further improve identification. For
example, a “soft switching” mechanism [117] can be employed to weigh linear
and nonlinear models [7], and an online iterative ensemble structure[68] can be
used to combine small-signal physical models and large-signal models. Nonethe-
31
less, the stability and optimality of online ensemble control remain to be inves-
tigated. Third, the Koopman-enabled linear structure provides opportunities
for the applications of distributed and cooperative MPC [168–170], which are
mature control techniques with well-characterized stability and robustness prop-
erties. This shed light on potential practical solutions to scalable integration of
distributed energy resources into modern power grids.
5.4.4. Data Sources
To the best of our knowledge, almost all the existing RL-based control are
based on power system simulators. The supervised learning-based modeling and
Koopman-based modeling for control are based on simulation data, historical
experimental offline data or online data. Real-time simulation is an emerging
trend that enables the development of high-fidelity simulators for complex mod-
ern power systems, offering the potential for future implementation of “digital
twins” for online monitoring and control. The digital twin is an up-to-date
representation of an actual current asset in operation that includes the asset’s
condition and relevant historical data [171]. The real-time simulator and ulti-
mately the digital twin can be used as the simulator for RL and the full-state
data generator for supervised learning. However, it’s important to note that
even with high-fidelity simulators or experimental data, there may exist a small
gap between these representations and the real-world systems, leading to un-
certainty propagation and thus compromising control performance. To miti-
gate this, some countermeasures may be taken such as robust (adversarial) RL
methods in simulator-based policy training [2, 172, 173], and data augmenta-
tion techniques, such as adversarial NNs [174], can generate sufficient training
data that includes necessary information for robust supervised learning-based
modeling for control.
6. Conclusion
This paper provides a comprehensive review on emerging data-driven control
for the applications in modern power grids. The data-driven control mainly con-
sists of two ingredients — identification and control, which should be conducted
jointly in a either sequential or simultaneous manner. To realize real-world ap-
plications for safety-critical power grids of high complexity, physics-informed
methodology is necessary while it remains to be studied to realize a practical
solution that is theoretically sound with performance guarantee. In general, one
needs to consider the physical constraints and find elegant ways to incorporate
them in identification and control. To this end, combining learning-based meth-
ods, physical domain knowledge, and the Koopman-based model structure seems
promising as mature linear system control methods with well-characterized sta-
bility and robustness can be readily applied to the linear Koopman-based model.
Besides, various machine learning methods including supervised, unsupervised
learning and RL may be used jointly in exploring and exploiting model struc-
ture and parameters with proper physics-informed learning objectives and con-
straints defined by users.
32
References
[1] P. D. Lund, J. Byrne, R. Haas, D. Flynn, Advances in Energy Systems:
The Large-scale Renewable Energy Integration Challenge, 1st Edition,
John Wiley & Sons, google-Books-ID: cA6GDwAAQBAJ.
[2] X. Chen, G. Qu, Y. Tang, S. Low, N. Li, Reinforcement learning for
selective