ArticlePDF Available

A multiplex, multi-timescale model approach for economic and frequency control in power grids


Abstract and Figures

Power systems are subject to fundamental changes due to the increasing infeed of decentralized renewable energy sources and storage. The decentralized nature of the new actors in the system requires new concepts for structuring the power grid and achieving a wide range of control tasks ranging from seconds to days. Here, we introduce a multiplex dynamical network model covering all control timescales. Crucially, we combine a decentralized, self-organized low-level control and a smart grid layer of devices that can aggregate information from remote sources. The safety-critical task of frequency control is performed by the former and the economic objective of demand matching dispatch by the latter. Having both aspects present in the same model allows us to study the interaction between the layers. Remarkably, we find that adding communication in the form of aggregation does not improve the performance in the cases considered. Instead, the self-organized state of the system already contains the information required to learn the demand structure in the entire grid. The model introduced here is highly flexible and can accommodate a wide range of scenarios relevant to future power grids. We expect that it is especially useful in the context of low-energy microgrids with distributed generation.
Content may be subject to copyright.
Chaos 30, 033138 (2020); 30, 033138
© 2020 Author(s).
A multiplex, multi-timescale model approach
for economic and frequency control in power
Cite as: Chaos 30, 033138 (2020);
Submitted: 18 October 2019 . Accepted: 12 March 2020 . Published Online: 27 March 2020
Lia Strenge, Paul Schultz , Jürgen Kurths , Jörg Raisch, and Frank Hellmann
A multiplex, multi-timescale model approach for
economic and frequency control in power grids
Cite as: Chaos 30, 033138 (2020); doi: 10.1063/1.5132335
Submitted: 18 October 2019 ·Accepted: 12 March 2020 ·
Published Online: 27 March 2020
View Online
Export Citation
Lia Strenge,1Paul Schultz,2Jürgen Kurths,2Jörg Raisch,1and Frank Hellmann2,a)
1Control Systems Group at Technische Universität Berlin, Einsteinufer 17, 10587 Berlin, Germany
2Research Department 4 Complexity Science, Potsdam Institute for Climate Impact Research, Telegraphenberg A 31,
14473 Potsdam, Brandenburg, Germany
Note: This paper is part of the Focus Issue on the Dynamics of Modern Power Grids.
a)Author to whom correspondence should be addressed:
Power systems are subject to fundamental changes due to the increasing infeed of decentralized renewable energy sources and storage. The
decentralized nature of the new actors in the system requires new concepts for structuring the power grid and achieving a wide range of control
tasks ranging from seconds to days. Here, we introduce a multiplex dynamical network model covering all control timescales. Crucially, we
combine a decentralized, self-organized low-level control and a smart grid layer of devices that can aggregate information from remote
sources. The safety-critical task of frequency control is performed by the former and the economic objective of demand matching dispatch
by the latter. Having both aspects present in the same model allows us to study the interaction between the layers. Remarkably, we find that
adding communication in the form of aggregation does not improve the performance in the cases considered. Instead, the self-organized state
of the system already contains the information required to learn the demand structure in the entire grid. The model introduced here is highly
flexible and can accommodate a wide range of scenarios relevant to future power grids. We expect that it is especially useful in the context of
low-energy microgrids with distributed generation.
Published under license by AIP Publishing.
Highly decentralized power grids, possibly in the context of pro-
sumer systems, require new concepts for their stable operation.
We expect that both self-organized systems and intelligent devices
with communication capability that can aggregate information
from remote sources will play a central role. Here, we introduce
a multiplex network model that combines both aspects and use it
in a basic scenario and uncover surprising interactions between
the layers.
Power systems are subject to fundamental changes caused by
the increasing infeed of decentralized and fluctuating renewable
energy sources. One key change is that the conventional energy pro-
ducers, i.e., big central power plants, are currently also the locus of
control resources for the power grid. The challenge facing future
power grids lies in achieving a stable and robust operation of the
grid without such centrally controlled actors.
In general, the objective for a stable operation is to maintain
the frequency and voltage of the system and to keep the system in
an economically desired state. To do so, it is necessary to achieve
an instantaneous balance between electricity generation and con-
sumption. Any imbalance is directly linked to a deviation of the
grid frequency from its nominal value (50 Hz/60 Hz). The control
and stabilization of frequency are traditionally divided into pri-
mary, secondary, and tertiary control. They address instantaneous
frequency stabilization, i.e., keeping frequency within given bounds
(primary), restoring the nominal frequency (secondary), and achiev-
ing or restoring a desired economic state (tertiary). These three
control layers also typically come with a temporal hierarchy, with
primary control acting at the scale of seconds, secondary in minutes,
and tertiary in quarter-hours.
In addition, the corresponding tasks require an increasing level
of explicit communication and coordination of the actors, who par-
ticipate in various markets to ensure sufficient control resources
in an economically feasible way. While primary control might
be achieved through automated reactions to frequency deviations,
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-1
Published under license by AIP Publishing.
controls on slower timescales are typically subject to active human
The power required to operate the safety-critical primary and
secondary control as well as to balance out unforeseen load vari-
ations is typically held in reserve and has to be provided at short
notice. Both reserved capacity and energy drawn cause costs. On
the other hand, the energy required to service the expected load is
bought days, weeks, or even years in advance, based on past expe-
rience. This energy can be dispatched by the cheapest provider, and
technical constraints play less of a role here. It can thus be expected
to be cheaper overall than primary or secondary control energy.
Following a fault, it is the role of tertiary control to restore this
economically favorable state.
Control concepts for (prosumer-based) microgrids, which
could form a key part of future grid designs, especially in emerging
markets, follow the same hierarchy of tasks.1,2Again, primary and
secondary controls are required for the proper functioning of the
grid itself, whereas tertiary control (also called energy management)
chooses the economically desirable source of energy. With this con-
text in mind, we explore scenarios for achieving frequency stability
as well as economically optimal balancing. We focus on distributed,
self-organized control actors, assuming that an analogous detailed
market design for single microgrids is not feasible due to their small
sizes, decentralized power provision, and low inertia.
In future power grids, the control tasks will have to be per-
formed by new distributed actors. A key question in their grid design
is how much coordination these actors will require and how much
can be achieved through self-organized means. To obtain a system
that can function in the face of communication failures, it is natural
to require that primary and secondary control should be achieved
in a fully decentralized and self-organized fashion. On the other
hand, tertiary control is an optimization task that can make use of
communication and coordination infrastructure safely.
To understand whether such a communication and coordina-
tion layer is required and how it performs with respect to control and
stabilization, it is necessary to study the interaction of the different
layers of the control hierarchy, which are typically studied only sep-
arately. Most literature studies on hierarchical control are reviewing
existing approaches on their respective timescales without explicitly
studying their interaction.39
In this paper, we introduce a model for the hierarchical opera-
tion of a power grid that aims to achieve two goals, robust control in
the face of communication failures and some notion of an econom-
ically optimal dispatch under operational constraints. We consider
the power grid as a two layer network,1012 where the layers are given
by (i) the physical electricity network together with a fully decen-
tralized real-time distributed primary/secondary control of the grid
frequency and (ii) an energy management layer. If we allow for com-
munication in the energy management layer, this can be described
as a multiplex network1315 of a physical and a control layer, i.e., both
layers have an identical set of nodes. In the latter, links correspond
to a directed information transfer between controllers. This model
allows us to study the interaction of timescales ranging from seconds
to days.
We use this model to analyze a simple but illustrative scenario,
where a subset of nodes in the system has the ability to dispatch
energy in hourly intervals and optimize for an unknown periodic
background demand that is inferred from the control actions
required by the primary/secondary layer. Using this scenario, we can
compare the effect of various communication strategies.
In order to study the performance of various approaches, we
make use of probabilistic methods,1624 that is, we define a scenario
ensemble and evaluate the expected performance of the system with
respect to the ensemble average by sampling over the ensemble. This
approach allows us to study the properties of the system for the
whole ensemble, rather than in individual case studies. As our main
aim here is to introduce the model and understand qualitatively the
performance of the control layers, our setup is rather conceptual and
does not capture real power systems in detail. While the inspira-
tion is drawn from the context of prosumer-based microgrids, we
consider this to provide a broader perspective as well.
A. Multiplex aspects of power grids
The study of multilayer networks as (dynamical) systems with
an additional mesoscale structure experienced an active develop-
ment in recent years (see Ref. 12 for a review). Subsequently, a
variety of statistical network characteristics (e.g., Ref. 25) has been
developed to quantify the multilayer structure. A special multi-
layer structure is the multiplex (or multilevel26) network, which we
employ in our model. Nodes are identical across the layers; hence,
the topology between the layers is fixed.
We identify the network layers with their different functional
roles within the system, i.e., electricity distribution and control.
Likewise, the coupling mechanism is different and given by the
physical power flow and communication, respectively. Introducing
an interdependence between the layers affects the overall system’s
resilience to failures, a popular example is the 2003 Italian blackout27
with a multilayer cascading failure. In particular, it has been shown
that the interconnection of different networks can promote network
breakdown in discontinuous first-order transitions.28,29
Note that there are a number of works that study consensus-
based methods for achieving certain objectives (see, e.g., Refs. 30
and 31) by introducing an additional communication layer. These
multiplex networks differ in various ways from the setup stud-
ied here. Most importantly, in the fact that the layers cooperate
on a single control objective, it is mostly secondary control8,32 or
quasi-stationary tertiary control33 that is considered.
B. Energy management
For a decentralized primary and secondary control, we make
the most simple choice and use a lag element, which can also be
described as adapted distributed proportional-integral (PI) control.
The main adaptation, discussed thoroughly in Ref. 34, is that the
integral controller includes an exponential decay term. While this
means that there remains a residual steady-state error, it makes the
setup robust to unavoidable systematic errors in the implementation
of the integrator.
The question of how to model energy management is far more
challenging and less settled. A variety of approaches3537 model
energy markets directly. For tertiary control (here referring to the
technical implementation) directly, we are not aware of any gen-
eral models, though more concrete studies for the microgrid context
exist.38 There are also various works studying the relationship to
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-2
Published under license by AIP Publishing.
congestion management and frequency control.39 Our approach
here is to side step the question of how exactly the dispatch is cho-
sen, but instead focus on studying the steady state emerging once the
economic optimization (given the available information) has been
performed. We do so by introducing an iterative learning control
(ILC)40,41 that considers the previous days performance and attempts
to iteratively improve it by scheduling a different dispatch for the
next day. As the consumption/production fluctuates from day to
day, this iterative process should converge to an optimum dispatch.
Note that the energy scheduled for the next day is known a day ahead
but not generated a day ahead.
ILC is a control method which can be applied to track a peri-
odic output or reject periodic disturbances. The error is reduced
over the iteration cycles, and it can easily be combined with feed-
back controllers. ILC has previously been used in power systems in
other contexts, mainly for inverter control, e.g., Refs. 42 and 43 In
addition, ILC is applied to an uninterruptible power supply6and
for optimal residential load scheduling.44 In building automation,
data-driven methods for demand response in the residential build-
ing sector are taken into account;45 ILC also addresses frequency
control with high penetration of wind integration;46 it is further
applied to energy management in electric vehicles.47 Hence, most of
the literature combining energy management and ILC focus on sin-
gle nodes in a grid without emphasis on the overall grid perspective.
However, a review on ILC for energy management in multi-agent
systems states that the applicability of ILC to the topic including
physical constraints has a high research potential due to its (peri-
odic) disturbance rejection capacity and distributed architecture for
large-scale systems.48 ILC for physically interconnected linear large-
scale systems is studied and applied to economic dispatch in power
In larger grids, we expect that this learning would be replaced,
for example, by a market-based system. However, in autonomous
microgrids, without the resources necessary to implement a market-
based solution, the ILC itself is a viable way of choosing dispatch.
We leave a detailed discussion of the design, as well as a proof of
linearized asymptotic stability in the iteration domain of the ILC in
such a scenario to a companion paper.50
This paper is structured as follows. In Sec. II, the overall model
with two control layers is presented. In Sec. III, we compare the
performance of the system for different multiplex topologies with
sampling based numerical experiments. Finally, in Sec. IV, we dis-
cuss our main result that an additional communication layer is
not needed in the proposed setting and suggest further research
C. Notation
Let Nbe the set of nodes in the electricity network. Then, we
have the two graph layers. First, the electricity network G=(N,E)
is an undirected graph, i.e., EN×Nwith (i,j)E(j,i)E.
Second, the communication layer is represented by a directed graph
GC=(N,EC), with a bipartition of Ninto NCwith higher-layer
control present and Nnwithout higher-layer control. NC˙
=N, where ˙
is the disjoint union. Note that EC= {(i,j)|iN,j
NC}(edges directed from ito j) is not necessarily a subset of
E, and data are available from all nodes in N. We call SN
a maximal independent set in N, i.e., iN:iSN(i)S
6= ∅, where N(i)denotes the neighbors of i. We label the nodes
jN= {1, ...,N}. card(·) denotes the cardinality of a set and
×the Cartesian product of two sets.
As noted above, we use a straightforward and well studied
model for providing the basic frequency control of the system with
bounded frequency deviation, cp Refs. 2and 34.
A. Lower layer
Our aim is to use a conceptual model for the dynamics of
a single node that can capture a variety of behaviors. In keeping
with the inspiration of a fully distributed microgrid, we assume
that all nodes have control capability. We further assume that the
distributed control mimics the relationship between synchronous
frequency and the power balance of generation and demand found
in traditional synchronous machines.51,52 Neglecting voltage dynam-
ics, this leads us to the formulation of the Kuramoto model with
inertia,2,53 with the input power (i.e., basically the natural frequency
of the oscillating units) controlled by the distributed control. Fur-
thermore, we assume that there are some nodes that allow for slower,
dispatchable energy that is controlled by the ILC in the higher layer.
The open-loop system equations for node jare then given by
φj(t)= −Pd
j(t)+Fj(t), (1a)
Fj(t)= − X
VjVkYjk sin φj(t)φk(t), (1b)
with the time tR,φjis the voltage phase angle of node jin the
co-rotating frame, and ωj:=˙
φjbeing its instantaneous frequency
deviation from the rated grid frequency. Fjdenotes the AC power
flow from neighboring nodes under the assumption of purely induc-
tive lines. In general, our approach is not limited to this assumption
though. The parameters are the effective inertia Mj, the steady-
state voltages Vj, and the nodal admittances Yjk, which encode the
network topology.
The system is driven by the balance between power demand
and generation; here, PILC
jand PLI
jare the power dispatched by
the higher layer (ILC) if available, and the distributed control (LI),
respectively. While we assume that PLI
jcan be set arbitrarily, PILC
restrictions. That is, PILC
jhas to be chosen to lie within the achiev-
able behavior PILC
jof the dispatchable energy at node j. This
can encode a variety of constraints such as minimal run times, max-
imum ramp rates, and finite storage. In order to mimic the behavior
of trading markets, with their hourly or quarter-hourly dispatch, we
here take BILC
jto be the space of functions that are constant during
each hour.
The energy demand Pd
jis a priori unknown. In order to effi-
ciently study the hierarchical control performance with regard to
communication structure and the stochastic nature of the sys-
tem, we use well-defined synthetic demand curves. In particular,
jis composed of Pp
j, which is a periodic baseline demand
with randomly selected amplitude at each node (period Tdof a single
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-3
Published under license by AIP Publishing.
FIG. 1. Exemplary demand curve (periodic and fluctuating component).
day) and Pf
jis composed of additive white noise with zero mean with
a piece-wise linear interpolation in intervals of 15 min. This is visu-
alized in Fig. 1. See Appendix B for an explicit formulation. While
for simplicity, we here use a traditional separation of dispatch and
demand, the model naturally accommodates fluctuating production
as well. Note that all power-related quantities here are scaled with a
rated power.
The decentralized control in the lower layer is responsible for
primary and secondary control tasks, which is frequency stability
and restoration. Hence, the addressed control objective for nodes
jNis to achieve a bounded frequency deviation,
tjNωj(t)[ωmin,ωmax ], (2)
where we consider ωmin =49.8 Hz =306.62 rad/s and ωmax
=50.2 Hz =315.42 rad/s. The lower-layer controller34 is chosen for
this purpose. It has a small steady-state error, but it avoids instability
caused by parallel integrators. The dynamics are as follows:
j(t)= −kp,jωj(t)+χj(t), (3)
Tj˙χj(t)= −ωj(t)kI,jχj(t), (4)
where χjis the controller state, Tj,kI,j, and kP,jare parameters of the
lower layer controller.
The choice of the control parameters should be in com-
pliance with known design criteria (Ref. 34, Corollary 4). In
order to specifically choose the parameters within the remaining
degrees of freedom, we select them by probabilistic methods. The
results are shown in Figs. 911 in Appendix C. Hence, we choose
kP,j=525 s/rad and kI,j=0.005 rad/s. The steady-state error
(Ref. 34, Corollary 4) for the maximum possible demand in this
setting is 0.001 655 rad/s.
Concretely, we use two complementary measures of the per-
formance of the primary and secondary control, the maximum
observed frequency deviation
ωtop :=max
t[tobs,start;tend ]ωj(t)(5)
and the exceedance, which is the total time that the frequency is out
of bounds within an observed interval, i.e.,
tend tobs,start Ztend
2(|ωj(t)| − 1ω)dt. (6)
In this terminology, exc >0 would mean that objective Eq. (2) has
not been achieved. However, to better resolve performance differ-
ences between the different control designs discussed below, we
choose a tighter bound =0.0005 rad/s for the defining critical
frequency deviation.
Since the learning control introduced in Sec. II B takes some
time to converge to a steady state, we discard the transients and focus
on the performance over an interval tobs,start;tend .
B. The ILC control layer
The aim of the higher-layer controller is to achieve a state that
is in some sense economically optimal. As stated above, we study
the equilibrium state rather than the convergence to that state. We
consider the former a sensible stand-in for other methods that try
to achieve an economically optimal situation. The design and per-
formance of the ILC itself as a method for microgrids is treated in a
companion paper.50
Concretely, the higher-layer controller looks at the previous
day and adjust the dispatch chosen from BILC
jin such a way as to
minimize the overall system cost. Therefore, the economic objective
translates to minimizing
Ctotal =
j(τ )| + (1λ)|PILC
j(τ )|dτ, (7)
with the (here constant) cost factor λ(0, 1]. Hence, λ
1λis a notion
of the price relation between PLI and PILC. As noted above, the mod-
eling assumption here is that power planned a day in advance is
technically less challenging, and thus cheaper than control power
that needs to be provided as an instantaneous reaction, and thus
λ > 0.5. This is aligned with current market pricing where day-
ahead markets trade energy at a much lower price than primary
and secondary control energy markets. A similar relation can be
expected in microgrids. Figure 8 in Appendix A shows how the
average overall system cost changes with the cost factor λ. For our
analysis, we choose λ=0.8.
An update is chosen within BILC
j, and the next day’s
jare adjusted accordingly, possibly aggregating the updates
from communicating nodes as well. Concretely, we adjust PILC
proportionally to
j(τ ))dτ+¯
T1), (8)
where th=(h1)twith t[tstart,tend ], tstart,tend R0is the
beginning of hour hN, sgn(·) is the sign function, and ¯
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-4
Published under license by AIP Publishing.
TABLE I. Simulation parameters (node j=1, . .. , N). W/W units refer to scaled quantities.
Parameter Value Unit Description
kI,j0.005 (W rad)/(W s) Lower-layer control parameter
kP,j525 W s/(W rad) Lower-layer control parameter
κj0.15/3600 1/s Learning parameter
Mj5 W s2/(W rad) Inertia
N24 . . . Number of nodes
1/Tj1/0.05 1/rad Lower-layer control parameter
Yjk VjVk6 W/W Coupling
0.0005 rad/s Frequency threshold for the exceedance
tend 50 days Number of simulated days
tobs,start 40 days Start of observation interval
S100 . . . Number of simulations in one experiment
The iteration step Eq. (8) we perform is motivated in Appendix A
and minimizes the cost Eq. (7) under certain assumptions.
Concretely, in the scenario where all nodes have dispatchable
energy, and the ILC only updates based on the local informa-
tion, this implies that we have for each node j=1, ...,Nand each
hour h
ctrl,j, (9)
where PILC,h
jis the value of the hourly constant PILC
j(t)for tht<
th+1, that is, during the hour starting at thand κjRis the learning
Let us now consider the genuine multiplex case, where we
allow for communication, that is, the ILC nodes aggregate infor-
mation about the expended control energy at different nodes and
update using the total. Then, using the adjacency matrix ACof the
communication layer, we get
for all jNC, where dj=PkNAC
jk is the degree. In terms of
control, this means we use a P-type (i.e., proportional) ILC con-
troller. A Q-filter, implying a forgetting filter regarding previous
and upcoming hours of a day or a more sophisticated interaction
between the nodes, is not considered here, hence Q=I, where Iis
the identity matrix. A Q-filter may become beneficial if the demand
amplitudes vary with the days. Furthermore, we simplify and use the
same learning gain for all nodes, i.e., κj=κfor all jN.
Our main focus is to study whether the communication net-
work is required in order to achieve sensible economic outcomes or
whether the decentralized robust operation of the grid implicitly car-
ries enough information between the nodes to achieve an acceptable
outcome without added communication infrastructure.
A. Grid ensemble
As noted above, we apply a probabilistic approach. That is, we
define a class of grids consisting of a random peak demand per node
as well as random topology and evaluate the expected performance
of the design for this grid class. For simplicity, and to focus on the
effects of the topology in the higher layer, we use a simple random
regular graph with degree three for the topology in the lower layer.
This is not intended to be a realistic choice for most power grids but
provides a homogeneous backdrop on which the higher layer can
operate. The demand amplitudes are chosen uniformly at random.
For details on the demand model, see Appendix B.
We consider a sample size of S=100 grids à card(N)=24
nodes with random demand from the ensemble and then integrate
the system for 50 days. Investigation of individual trajectories reveals
that this is highly sufficient to achieve equilibrium for the ILC in
all cases considered (compare Figs. 1216 in Appendix D, which
show example trajectories for the different higher-layer scenarios).
We then use the performance on the last 10 days to study the steady-
state properties. All initial conditions are set to zero, i.e., the ILC
update sets in after the first day. All relevant simulation parameters
are found in Table I.
We evaluate the expected value of three quantities already
introduced above. First, the maximum frequency deviation ωtop to
see how far the system deviates from the desired frequency; sec-
ond, the frequency exceedance excjwhich indicates the quality of the
control achieved; and the third quantity is the total cost of higher-
layer and lower-layer control energy in the system Ctotal defined in
Eq. (7).
B. Higher-layer topologies
We consider five different topologies chosen to illustrate differ-
ent designs of communication and control infrastructure.
The first baseline scenario (scenario 0) is to study the perfor-
mance of the decentralized control by itself, without any ILC, that is,
the higher layer is simply the empty graph, NC= ∅.
The second baseline scenario (scenario I) is to assume that
every node has the ability to dispatch energy and optimizes to sat-
isfy its demand locally, without taking the neighbors into account,
NC=Nbut EC= ∅ and thus AC=0.
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-5
Published under license by AIP Publishing.
FIG. 2. An illustration of the multiplex network consisting of a physical layer
(“grid,” bottom) and a control layer (“control,” top). Directed edges are indi-
cated with arrows. The vertical dashed lines identify the nodes in the two layers.
In the upper layer, filled circles indicate the nodes vNCwhere control is
The three main scenarios we want to consider take ILC at a
subset of nodes in such a way as to mimic three different potential
designs. The first two (II and III) scenarios assume that the location
of the dispatchable power is chosen in some sense to be central in
the underlying physical grid. To model this, we construct a maximal
independent set NC, that is, a (non-unique) maximal set of vertices
such that two nodes in NCare never adjacent. Hence, every node
in the graph GCis either in NCor neighbor to one node in NC,
cp. Fig. 2(top). The edges are directed, accounting for the directed
information transfer. By assigning the ILC to NC, every other node
in Nnis adjacent to a node with dispatchable energy. With this set
of nodes, we can now define and compare the scenarios II and III.
Scenario II has communication from these neighbors, and scenario
III has no communication. In the former case, we have a directed
communication graph with the adjacency matrix elements AC
jk =1
if jNCand Yjk >0. Otherwise, we set AC
jk =0.
This can be seen as a highly conceptual model where local
regions in the network are responsible for the energy balance in their
respective areas. To contrast this with a random topology, we finally
consider the case (scenario IV) that the ILC is positioned at half
the nodes at random and communicate at random with three other
The three scenarios II,III, and IV thus represent sparse
ILC with no communication, structured communication, and
Max frequency deviation [rad/s]
FIG. 3. The maximum frequency deviations of the nodes for the various higher
layers. The box plots show the quartiles and outliers of the system. The colored
box covers the second and third quartile, the middle line gives the median. The T
bars give the extrema of the distribution up to outliers. See Table II for the scenario
random communication. The scenarios are summarized in
Table II.
C. Results of the numerical experiments
We first consider the maximum frequency deviation to see
how far the system deviates from the desired frequency and the
exceedance of the frequency. These observables show the quality of
the control achieved by the lower layer in the presence of the various
higher-layer topologies. The first plot in Fig. 3 shows the distribution
of ωtop across the grid ensemble. In each scenario, ωmin ωtop
ωmax, i.e., the lower-layer control objective Eq. (2) is always achieved.
We see that the performance is very similar across the communica-
tion scenarios, it is slightly better for the no-ILC case (0) and the
local ILC at all nodes without communication (I). Figure 4 depicts
the exceedance excjacross the grid ensemble for all nodes j. We can
observe that adding a higher-layer control reduces the exceedance
drastically (IIV). The scenarios with communication (II,IV) per-
form slightly better than without (I,III). It is apparent from the
observation of both ωtop and excjthat the addition of the higher layer
the amplitude of transient deviations but at the same time short-
ens their duration. This indicates that the ILC control suppresses
TABLE II. Overview of the studied higher-layer topologies.
Expt. Description Communication graph
0No-ILC GCis the empty graph
ILocal ILC at all nodes GC=(N,), card(NC)=card(N)
II ILC at nodes in a max. independent set in the network GC=(N,EC),EC=ENC×Nn,NC
graph and averaged update with all neighboring nodes max. independent set
III Local ILC at nodes in a max. independent set in the network graph GC=(N,),NCmax.independent set
IV ILC at 50% of the nodes with averaged update with GC=(N,EC), EC= {(i,j)|iNC,jN\ {i}},
3 random other nodes card(NC) = card(Nn)
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-6
Published under license by AIP Publishing.
Exceedance [%]
FIG. 4. The exceedance of the nodes for the various higher layers, see Table II
for the scenario definitions.
demand fluctuations that drive the system out of the bounds given
by .
More interestingly, we can now consider the total system cost
from Eq. (7) over the course of the last 10 days of the simulations,
given the various choices of higher-layer control. We chose λ=0.8,
i.e., instantaneous control energy being four times more expensive
than energy at the day-ahead market. The baseline scenario 0, with
no-ILC at all, in the left column of Fig. 5, gives us an idea of the
total cost in the absence of dispatch. Providing dispatchable energy
at every node in scenario Ireduces the total cost by a factor of
almost three with the parameters chosen, see the second column in
Fig. 5.
Turning now to the three main scenarios IIIV, we see that
they all manage to reduce the cost even further compared to scenario
I. For those scenarios, we obtain a cost reduction factor of around
four compared to the non-ILC scenario 0, consistent with our choice
of λ(λ/1λ=0.8/0.2 =4).
Overall cost [a.u.]
FIG. 5. The overall energy cost, sum over all nodes, see Table II for the scenario
2 4 6 8 10 12 14 16 18 20 22 24
Number of ILC nodes
Max frequency deviation [rad/s]
FIG. 6. Maximum frequency deviation for λ=0.8 with variation of ILC nodes,
50 days simulated, last 10 days obser ved.
Presumably, the increased cost in scenario Iis due to conflict-
ing actions of the decantralized controllers that are eliminated by
the communication infrastructure in the scenarios IIIV. If, how-
ever, the communication topology is not adapted to the unerlying
physical network but random, we also observe a slight cost increase,
showing that random aggregation is generally not beneficial over
placing control at a maximal independent set in scenario II.
Another distinction between Iand IIIV is the number of
nodes in the control set NC. To investigate the influence on the
overall performance, we systematically varied the number of ILC
nodes from 1 to card(N)=24. The controlled nodes were ran-
domly drawn in each run and not connected to any other nodes
in the communication layer similar to scenario III. The results
are twofold. The maximum frequency deviation in Fig. 6 decreases
monotonically, indicating that the best performance is achieved with
the highest control effort. Contrarily, Fig. 7 shows that the system
is cost-optimal when about one third of nodes are equipped with
2 4 6 8 10 12 14 16 18 20 22 24
Number of ILC nodes
Overall cost (all nodes) [a.u.]
FIG. 7. Overall cost for λ=0.8 with variation of ILC nodes, 50 days simulated,
last 10 days observed.
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-7
Published under license by AIP Publishing.
ILC control. Whereas the significant improvement compared to 0
is evident for a small size of NC, a further distribution of control
action across more nodes leads to slightly higher costs, still remain-
ing well below the baseline scenario. Interestingly, when NC=N,
the expected cost is higher than in scenario Iwithout communica-
tion. Concerning the cost difference between Iand IIIV in Fig. 5,
this experiment implies that the reduction is mainly achieved by a
smaller-sized control set, whereas the addition of communication
links actually increases the costs.
In this paper, we introduced a multiplex hierarchical model of
power grids that covers timescales from seconds to days and allows
studying the interaction of energy management/tertiary control and
self-organized primary and secondary control.
Remarkably, we find that a basic but natural communication
and aggregation scheme in the higher layer does not improve the
performance. The results indicate that the self-organized distributed
control of the lower layer already carries sufficient information to
learn the appropriate dispatch at those nodes that are dispatchable.
Adding explicit communication does not visibly reduce the cost of
the system while the performance with respect to the lower-layer
control objective is expected to decrease slightly.
Actually, there already exists implicit communication through
the power flows managed by the primary and secondary control
in each scenario. It seems that any additional communication that
tries to aggregate the local deviations needs to take this into account.
Also, a carefully designed Q-filter and learning matrix may improve
the performance of additional communication.
Clearly for more complex and interesting control objectives
that a realistic energy management system has to achieve commu-
nication is necessary. However, our results show that even in these
cases it might be worthwhile investigating the implicit communica-
tion already present in the system, and taking it into account since
it leads to inputs taken from different parts of the system to be
correlated, thus potentially causing an overcompensation.
More broadly, we saw that for the parametrization of the decen-
tralized control layer, probabilistic methods are a useful comple-
ment to analytic bounds. The selection of kI,jis chosen according to
the analytical bounds in Ref. 34, Corollary 4 and to avoid a frequency
deviation independently of the choice of kP,j(cp. Fig. 9).
Furthermore, the presence of a higher layer with separate con-
trol objectives certainly has the potential to affect the performance
of the lower layer. For any added ILC, there is a large effect on the
exceedance (Fig. 4). This is not unexpected as the control funda-
mentally changes the nature of all nodes. Still, it indicates that it
is interesting to further study the interactions of the layers in the
future. In fact, the interaction between higher-layer energy manage-
ment and the frequency dynamics of the power grid is an effect that
is observed in real power grids, where trading intervals are very vis-
ible in the statistics of power grid frequency signals.54,55 We expect
that the type of model we have set up here is highly useful to study
and reproduce some of these results without having to go to highly
specific market models.
The general setup we have chosen can serve as a wide rang-
ing basis for the study of the interaction of dynamics and dispatch
constraints. The overall model can easily be extended to include local
limits on available storage and ramping times. Adding individual
and time-varying prices for various forms of energy to give the ILC
a more realistic target function to optimize is also straightforward.
Furthermore, a more sophisticated higher layer can use the
distributed controllers to achieve more challenging goals than
merely minimizing price, or prioritizing one type of energy over
the other, i.e., in future work, we may consider, e.g., the set
j(t),Tj(t),kp,j(t),kI,j(t)] as an input to the ILC.
Finally, we note that the type of system we introduced here can
be of independent interest in the context of theoretical physics. For
example, Nicosia56 analyzes the coupling between different dynam-
ics in a multiplex network, i.e., between a network of Kuramoto
oscillators and a random walk. Under certain conditions, the cou-
pling between the layers then induces spontaneous explosive syn-
chronization transitions. Since we study the synchronization of
Kuramoto oscillators with inertia [Eq. (1)] and use proportional ILC
that is also linear, this is mathematically similar to our model. Thus,
it would be interesting under which conditions models like ours can
exhibit such properties as well.
This work was funded by the Deutsche Forschungsgemein-
schaft (DFG, German Research Foundation)—No. KU 837/39-1/RA
516/13-1. All code was written in Julia and is available on request or
on the first authors github.57 The simulations were performed using
the DifferentialEquations.jl package58 and the Rodas4p solver.59
We want to motivate the precise form of the update law used
in the ILC above. If we assume that the system is roughly held in
equilibrium, despite the low-amplitude fluctuations, Eq. (1) gives us
0≈ −Pd
j(t)+Fj(t), (A1)
for each node j=1, ...,N. In the following, we omit the node
index for readability. If we neglect changes to the flows, we can
approximately assume
PILC ≈ −1, (A2)
i.e., a decrease in PLI is directly proportional to an increase in PILC.
The aim of the ILC is adapting PILC to optimize an observable
O(PLI). Take, for instance,
|PLI|2dt, (A3)
as the quadratic norm of the lower-layer control power. If we change
PILC by a constant shift δPILC that does not depend on t, we find that
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-8
Published under license by AIP Publishing.
the variation of O2(PLI)is approximated as
δO2(PLI)=O2(PLI )
≈ −2ZT
Thus, if we choose the ILC update to be δPILC =RPLIdt, then
the change in O2is always negative, and we gradient descend toward
a local minimum. As our PILC are constant on the hour, this update
law is sufficient to make sure that we minimize the square norm of
PLI for each hour.
A more economic objective function could be to integrate the
total cost of energy used, taking into account that there are differ-
ent price points for energy bought (and scheduled) a day ahead or
requested from a standing reserve of control energy. This suggests
the objective function
0λ|PLI| + (1λ)|PILC |dt
0λPLIsgn(PLI )+(1λ)PILCsgn(PILC )dt,
where λ[0; 1] is a real number. It should be noted that in reality
the composition of costs is considerably more complex. For exam-
ple, capacity markets reward keeping a certain amount of generation
available, whether it is used or not. The objective function Eq. (A4)
only reflects the presence of different price levels for different levels
of flexibility.
In order to calculate the variation of Oλwith respect to a small
constant shift δPILC we make the further assumption that the contri-
bution from the shift in the sgn functions is of higher order, as can
be expected if PLI is sufficiently smooth. Then, we obtain
λsgn(PLIPLIdt +T(1λ)sgn(PILC PILC
= λZT
sgn(PLI)dt +T1)sgn(PILC )δPILC. (A5)
For λ=1, that is, day-ahead energy is infinitely cheaper than
instantaneous energy, we can choose the ILC update
sgn(PLI)dt, (A6)
(where means “proportional to”), which guarantees that δOλ=1is
negative, and we again descend to a sensible local minimum. Intu-
itively, this makes sense since we should increase the background
power when there are more times when positive control energy is
For general λ, first note that as the ILC compensates a posi-
tive background demand, we can always assume that sgn(PILC)=1.
0.2 0.4 0.6 0.8 1.0
Average total cost [a.u.]
I: local ILC all nodes
II: local ILC at vc + neighbor. com
III: local ILC at vc
IV: local ILC rand 50% + rand com
FIG. 8. Average mean cost over thecost factor λ(50 days simulated, last 10 days
Then, we have
δOλ= λZT
sgn(PLI)dt +T1)δPILC . (A7)
Therefore, we want to chose the update law
sgn(PLI)dt +T1)(A8)
to obtain an appropriate gradient descend.
In summary, we have the following economic update law:
0λsgn(PLI)+1)dt. (A9)
Figure 8 shows the average total cost for all numerical experi-
ments performed in this paper. λ > 0.5 are realistic scenarios, i.e.,
with a cheaper higher layer dispatched energy than lower-layer
control energy.
For every node jNin the network, we assume the demand
is given by a periodic baseline Pp
jsubject to fluctuations Pf
Td, (B1)
j(t)=tmod TqBj1+ bt/Tqc+Bjbt/Tqc, (B2)
where the period Td(s) is the duration of a day and the demand
amplitudes AjU([0; 1])are uniform i.i.d. random numbers. The
fluctuation amplitudes Bjvary over time in a Gaussian random walk
with zero mean and a variance of 0.2. Pf
jis linearly interpolated
between two consecutive updates, spaced apart by Tq=15min, as
given above. Note that both Ajand Bjare normalized with a rated
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-9
Published under license by AIP Publishing.
0 200 400 600 800 1000
Maximum frequency deviation [rad/s]
FIG. 9. Selection of control parameters: maximum frequency deviation; simulated
and observed for 1 day, other parameters from Table I except for kI,jand kP,j.
The resulting maximum frequency deviation is 0.001 34 rad/s.
The control parameters kp,j[0, 1000] s/rad and kI,j
[0.001, 1] rad/s are varied in numerical experiments with 41 and
40 values for each parameter, respectively. This results in 41 ×40
=1640 batches. We use 100 simulations—also called runs—in each
batch. Each run has a randomly chosen 3-degree graph with 24
nodes as power network and a random demand as illustrated in
Fig. 1.Figures 911 show the maximum frequency deviation, the
exceedance, and the frequency variance over the relevant ranges for
kP,jand kI,j. We are interested in a low value for all given quantities.
In combination with the analytic bounds given in the literature,34
we choose kP,j=525 s/rad and kI,j=0.005 rad/s for all nodes
FIG. 10. Selection of control parameters: exceedance; simulated and observed
for 1 day, other parameters from Table I except for kI,jand kP,j. The resulting
exceedance is 4.53 ×104%.
0 200 400 600 800 1000
log10(frequency variance [rad/s])
- 10.0
- 7.5
- 5.0
- 2.5
FIG. 11. Selection of control parameters: frequency variance; simulated and
observed for 1 day, other parameters from Table I except for kI,jand kP,j. The
resulting frequency variance is 1.0586 rad/s.
Figures 1216 show exemplary trajectories of cumulative
lower-layer control energy used for 24 nodes for cases 0IV over
a time of 20 days including the initial learning phase. In case 0, the
energy is cumulative over the whole time span, while in the cases
I–IV, it is cumulative over every hour only and then reset. Recall
that for the equilibrium state analysis above, days 20–30 are chosen
which are not shown here in detail. It can be observed that ILC at all
nodes learns faster but the equilibrium performance is worse than in
the other cases. Other experiments performed show that during the
learning process, faster learning leads to lower costs.
FIG. 12. Exemplary trajectories of lower-layer control energy for 24 nodes
for case 0 over a time of 20 days including the initial learning phase. Note that
in case 0 this is the total cumulative energy over the whole time span since there
are no resets.
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-10
Published under license by AIP Publishing.
FIG. 13. Exemplary trajectories of lower-layer control energy for 24 nodes for
case I over a time of 20 days including the initial learning phase.
FIG. 14. Exemplary trajectories of lower-layer control energy for 24 nodes for
case II over a time of 20 days including the initial learning phase.
FIG. 15. Exemplary trajectories of lower-layer control energy for 24 nodes for
case III over a time of 20 days including the initial learning phase.
FIG. 16. Exemplary trajectories of lower-layer control energy for 24 nodes for
case IV over a time of 20 days including the initial learning phase.
1J. M. Guerrero, J. C. Vasquez, J. Matas, L. G. De Vicuña, and M. Castilla, “Hier-
archical control of droop-controlled AC and DC microgrids—A general approach
toward standardization,” IEEE Trans. Ind. Electron. 58, 158–172 (2010).
2J. Schiffer, D. Zonetti, R. Ortega, A. M. Stankovi´
c, T. Sezi, and J. Raisch, “A survey
on modeling of microgrids—From fundamental physics to phasors and voltage
sources,” Automatica 74, 135–150 (2016).
3A. Bidram and A. Davoudi, “Hierarchical structure of microgrids control
system,” IEEE Trans. Smart Grid 3, 1963–1976 (2012).
4F. Dörfler, J. W. Simpson-Porco, and F. Bullo, “Plug-and-play control and opti-
mization in microgrids,” in 53rd IEEE Conference on Decision and Control (IEEE,
2014), pp. 211–216.
5D. E. Olivares, A. Mehrizi-Sani, A. H. Etemadi, C. A. Cañizares, R. Iravani,
M. Kazerani, A. H. Hajimiragha, O. Gomis-Bellmunt, M. Saeedifard, R. Palma-
Behnke et al., “Trends in microgrid control,” IEEE Trans. Smart Grid 5,
1905–1919 (2014).
6M. Aamir, K. A. Kalwar, and S. Mekhilef, “Uninterruptible power supply (UPS)
system,” Renew. Sustain. Energy Rev. 58, 1395–1410 (2016).
7H. Xin, R. Zhao, L. Zhang, Z. Wang, K. P. Wong, and W. Wei, “A decentralized
hierarchical control structure and self-optimizing control strategy for FP type DGs
in islanded microgrids,” IEEE Trans. Smart Grid 7, 3–5 (2015).
8Y. Han, H. Li, P. Shen, E. A. A. Coelho, and J. M. Guerrero, “Review of active
and reactive power sharing strategies in hierarchical controlled microgrids,” IEEE
Trans. Power Electron. 32, 2427–2451 (2016).
9Z. Li, C. Zang, P. Zeng, H. Yu, and S. Li, “Fully distributed hierarchical control
of parallel grid-supporting inverters in islanded AC microgrids,” IEEE Trans. Ind.
Inf. 14, 679–690 (2017).
10J. Gao, S. V. Buldyrev, H. E. Stanley, and S. Havlin, “Networks formed from
interdependent networks,” Nat. Phys. 8, 40–48 (2012).
11D. M. Gale, S. Kariv, and U. C. Systems, American Economic Review, edited
by G. D’Agostino and A. Scala, Understanding Complex Systems (Springer
International Publishing, Cham, 2014), Vol. 97, pp. 99–103.
12M. Kivelä, A. Arenas, M. Barthelemy, J. P. Gleeson, Y. Moreno, and M. A.
Porter, “Multilayer networks,” J. Complex Netw. 2, 203–271 (2014).
13M. De Domenico, A. Solé-Ribalta, E. Cozzo, M. Kivelä, Y. Moreno, M. A. Porter,
S. Gómez, and A. Arenas, “Mathematical formulation of multilayer networks,”
Phys. Rev. X 3, 041022 (2013).
14V. Nicosia, G. Bianconi, V. Latora, and M. Barthelemy, “Growing multiplex
networks,” Phys. Rev. Lett. 111, 1–5 (2013).
15S. Boccaletti, G. Bianconi, R. Criado, C. I. del Genio, J. Gómez-Gardeñes,
M. Romance, I. Sendina-Nadal, Z. Wang, and M. Zanin, “The structure and
dynamics of multilayer networks,” Phys. Rep. 544, 1–122 (2014).
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-11
Published under license by AIP Publishing.
16P. J. Menck, J. Heitzig, N. Marwan, and J. Kurths, “How basin stability comple-
ments the linear-stability paradigm,” Nat. Phys. 9, 89 (2013).
17P. J. Menck, J. Heitzig, J. Kurths, and H. J. Schellnhuber, “How dead ends
undermine power grid stability,” Nat. Commun. 5, 3969 (2014).
18P. Schultz, J. Heitzig, and J. Kurths, “Detours around basin stability in power
networks,” New J. Phys. 16, 125001 (2014).
19F. Hellmann, P. Schultz, C. Grabow, J. Heitzig, and J. Kurths, “Survivability of
deterministic dynamical systems,” Sci. Rep. 6, 29654 (2016).
20P. Schultz, F. Hellmann, K. N. Webster, and J. Kurths, “Bounding the first exit
from the basin: Independence times and finite-time basin stability,” Chaos 28,
043102 (2018).
21M. Lindner and F. Hellmann, “Stochastic basins of attraction and generalized
committor functions,” Phys. Rev. E. 100, 022124 (2019).
22H. Kim, S. H. Lee, and P. Holme, “Building blocks of the basin stability of power
grids,” Phys. Rev. E. 93, 062318 (2016).
23H. Kim, S. H. Lee, J. Davidsen, and S.-W. Son, “Multistability and variations in
basin of attraction in power-grid systems,” New J. Phys. 20, 113006 (2018).
24M. F. Wolff, P. G. Lind, and P. Maass, “Power grid stability under perturbation
of single nodes: Effects of heterogeneity and internal nodes,” Chaos 28, 103120
25J. F. Donges, H. C. Schultz, N. Marwan, Y. Zou, and J. Kurths, “Investigating
the topology of interacting networks: Theory and application to coupled climate
subnetworks,” Eur. Phys. J. B 84, 635–651 (2011).
26R. Criado, J. Flores, A. García Del Amo, J. Gómez-Gardeñes, and M. Romance,
“A mathematical model for networks with structures in the mesoscale,” Int. J.
Comput. Math. 89, 291–309 (2012).
27C. M. Schneider, N. Yazdani, N. A. Araújo, S. Havlin, and H. J. Herrmann,
“Towards designing robust coupled networks,” Sci. Rep. 3, 1969 (2013).
28S. V. Buldyrev, R. Parshani, G. Paul, H. E. Stanley, and S. Havlin, “Catas-
trophic cascade of failures in interdependent networks,” Nature 464, 1025–1028
29A. Vespignani, “Complex networks: The fragility of interdependency,” Nature
464, 984–985 (2010).
30A. Krishna, J. Schiffer, N. Monshizadeh, and J. Raisch, “A consensus-based
voltage control for reactive power sharing and PCC voltage regulation in micro-
grids with parallel-connected inverters,” in 2019 18th European Control Conference
(ECC) (IEEE, 2019), pp. 542–547.
31M. S. Mahmoud, M. S. U. Rahman, and M.-S. Fouad, “Review of microgrid
architectures—A system of systems perspective,” IET Renew. Power Gener. 9,
1064–1078 (2015).
32D. Wu, T. Dragicevic, J. C. Vasquez, J. M. Guerrero, and Y. Guan, “Secondary
coordinated control of islanded microgrids based on consensus algorithms,”
in 2014 IEEE Energy Conversion Congress and Exposition (ECCE) (IEEE, 2014),
pp. 4290–4297.
33S. Mao, Z. Dong, P. Schultz, Y. Tang, K. Meng, Z. Y. Dong, and F. Qian,
“A finite-time distributed optimization algorithm for economic dispatch in
smart grids,” IEEE Trans. Syst. Man Cybern. Syst. (published online 2019).
34E. Weitenberg, Y. Jiang, C. Zhao, E. Mallada, F. Dörfler, and C. De Persis,
“Robust decentralized frequency control: A leaky integrator approach,” in 2018
European Control Conference (ECC) (IEEE, 2018), pp. 764–769.
35A. Weidlich and D. Veit, “A critical survey of agent-based wholesale electricity
market models,” Energ. Econ. 30, 1728–1759 (2008).
36P. Ringler, D. Keles, and W. Fichtner, “Agent-based modelling and simulation
of smart electricity grids and markets—A literature review,” Renew. Sustainable
Energy Rev. 57, 205–215 (2016).
37M. Mureddu and H. Meyer-Ortmanns, “Extreme prices in electricity balancing
markets from an approach of statistical physics,” Phys. A Stat. Mech. Appl. 490,
1324–1334 (2018).
38F. Claessen, B. Claessens, M. Hommelberg, A. Molderink, V. Bakker, H. Toer-
sche, and M. van den Broek, “Comparative analysis of tertiary control systems for
smart grids using the flex street model,” Renew. Energy 69, 260–270 (2014).
39C. Zhao, E. Mallada, S. Low, and J. Bialek, “A unified framework for frequency
control and congestion management,” in 2016 Power Systems Computation
Conference (PSCC) (IEEE, 2016), pp. 1–7.
40N. Amann, D. H. Owens, and E. Rogers, “Predictive optimal iterative learning
control,” Int. J. Control. 69, 203–226 (1998).
41D. A. Bristow, M. Tharayil, and A. G. Alleyne, “A survey of iterative learning
control,” IEEE Control Syst. Mag. 26, 96–114 (2006).
42Z. Zeng, H. Yang, R. Zhao, and C. Cheng, “Topologies and control strategies
of multi-functional grid-connected inverters for power quality enhancement: A
comprehensive review,” Renew. Sustainable Energy Rev. 24, 223–270 (2013).
43K.-T. Teng, “Repetitive and iterative learning control for power converter and
precision motion control,” Ph.D. thesis (UCLA, 2014).
44B. Chai, Z. Yang, K. Gao, and T. Zhao, “Iterative learning for optimal residential
load scheduling in smart grid,” Ad Hoc Netw. 41, 99–111 (2016).
45A. Bampoulas, M. Saffari, F. Pallonetto, E. Mangina, and D. P. Finn, “Self-
learning control algorithms for energy systems integration in the residential
building sector,” in 2019 IEEE 5th World Forum on Internet of Things (WF-IoT)
(IEEE, 2019), pp. 815–818.
46W. Guo, F. Liu, J. Si, D. He, R. Harley, and S. Mei, “Online supplementary ADP
learning controller design and application to power system frequency control with
large-scale wind energy integration,” IEEE Trans. Neural Netw. Learn. Syst. 27,
1748–1761 (2015).
47H.-Q. Guo, C.-Z. Liu, J.-W. Yong, X.-Q. Cheng, and F. Muhammad, “Model
predictive iterative learning control for energy management of plug-in hybrid
electric vehicle,” IEEE Access 7, 71323 (2019).
48D. H. Nguyen and D. Banjerdpongchai, “Iterative learning control of energy
management system: Survey on multi-agent system framework,” Eng. J. 20, 1–4
49J.-X. Xu and S. Yang, “Iterative learning based control and optimization for
large scale systems,” IFAC Proc. 46, 74–81 (2013).
50L. Strenge, X. Jing, T. Seel, F. Hellmann, and J. Raisch, “Iterative learning control
in prosumer-based microgrids with hierarchical control,” in IFAC World Congress
2020 (to be published).
51D. J. Hill and G. Chen, “Power systems as dynamic networks,” in 2006 IEEE
International Symposium on Circuits and Systems (IEEE, 2006), p. 4.
52J. Machowski, J. Bialek, and J. Bumby, Power System Dynamics: Stability and
Control (John Wiley & Sons, Ltd., 2011).
53J. Schiffer, R. Ortega, A. Astolfi, J. Raisch, and T. Sezi, “Conditions for stabil-
ity of droop-controlled inverter-based microgrids,” Automatica 50, 2457–2469
54B. Schäfer, C. Beck, K. Aihara, D. Witthaut, and M. Timme, “Non-Gaussian
power grid frequency fluctuations characterized by lévy-stable laws and super-
statistics,” Nat. Energy 3, 119 (2018).
55M. Anvari, L. R. Gorjão, M. Timme, D. Witthaut, B. Schäfer, and H. Kantz,
“Stochastic properties of the frequency dynamics in real and synthetic power
grids,” arXiv:1909.09110 (2019).
56V. Nicosia, P. S. Skardal, A. Arenas, and V. Latora, “Collective phenom-
ena emerging from the interactions between dynamical processes in multiplex
networks,” Phys. Rev. Lett. 118, 138302 (2017).
57See for
“Repository Controlled-Multi-Timescale-Powergrid.”
58C. Rackauckas and Q. Nie, “Differentialequations. jl—A performant and
feature-rich ecosystem for solving differential equations in julia,” J. Open Res.
Softw. 5, 15 (2017).
59G. Wanner and E. Hairer, Solving Ordinary Differential Equations II (Springer,
Berlin, 1996).
Chaos 30, 033138 (2020); doi: 10.1063/1.5132335 30, 033138-12
Published under license by AIP Publishing.
... Related output approaches are investigated in Strenge et al. (2020). For stability over the cycles, we are interested in the behavior of the disturbance-free system and make use of the formal solution of (4). ...
Full-text available
Power systems are subject to fundamental changes due to the increasing infeed of renewable energy sources. Taking the accompanying decentralization of power generation into account, the concept of prosumer-based microgrids gives the opportunity to rethink structuring and operation of power systems from scratch. In a prosumer-based microgrid, each power grid node can feed energy into the grid and draw energy from the grid. The concept allows for spatial aggregation such that also an interaction between microgrids can be represented as a prosumer-based microgrid. The contribution of this work is threefold: (i) we propose a decentralized hierarchical control approach in a network including different time scales, (ii) we use iterative learning control to compensate periodic demand patterns and save lower-layer control energy and (iii) we assure asymptotic stability and monotonic convergence in the iteration domain for the linearized dynamics and validate the performance by simulating the nonlinear dynamics.
Due to the increase in volatile power generation facilities, the need for flexible modeling options of an energy network is growing. One approach consists of a cellular architecture whose hierarchy levels are less pronounced. Such an architecture is provided by the Loop Circle Arc theory (LoCA theory). Each cell consists of essentially uniform basic building blocks, such as a storage unit, an energy converter, and a source and load, as well as an interface to the next cell. Based on this theory, a model of N households connected to a Circle is created. In order to report the demand of the connected households to the next cell, the Arc, via the interface, it is necessary to know the summed power values. Since the households generally represent stochastic processes, the densities associated with the households are estimated under the assumption of measured consumption values over a 24-hour period. Using the EM-Algorithm, mixed distribution densities are estimated based on normal distribution densities for each household and superimposed accordingly. In this way, in addition to the expected total power consumption, a variance can be given at the same time. This allows not only an estimation of the energy to be made available at certain times. It is also possible to simplify the network, since the N households can be approximated by the time evolution of the expected overall power consumption values.
Full-text available
The frequency constitutes a key state variable of electrical power grids. However, as the frequency is subject to several sources of fluctuations, ranging from renewable volatility to demand fluctuations and dispatch, it is strongly dynamic. Yet, the statistical and stochastic properties of the frequency fluctuation dynamics are far from fully understood. Here we analyze properties of power-grid frequency trajectories recorded from different synchronous regions. We highlight the non-Gaussian and still approximately Markovian nature of the frequency statistics. Furthermore, we find that the frequency displays significant fluctuations exactly at the time intervals of regulation and trading, confirming the need of having a regulatory and market design that respects the technical and dynamical constraints in future highly renewable power grids. Finally, employing a recently proposed synthetic model for the frequency dynamics, we combine our statistical and stochastic analysis and analyze in how far dynamically modeled frequency properties match the ones of real trajectories.
Full-text available
A novel optimal energy management strategy (EMS) for plug-in hybrid electric vehicle (PHEV) is proposed in this paper, which takes the battery health into consideration for prolonging its service life. The integrated control framework combines batch-wise iterative learning control (ILC) and time-wise model predictive control (MPC), referred as 2D-MPILC. The major advantages of the proposed method are shown with better performance as well as faster convergence speed by taking into account the time-wise feedback control with-in the current batch. Then, the MPILC method is applied for PHEV with the ability to make continuous period-to-period improvements. Its performances will approach to DP-based method after a learning process with a satisfying real-time processing capacity. The results in real-world city bus routines verify the effectiveness of the proposed EMS for greatly improving the performance of PHEV.
Full-text available
Power grids sustain modern society by supplying electricity and thus their stability is a crucial factor for our civilization. The dynamic stability of a power grid is usually quantified by the probability of its nodes' recovery to phase synchronization of the alternating current it carries, in response to external perturbation. Intuitively, the stability of nodes in power grids is supposed to become more robust as the coupling strength between the nodes increases. However, we find a counterintuitive range of coupling strength values where the synchronization stability suddenly droops as the coupling strength increases, on a number of simple graph structures. Since power grids are designed to fulfill both local and long-range power demands, such simple graph structures or graphlets for local power transmission are indeed relevant in reality. We show that the observed nonmonotonic behavior is a consequence of transitions in multistability, which are related to changes in stability of the unsynchronized states. Therefore, our findings suggest that a comprehensive understanding of changes in multistability are necessary to prevent the unexpected catastrophic instability in the building blocks of power grids. © 2018 The Author(s). Published by IOP Publishing on behalf of the Institute of Physics and Deutsche Physikalische Gesellschaft.
Full-text available
Multiple types of fluctuations impact the collective dynamics of power grids and thus challenge their robust operation. Fluctuations result from processes as different as dynamically changing demands, energy trading and an increasing share of renewable power feed-in. Here we analyse principles underlying the dynamics and statistics of power grid frequency fluctuations. Considering frequency time series for a range of power grids, including grids in North America, Japan and Europe, we find a strong deviation from Gaussianity best described as Lévy-stable and q-Gaussian distributions. We present a coarse framework to analytically characterize the impact of arbitrary noise distributions, as well as a superstatistical approach that systematically interprets heavy tails and skewed distributions. We identify energy trading as a substantial contribution to today’s frequency fluctuations and effective damping of the grid as a controlling factor enabling reduction of fluctuation risks, with enhanced effects for small power grids. Electric power grids exhibit frequency fluctuations brought on by changes in demand, trading and intermittency of renewable sources. Schäfer et al. analyse fluctuations in real power grids from North America, Japan and Europe and find deviations from Gaussianity and substantial contributions due to trading.
The economic dispatch problem (EDP) is one of the fundamental and important problems in power systems. The objective of EDP is to determine the output generation of generators to minimize the total generation cost under various constraints. In this article, a finite-time consensus-based distributed optimization algorithm is proposed to solve EDP. It is only required that each device in the communication network has access to its own local generation cost function, designed virtual local demand and its neighbors' local optimization variables. The proposed finite-time algorithm can solve EDP, if the gain parameters in the algorithm satisfy some conditions under undirected and connected time-varying graphs. Moreover, the bounded or linear increasing assumption on the gradient and subgradient of objecive functions is relaxed in this algorithm. Examples under several cases are provided to verify the effectiveness of the proposed distributed optimization algorithm.
Conference Paper
This paper provides the research plan focusing in the application of self-learning techniques for energy systems integration in the residential building sector. Demand response (DR) is becoming increasingly important in the evolution of the power grid since demand no longer necessarily determines system supply but is now more closely constrained by generation profiles. DR can offer energy flexibility services across wholesale and balancing markets. Different applications have focused on IoT in DR to assist customers, aggregators and utility companies to manage the energy consumption and energy usage through the adjustment of consumer behaviour. Even though there is extensive work in the literature regarding the potential of the commercial and the residential building sectors to provide flexibility, to date there is no standardized framework to evaluate this flexibility in a customer-tailored way. At the same time, DR events may jeopardize occupant comfort expectations hindering the utilization of flexibility that buildings energy systems can provide. In this research, the integration of machine learning algorithms into building control systems is investigated, in order to unify the monitoring and control of the separate systems under a holistic approach. This will allow the operation of the systems to be optimized with respect to reducing their energy consumption and their environmental footprint in tandem with the maximization of flexibility and occupant comfort.
Power flow dynamics in electricity grids can be described by equations resembling a Kuramoto model of non-linearly coupled oscillators with inertia. The coupling of the oscillators or nodes in a power grid generally exhibits pronounced heterogeneities due to varying features of transmission lines, generators, and loads. In studies aiming at uncovering mechanisms related to failures or malfunction of power systems, these grid heterogeneities are often neglected. However, over-simplification can lead to different results away from reality. We investigate the influence of heterogeneities in power grids on stable grid functioning and show their impact on estimating grid stability. Our conclusions are drawn by comparing the stability of an Institute of Electrical and Electronics Engineers test grid with a homogenized version of this grid.
We generalize the concept of basin of attraction of a stable state in order to facilitate the analysis of dynamical systems with noise and to assess stability properties of metastable states and long transients. To this end we examine the notions of mean sojourn times and absorption probabilities for Markov chains and study their convergence to the basin of attraction in the limiting cases. Since any dynamical system described by a transfer operator on a compact domain can be approximated by a Markov chain our approach is applicable to a large variety of problems.