Available via license: CC BY 4.0

Content may be subject to copyright.

Citation: Strake, J.; Döhring, D.;

Benigni, A. MGRIT-Based

Multi-Level Parallel-in-Time

Electromagnetic Transient Simulation.

Energies 2022,15, 7874.

https://doi.org/10.3390/en15217874

Academic Editors: José Matas and

Rossano Musca

Received: 29 September 2022

Accepted: 17 October 2022

Published: 24 October 2022

Publisher’s Note: MDPI stays neutral

with regard to jurisdictional claims in

published maps and institutional afﬁl-

iations.

Copyright: © 2022 by the authors.

Licensee MDPI, Basel, Switzerland.

This article is an open access article

distributed under the terms and

conditions of the Creative Commons

Attribution (CC BY) license (https://

creativecommons.org/licenses/by/

4.0/).

energies

Article

MGRIT-Based Multi-Level Parallel-in-Time Electromagnetic

Transient Simulation

Julius Strake1,2,* , Daniel Döhring 1and Andrea Benigni 1,2,3

1Institut für Energie- und Klimaforschung (IEK), Energiesystemtechnik (IEK-10), Forschungszentrum Jülich,

Wilhelm-Johnen-Straße, 52428 Jülich, Germany

2Faculty 4—Mechanical Engineering, RWTH Aachen University, 52056 Aachen, Germany

3JARA-Ofﬁce Jülich, Wilhelm-Johnen-Straße, 52425 Jülich, Germany

*Correspondence: j.strake@fz-juelich.de

Abstract:

In this paper, we present an approach for multi-level parallel-in-time (PinT) electromagnetic

transient (EMT) simulation. We evaluate the approach in the context of power electronics system-level

simulation. While PinT approaches to power electronics simulations based on two-level algorithms

have been thoroughly explored in the past, multi-level PinT approaches have not yet been investigated.

We use the multigrid-reduction-in-time (MGRIT) method to parallelize a dedicated EMT simulation

tool which is capable of switching between different converter models as it operates. The presented

approach yields a time-parallel speed-up of up to 10 times compared to the sequential-in-time

implementation. We also show that special care has to be taken to synchronize the time grids with the

electronic components’ switching periods, indicating that further research into the usage of different

models from adequate model hierarchies is necessary.

Keywords: multi-level; parallel-in-time; MGRIT; simulation; power electronics

1. Introduction

In recent years, partly due to the introduction of larger and more complex converter-

level solutions, the execution speed of power electronics simulations has become a con-

cern [1–3].

To allow for the growing size of analyzed systems, while maintaining a high resolution

in the time domain required by switching devices, many simulations employ techniques of

parallel computing [

4

–

9

], such as parallelization of calculations of different components [

10

,

11

].

Modern power electronics devices are increasingly penetrating power systems, while

requiring even smaller time-steps and, thus, slowing down simulations [12].

Physical systems like these introduce restrictions on the simulation’s time-step and,

thus, slow down computation. Within power electronics simulations, the smallest accept-

able time-step is often determined by switching frequencies of converter devices. Increasing

the complexity of the simulated system results in longer execution times, assuming that

the time-step is already set to the largest meaningful value. This issue is usually addressed

by exploiting parallelism of the model or in the simulation algorithm in some way and,

thus, distributing the computations among multiple computing units. This approach is

supported by the fact that speed-up cannot be expected to originate from semi-conductor

improvements alone, but rather from improving algorithms and hardware architecture

simultaneously [13].

Dynamic simulations, even with a high degree of spatial parallelism, are still in-

herently bounded by the sequential nature of the time-stepping involved. To address

this, even before multicore architectures were the standard, methods for parallelization

of the temporal dimensions have been proposed [

14

], and subsequently, many new tech-

niques have been explored [

15

]. Based on the Parareal algorithm [

16

], published in 2001,

various sophisticated PinT simulation techniques have been developed. In conjunction

Energies 2022,15, 7874. https://doi.org/10.3390/en15217874 https://www.mdpi.com/journal/energies

Energies 2022,15, 7874 2 of 16

with spectral-deferred-correction (SDC) methods [

17

], which apply an iterative solver

to a collocation-like problem [

18

–

21

], hybrid parareal spectral-deferred-correction (SDC)

methods were presented in [

22

,

23

]. Further developments evolved into the parallel full-

approximation scheme in space and time (PFASST) [

24

]. A somewhat different founda-

tion for parallel-in-time (PinT) methods is provided by multigrid-based methods [

25

,

26

].

However, both Parareal and PFASST can be perceived as special cases of the multigrid

approach [

25

], as discussed in [

27

,

28

]. Besides the multigrid-reduction-in-time (MGRIT)

algorithm [

29

], space-time multigrid methods [

30

,

31

] and multigrid wave-form relaxation

methods [32,33] have been developed.

The MGRIT method has been shown to be applicable in high-performance computing

environments, where thousands of computing units are available for simulations [34,35].

At the cost of increasing the overall number of computations, speed-up can be achieved

given a sufﬁcient number of computing units. This serves as the main motivation to adapt

the MGRIT technique developed for the numerical solution of differential equations [

29

] to

the simulation of electronic circuits usually described by differential algebraic systems of

equations (DAEs).

Recently, the parareal approach was shown to be applicable to electromagnetic tran-

sient (EMT) simulations of power systems [

36

], DC and AC/DC grids with device-level

switch modelling [

37

–

39

] and simulations of electric vehicles [

40

]. Further implementations

of a two-level approach have been published [

41

,

42

], showing the continued interest in

parallelization-in-time for power system simulations.

Multi-level approaches have been successfully applied to power system simulations

with scheduled event detection [

43

] and to power delivery networks with non-linear

load-models [44].

The multi-level parallel-in-time (PinT) simulation of device-level, switching-model

converters has not yet been demonstrated to the authors’ knowledge. Since it promises

further speed-up if sufﬁcient computing resources are available, while providing similar

levels of accuracy [

45

], the combination of MGRIT with power electronics simulation and

control will be explored in this publication. We focus on the comparison between sequential,

two-level, and multi-level versions of the same algorithm, analyzing the inﬂuence of further

time-parallel levels beyond the ﬁrst, while exploring the limitations on coarse level time-

step size due to interference with the switching periods of the modelled devices.

Section 2gives an overview of the MGRIT algorithm. Section 3describes the imple-

mented algorithm and test cases. The resulting simulation data are presented in Section 4

along with a comparison with time-sequential simulation techniques. We show that the pre-

sented multi-level approach is able to provide a speed-up of between three and four times

compared to two-level versions, and up to 10 times compared to the fully sequential

version. Section 5concludes the article with some summarizing thoughts, and points to

opportunities for further research.

2. The Parallel-in-Time (PinT) Approach

The well-researched Parareal algorithm [

16

] forms the basis of parallel-in-time sim-

ulations and can be interpreted as a two-level version of multigrid-reduction-in-time

(MGRIT) [

46

]. The basic idea of Parareal is an iteration between a less accurate, but quick,

simulation with long time-steps and calculating the exact solution (on the different time

slices in-between) in parallel, which is, in turn, used to update the approximation on the

coarse grid. Using the MGRIT algorithm, an existing time-stepping scheme can be modiﬁed

to be executed in a

PinT

fashion [

29

,

47

]. Recursive application of such a two-level approach

leads to multi-level variants of MGRIT [29].

In this chapter, we brieﬂy introduce the

PinT

-algorithm MGRIT [

29

,

45

]. We start with

some deﬁnitions and then give an overview of the MGRIT scheme, as presented in [

29

,

46

].

Energies 2022,15, 7874 3 of 16

Assume a given initial value problem (IVP) of the form

˙

x(t) = fx(t),t, for t∈[t0,tf],

x(t0) = x0,(1)

where

x

is an at least once continuously differentiable, complex, vector-valued function of

time

t

,

x0

is its initial value at time

t0

, and

f

describes the ﬁrst derivative of

x

at time

t

in

terms of x(t)and t.

To discretize the simulation interval

[t0

,

tf]

, choose a time-step

δ= (tf−t0)/k

for

some k∈N. With this, we deﬁne the ﬁne time grid

Θδ={ti∈R|ti=t0+iδ, for i=0, . . . , k}.

We now introduce the propagator

φ

, an operator that approximates

x

at time

t+δ

based on a previous value x(t),

x(t+δ)≈φx(t),t.

Starting with the initial condition

x0

, and applying the propagator

φ

iteratively, we

could compute an approximate solution

xi≈x(ti)

, for

i=

0,

. . .

,

k

, of the system of

Equation (1) on the ﬁne time grid Θδ. This approximate solution is given by the sequence

xi+1=φ(xi,ti), for i=0, . . . , k−1. (2)

The above approach describes a standard (sequential) numerical integration method.

For

PinT

, we now introduce a second coarse time grid

Θ∆

with coarse time-step

∆:=cf ·δ

,

where

cf :=k/K

for some

K∈N

is the coarsening factor between the two grids. Then, we

can express the coarse time-step as ∆= (tf−t0)/Kand the coarse time grid as

Θ∆={Tj∈R|Tj=t0+j∆for j=0, . . . , K}.

Here, we introduce the notation

Tj

for denoting points on the coarse time grid. To

distinguish the two grids,

Θδ

will be called the ﬁne grid. Time points that are only on

the ﬁne grid,

ti∈Θδ\Θ∆

, are called F-points. Points on the coarse grid,

ti

,

Tj∈Θ∆

are

called C-points. An illustration of both time grids and the involved notation can be found

in Figure 1.

∆

δ

t0t8=tf

t4

T0=t0T1=t4T2=tf

t1t2t3t5t6t7

Figure 1.

Visualization of the time interval

[t0

,

tf]

and notation for two different time grids. Step sizes

are

δ

(ﬁne grid) and

∆

(coarse grid), the coarsening factor is

cf =

4. C-points (F-points) are drawn as

long (short) vertical lines which deﬁne the time grids Θ∆(Θδ).

Analogous to the ﬁne propagator

φ

, we introduce the coarse propagator

Ψ

. It repre-

sents an integration algorithm that approximates the solutions on the coarse grid,

x(t+

Energies 2022,15, 7874 4 of 16

∆)≈Ψ(x(t)

,

t)

. Starting with

X0=x0

, we can generate the approximate solution

Xj≈x(Tj)on the C-points iteratively:

Xj+1=Ψ(Xj,Tj), for j=0, . . . , K.

In principal, parallelism is achieved in the following way: First, the evolution of the initial

condition

x0=X0

over the coarse grid

Θ∆

is computed sequentially, yielding approxima-

tions

Xj

. Then, these

Xj

serve as initial conditions for the ﬁne-grid propagation on the

F-points in the different time slices,

ti∈[Tj

,

Tj+1)

, which can be computed independently

of one another.

The following paragraph formalizes this general intuition by introducing the MGRIT

algorithm.

Multigrid Reduction in Time

This group of algorithms recursively applies a two-level integration scheme to yield a

multi-level approach and, thus, allows for a higher degree of parallelism compared to purely

parallel-in-space (PinS) approaches [

29

,

46

]. For simplicity, we only present the two-level

version here. Higher-level versions are easily derived from this by recursively introducing

additional time grids (although convergence considerations are more complicated in the

multi-level case [48]).

The approximate solution of the initial value problem (IVP)

(1)

on a given ﬁne time grid

Θδ

with propagator

φ

may be written in a more succinct way. Note that

φ

, as introduced

above, may be a non-linear and explicitly time-dependent function. For simplicity, we

restrict ourselves here to linear and time-independent propagators

φ

and

Ψ

. Thus, we can

rewrite the iterative update given by Equation

(2)

into one simultaneous linear equation

system:

Ax =

1

−φ1

......

−φ1

x0

x1

.

.

.

xk

!

=

x0

0

.

.

.

0

=:g(3)

Introducing the coarse time-grid

Θ∆

and denoting the cf-times successive application

of φwith φcf, we may rewrite the above equation as

A∆X=

1

−φcf 1

......

−φcf 1

X0

X1

.

.

.

XK

!

=

X0

0

.

.

.

0

=:g∆. (4)

Solving this system yields the solution on the coarse grid as approximated by the ﬁne

propagator. Now, replacing each

φcf

in

A∆

by the coarse-grid propagator

Ψ

, we gain the

coarse-grid approximation:

BX =

1

−Ψ1

......

−Ψ1

X0

X1

.

.

.

XK

!

=

X0

0

.

.

.

0

=g∆. (5)

The classical residual-correction method [

49

] forms the basis of the multigrid procedure

for linear systems of equations [

50

]. In the following, we present the residual-correction

method within MGRIT. Let

X(l)

denote the approximate solution after the

l

-th iteration,

Energies 2022,15, 7874 5 of 16

with some initial condition

X(0)

(this may, for example, simply be the initial value

x0

at all

C-points). By deﬁning the residual of the l-th iteration R(l)at the C-points,

R(l):=g∆−A∆X(l), (6)

the coarse grid correction C(l)may be introduced:

C(l):=B−1R(l). (7)

This correction is used to update the states X(l)at the C-points in an iterative manner:

X(l+1)=X(l)+C(l). (8)

Plugging Equations (6) and (7) into (8), the update rule becomes

X(l+1)=X(l)+B−1g∆−A∆X(l), (9)

which can be interpreted as a pre-conditioned stationary iteration. The ﬁne-solution term

A∆X(l)

can be computed in parallel, using the results from the preceding coarse solve as

initial values.

The

(j+

1

)

-th row of Equation

(9)

, with

j=

0,

. . .

,

K−

1, corresponds to a given

C-point and may be written as

X(l+1)

j+1=ΨX(l+1)

j−ΨX(l)

j+φcfX(l)

j

by multiplying Equation

(9)

with

B

from the left. It is easy to see that the ﬁne-grid

approximation is recovered at the C-points if the terms

ΨX(l+1)

j

and

ΨX(l)

j

converge towards

each other as lgrows.

Algorithm 1summarizes the procedure for two levels as published in [

46

]. This two-

level version is equivalent to the Parareal approach [

16

]; recursive application yields a

multi-level integration scheme [

45

]. Note that a (potentially signiﬁcantly) reduced amount

of sequential time-stepping is still needed on the individual levels, at least in between

respective C-points or, on the coarsest level, in the process of computing the residual

R(l)

.

For a maximally possible amount of levels with coarsening factor

cf =

2, the highest

number of successive sequential steps is also two.

Algorithm 1 2-level MGRIT algorithm

1: repeat

2: Propagate approximate solution x(l), cf. Equation (3)

3: Compute residual R(l)on coarse grid, cf. Equation (6)

4: Solve coarse grid correction problem, cf. Equation (7)

5: Correct approximate solution X(l+1)at C-points, cf. Equation (8)

6: until norm of residual Ris sufﬁciently small.

7: Update solution x(l+1)at F-points, cf. Equation (3)

3. Implementation

In this section, we present the conceptual combination of the resistive companion

(RC) method [

51

] with the MGRIT algorithm. We use the MGRIT implementation from

the XBraid software package [

52

]. For simplicity, we implement only a sequential-in-space

version of the RC method, dubbed here resistive companion solver (RCS). This solver

implements some features not commonly found in power electronics simulation software:

Notably, the representation of the system state (including control logic) accommodates the

multi-level nature of the MGRIT approach by allowing for changes of the time-step and

other simulation parameters as it operates. This is enabled, in part, by employing physical

Energies 2022,15, 7874 6 of 16

currents as system variables instead of the current injections resulting from discretizations

of differential equations, which are commonly used in ﬁxed time-step implementations

of electromagnetic transient (EMT)-solvers. This is depicted, e.g., in Equation (4) of [

53

],

where the current injections are usually calculated in a post-step and used directly for

the calculation of the next time-step. Instead, we use the physical current to calculate the

required current injection based on the currently applicable time-step length. This proof-

of-concept sequential-in-space time-stepping scheme can be replaced with parallelized

versions without needing to change the PinT algorithm.

A ﬂowchart of the two-level version of the MGRIT algorithm used in this article is

given in Figure 2.

Version September 28, 2022 submitted to Energies 7 of 16

Start

Read conﬁg

and netlist

Build initial

system state

X(0)

0

,

set

l=0

,

R(0)=0

Calculate vector

of C-point states

X(l)

, correcting

with residual

R(l)

Fine solve on

[T1,T2]

:

ϕcfX(l)

1

Fine solve on

[T0,T1]

:

ϕcfX(l)

0

Fine solve on

[TK−1,TK]

:

ϕcfX(l)

K−1

Calculate resid-

ual

R(l)

2=

X(l)

2−ϕcfX(l)

1

Calculate resid-

ual

R(l)

1=

X(l)

1−ϕcfX(l)

0

Calculate resid-

ual

R(l)

K=

X(l)

K−ϕcfX(l)

K−1

· · ·

· · ·

||R(l)|| ?

<ε

Output

simulation

data

Stop

yes

no

Figure 2. Execution ﬂow-chart of the implemented

PinT

version of the resistive-companion type

solver in the 2-level version.

Figure 2.

Execution ﬂowchart of the implemented

PinT

version of the resistive-companion type

solver in the two-level version.

3.1. Special Considerations for the Solver

To keep track of the different parameters, we use a system-state object that contains all

independent parameters of the system which are subject to change during the simulation.

This includes, but is not limited to, nodal voltages, branch currents, control parameters, such

as current and accumulated error, duty cycles, etc. All of these can easily be overwritten at

any time to allow for the injections necessary in the algorithm.

Energies 2022,15, 7874 7 of 16

Furthermore, special care has to be taken of the controller’s duty cycle. Resolving

this duty cycle with an appropriate time-step is necessary to accurately reproduce the

controller’s behavior in electromagnetic transient situations. Thus, the coarse step should

not be too large.

Note that the combination of methods described here does not lead to a gain in com-

putation speed compared to the sequential implementation for all possible combinations

of parameters. The actual speed-up is highly dependent on multiple factors, such as the

coarsening factor, number of levels, and the test case itself, as will be shown in Section 4

below. This article aims to provide a proof of concept for simulating power electronics in a

multi-level

PinT

fashion. It exhibits speed-ups of up to 10 times compared to sequential

simulation. Theoretically, the potential speed-up is mainly bounded by the number of

available processors in the most simple cases.

The resistive companion solver (RCS) developed for this article is implemented in C++

and uses the C++ interface provided by the XBraid software [

52

]. In order to enable a PinT

execution of the sequential RCS utilizing the XBraid package, only a few additional wrap-

ping routines and data structures have to be provided. This allows for direct comparison of

the unchanged sequential program and the corresponding MGRIT counterpart. For each

MGRIT level, distinct discretization schemes may be used, resulting in a level-dependent

propagator

φ(l)

. Similarly, different techniques for the matrix decomposition may be in-

voked on different levels. At the time of writing, three different LU decomposition methods

from the Eigen3 library [

54

] are available; the implementation of further factorization

techniques, e.g., iterative solvers, is possible.

As is usual for a resistive companion (RC) approach, the modular implementa-

tion allows for dynamically reading in netlist and simulation parameters at runtime.

Among the additional parameters needed for

PinT

execution are the number of differ-

ent levels, the coarsening factors between the levels, and additional options, such as a

halting tolerance for the residual between solutions on different levels.

The setup of the system matrix is performed analogously to classical EMT-type ap-

proaches and will not be shown explicitly here. We refer the interested reader to [

51

] for

more details. Note that the use of an implicit method, such as implicit Euler or the implicit

midpoint-rule, is recommended, since MGRIT is known to perform better with L-stable

methods [55].

3.2. Modeling of Converters

For the converters, both a traditional switching model and an averaged model were

implemented. For the latter, each converter is replaced by a number of voltage sources,

current sources, and resistors. The parameters of these substitute elements are updated in

every time-step to reﬂect the behavior of the emulated component.

For simplicity, a single proportional-integral (PI) controller is used to control either

the output voltage or the output current of the converter. Depending on the switching

frequency

fsw

, the duty cycle is recalculated in periods given by

Tsw =

1

/fsw

. The control

signal is given by

d(t) = kintε(t) + kprop Rt

t0ε(τ)dτ

. The cumulative error given by the

integral is dependent on the history of the system and, thus, needs to be updated properly

respecting the used time-step and previously accumulated error. For a given time-step

δt

,

the integral component is approximated by

εacc(t) = Rt

t0ε(τ)dτ≈εacc(t−δt) + 1

2[ε(t−

δt) + ε(t)].

4. Evaluation

In this section, we present a number of test cases and analyze the performance of our

PinT

solver in comparison with sequential time-stepping. The smaller cases also function

as building blocks for a scalable DC-microgrid test case.

All calculations were executed on a machine with two AMD EPYC 7H12 CPUs with

2.60–3.30 GHz clock speed and 64 cores each. For the parallel calculations, if not otherwise

indicated, 128 processing units were used independent of the number of coarse intervals.

Energies 2022,15, 7874 8 of 16

This was due to the overall execution time being lowest with the maximum available

processors.

The halting criterion used is a relative tolerance of 1

×

10

−4

on the normalized

residuum

||R(l)||

. All given timings are average values from 10 different executions.

Standard deviations of all values are below 0.8% of the given value.

4.1. Pi-Model Line

As a ﬁrst example, we consider a simple pi-model line, as illustrated in Figure 3.

Adding a voltage source supplying a voltage

VS

(with internal resistance

RV

) on the

terminals of capacitor

C1

and a resistive load

RL

on those of

C2

, we obtain a ﬁrst simple

test case. The simulation results in comparison with sequential time-stepping for different

simulation lengths

tf

, coarsening factors

cf

, and number of levels

Nlvl

are summarized

in Table 1. The time-step on the ﬁnest level was chosen as

δ=

1

×

10

−2

s, which was

also used for the sequential simulation. Since our model here is equivalent to a system of

well-behaved ordinary differential equations (ODEs), the results conﬁrm that, as expected,

the MGRIT algorithm converges for all combinations of meta-parameters. While higher

numbers of levels and higher coarsening factors lead to an increase in the required number

of iterations until convergence, the speed-up seems to be more or less independent as single

iterations are faster. For the most effective combination of parameters, a speed-up of about

one order of magnitude can be observed.

RπLπ

CAC1

Vin,A Vout,1

Figure 3.

Schematic of the pi-line. For testing the components individually in a simple test case,

voltage source and resistive load were added at the respective terminals.

Table 1.

Runtime of multi-level

PinT

compared to two-level and sequential time-stepping for the

pi-line test case.

Simulated timespan tf[s]1 10 100

Sequential runtime [s] 0.010 55 0.081 73 0.7922

Parallel runtime and number of iterations

[% of sequential time/% of 2-lvl time/#iter]

Nlvl =2

cf =2 73.82/-/3 50.24/-/3 42.94/-/3

cf =4 50.38/-/3 30.95/-/3 22.66/-/3

cf =10 50.58/-/4 20.43/-/4 12.95/-/4

Nlvl =3

cf =2 50.89/68.94/3 30.30/60.31/3 23.59/54.94/3

cf =4 47.65/94.58/4 16.62/53.70/4 11.70/51.63/4

cf =10 52.21/103.2/4 13.58/66.47/5 11.63/89.81/9

Nlvl =4

cf =2 55.76/75.54/5 25.60/50.96/4 18.14/42.24/4

cf =4 — 13.25/42.81/5 12.16/53.66/6

cf =10 — — 9.567/73.88/8

4.2. Converter

As a second test case, we considered a single leg of a power converter as indicated

in Figure 4. We used the latency-based linear multistep compound (LB-LMC) modelling

approach [

10

] to ensure that our results were compatible with parallel-in-space (PinS)

execution of individual time steps.

Energies 2022,15, 7874 9 of 16

Vin,A

Vin,B

CA

CB

S1

L1,G1Vout,1

C1

Figure 4.

Schematic of the converter with output LC-ﬁlter. For testing the components individually

in a simple test case, voltage source and resistive load were added at the respective terminals.

The simulation results in comparison with sequential time-stepping for different

simulation lengths

tf

, coarsening factors

cf

, and number of levels

Nlvl

are summarized in

Table 2. For the converter, we used a time-step of

δ=

1

×

10

−6

s and a switching frequency

of

fsw =

20

kHz

. Again, we see a higher number of iterations for higher coarsening factors

and numbers of levels, by which the overall reduction in runtime seems unaffected. The

speed-up for the best combination of parameters is about one order of magnitude, as before.

Table 2.

Runtime of multi-level

PinT

compared to two-level and sequential time-stepping for the

converter test case. Deterioration in convergence occurs for cases in which the coarsest time-step and

switching period of the converter are not well-aligned.

Simulated timespan tf[s]0.1 1 10

Sequential runtime [s] 3.909 39.27 394.3

Parallel runtime and number of iterations

[% of sequential time/% of 2-lvl time/#iter]

Nlvl =2

cf =2 136.0/-/3 133.8/-/3 133.8/-/3

cf =5 76.26/-/3 74.00/-/3 73.89/-/3

cf =10 41.83/-/4 38.82/-/4 38.70/-/4

Nlvl =3

cf =2 NC NC NC

cf =5 24.21/31.75/6 19.86/26.84/6 19.67/26.62/6

cf =10 NC NC NC

Nlvl =4

cf =2 NC NC NC

cf =5 NC NC NC

cf =10 NC NC NC

Due to the switching behavior of the converter, we expect impaired convergence for

cases where the maximum time-step is greater than the switching period of the converter

or does not align with it. Considering, for example, the case of

Nlvl =

4 and

cf =

2,

the coarsest time-step would be

∆t=

2

3·δt=

8

µs

. The switching period of

Tsw =

50

µs

is

not an integer multiple of this time-step, and, thus, the coarsest steps do not align with the

switching events. For all combinations of parameters in which the switching period is not

an integer multiple of the coarsest time-step, we see that the approximation on the coarse

grid is not able to appropriately capture the development of the duty cycle and cumulative

error, leading to non-convergence.

Energies 2022,15, 7874 10 of 16

4.3. Microgrid

As a ﬁnal and more comprehensive test case, we consider a residential microgrid.

The schematic of the microgrid can be found in Figure 5. The microgrid is structured

around a DC bus where household, storage and generation units are interfaced by means

of DC/DC converters. The number of household-type elements is not ﬁxed and can be

used to scale the computational burden of the test case (cf. Section 4.5). As an example,

the electrical current for 16 households, ramping up their consumption over one second,

with randomly selected initial times, is shown in Figure 6.

Figure 5.

Schematic of the microgrid. Converters are marked by the label “DC/DC”, while the labels

“Grid”, "Battery”, and “Household” represent simple models of the given elements. Black boxes

represent a pi-model line, as described in Section 4.1.

(a)

Figure 6. Cont.

Energies 2022,15, 7874 11 of 16

(b)

Figure 6.

Simulation results of a 16-household microgrid with ramps in each household for (

a

)

PinT

and (

b

) sequential execution. The simulation was performed with a cold-start (all voltages and

currents equal to zero). The initial transient of the main bus voltage is shown in the lower left plots,

respectively. The development of the current over the whole simulation interval is shown in the

respective upper left plots (note the shift of the abscissa by the target voltage,

1000 V

). The main

bus voltage stays well within an error interval of

(

1000

±

0.1

)V

. The currents in the households are

summarized in the right-hand side plots. Each household ramps up its power consumption from zero

to

8.450 kW

over one second, with randomly chosen switch-on times. (

a

) Results of

PinT

simulation.

Some minor artifacts resulting from iterating only until the chosen error tolerance was reached are

visible. (b) Results of sequential simulation for comparison.

The simulation results in comparison with sequential time-stepping for different

simulation lengths

tf

, coarsening factors

cf

, and number of levels

Nlvl

are summarized in

Table 3. As for the single converter, we used a time-step

δ=

1

×

10

−6

s and a switching

frequency

fsw =

20

kHz

. Non-convergence of the residuals is marked with “NC” for

the applicable parameter combinations. As with the converter model, the reason for this

non-convergence is that, due to the internal switching cycle of the proportional-integral

(PI)-controlled converter, some combinations of coarsening factors

cf

and number of coarse

intervals

Nlvl

lead to control events that fall between coarse time-grid points and, thus,

cannot be calculated correctly on the coarse grids. As in the previous cases, an increase

in the required number of iterations for certain parameter combinations does not lead to

less speed-up.

To test the multi-level capabilities of our approach, a second series of measurements

with a much smaller time-step of

δt=

2.0

×

10

−7

s and

δt=

2.5

×

10

−9

s was performed.

While this is not a step size that would usually be used, the results summarized in Table 4

show that higher amounts of coarse levels are not only possible with the right combination

of parameters but even lead to better speed-ups. Nevertheless, the problem remains that

the coarsest levels have to coincide with the switching intervals. The use of an averaged

model of the controlled converter might resolve the issue but this is not within the scope of

this article.

Energies 2022,15, 7874 12 of 16

Table 3.

Runtime of multi-level

PinT

compared to two-level and sequential time-stepping for the

microgrid test case with four household-type elements. The results summarized here display the

expected behavior of no convergence whenever the coarsest time-step does not align with the

switching period. On the other hand, when alignment is given, the algorithm converges and, for

higher coarsening factors and amounts of levels, speeds up the simulation somewhat.

Simulated timespan tf[s]0.1 1 10

Sequential runtime [s] 10.10 100.4 1000

Parallel runtime and number of iterations

[% of sequential time/% of 2-lvl time/#iter]

Nlvl =2

cf =2 184.2/-/4 184.7/-/4 186.0/-/4

cf =5 95.93/-/6 95.99/-/6 96.74/-/6

cf =10 61.59/-/20 60.83/-/20 61.32/-/20

Nlvl =3

cf =2 NC NC NC

cf =5 32.84/34.23/8 36.69/38.22/8 36.97/38.22/8

cf =10 NC NC NC

Nlvl =4

cf =2 NC NC NC

cf =5 NC NC NC

cf =10 NC NC NC

Table 4.

Runtimes for sufﬁciently small time-step to allow convergence in the microgrid test case.

Time-steps of this size would usually not be used in applications, but the results prove that better

speed-ups are possible if the alignment of coarse step size and switching period is respected.

Timestep δt[s]Sim. time tf[s]Seq. runtime [s]

Par. runtime [% seq. time/#iter]

Nlvl =4

cf =2 cf =5 cf =10

2.0 ×10−70.1 51.63 NC 15.04/33 NC

2.5 ×10−90.1 4109 25.45/3 13.49/4 13.05/7

4.4. Multi-Level Scaling Advantage

The scaling potential of the multi-level approach becomes apparent when reducing

the amount of coarse steps to correspond to the number of processing units available, such

that the available two-level parallelism is exhausted. When using a two-level approach,

the number of processing units that can reasonably be used corresponds to the number

of time slices whose ﬁne solution can theoretically be computed in parallel. This means

that the number of C-points should correspond to the number of processing units for the

optimal two-level speed-up.

A multi-level approach, on the other hand, can enable much more parallelism to

exploit any further processing capabilities.

To illustrate this, we solve the microgrid test case with a two-level and a ﬁve-level

algorithm for simulation durations corresponding to 64, 96, 128, and 256 coarsest time-steps,

respectively. In both the two-level and the ﬁve-level version, the coarsest step size is chosen

to correspond to

∆=

12.5

µs

, while the ﬁnest time-step was set to

δ=

0.78125

µs

. The

two-level version employs a coarsening factor

cf2=

16 to reach

∆

on the coarse level, while

the ﬁve-level version’s uniform coarsening factor was chosen as

cfmulti =

2, resulting in

each level’s step length being twice that of the previous level, and ultimately reaching

∆

on the coarsest level. As mentioned above, we expected the speed-up of the two-level

version to plateau when the number of processes reached the number of coarse levels.

Figure 7a,b

demonstrate that this expected behavior does indeed occur, showcasing the

increased amount of parallelism that multi-level approaches can offer.

Energies 2022,15, 7874 13 of 16

(a) (b)

Figure 7.

Multi-level allows for higher rates of parallelism, resulting in more speed-up in case the

available resources are already exhausted by a two-level

PinT

algorithm. The plots illustrate this

comparing speed-ups between the two-level and ﬁve-level versions of the MGRIT algorithm for

64, 96, 128, and 256 time-steps on the coarsest level. In all cases, a microgrid with four household-

type elements was simulated with a coarsest time-step of

∆t=

12.5

µs

and a ﬁne time-step of

δt=

0.78125

µs

. For the ﬁve-level version, three intermediate coarsening levels with uniform

coarsening factor

cf =

2 were added, resulting in time-steps of

∆tlvl =δt·cflvl

. (

a

) Absolute

runtime per coarse step. For all cases, the multi-level version scales better with increasing number

of processors. (

b

) Illustration of speed-up only resulting from adding processors. Relative runtime

compared to that of the ﬁrst datapoint.

4.5. Scalability of the Test Case

Testing the scalability (in space) of our approach, we added varying amounts of

household-type elements to the microgrid studied in Section 4.3, connected via pi-model

lines, as shown in Figure 5. As meta-parameters, we used

Nlvl =

3 differently spaced time

grids with a coarsening factor of cf =5. The results can be found in Table 5.

Table 5.

Microgrid: Runtime Comparison Between

PinT

and Sequential Timestepping for Different

Grid Sizes.

Households

Runtime

seq. [s] PinT [% seq.]

4 29.71 33.24

8 57.23 32.86

16 148.8 33.72

32 545.0 34.63

64 2798 35.31

The percentage speed-up compared to sequential execution remains stable even for

much larger grids.

While this indicates the workability of our approach, the execution time itself remains

dependent on the grid-size. Different components are still solved sequentially per time-

step, leading to increased execution times on a component level and, thus, higher overall

execution times. To tackle this, parallelization techniques in space, such as the latency-based

linear multistep compound (LB-LMC) approach, are needed. The simultaneous application

of both a PinS and a multi-level PinT method will be analyzed in a future publication.

5. Conclusions

Multi-level approaches provide further opportunities for parallelization when the

per-step parallelity is already fully exhausted. Application to simulations of converters

that are modelled on the switch-level may enable faster-than-real-time simulations of

Energies 2022,15, 7874 14 of 16

power systems at a level of accuracy that, until now, has only been reached by slower

simulation approaches.

In this paper, we have presented a multi-level

PinT

approach for simulating power

electronics devices in DC microgrids. The approach has been shown to provide further

parallelization opportunities and, thus, better scaling with processor number than simple

two-level approaches, while already reaching speed-ups of up to four times compared to

the two-level version when executed with a relatively small number of processing units

(cf. Table 2). With the right meta-parameters, which depend upon the simulated case, we

were able to reduce the

PinT

simulation time to between 33 % and 10 % of the sequential

simulation time, as shown in Tables 1and 3. Overall, a higher coarsening factor and

increased number of levels seem to lead to a higher number of required iterations until the

algorithm converges, but, due to faster iterations, an improved speed-up is still possible,

as shown, for example, in Table 3.

While a good choice of the meta-parameters can increase performance, a poor choice

can also lead to severe deterioration of performance. These cases seem to occur especially

when the internal switching cycle and the coarser time-steps are not synchronized, both for

small and large numbers of switch-level modelled power-converter devices. The use of

averaged models for the coarse propagators may help mitigate these effects and improve

convergence. Applicable model hierarchies that enable larger course-level time-steps are,

thus, an interesting avenue for future research.

Of course, a pure

PinT

approach cannot speed up the individual time-steps. In general,

PinT

approaches are most effectively used together with highly optimized parallel-in-space

(PinS) approaches, when the latters’ potential speed-up is already exhausted and further

computing resources are available. The scaling study in Section 4.5 showcases that the

speed-up via multi-level

PinT

is relatively independent of the system size, which suggests

that any speed-up gained via single-step-based parallel methods will not diminish the

additional potential for

PinT

speed-up. Thus, a combination with the latency-based linear

multistep compound (LB-LMC) method of achieving high spatial parallelism represents a

promising candidate for future research.

Author Contributions:

Conceptualization, J.S. and A.B.; methodology, J.S.; software, J.S. and D.D.;

validation, J.S. and D.D.; formal analysis, J.S.; investigation, J.S.; resources, A.B.; writing—original

draft preparation, J.S. and D.D.; writing—review and editing, J.S. and A.B.; visualization, J.S.; super-

vision, A.B.; project administration, A.B.; funding acquisition, A.B. All authors have read and agreed

to the published version of the manuscript.

Funding:

This research was funded by the Deutsche Forschungsgemeinschaft (DFG, German Re-

search Foundation)—project number 313504828.

Institutional Review Board Statement: Not applicable.

Informed Consent Statement: Not applicable.

Data Availability Statement: Not applicable.

Conﬂicts of Interest:

The authors declare no conﬂict of interest. The funders had no role in the design

of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or

in the decision to publish the results.

References

1.

Lin, N.; Dinavahi, V. Dynamic Electro-Magnetic-Thermal Modeling of MMC-Based DC–DC Converter for Real-Time Simulation

of MTDC Grid. IEEE Trans. Power Deliv. 2018,33, 1337–1347. https://doi.org/10.1109/TPWRD.2017.2774806.

2.

Xu, J.; Zhao, C.; Liu, W.; Guo, C. Accelerated Model of Modular Multilevel Converters in PSCAD/EMTDC. IEEE Trans. Power

Deliv. 2013,28, 129–136. https://doi.org/10.1109/TPWRD.2012.2201511.

3.

Montano, F.; Ould-Bachir, T.; David, J.P. An Evaluation of a High-Level Synthesis Approach to the FPGA-Based Submicrosecond

Real-Time Simulation of Power Converters. IEEE Trans. Ind. Electron.

2018

,65, 636–644. https://doi.org/10.1109/TIE.2017.2716880.

4.

Marti, J.R.; Linares, L.R. Real-time EMTP-based transients simulation. IEEE Trans. Power Syst.

1994

,9, 1309–1317.

https://doi.org/10.1109/59.336135.

Energies 2022,15, 7874 15 of 16

5.

Devaux, O.; Levacher, L.; Huet, O. An advanced and powerful real-time digital transient network analyser. IEEE Trans. Power

Deliv. 1998,13, 421–426. https://doi.org/10.1109/61.660909.

6.

Hollman, J.A.; Marti, J.R. Real time network simulation with PC-cluster. IEEE Trans. Power Syst.

2003

,18, 563–569.

https://doi.org/10.1109/TPWRS.2002.804917.

7.

Lok-Fu, P.; Faruque, M.O.; Xin, N.; Dinavahi, V. A versatile cluster-based real-time digital simulator for power engineering

research. IEEE Trans. Power Syst. 2006,21, 455–465. https://doi.org/10.1109/TPWRS.2006.873414.

8.

Zhou, Z.; Dinavahi, V. Parallel Massive-Thread Electromagnetic Transient Simulation on GPU. IEEE Trans. Power Deliv.

2014

,

29, 1045–1053. https://doi.org/10.1109/TPWRD.2013.2297119.

9.

Le-Huy, P.; Woodacre, M.; Guérette, S.; Lemieux, É. Massively Parallel Real-Time Simulation of Very-Large-Scale Power Systems.

In Proceedings of the IPST Conference IPST2017, Seoul, Korea, 26–29 June 2017.

10.

Benigni, A.; Monti, A. A parallel approach to real-time simulation of power electronics systems. IEEE Trans. Power Electron.

2014

,

30, 5192–5206.

11.

Razik, L. High-Performance Computing Methods in Large-Scale Power System Simulation. Ph.D. Thesis, RWTH Aachen

University, Aachen, Germany, 2020.

12.

Ou, K.; Rao, H.; Cai, Z.; Guo, H.; Lin, X.; Guan, L.; Maguire, T.; Warkentin, B.; Chen, Y. MMC-HVDC Simulation and Testing

Based on Real-Time Digital Simulator and Physical Control System. IEEE J. Emerg. Sel. Top. Power Electron.

2014

,2, 1109–1116.

https://doi.org/10.1109/JESTPE.2014.2337512.

13.

Leiserson, C.E.; Thompson, N.C.; Emer, J.S.; Kuszmaul, B.C.; Lampson, B.W.; Sanchez, D.; Schardl, T.B. There’s plenty of room at

the Top: What will drive computer performance after Moore’s law? Science 2020,368, eaam9744.

14. Nievergelt, J. Parallel methods for integrating ordinary differential equations. Commun. ACM 1964,7, 731–733.

15.

Gander, M.J. 50 Years of Time Parallel Time Integration. In Multiple Shooting and Time Domain Decomposition Methods; Carraro, T.,

Geiger, M., Körkel, S., Rannacher, R., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 69–113.

16.

Lions, J.L. Résolution d’EDP par un schéma en temps “pararéel” A “parareal” in time discretization of PDE’s. CRASM

2001

,

332, 661–668.

17.

Dutt, A.; Greengard, L.; Rokhlin, V. Spectral deferred correction methods for ordinary differential equations. BIT Numer. Math.

2000,40, 241–266.

18.

Canuto, C.; Hussaini, M.Y.; Quarteroni, A.; Thomas, A., Jr. Spectral Methods in Fluid Dynamics; Springer Science & Business Media:

Berlin/Heidelberg, Germany, 2012.

19.

Vlassenbroeck, J.; Van Dooren, R. A Chebyshev technique for solving nonlinear optimal control problems. IEEE Trans. Autom.

Control 1988,33, 333–340.

20. Reddien, G. Collocation at Gauss points as a discretization in optimal control. SIAM J. Control Optim. 1979,17, 298–306.

21.

Speck, R.; Ruprecht, D.; Emmett, M.; Minion, M.; Bolten, M.; Krause, R. A multi-level spectral deferred correction method. BIT

Numer. Math. 2015,55, 843–867.

22.

Minion, M.L.; Williams, S.A. Parareal and spectral deferred corrections. In AIP Conference Proceedings; American Institute of

Physics: Melville, NY, USA, 2008; Volume 1048, pp. 388–391.

23. Minion, M. A hybrid parareal spectral deferred corrections method. Commun. Appl. Math. Comput. Sci. 2011,5, 265–301.

24.

Emmett, M.; Minion, M. Toward an efﬁcient parallel in time method for partial differential equations. Commun. Appl. Math.

Comput. Sci. 2012,7, 105–132.

25. Trottenberg, U.; Oosterlee, C.W.; Schuller, A. Multigrid; Elsevier: Amsterdam, The Netherlands, 2000.

26.

Hackbusch, W. Parabolic multigrid methods. In Computing Methods in Applied Sciences and Engineering; Glowinski, R., VI, Lions,

J.-L., Eds.; North-Holland Publishing Co.: Amsterdam, The Netherlands, 1984.

27.

Gander, M.J.; Vandewalle, S. Analysis of the parareal time-parallel time-integration method. SIAM J. Sci. Comput.

2007

,

29, 556–578.

28.

Bolten, M.; Moser, D.; Speck, R. A multigrid perspective on the parallel full approximation scheme in space and time. Numer.

Linear Algebra Appl. 2017,24, e2110. https://doi.org/10.1002/nla.2110.

29.

Falgout, R.D.; Friedhoff, S.; Kolev, T.V.; MacLachlan, S.P.; Schroder, J.B. Parallel time integration with multigrid. SIAM J. Sci.

Comput. 2014,36, C635–C661. https://doi.org/10.1137/130944230.

30.

Horton, G.; Vandewalle, S. A space-time multigrid method for parabolic partial differential equations. SIAM J. Sci. Comput.

1995

,

16, 848–864.

31.

Gander, M.J.; Neumuller, M. Analysis of a new space-time parallel multigrid algorithm for parabolic problems. SIAM J. Sci.

Comput. 2016,38, A2173–A2208.

32. Vandewalle, S.; Van de Velde, E. Space-time concurrent multigrid waveform relaxation. Ann. Numer. Math. 1994,1, 335–346.

33. Lubich, C.; Ostermann, A. Multi-grid dynamic iteration for parabolic equations. BIT Numer. Math. 1987,27, 216–234.

34.

Falgout, R.D.; Manteuffel, T.A.; O’Neill, B.; Schroder, J.B. Multigrid reduction in time for nonlinear parabolic problems: A case

study. SIAM J. Sci. Comput. 2017,39, S298–S322.

35. Friedhoff, S.; Hahne, J.; Schöps, S. Multigrid-reduction-in-time for Eddy Current problems. PAMM 2019,19, e201900262.

36.

Cheng, T.; Duan, T.; Dinavahi, V. Parallel-in-Time Object-Oriented Electromagnetic Transient Simulation of Power Systems. IEEE

Open Access J. Power Energy 2020,7, 296–306. https://doi.org/10.1109/OAJPE.2020.3012636.

Energies 2022,15, 7874 16 of 16

37.

Pels, A.; Kulchytska-Ruchka, I.; Schöps, S. Parallel-in-Time Simulation of Power Converters Using Multirate PDEs. In Scientiﬁc

Computing in Electrical Engineering; van Beurden, M., Budko, N., Schilders, W., Eds.; Springer International Publishing: Cham,

Switzerland, 2021; pp. 33–41.

38.

Cheng, T.; Lin, N.; Liang, T.; Dinavahi, V. Parallel-in-time-and-space electromagnetic transient simulation of multi-terminal DC

grids with device-level switch modelling. IET Gener. Transm. Distrib. 2022,16, 149–162.

39. Cheng, T.; Lin, N.; Dinavahi, V. Hybrid Parallel-in-Time-and-Space Transient Stability Simulation of Large-Scale AC/DC Grids.

In IEEE Transactions on Power Systems; IEEE: New York, NY, USA, 2022; p. 1. https://doi.org/10.1109/TPWRS.2022.3153450.

40.

Lyu, C.; Lin, N.; Dinavahi, V. Device-Level Parallel-in-Time Simulation of MMC-Based Energy System for Electric Vehicles. IEEE

Trans. Veh. Technol. 2021,70, 5669–5678.

41.

Park, B.; Sun, K.; Dimitrovski, A.; Liu, Y.; Simunovic, S. Examination of Semi-Analytical Solution Methods in the

Coarse Operator of Parareal Algorithm for Power System Simulation. IEEE Trans. Power Syst.

2021

,36, 5068–5080.

https://doi.org/10.1109/TPWRS.2021.3069136.

42.

Cai, M.; Mahseredjian, J.; Kocar, I.; Fu, X.; Haddadi, A. A parallelization-in-time approach for accelerating EMT simulations.

Electr. Power Syst. Res. 2021,197, 107346. https://doi.org/https://doi.org/10.1016/j.epsr.2021.107346.

43.

Schroder, J.B.; Falgout, R.D.; Woodward, C.S.; Top, P.; Lecouvez, M. Parallel-in-time solution of power systems with scheduled

events. In Proceedings of the 2018 IEEE Power & Energy Society General Meeting (PESGM), Portland, OR, USA, 5–10 August

2018; pp. 1–5.

44.

Cheng, C.K.; Ho, C.T.; Jia, C.; Wang, X.; Zen, Z.; Zha, X. A Parallel-in-Time Circuit Simulator for Power Delivery Networks with

Nonlinear Load Models. In Proceedings of the 2020 IEEE 29th Conference on Electrical Performance of Electronic Packaging and

Systems (EPEPS), San Jose, CA, USA, 5–7 October 2020; pp. 1–3. https://doi.org/10.1109/EPEPS48591.2020.9231406.

45.

Friedhoff, S.; Falgout, R.D.; Kolev, T.V.; MacLachlan, S.P.; Schroder, J.B. A Multigrid-in-Time Algorithm for Solving Evolution

Equations in Parallel. In Proceedings of the Sixteenth Copper Mountain Conference on Multigrid Methods, Copper Mountain,

CO, USA, 17–22 March 2013.

46.

Dobrev, V.A.; Kolev, T.; Petersson, N.A.; Schroder, J.B. Two-Level Convergence Theory for Multigrid Reduction in Time (MGRIT).

SIAM J. Sci. Comput. 2017,39, S501–S527. https://doi.org/10.1137/16M1074096.

47.

Günther, S.; Gauger, N.R.; Schroder, J.B. A Non-Intrusive Parallel-in-Time Adjoint Solver with the XBraid Library. Comput. Vis.

Sci. 2018,19, 85–95. https://doi.org/10.1007/s00791-018-0300-7.

48.

Hessenthaler, A.; Southworth, B.S.; Nordsletten, D.; Röhrle, O.; Falgout, R.D.; Schroder, J.B. Multilevel convergence analysis of

multigrid-reduction-in-time. SIAM J. Sci. Comput. 2020,42, A771–A796.

49.

Beilina, L.; Karchevskii, E.; Karchevskii, M. Numerical Linear Algebra: Theory and Applications; Springer: Berlin/Heidelberg,

Germany, 2017.

50.

Hackbusch, W. Multi-Grid Methods and Applications; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013;

Volume 4.

51.

Dommel, H.W. Digital computer solution of electromagnetic transients in single-and multiphase networks. In IEEE Transactions

on Power Apparatus and Systems; IEEE: New York, NY, USA, 1969; pp. 388–399.

52. XBraid: Parallel Multigrid in Time. Available online: https://github.com/XBraid/xbraid(accessed on 24 August 2022).

53.

Wu, B.; Dougal, R.; White, R.E. Resistive companion battery modeling for electric circuit simulations. J. Power Sources

2001

,

93, 186–200.

54.

Guennebaud, G.; Steiner, B.; Larsen, R. M.; Sánchez, A; Hertzberg, C.; Zhulenev, E.; Goli, M.; Tellenbach, D.; Jacob, B.; Margaritis,

K.; et al. Eigen v3. 2010. Available online: http://eigen.tuxfamily.org (accessed on 25 January 2022).

55.

Friedhoff, S.; Southworth, B.S. On “Optimal” h-independent convergence of Parareal and multigrid-reduction-in-time using

Runge-Kutta time integration. Numer. Linear Algebra Appl. 2021,28, e2301.