Conference PaperPDF Available

Towards Proper Assessment of QP Algorithms for Embedded Model Predictive Control

Authors:

Abstract and Figures

With model predictive control (MPC) becoming a viable approach for advanced feedback control at very fast sampling times, a plethora of methods for solving quadratic programming (QP) problems on embedded computing hardware has been proposed. While most of these methods seem to be useful and superior to competing approaches on particular problem instances, very little effort has been put into a proper benchmarking on a non-trivial number of MPC problems. This paper is intended to help filling this gap by (i) briefly discussing the most important aspects for assessing the suitability of a certain QP method for an MPC problem at hand, (ii) providing a concise overview of about a dozen different QP algorithms that have been proposed for use in MPC, (iii) describing a general benchmarking framework for comparing the numerical performance of the different QP algorithms, and (iv) providing preliminary benchmarking results based on different performance metrics. Numerical performance of the various algorithms is assessed by means of a suitable collection of benchmark problems, both taken from academic studies and industrial applications of MPC.
Content may be subject to copyright.
Towards Proper Assessment of QP Algorithms
for Embedded Model Predictive Control
D. Kouzoupis, A. Zanelli, H. Peyrl, H. J. Ferreau
Abstract With model predictive control (MPC) becoming
a viable approach for advanced feedback control at very fast
sampling times, a plethora of methods for solving quadratic
programming (QP) problems on embedded computing hard-
ware has been proposed. While most of these methods seem to
be useful and superior to competing approaches on particular
problem instances, very little effort has been put into a proper
benchmarking on a non-trivial number of MPC problems. This
paper is intended to help filling this gap by (i) briefly discussing
the most important aspects for assessing the suitability of
a certain QP method for an MPC problem at hand, (ii)
providing a concise overview of about a dozen different QP
algorithms that have been proposed for use in MPC, (iii)
describing a general benchmarking framework for comparing
the numerical performance of the different QP algorithms,
and (iv) providing preliminary benchmarking results based on
different performance metrics. Numerical performance of the
various algorithms is assessed by means of a suitable collection
of benchmark problems, taken from both academic studies and
industrial applications of MPC.
I. INTRODUCTION
Model predictive control (MPC) is an advanced feedback
control strategy that allows for direct handling of constraints
and is well-suited for multi-variable control [1]. It requires
to solve a constrained optimization problem online, which
has been considered a main practical drawback for apply-
ing MPC to processes with fast dynamics. However, this
drawback has become less and less important since the
first industrial use of MPC in the late 1970s. On the one
hand, computational power has increased by many orders
of magnitude during the last decades and still keeps on
growing. On the other hand, highly efficient and reliable
numerical algorithms have been developed for solving the
resulting optimization problems in real-time. These advances
made it possible to solve linear MPC problems of nontrivial
sizes within microseconds, even on computing hardware with
limited resources.
Despite this huge progress, practitioners still need to take
a lot of care in order to select a suitable MPC algorithm
that meets their requirements. This is because most of the
recently proposed MPC algorithms perform particularly well
for certain problem setups, while they tend to be unneces-
sarily slow in other situations. Moreover, all algorithms have
Dimitris Kouzoupis is with Institute of Microsystems Engineering
(IMTEK), University of Freiburg, Georges-Koehler-Allee 102, D-79110
Freiburg, Germany.
Andrea Zanelli is with Automatic Control Laboratory, ETH Zurich,
Physikstrasse 3, CH-8092 Zurich, Switzerland.
Helfried Peyrl and Hand Joachim Ferreau are with ABB Corporate
Research, Segelhofstrasse 1K, 5405 Baden-D
¨
attwil, Switzerland, e-Mail:
joachim.ferreau@ch.abb.com
certain implementation characteristics that may favour one
over the other for a given hardware configuration.
This paper aims at assessing the suitability of different
MPC algorithms in a structured way. For the present study,
we focus on linear MPC formulations of the following form:
min
x, u
N1
X
k=0
y
k
y
r
k
u
k
u
r
k
T
Q
k
S
k
S
T
k
R
k

y
k
y
r
k
u
k
u
r
k
+
g
y
k
g
u
k
T
y
k
y
r
k
u
k
u
r
k
+ (x
N
x
r
N
)
T
P (x
N
x
r
N
) (1a)
s. t. x
0
given, (1b)
x
k+1
= A
k
x
k
+ B
k
u
k
+ f
k
k {0, . . . , N 1} , (1c)
y
k
= C
k
x
k
+ D
k
u
k
+ e
k
k {0, . . . , N 1} , (1d)
u
l
k
u
k
u
u
k
k {0, . . . , N 1} , (1e)
y
l
k
y
k
y
u
k
k {0, . . . , N 1} , (1f)
d
l
k
M
k
y
k
+ N
k
u
k
d
u
k
k {0, . . . , N 1} , (1g)
d
l
N
T x
N
d
u
N
. (1h)
Herein, x
k
R
n
x
, y
k
R
n
y
and u
k
R
n
u
denote the
differential state, the process output and the control input
at the discrete time instant k, respectively. We assume an
affine linear time-varying dynamic model which is subject
to linear constraints that might also change in time. Aim is
to track possibly time-varying reference values y
r
k
and u
r
k
,
while we assume the weighting matrices Q
k
R
n
y
×n
y
and
R
k
R
n
u
×n
u
to be symmetric and positive semi-definite.
Moreover, a symmetric positive definite penalty P R
n
x
×n
x
as well as linear constraints on the terminal state x
N
may
be imposed for stability reasons.
Formulation (1) generalizes common linear time-invariant
(LTI) MPC problem formulations to affine linear time-
varying ones as they may arise within nonlinear MPC
algorithms when linearizing the problem at the current it-
erate. Also MPC problems based on a u formulation can
be posed within this framework by introducing auxiliary
differential states. Nevertheless, note that the vast majority
of the benchmarking problems considered in this study are
based on LTI formulations.
Problem formulation (1) is a specially structured quadratic
programming (QP) problem and many different algorithms
have been proposed to exploit this structure for solving it
efficiently. Among them are interior-point methods [2], [3],
[4], active-set methods [5], [6], [7], gradient methods [8], [9],
[10] and explicit methods [11], [12]. When discussing their
respective numerical properties, we will focus on small- to
medium-scale MPC problems that are intended to run on
“embedded” controller hardware. Such hardware typically
features rather limited resources in terms of memory, clock
speed or number representation (see Section II-C) and re-
quires the QP/MPC algorithm to run highly reliably without
any user-interaction. The present work discusses properties
of a variety of different algorithms that are suited for at
least some kind of embedded hardware and assesses their
respective numerical performance in a PC-based framework.
Experimental comparisons on actual embedded devices is
envisaged but beyond the scope of the current study.
This paper is organized as follows: Section II briefly
discusses the most important aspects for assessing the suit-
ability of a certain QP method for an MPC problem at
hand. Section III provides a concise overview different
QP algorithms that have been proposed for use in MPC.
Section IV presents a general benchmarking framework for
comparing the numerical performance of these algorithms
based on different performance metrics, while preliminary
benchmarking results are provided in Section V. Section VI
concludes the paper and points towards directions of future
research.
II. PROPERTIES OF EMBEDDED QP ALGORITHMS
This section reviews the main properties of QP algorithms
that are important when assessing their suitability for a given
embedded controller setup.
A. QP Formulation
Problem formulation (1) is a specially structured QP
problem that can be written in the following form:
min
z
1
2
z
T
H
s
z + g
T
s
z + c
s
(2a)
s. t. B
s
z = b
s
(x
0
) (2b)
a
l
s
A
s
z a
u
s
(2c)
z
l
s
z z
u
s
, (2d)
where the vector of optimization variables z
def
= (x, u) com-
prises both the differential states and the control inputs.
The Hessian matrix H
s
is symmetric and positive semi-
definite, i.e. QP problem (2) is convex (provided S
k
is chosen
appropriate, e.g. S
k
= 0). All matrices H
s
, B
s
and A
s
are
sparse with a well-known block-structure. Also note that
problem (2) is parametric in the initial differential state x
0
and possibly further quantities that enter the formulation
linearly.
It is also well-known that all state variables are actually
uniquely determined by the initial state x
0
and the control
inputs u. Thus, it is possible to eliminate all states from
formulation (2) to yield a smaller-scale, condensed QP
problem of the form:
min
z
1
2
z
T
H
d
z + g
d
(x
0
)
T
z + c
d
(x
0
) (3a)
s. t. a
l
d
(x
0
) A
d
z a
u
d
(x
0
) (3b)
z
l
d
z z
u
d
, (3c)
where the vector of optimization variables z
def
= u now only
comprises the control inputs. However, by doing so, the QP
matrices H
d
, A
d
become dense.
Whether it is computationally more favorable to work with
the sparse formulation (2) or the condensed formulation (3)
mainly depends on the number of states n
x
, control inputs n
u
and the length of the control horizon N. By analyzing the
number of non-zero entries in the respective QP matrices,
one can derive the following rule-of-thumb: The sparse
formulation (2) is clearly preferable when a) the number of
states is small, b) the ratio
n
x
n
u
is small (i.e. a relatively large
number of control inputs) and c) the control horizon N is
long. In the opposite case, i.e. when there are relatively few
control inputs and the control horizon is short, the condensed
formulation (3) is typically more efficient. Moreover, the best
choice may depend on whether state-elimination only needs
to be done once in a pre-processing step (as it is the case
in an LTI setup), or whether this step needs to be performed
online as in a nonlinear MPC setup. For the latter case,
a quantitive comparison of both formulations based on an
academic nonlinear MPC case study can be found in [13].
Most proposed MPC algorithms are tailored to either of
these two QP formulations, which may already rule out cer-
tain algorithms in terms of efficiency if the MPC formulation
strongly favours one of these two QP formulations.
B. Further Numerical Aspects
When assessing suitability of a given MPC algorithm in
general or for a specific MPC problem at hand, further
numerical aspects need to be considered.
First, it is possible to classify MPC problems into different
problem classes depending on the following criteria:
Type of inequality constraints
1
: whether problem (1)
comprises a) only input bounds (1e), or b) also output
bounds (1f) or c) also general polytopic constraints
(1g)–(1h), has a significant impact on the choice of
algorithm. Several of the proposed MPC algorithms
cannot handle all types of constraints efficiently.
Type of objective function: if all matrices Q
k
, R
k
, P
are positive definite, S
k
is chosen appropriately and C
k
has rank n
x
, the resulting QP is strongly convex, which
is a common assumption for most MPC algorithms to
work reliably. If this is not that case, one may need to
regularize the QP problem, though it is not always trivial
to find a good balance between the original control
objectives and a reasonable numerical conditioning.
System dynamics: if system dynamics are unstable, the
dense QP problem formulation (3) tends to become ill-
conditioned. Ill-conditioning of both sparse and dense
formulations may also be caused by unbalanced weights
in the objective function or different timescales in the
process dynamics.
Constraint feasibility: if the problem formulation not
only comprises bounds on the control inputs but also
more general inequality constraints as in Eqs. (1f)–(1h),
it can be difficult to guarantee upfront that all resulting
QP problems remain feasible. In that case, it may be
1
We do not consider unconstrained MPC problems that do not comprise
any inequality constraints (1e)–(1h), as they are usually trivial to solve.
crucial to employ an algorithm that can detect infeasible
QP problems reliably in order to implement a backup
strategy. Alternatively, turning some of the inequalities
into soft constraints, e.g. by introducing slack variables
and a suitable penalty function, can guarantee that all
QP instances remain feasible
2
.
Ill-conditioning of the problem makes all algorithms more
sensitive to the finite accuracy of any practical number
representation. But it is important to note that there are
algorithms that are hardly affected while others may simply
fail to converge if conditioning becomes too bad. Also note
that the condensed QP formulation (3) is more susceptible
to accumulate numerical errors than the sparse one if N
becomes large.
Another obvious criterion when comparing different MPC
algorithms is their numerical performance in terms of com-
putational speed. Here, worst-case execution times are of
highest importance as they ensure that the algorithm can run
online reliably within each sampling period. However, also
average execution can be of significance for several reasons:
a) if feedback can be given immediately once computation
has finished (without waiting for the next sampling instant),
a faster feedback may lead to improved control performance;
b) a lower average computational load may be favorable in
terms of power consumption of the processing unit, which
often is an important aspect for embedded controllers; c)
multi-threaded controllers may use the idle time to perform
background tasks of low priority. Warm-start techniques, that
re-use solution information of previous problem instances,
almost always reduce average computational load, whereas
they often do not improve worst-case performance a lot.
A related question is whether an algorithm tends to re-
quire a rather constant number of iterations or whether the
number of iterations typically varies significantly. Moreover,
some algorithms may provide a reasonably tight theoretical
performance guarantee, while the worst-case performance of
others need to be assessed in a heuristical way.
Finally, it is of uttermost importance that MPC algorithms
run reliably without any user-interaction. This requires both
an MPC algorithm that is known to solve all problem
instances of the given MPC formulation from a theoretical
point of view and a carefully designed, developed and tested
source code implementation.
C. Implementation Aspects
Even though a certain MPC algorithm might be a perfect
choice in view of its numerical properties, there are a number
of further implementation aspects to be considered when
choosing an algorithm for a given control application. In
particular, the target hardware of the controller is of great
importance. For example, the following embedded hardware
setups may occur in practice:
PC-like single or multi-core processor,
2
The benchmarking framework considered in this paper allows the user
to introduce soft constraints, but we do not further elaborate on this feature
for sake of brevity of presentation.
microcontroller with RISC processor and limited
caching capabilities,
field programmable gate array (FPGA),
graphics processing unit (GPU),
hybrid architectures combining some of the above-
metioned hardware.
Traditionally, MPC algorithms were run sequentially on a
single-core processor. However, as multi-core processors are
becoming more wide-spread even for embedded computing
devices, it may become crucial to employ an algorithm that
can be run in parallel with a reasonable speed-up. This
becomes even more obvious when implementing algorithms
on an FPGA where computations are inherently parallel
and exploiting this fact is crucial for efficiency in order to
compensate for the slower clock speed compared to PC-like
hardware. Moreover, certain operations such as divisions or
computation of square-roots are particularly demanding on
FPGAs. Also GPUs offer great opportunities to dramatically
speed-up computations by means of massive parallelization.
GPUs used to exhibit relatively high communication laten-
cies, but recent hardware developments may render them
appropriate for high-speed applications with sampling times
in the order milliseconds or below. All of these embedded
hardware platforms have in common that both clock speed
and, in particular, memory size are usually significantly lower
than that of a desktop PC.
It is also worth mentioning that the choice of programming
language may be of great relevance in practical setups. Even
though implementations in C or VHDL are the de facto stan-
dard in academic implementations, industrial practice may
require implementations according to the IEC standard 61131
(for programmable logic controllers) or embedded MATLAB
(in order to facilitate code maintenance), for example. Auto-
generated algorithms can be an interesting option to provide
the same algorithm in different programming languages, but
may also increase the effort to validate the generated code.
Finally, the amount of source code required to code the
algorithm is of interest, where smaller code sizes are obvi-
ously preferable for a number of reasons, e.g. reduced devel-
opment/maintenance effort, simplified validation as likeliness
of bugs is reduced, and reduced object code size (which can
be critical on low-cost hardware).
III. EXISTING METHODS FOR EMBEDDED QP
Many different algorithms for solving QP problems arising
in MPC applications have been proposed. Some of them only
in form of a conceptual description of the method along with
a limited number of numerical show cases, others in form
of publicly available or open-source code. We briefly survey
the ones that seem to have had most impact in the academic
community, which naturally favours the ones that exist as
publicly available software implementations.
Algorithmically, one usually distinguishes between so-
called explicit methods and iterative methods. The former
pre-compute for a given MPC formulation all potentially
arising QP problems such that the only computational task
to be done online is to look-up the solution corresponding to
current system state in an efficient way. The multi-parametric
toolbox (MPT) [14] is a mature and popular MATLAB toolbox
that implements various explicit methods such as multi-
parametric quadratic programming and variants that aim at
providing an only approximative solution to the QP problems
at the benefit of reduced off-line or online complexity.
In contrast, iterative methods solve each QP problem
online and most of them can be naturally classified from an
algorithmic point of view according to the way they handle
the inequality constraints (1e)–(1h).
Gradient methods iteratively compute a step towards the
solution of the unconstrained QP problem and then project
this step onto the feasible set. As this projection is difficult
in case of output or polytopic constraints (in fact, exact
projection may be as difficult as solving a full QP prob-
lem), several variants to dualize some parts of the prob-
lem formulation have been proposed: the MATLAB toolbox
FiOrdOs
3
[16] auto-generates code for either primal fast
gradient methods (if only simple constraints are present)
or dual fast gradient methods by applying a Lagrangian
relaxation to equality and polytopic constraints as proposed
in [8]. The GPAD algorithm proposed in [9] dualizes only
the inequality constraints, which allows to either eliminate or
keep the states in the primal problem. Recently, generalized
versions of these fast gradient methods were proposed by
Giselsson [10] and shown to require a significantly lower
number of iterations on certain problems. Also gradient
methods based on inexact first-order information have been
proposed [17] and implemented in the open-source toolbox
DuQuad [18].
Active-set methods are based on the fact that solving an
equality-constrained QP problem is equivalent to solving a
single linear system, which is rather trivial. Thus, they start
with a guess which inequality constraint will hold with equal-
ity at the optimal solution and solve the resulting equality-
constrained QP problem. If it turns out that this guess was
not correct, it is updated until the optimal solution is found.
MATLABs quadprog function implements a primal active-
set method. Dual active-set methods are available in the code
qpas [5] and QPSchur [6]. A parametric active-set strategy
designed for use in MPC is implemented in the open-source
software qpOASES [7], [19].
The basic idea of interior-point methods is to remove the
inequality constraints from the formulation and to penalize
constraint violations in the objective function instead. How-
ever, in order to make this work reliably, a non-quadratic
penalty term (e.g. logarithmic) needs to be used. The re-
sulting equality-constrained nonlinear programming (NLP)
problem is then solved by means of Newton’s method. Struc-
ture exploiting interior-point methods have been proposed
for use in MPC by several authors in the literature. The
generic QP solver CVXGEN [3] exploits data sparsity by pre-
computing a permutation such that the LDL
T
factorization
of the linear system underlying the Newton step is sparse.
3
Presentation and results are based on version 1.2; the most recent version
2.0 also features a novel primal-dual approach [15].
While CVXGEN exploits sparsity at a general level, efforts
were made to specifically exploit the block-banded struc-
ture arising from the sparse MPC formulation (2b): in [2],
the linear equation system underlying the Newton step is
solved using Riccati recursions while the primal method
presented [20] employs a block-wise Cholesky factorization.
The latter paper inspired the development of the primal-dual
solver FORCES [4] (currently being replaced by FORCES
PRO), which can handle generic multi-stage control problems
and also supports quadratic inequalities. Further efficient
implementations have been proposed in [21], [22] and also
MATLABs quadprog offers an interior-point method as
default choice for solving larger-scale QPs.
From an implementation point of view, a valuable al-
ternative classification asks for the complexity of a single
iteration. Simple iteration algorithms only perform matrix-
vector multiplications or even more basic operations in each
step. In contrast, complex iteration algorithms require in each
step the solution of a linear system of equations (hence some
kind of matrix inversion technique), which is computation-
ally more demanding and potentially numerically less robust
(in particular in fixed-point arithmetic).
Simple iteration algorithms comprise the above-mentioned
gradient methods as well as the multiplicative update dual
optimization algorithm PQP [23] (whose scheme is simple,
highly parallelizable but not division-free). Also the alter-
nating direction method of multipliers (ADMM) [24], that
recently started to draw a lot of attention from the MPC
community falls into this category (see [25] for a tutorial in-
troduction). In contrast, interior-point and active-set methods
as well as the dual Newton-type code qpDUNES [26] or the
piecewise-smooth Newton method proposed in [27] can be
classified as complex iteration algorithms.
Table I summarizes some of the numerical properties of
MPC algorithms that have been coupled to the benchmarking
suite presented in Section IV. Namely, it describes the type
of inequalities (input, output, polytopic) that can be handled,
the type of objective functions (convex, strongly convex (or
at least on a reduced set of variables, s
)), typical variations
in number of iterations (high ‘+’, medium ‘o’, low ‘–’) and
the type of QP formulation that can be dealt with (sparse (2),
dense (3)).
Table II summarizes implementation aspects of the dif-
ferent MPC algorithms in a rather rough way. Namely, it
mentions the programming language, gives some indication
on the code size (large, medium, small) of the respective
algorithm and rates its suitability for parallelization or im-
plementation in fixed-point arithmetic (high ‘+’, medium ‘o’,
low ‘–’).
IV. A GENERAL BENCHMARKING SUITE
A major shortcoming of most publications introducing
novel or surveying existing MPC algorithms is the lack of
sufficiently detailed numerical assessment or comparisons. It
is rather typical to illustrate solver performance on just one
or two academic examples. While this is perfectly justified
TABLE I
SUMMARY OF NUMERICAL PROPERTIES OF DIFFERENT MPC
ALGORITHMS (SEE SECTION III FOR NOTATION).
ineq. obj. var. iter. formulation
ADMM i/o/p c + s/d
primal FiOrdOs i c + d
dual FiOrdOs i/o/p s + s/d
FORCES i/o/p c s
GPAD i/o/p s
+ s/d
MPT i/o/p c o d
PQP i/o/p s + d
qpDUNES i/o/(p) s o s
qpOASES i/o/p c o d
quadprog i/o/p c o s/d
TABLE II
SUMMARY OF IMPLEMENTATION ASPECTS OF DIFFERENT MPC
ALGORITHMS (SEE SECTION III FOR NOTATION).
lang. code size parallel. fixed-point
ADMM C s + +
primal FiOrdOs C s + +
dual FiOrdOs C s/m + o
FORCES C m/l o
GPAD C s + +
MPT C s/m/l + +
PQP C s + o
qpDUNES C m o
qpOASES C/C++ m
quadprog Matlab m
when discussing theoretical properties of the method in
question, it tells little about its practical usefulness.
Practitioners may be faced with either of the two following
situations: on the one hand, one may need to tackle a specific
MPC application and looks for assessing the performance
of many different algorithms on this particular problem
instance, preferably by exploiting problem-specific structure
as much as possible. On the other hand, one may want to
employ MPC for a range of problems and needs to assess
whether there is a single approach that can satisfy perfor-
mance requirements on all of these problems or whether
different (classes of) problems ask for different solvers. In
order to setup a proper assessment in both of these cases, a
benchmarking suite needs to be designed in a general way
such that it accommodates the use of many different MPC
algorithms for solving a large variety of MPC problems.
This section introduces a general MATLAB-based bench-
marking suite that is currently being developed within ABB
Corporate Research and also discusses guidelines on how to
run such an assessment in a fair way (though full fairness
is very difficult to achieve). It is based on the presentation
given in [28, Ch. 5].
A. Collection of Benchmark Problems
The benchmarking suite is intended to collect many dif-
ferent MPC problem formulations arising from different
application domains and with a great variety of properties.
All MPC problems are described in a common way by an
TABLE III
OVERVIEW OF ACADEMIC AND INDUSTRIAL BENCHMARK PROBLEMS.
n
x
n
u
N constr. stable? #QPs
AIRCRAFT 4 2 10 i/o no 240
BALLONPLATE 2 1 25 i/o yes 150
DCMOTOR 4 1 20 i/o/p yes 180
TOYEXAMPLE 2 1 20 i yes 200
FORCESEXAMPLE 2 1 10 i/o no 100
HELICOPTER 6 2 3 i/o yes 150
NONLINEARCSTR 4 2 20 i yes 80
PENDULUM 3 1 15 i no 150
QUADCOPTER 12 4 20 i/o/p yes 150
SHELL 9 3 40 i/o yes 150
SPACECRAFT 7 4 10 i/o no 500
SPRINGMASS 6 2 20 i/o no 300
COMPRESSOR 5 2 100 i yes 100
NONLINEARDRIVE 5 4 10 i/o yes 400
appropriate MATLAB data structure. This structure allows
to formulate a general, possibly time-varying affine linear
MPC formulation (1) with the additional possibility to have
a control grid that is coarser than the discretization grid of
the differential state (thus allowing for a control horizon of
length N
c
< N or move blocking). The MPC formulation
is complemented by a certain control scenario, e.g. a se-
quence of initial states or changes in the reference values or
constraint limits.
The benchmark problems originate from three different
types of sources:
1) academic examples presented in scientific publications,
2) industrial examples and case studies,
3) randomly generated examples.
Randomly generated examples are commonly used for
benchmarking purposes. However, there is evidence that real-
world system dynamics typically exhibit much more struc-
ture such as integrating behaviour or partially independent
sub-modules that make them behave significantly different
from naively generated random examples. Moreover, ran-
domly generated examples tend to be well-conditioned, while
realistic problems often inherit ill-conditioning as sketched
in Section II-B. Consequently, the focus is on collecting
as many academic and industrial benchmark examples as
possible and randomly generated ones are purely provided
as a reference.
The results shown in this paper are based on twelve
academic examples that can be found in literature and two
examples from industrial research projects (i.e. results from
random problem instances are not included). For each of
these examples, we have setup different control scenarios,
each resulting in a sequence of about 50 to 500 QP problems.
An overview of all benchmark problems used for the present
study is given in Table III, which collects some of their key
properties (where N
c
= N for most examples). We intend
to further extend this collection and to make it publicly
available to other researchers (see [29]).
B. Solving the MPC Problems
All benchmark problems are given as a general MPC
formulation (1) and may need further pre-processing before
actually solving them. This can be done by either:
directly passing the MPC formulation to the respective
solver;
minor pre-processing to obtain a structured sparse QP
problem similar to (2) (see [28] for more details) such
that it can be passed to a solver capable of tackling
sparse QP formulations;
elimination of all differential states to obtain a dense QP
problem similar to (3) such that it can be passed to a
solver capable of tackling condensed QP formulations.
Any of these steps may be complemented by calling the
respective software tool to obtain and compile auto-generated
code implementing a customized linear MPC algorithm.
Afterwards, the MPC problem may be actually solved in
either an open-loop or closed-loop mode. In the first case,
one assumes that all QP problems are given beforehand in
order to make sure that all solvers face exactly the same
QP problems. In the second case, only the very first initial
differential state x
0
is given and all further initial states
are obtained by simulating closed-loop (potentially including
plant-model mismatch) using the first piece of the optimized
control input trajectory. This second way is more suited to
assess the actual control performance of a given controller
setup but renders direct comparisons of different solvers at
the level of QP solutions less meaningful.
For solving the benchmark problems, about a dozen dif-
ferent MPC algorithms have been coupled to the bench-
marking suite so far. We have either interfaced existing,
publicly available codes (such as MPT, FiOrdOs, FORCES,
qpDUNES, qpOASES, quadprog) or prototyped simple
schemes ourselves as C or embedded Matlab code (such
as primal/dual fast gradient methods (FGM) in different
flavours, two variants of the ADMM method, or the PQP
algorithm). For all of these solvers, different algorithmic
options can be set such as defining the maximum number
of iterations or enabling/disabling warm-starts of the QP
solution.
C. Comparing Performance
Comparing different QP solvers in a fair way is not a trivial
procedure and most of the times the comparison strongly
depends on the application. For instance, as discussed in
Section II, there are cases where the solver is given a fixed
amount of time to solve an optimization problem and others
where an early termination is considered advantageous. To
offer a wide range of options, the benchmarking suite ends
the iterative process in one of the following ways:
Solver’s own termination condition. A reliable stopping
criterion is an important property of the solver. Running
a simulation where each solver is stopped by its own
stopping criterion might seem trivial but typically this
condition depends on one or more tuning parameters
(such as tolerances) and the different nature of the
criteria renders a direct comparison difficult. A fair way
to compare solvers with different stopping criteria is
proposed in [30]. Therein, the authors suggest to first
solve the QP problem with a default tuning and, if
the solution is not accurate enough, repeatedly adjust
the available tolerances until the desired accuracy is
reached, or the solver exceeds the maximum compu-
tation time.
Fixed number of iterations. In many applications, a fixed
amount of time for solving the optimization problem
might be provided and the solver is called to return
a solution with the highest possible accuracy. In this
setup, the computational burden of checking the termi-
nation condition in each iteration (or every n iterations)
is skipped and the solvers perform a fixed number of
iterations. Of course, for a fair comparison, the number
of iterations must be scaled according to the cost-per-
iteration of each solver.
Known a priori optimal solution. Another common
way to compare the efficiency of different solvers
is to measure how fast they converge to a known
optimal solution. In the case of a strongly convex QP,
the solution is unique and therefore a metric based
on the relative error (and feasibility) of the solution
vector can be used to stop the solver. If the problem is
not strongly convex, the relative error in the optimal
objective value can be compared.
V. ILLUSTRATIVE RESULTS
We use the benchmarking suite to compare the numerical
performance of different MPC algorithms on a collection of
academic and industrial examples as listed in Table III. All
computations have been performed using MATLAB R2013b
on an Intel Core i7 processor with eight cores running at
2.4 GHz each and 16 GB of RAM. Automatic performance
assessment on embedded controller hardware is currently
under development.
We do not aim at drawing final conclusions on the com-
petitiveness of the different algorithms, but rather present
preliminary results that illustrate different possibilities to
profile the algorithms.
A. Performance on Benchmark Collection
We first assess the relative runtimes of different MPC
algorithms by means of performance graphs [31]. This is
a commonly used way to compare a number of solvers
on a large number of benchmark problems. It depicts what
fraction of the test problems (ordinate) are solved by each
solver within a certain time factor (abscissa) of the fastest
method on each problem. Generally, fast methods have their
graph in the left hand part, reliable methods in the upper part
of the graph.
The performance graphs are defined as follows: given a
set of problems P and a set of solvers S, let t
p,s
denote
the time
4
solver s S needs for solving problem p P.
4
or any suitable quantity such as number of iterations or solution accuracy
0 1 2 3 4 5 6 7 8
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
τ
ρ
s,2
(τ)
FiOrdOs
FORCES
qpOASES
quadprog
Fig. 1. Performance graph comparing execution times of different solvers
with their standard settings on all benchmark problems.
The performance ratio compares the time solver s needs for
solving p with the fastest solver:
r
p,s
:=
t
p,s
min t
p,s
: s S
. (4)
The performance graph of a solver s is then obtained by
plotting the cumulative distribution function of r
p,s
defined
by
ρ
s,b
(τ) :=
1
|P|
|{p P : log
b
(r
p,s
) τ }| , (5)
where | · | denotes the number of elements in a set.
Fig. 1 compares the CPU time required by publicly avail-
able C/C++ code implementations and the active-set method
of MATLABs quadprog on all benchmark examples of
Table III featuring a broad range of numerical properties.
FORCES and FiOrdOs have been used to auto-generate
custom implementations for each of the examples, qpOASES
and FiOrdOs are run with the warm-start option disabled,
i.e. performing cold-starts, and FiOrdOss pre-conditioning
feature has been turned on. All solvers are run until their
internal convergence criterion is met or a given maximum
number of iterations is reached
5
. Problem instances where
the obtained solution exhibits a significant amount of sub-
optimality or infeasibility are considered not solved, imply-
ing that the corresponding curve in the performance plot does
not reach 1 even at very high solution time ratios. As all
experiments are rather preliminary, we refrain from drawing
further conclusions from the results.
Second, we look at computational performance from a
different point of view. Instead of comparing CPU times,
we specify a fixed number of iterations and compare the
accuracy of the solution (or intermediate iterate) that could
be obtained by different solvers. This mimics an embedded
MPC setup where one needs to obey a strict runtime limit
and hopes to obtain a solution that is as accurate as possible.
Fig. 2 shows such an alternative performance graph based
5
In order to make results more comparable as discussed in Section IV-B,
all simulations are run open-loop with the respective optimal input sequence
obtained beforehand by running qpOASES in closed-loop.
0 5 10 15 20
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
τ
ρ
s,10
(τ)
ADMM
DFGM (g)
GPAD (g)
PQP
Fig. 2. Performance graph comparing solution accuracy obtained with
different MPC algorithms after a fixed number of iterations.
on the solution accuracies achieved after 100 iterations of
different simple iteration algorithms (where ‘(g)’ denotes
generalized variants as proposed in [10]), which have been
prototyped in Matlab and have roughly comparable CPU
times per iteration. In this graph, the solution accuracy is
measured in terms of the relative error in the optimal QP
objective function value (obtained by means of qpOASES).
To avoid numerical artefacts, τ = 0 (i.e. the denominator
in Eq. (4)) is defined as a relative error of 10
8
. Thus,
the dotted vertical line τ = 6 helps to identify for each
method the fraction of QP instances where 100 iterations
were sufficient to obtain a (feasible) solution with less than
1 % inaccuracy. It is worth mentioning that results for the
ADMM are somewhat misleading as we did not employ any
problem-specific tuning, which is particularly crucial for this
method.
B. Performance on Specific Example
In this section, we illustrate how the benchmarking suite
can be used to analyse the qualitative numerical behaviour of
different MPC algorithms on a specific application (here, the
SPACECRAFT example) besides plotting usual information
such as trajectories, number of iterations or KKT tolerances.
Fig. 3 depicts how the number of iterations required to
solve the SPACECRAFT problem depends on the number of
inequality constraints active at the optimal solution. Not
surprisingly, it can be seen that the number of iterations
is basically constant for an interior-point method, while it
seems to grow linearly for a (cold-started) active-set method.
The employed gradient method shows a positive correlation
between the number of active constraints and the number of
iterations although the variance is much higher than for the
active-set method.
VI. CONCLUSIONS AND FUTURE WORK
This paper provided guidelines for assessing different
embedded MPC algorithms in a fair and practically relevant
manner. A number of MPC algorithms has been surveyed
along with a concise discussion on their numerical and
5 10 15 20 25 30 35 40
0
50
100
150
200
250
300
350
active constraints [#]
iterations [#]
FORCES
GPAD (g)
qpOASES
Fig. 3. Number of iterations vs. number of constraints active at the solution
when solving the SPACECRAFT benchmark with different MPC algorithms.
implementation-related aspects. Moreover, a general bench-
marking suite has been described that allows for comparing
the numerical behavior and performance of different algo-
rithms on a large number of benchmark examples. Also
preliminary computational results have been presented in
order to highlight some of the different point of views when
assessing the suitability of a certain algorithm for solving
MPC problems.
Future work will mainly aim at extending the presented
benchmarking framework by adding more benchmark exam-
ples and including further interesting MPC algorithms (or
replacing prototype implementations by more efficient ones).
Moreover, we will add the possibility to assess numerical
performance and memory footprint directly on embedded
controller hardware.
REFERENCES
[1] J. Rawlings and D. Mayne, Model Predictive Control: Theory and
Design. Nob Hill, 2009.
[2] C. Rao, S. Wright, and J. Rawlings, “Application of Interior-Point
Methods to Model Predictive Control,Journal of Optimization Theory
and Applications, vol. 99, pp. 723–757, 1998.
[3] J. Mattingley and S. Boyd, Convex Optimization in Signal Process-
ing and Communications. Cambridge University Press, 2009, ch.
Automatic Code Generation for Real-Time Convex Optimization.
[4] A. Domahidi, A. U. Zgraggen, M. N. Zeilinger, M. Morari, and C. N.
Jones, “Efficient interior point methods for multistage problems arising
in receding hoirzon control, in Decision and Control (CDC), 2012
51th IEEE Conference on, Maui, HI, USA, Dec. 2012, pp. 668–674.
[5] A. Wills, D. Bates, A. Fleming, B. Ninness, and S. Moheimani,
Application of MPC to an active structure using sampling rates up
to 25kHz. 44th IEEE Conference on Decision and Control and
European Control Conference ECC’05, Seville, 2005.
[6] R. Bartlett and L. Biegler, “QPSchur: A Dual, Active Set, Schur
Complement Method for Large-scale and Structured Convex Quadratic
Programming Algorithm, Optimization and Engineering, vol. 7, pp.
5–32, 2006.
[7] H. J. Ferreau, H. G. Bock, and M. Diehl, “An online active set strategy
to overcome the limitations of explicit MPC, International Journal
of Robust and Nonlinear Control, vol. 18, no. 8, pp. 816–830, May
2008.
[8] S. Richter, M. Morari, and C. N. Jones, “Towards computational com-
plexity certification for constrained MPC based on lagrange relaxation
and the fast gradient method, in Decision and Control and European
Control Conference (CDC-ECC), 2011 50th IEEE Conference on.
IEEE, 2011, pp. 5223–5229.
[9] P. Patrinos and A. Bemporad, An accelerated dual gradient-projection
algorithm for linear model predictive control, in Proc. IEEE 51st
Conf. on Decision and Control (CDC), 2012, pp. 662–667.
[10] P. Giselsson, “Improved fast dual gradient methods for embedded
model predictive control, in Proceedings of the 19th IFAC World
Congress, Cape Town, South Africa, 2014, pp. 2303–2309.
[11] A. Bemporad, M. Morari, V. Dua, and E. Pistikopoulos, “The ex-
plicit linear quadratic regulator for constrained systems, Automatica,
vol. 38, pp. 3–20, 2002.
[12] A. Alessio and A. Bemporad, Nonlinear model predictive control, ser.
Lecture Notes in Control and Information Sciences. Springer, 2009,
vol. 384, ch. A Survey on Explicit Model Predictive Control, pp. 345–
369.
[13] M. Vukov, A. Domahidi, H. Ferreau, M. Morari, and M. Diehl, “Auto-
generated Algorithms for Nonlinear Model Predictive Control on Long
and on Short Horizons, in Proceedings of the 52nd IEEE Conference
on Decision and Control, Firenze, Italy., 2013.
[14] M. Kvasnica, P. Grieder, M. Baoti, and M. Morari, “Multi parametric
toolbox (MPT), in HSCC (Hybrid Systems: Computation and Con-
trol), ser. Lecture Notes in Computer Science, R. Alur and G. Pappas,
Eds. Springer Verlag, 2004, vol. 2993, pp. 448–462.
[15] D. M. Kufoalor, S. Richter, L. Imsland, T. Johansen, M. Morari, and
G. Eikrem, “Embedded model predictive control on a PLC using a
primal-dual first-order method for a subsea separation process, in
Proceedings of the 22nd Mediterranean Conference on Control and
Automation, Palermo, Italy, 2014.
[16] F. Ullmann, “FiOrdOs: A MATLAB toolbox for c-code generation for
first order methods, Master thesis, ETH Zurich, Switzerland, 2011.
[17] V. Nedelcu, I. Necoara, and D. Q. Quoc, “Computational complexity
of inexact gradient augmented lagrangian methods: Application to con-
strained MPC, SIAM Journal on Control and Optimization, vol. 52,
no. 5, pp. 3109–3134, 2014.
[18] S. Kvamme, “DuQuad Webpage, http://sverrkva.github.io/duquad/,
2014–2015.
[19] H. Ferreau, C. Kirches, A. Potschka, H. Bock, and M. Diehl,
“qpOASES: A parametric active-set algorithm for quadratic program-
ming, Mathematical Programming Computation, vol. 6, no. 4, pp.
327–363, 2014.
[20] Y. Wang and S. Boyd, “Fast model predictive control using online
optimization, IEEE Transactions on Control Systems Technology,
vol. 18, no. 2, pp. 267–278, Mar. 2010.
[21] M. K
¨
ogel and R. Findeisen, “On efficient predictive control of linear
systems subject to quadratic constraints using condensed, structure-
exploiting interior point methods, in Proceedings of the 12th Euro-
pean Control Conference, Zurich, Switzerland, 2013, pp. 27–34.
[22] G. Frison, H. B. Sørensen, B. Dammann, and J. Jørgensen, “High-
performance small-scale solvers for linear model predictive control,
in IEEE European Control Conference. IEEE, 2014, pp. 128–133.
[23] S. D. Cairano and M. Brand, “On a multiplicative update dual
optimization algorithm for constrained linear MPC, in Proceedings
of the 52nd IEEE Conf. on Decision and Control, Firenze, Italy, 2013,
pp. 1696–1701.
[24] B. O’Donoghue, G. Stathopoulos, and S. Boyd, “A splitting method for
optimal control, IEEE Transactions on Control Systems Technology,
vol. 21, no. 6, pp. 2432–2442, Nov. 2013.
[25] G. Stathopoulos, A. Sz
¨
ucs, Y. Pu, and C. Jones, “Splitting methods
in control, in Proceedings of the 13th European Control Conference,
Strassbourg, France, 2014, pp. 2478–2483.
[26] J. Frasch, M. Vukov, H. Ferreau, and M. Diehl, A new quadratic
programming strategy for efficient sparsity exploitation in SQP-based
nonlinear MPC and MHE, in Proceedings of the 19th IFAC World
Congress, Cape Town, South Africa, 2014, pp. 2945–2950.
[27] P. Patrinos, P. Sopasakis, and H. Sarimveis, A global piecewise
smooth newton method for fast large-scale model predictive control,
Automatica, vol. 47, pp. 2016–2022, 2011.
[28] D. Kouzoupis, “Complexity of first-order methods for fast embedded
model predictive control, Master’s thesis, ETH Zurich, Switzerland,
2014.
[29] H. Ferreau and H. Peyrl, “mpcBenchmarking Webpage,
https://github.com/ferreau/mpcBenchmarking, 2015.
[30] E. D. Dolan, J. J. Mor, T. S. Munson, E. D. Dolan, J. J. Mor, and T. S.
Munson, “Optimality measures for performance profiles, in Preprint
ANL/MCS-P1155-0504, Argonne National Lab, 2004.
[31] E. Dolan and J. Mor
´
e, “Benchmarking optimization software with
performance profiles, Math. Program., vol. 91, pp. 201–213, 2002.
... To write the QP in a form suitable for embedded implementation, the optimization problem has to be rewritten in a vector form. According to [36], a condensed QP formulation is preferable and typically computationally favorable when the size of the optimization problem is relatively small, as is the case with (22). ...
... In this section, the impact of the prediction horizon and carrier frequency on the closed-loop performance of the proposed MPC algorithm are investigated, both at steady-state operation and during transients. Moreover, since the choice of weighting factors affects the controller behavior as well as the conditioning of the QP problem (31) [36], the sensitivity of the closed-loop performance to variations in the matrix R is analyzed to provide a deeper insight in the controller design. ...
Article
Full-text available
This paper presents a model predictive control (MPC) algorithm for a three-level neutral point clamped (3L-NPC) converter connected to the grid via an LCL filter. The proposed long-horizon MPC method, formulated as a multi-criterion quadratic program (QP), simultaneously controls the grid and converter current as well as the filter capacitor voltage, while meeting the relevant grid standards. To achieve the latter, a carrier-based pulse width modulation (CB-PWM) stage is employed. Finally, soft constraints are included to ensure operation of the system within its safe operating limits, particularly with regards to a potential overcurrent or overvoltage trip during transient operation. The presented simulation results based on a medium-voltage system as well as experimental studies based on a scaled-down prototype verify the effectiveness of the proposed method.
... The term QP solver refers not only to an algorithm for solving a QP, but also to a particular software implementation of that algorithm. A variety of QP solvers with different features exist; a good review can be found in [48]. A brief review is also given in Section 4.1 of this thesis. ...
... A variety of QP solvers with different features exist, and here we review some of the ones which are often used in real-time MPC. More detailed review can be found in [48]. ...
Thesis
Accurate reproduction of the feeling of motion is the primary goal of motion simulation. The subjective experience of self-motion results from the processing of visual and inertial sensory information in the human central nervous system (CNS). In contrast to the real experience, the simulation capabilities are restricted by the motion simulator workspace, which makes the exact reproduction of the inertial sensory input generally impossible. It is possible, however, to minimize the mismatch between the desired and the actual inertial signal, while satisfying the constraints imposed by the motion simulator. This naturally leads to the formulation of the motion simulation problem as a constrained optimal control problem (OCP). If this OCP is solved in real-time on a moving horizon, we enter the field of model predictive control (MPC). The MPC-based motion simulation is computationally expensive due to the (large) size of the problem and the need to take the generally non-linear dynamics of the simulator into account. The solution time must be small enough to achieve the desired control rate. Use of appropriate algorithms and a thoughtful approach to software development are needed to make the MPC implementation real-time capable. Another important problem is the offline design of motion simulators. The optimal design parameters can be obtained from an optimization problem, which can be solved simultaneously with finding optimal control inputs in an offline context. This work focuses on developing efficient approaches for solving optimization problems arising in the field of motion simulation. In the offline context, a method to simultaneously optimize simulator trajectories and design parameters is developed. In the online context, real-time capable implementations for two motion simulators at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany are created and their performance is analyzed. One of the implementations is experimentally validated by measuring the inertial signal tracking error for realistic motion simulation scenarios. The efficiency of the software implementation plays a critical role in real-time MPC. In order to simplify the development of MPC controllers and to be able to reuse algorithms, a number of software packages exist. Such software packages must be efficient, convenient to use, and provide a sufficient degree of control over the implementation details to the user. To satisfy these conflicting demands, the tmpc library was developed in the context of this work and was used to implement the real-time controllers for the two motion simulators. The library is released as open source in the hope that it will be useful for developing further real-time MPC applications.
... Consequently, they may not be able to solve the MPC problem of interest in real time within a few hundred, or even tens, of microseconds, since they do not exploit its structure. Indeed, the execution time greatly depends on various factors of the optimization problem, such as the size of the state and input vectors, the number of the constraints and the geometry of the feasible region, see [26] for a comprehensive assessment of different QP solvers. Therefore, to facilitate the real-time implementation of the direct MPC algorithm, an efficient and highly reliable gradient-based QP solver is developed in this paper. ...
... For the convex QP (24) satisfaction of the KKT conditions (26) suffices for τ * to be a global solution [37]. In the following, it is shown howτ * can be found by solving the KKT conditions (26). First, it is noted that (26) can be split into two decoupled sets of equations 3 ...
Article
Full-text available
This paper presents a direct model predictive control (MPC) method for drive systems with superior steady-state and dynamic performance. Specifically, the discussed MPC algorithm achieves a steady-state behavior that is similar or better than that of a linear controller with a dedicated modulator, and fast transient responses that characterize direct controllers. Moreover, it ensures a fixed switching frequency by allowing for one switching transition per phase and sampling interval. Furthermore, the controller utilizes the stator current gradient to predict the evolution of the drive system within the prediction horizon. To find the optimal switching time instants-and thus ensure favorable performance-the control and modulation problems are formulated in one computational stage as a constrained quadratic program (QP). To solve the latter within a few microseconds, a computationally efficient QP solver based on a gradient method is proposed that enables the real-time implementation of the presented algorithm. To further alleviate the computational demands of the proposed method, a mechanism that can identify suboptimal switching sequences at the very early stages of the optimization process is proposed. The effectiveness of the proposed control scheme is experimentally verified on a 3 kW drive system consisting of a two-level inverter and an induction machine.
... A few of the most widespread ones include qpOASES [5], CVXGEN [6], FiOrdOs [7] and FORCES [8]. We refer the reader to [9] for an overview and comparison between these tools. ...
... where x k is the state at sample time k, b(x k ) is used to indicate that vector b is parametrized by x k , as shown in (9), θi for i ∈ Z 3 1 are given by (8), and Ai for i ∈ Z 3 1 are given by (9). To simplify the notation, let us define the vector of optimization variables w = (z1, z2, z3, λ). ...
Preprint
This article presents an implementation of a sparse, low-memory footprint optimization algorithm for the implementation of the model predictive control for tracking formulation in embedded systems. The algorithm is based on an extension of the alternating direction method of multipliers to problems with three separable functions in the objective function. One of the main advantages of the proposed algorithm is that its memory requirements grow linearly with the prediction horizon of the controller. Its sparse implementation is attained by identification of the particular structure of the optimization problem, and not by employing the common sparse algebra techniques, leading to a very computationally efficient implementation. We describe the controller formulation and provide a detailed description of the proposed algorithm, including its pseudocode. We also provide a simple (and sparse) warmstarting procedure that can significantly reduce the number of iterations. Finally, we show some preliminary numerical results of the performance of the algorithm.
... Furthermore, when input and/or state constraints are considered for the MPC formulation [37], then either efficient QP solvers can be employed (e.g., interior-point, active-set methods [57]), or the unconstrained solution can be projected onto the feasible set. In [58], a CCS-MPC strategy with input and state constraints was introduced for MPTC in PMSMs, in which an active-set algorithm to solve efficiently the associated QP problem is implemented in a low-cost control platform. ...
Article
Full-text available
This paper presents the application of Model Predictive Control (MPC) in high-performance drives. A wide variety of machines have been considered: induction machines, synchronous machines, linear motors, switched reluctance motors, and multiphase machines. The control of these machines has been done by introducing minor and easy-to-understand modifications to the basic predictive control concept, showing the high flexibility and simplicity of the strategy. The second part of the paper is dedicated to the performance comparison of MPC with classical control techniques such as field-oriented control and direct torque control. The comparison considers the dynamic behavior of the drive and steady-state performance metrics such as inverter losses, current distortion in the motor, and acoustic noise. The main conclusion is that MPC is very competitive concerning classic control methods by reducing the inverter losses and the current distortion with comparable acoustic noise.
... As evidences from the above references, research in this field is very extensive, encompassing a wide range of embedded systems, optimization methods and approaches. For additional references we refer the reader to the surveys [121,122,123,124]. This chapter presents sparse solvers for various MPC formulations in which the choice of formulation and optimization algorithm leads to an efficient solver that exploit the structure of the problem. ...
Preprint
This Ph.D. dissertation contains results in two different but related fields: the implementation of model predictive control (MPC) in embedded systems using first order methods, and restart schemes for accelerated first order methods (AFOM). We start by presenting three novel restart schemes for AFOM. These schemes can improve the convergence of the AFOM by suppressing the undesirable oscillations that they are prone to present. The schemes we develop have theoretical guarantees and do not require knowledge of difficult-to-obtain parameters of the optimization problem. Next, we present sparse solvers for various MPC formulations which take advantage of the structures of the optimization problems. The solvers have been made available in an open-source toolbox for Matlab called SPCIES (https://github.com/GepocUS/Spcies). Finally, we present a novel MPC formulation that displays a larger domain of attraction and better performance than other MPC formulations, especially when using small prediction horizons. This, along with its recursive feasibility and asymptotic stability, makes it especially suitable for its implementation in embedded systems.
... Several algorithms have been proposed in literature, such as interiorpoint (IP), active-set (AS), gradient, and explicit methods, as well as the aforementioned ADMM. A comprehensive assessment of different QP solvers is given in [185] and, with more focus on FPGA implementation, in [186]. Besides numerical accuracy, the computational speed of the solver is crucial. ...
Article
Full-text available
Model predictive control (MPC) has established itself as a promising control methodology in power electronics. This survey paper highlights the most relevant MPC techniques for power electronic systems. These can be classified into two major groups, namely, MPC without modulator, referred to as direct MPC, and MPC with a subsequent modulation stage, known as indirect MPC. Design choices and parameters that affect the system performance, closed-loop stability and controller robustness are discussed. Moreover, solvers and control platforms that can be employed for the real-time implementation of MPC algorithms are presented. Finally, the MPC schemes in question are assessed, among others, in terms of design and computational complexity, along with their performance and applicability depending on the power electronic system at hand.
Thesis
Full-text available
In recent years, model predictive control (MPC) formulations have received significant attention and wide acceptance by the power electronics community as a promising alternative to the established control strategies. At the same time, interest in high-power grid-connected converters has been growing, motivated by the need for a flexible interface in applications such as renewable energy and industrial drives and now vehicles to be connected to the grid. Depending on the considered controlled variable, this dissertation focuses on the development of a constrained long-horizon indirect—the manipulated variable is real-valued, thus, belonging to a continuous set—and direct—the manipulated variable corresponding to the converter switch positions, thus, belonging to an integer set—MPC-based controllers for high-power grid-connected three-level neutral-point-clamped (3L-NPC) converters with LCL filters. Medium voltage (MV) applications represent a particularly challenging field in which the nature of the semiconductor devices requires the minimization of the switching frequency (i.e. switching losses) of the converter, while grid codes impose tight limits on the current and voltage harmonics amplitudes injected at the point of common coupling (PCC). To this end, LCL filters are adopted. Nevertheless, the latter introduces additional control challenges due to the higher order of the resulting system and the filter resonance. Considering such case study, both methods are tested and show that they can achieve steady-state performance that can bring improvements compared with conventional control methods, such as voltage-oriented control with space vector modulation. This includes the intrinsic ability of MPC to damp resonating effects (i.e. without adding external damping loops). In particular, it is shown that indirect MPC exhibits superior steady-state performance with fast transient responses and lower output distortion while operating the converter within its safe operating limits (enhanced reliability) through the introduction of soft constraints. By appropriately formulating the constrained optimization problem underlying indirect MPC as a quadratic program (QP), the grid and converter currents as well as the filter capacitor voltage can be simultaneously and successfully controlled while keeping the online calculation effort low, even in the presence of longer horizon. This is mainly due to efficient QP solvers which can be adopted in such framework. This facilitates the further development of a feasible implementation of this approach on an embedded platform. Indeed, the focus of the dissertation is subsequently shifted to the real-time certification of the derived embedded MPC to provide a feasible and effective solution to be used in large-scale industrial applications. Based on the proposed computational burden analysis, it can be concluded that the proposed MPC algorithms meets the real-time certification and guarantees that the solution to the QP is always found at run-time without affecting its optimality. Throughout the dissertation, the performance of the proposed strategies is investigated within a selfdeveloped scaled-down experimental test-bench and a customized control platform based on a multiple processor system-on-chip field-programmable gate array (MPSoC FPGA) . Although the controllers that are presented represent concrete and feasible alternatives to the established control strategies, there remains scope for future work in terms of testing and implementation strategies.
Thesis
Full-text available
This thesis is about numerical methods for nonlinear model predictive control (NMPC) with a focus on system theoretic guarantees. Although considerable progress has been made since its early application in the late 1970s in the process industry, NMPC still requires a computational effort that is prohibitive for many applications due to fast dynamics or the low computational power available. For this reason, despite NMPC being nowadays the state-of-the-art control strategy in many applications, its applicability to a broader range of systems, still hinges on the development of efficient methods for numerical optimization. In particular, we propose inexact methods that can speed up the computations associated with the solution of the underlying nonconvex programs. Although drawing from rather diverse areas, from an abstract point of view, such methods exploit a common idea. In fact, in many cases, carefully chosen perturbations to exact solutions and formulations do not jeopardize stability properties and can be leveraged to alleviate the computational burden of NMPC. The algorithmic contributions of the thesis revolve around three main ideas. First, we analyze and extend a class of inexact methods, the so-called real-time methods. These strategies take inspiration from the popular real-time iteration (RTI) and rely on a limited number of iterations of the optimizer in order to compute an approximate solution to the underlying nonconvex programs. By warmstarting the iterates it is possible to “track” a parametric solution as the state of the system evolves. We prove asymptotic stability of the system- optimizer dynamics for a class of RTI-like methods that need not be based on standard sequential quadratic programming. This class of methods encompasses different strategies proposed in the thesis. Second, we propose and analyze numerical methods that exploit inexact first- and second-order derivatives in order to efficiently compute suboptimal, but feasible solutions to nonconvex programs. These methods, which have strong ties with the celebrated second-order corrections in numerical optimization, can be used to reduce the computational cost associated with derivative computation and linear algebra. We show that the feedback policy resulting from such methods is locally stabilizing and enjoys favorable asymptotic suboptimality bounds. The third class that we analyze is the one of progressive tightening methods. In this case, the main idea lies in exploiting stage-varying constraints and costs that give rise to increasingly conservative predictions the farther we look into the future. We prove asymptotic stability of progressive tightening NMPC and propose a numerical method that can exploit a specific barrier-based tightening in order to reduce the computational burden. Finally, real-time variants of the second and third class of methods are proposed and analyzed from a system theoretic point of view and numerically validated. In addition to the proposed algorithms and system theoretic results, we experimentally validate the classical RTI strategy as implemented in the software package acados on the task of controlling a reluctance synchronous machine. We present the experimental results of a current controller based on indirect NMPC running with a sampling time of 250 µs. Lastly, motivated by the high degree of interaction between software components involved in the implementation of complex numerical strategies, we propose a high-level software framework for embedded high-performance computing. This framework, named prometeo, provides a Python-to-C transpiler and a static analysis tool that allows one to combine the performance of low-level languages (such as C) with the user-friendliness of high-level ones.
Article
We present Optimization Engine (OpEn): an open-source code generation framework for real-time embedded nonconvex optimization, which implements a novel numerical method. OpEn combines the proximal averaged Newton-type method for optimal control (PANOC) with the penalty and augmented Lagrangian methods to compute approximate stationary points of nonconvex problems. The proposed method involves very simple algebraic operations such as vector products, has a low memory footprint and exhibits very good convergence properties that allow the solution of nonconvex problems on embedded devices. OpEn’s core solver is written is Rust — a modern, high-performance, memory-safe and thread-safe systems programming language — while users can call it from Python, MATLAB, C, C++, ROS or over a TCP socket.
Article
Full-text available
The results of a PLC implementation of embedded Model Predictive Control (MPC) for an industrial problem are presented in this paper. The embedded MPC developed is based on the linear MPC module in SEPTIC (Statoil Estimation and Prediction Tool for Identification and Control), and it combines custom ANSI C code generation with problem size reduction methods, embedded real-time considerations, and a primal-dual first-order method that provides a fast and light QP solver obtained from the FiOrdOs code generator toolbox. Since the primal-dual first-order method proposed in this paper is new in the control community, an extensive comparison study with other state-of-the-art first-order methods is conducted to underline its potential. The embedded MPC was implemented on the ABB AC500 PLC, and its performance was tested using hardware-in-the-loop simulation of Statoil's newly patented subsea compact separation process. A warm-start variant of the proposed first-order method outperforms a tailored interior-point method by a factor of 4 while occupying 40% less memory.
Conference Paper
Full-text available
Several algorithms based on Nesterov’s fast gradient method have been recently proposed in the literature for use in linear model predictive control (MPC). Their simple algorithmic schemes have attracted much attention for MPC applications on embedded hardware. The purpose of this paper is to investigate the suitability of these algorithms in a nonlinear MPC setup. We assess the necessary numerical modifications and analyze the additional online computational complexity. We illustrate our findings by combining different first-order methods with the real-time iteration (RTI) scheme for nonlinear MPC and using them to control a model of an inverted pendulum.
Article
Recently, several authors have suggested the use of first order methods, such as fast dual ascent and the alternating direction method of multipliers, for embedded model predictive control. The main reason is that they can be implemented using simple arithmetic operations only. In this paper, we present results that enable for significant improvements when using fast dual ascent for embedded model predictive control. These improvements rely on a new characterization of the set of matrices that can be used to describe a quadratic upper bound to the negative dual function. For many interesting formulations, it is shown that the provided bound cannot be improved and that it is tighter than bounds previously presented in the literature. The improved quadratic upper bound is used in fast dual gradient methods as an approximation of the dual function. Since it better approximates the dual function than previous upper bounds, the convergence is improved. The performance enhancement is most significant for ill-conditioned problems. This is illustrated by a numerical evaluation on a AFTI-16 aircraft model where the algorithm outperforms previous proposals of first order methods for embedded control with one to three orders of magnitude.
Article
A large class of algorithms for nonlinear model predictive control (MPC) and moving horizon estimation (MHE) is based on sequential quadratic programming and thus requires the solution of a sparse structured quadratic program (QP) at each sampling time. We propose a novel algorithm based on a dual two-level approach involving a nonsmooth version of Newton's method that aims at combining sparsity exploitation features of an interior point method with warm-starting capabilities of an active-set method. We address algorithmic details and present the open-source implementation qpDUNES. The effectiveness of the solver in combination with the ACADO Code Generation tool for nonlinear MPC is demonstrated based on set of benchmark problems, showing significant performance increases compared to the established condensing-based approach, particularly for problems with long prediction horizons.
Article
Although linear Model Predictive Control has gained increasing popularity for controlling dynamical systems subject to constraints, the main barrier that prevents its widespread use in embedded applications is the need to solve a Quadratic Program (QP) in real-time. This paper proposes a dual gradient projection (DGP) algorithm specifically tailored for implementation on fixed-point hardware. A detailed convergence rate analysis is presented in the presence of round-off errors due to fixed-point arithmetic. Based on these results, concrete guidelines are provided for selecting the minimum number of fractional and integer bits that guarantee convergence to a suboptimal solution within a pre-specified tolerance, therefore reducing the cost and power consumption of the hardware device.
Conference Paper
We present a code generation strategy for handling long prediction horizons in the context of real-time nonlinear model predictive control (NMPC). Existing implementations of fast NMPC algorithms use the real-time iteration (RTI) scheme and a condensing technique to reduce the number of optimization variables. Condensing results in a much smaller, but dense quadratic program (QP) to be solved at every time step. While this approach is well suited for short horizons, it leads to unnecessarily long execution times for problem formulations with long horizon. This paper presents a new implementation of auto-generated NMPC code based on a structure exploiting auto-generated QP solver. Utilizing such a QP solver, the condensing step can be avoided and execution times scale linearly with the horizon length instead of cubically. Our simulation results show that this approach significantly decreases the execution time of NMPC with long horizons. For a nonlinear test problem that comprises 9 states and 3 controls on a horizon with 50 time steps, an improvement by a factor of 2 was observed, reducing the execution time for one RTI to below 4 milliseconds on a 3 GHz CPU.
Conference Paper
We discuss a multiplicative update quadratic programming algorithm with applications to model predictive control for constrained linear systems. The algorithm, named PQP, is very simple to implement and thus verify, does not require projection, offers a linear rate of convergence, and can be completely parallelized. The PQP algorithm is equipped with conditions that guarantee the desired bound on suboptimality and with an acceleration step based on projection-free line search. We also show how PQP can take advantage of the parametric structure of the MPC problem, thus moving offline several calculations and avoiding large input/output dataflows. The algorithm is evaluated on two benchmark problems, where it is shown to compete with, and possibly outperform, other open source and commercial packages.
Conference Paper
In Model Predictive Control (MPC), an optimization problem needs to be solved at each sampling time, and this has traditionally limited use of MPC to systems with slow dynamic. In recent years, there has been an increasing interest in the area of fast small-scale solvers for linear MPC, with the two main research areas of explicit MPC and tailored on-line MPC. State-of-the-art solvers in this second class can outperform optimized linear-algebra libraries (BLAS) only for very small problems, and do not explicitly exploit the hardware capabilities, relying on compilers for that. This approach can attain only a small fraction of the peak performance on modern processors. In our paper, we combine high-performance computing techniques with tailored solvers for MPC, and use the specific instruction sets of the target architectures. The resulting software (called HPMPC) can solve linear MPC problems 2 to 8 times faster than the current state-of-the-art solver for this class of problems, and the high-performance is maintained for MPC problems with up to a few hundred states.
Conference Paper
The need for optimal control of processes under a restricted amount of resources renders first order optimization methods a viable option. Although computationally cheap, these methods typically suffer from slow convergence rates. In this work we discuss the family of first order methods known as decomposition schemes. We present three popular methods from this family, draw the connections between them and report all existing results that enable acceleration in terms of the convergence rate. The approach for splitting a problem into simpler ones so that the accelerated variants can be applied is also discussed and demonstrated via an example.
Article
Many practical applications lead to optimization problems that can either be stated as quadratic programming (QP) problems or require the solution of QP problems on a lower algorithmic level. One relatively recent approach to solve QP problems are parametric active-set methods that are based on tracing the solution along a linear homotopy between a QP problem with known solution and the QP problem to be solved. This approach seems to make them particularly suited for applications where a-priori information can be used to speed-up the QP solution or where high solution accuracy is required. In this paper we describe the open-source C++ software package qpOASES, which implements a parametric active-set method in a reliable and efficient way. Numerical tests show that qpOASES can outperform other popular academic and commercial QP solvers on small- to medium-scale convex test examples of the Maros-Mészáros QP collection. Moreover, various interfaces to third-party software packages make it easy to use, even on embedded computer hardware. Finally, we describe how qpOASES can be used to compute critical points of nonconvex QP problems.