Performance bounds for linear stochastic control

Abstract

We develop computational bounds on performance for causal state feedback stochastic control with linear dynamics, arbitrary noise distribution, and arbitrary input constraint set. This can be very useful as a comparison with the performance of suboptimal control policies, which we can evaluate using Monte Carlo simulation. Our method involves solving a semidefinite program (a linear optimization problem with linear matrix inequality constraints), a convex optimization problem which can be efficiently solved. Numerical experiments show that the lower bound obtained by our method is often close to the performance achieved by several widely-used suboptimal control policies, which shows that both are nearly optimal. As a by-product, our performance bound yields approximate value functions that can be used as control Lyapunov functions for suboptimal control policies.

Full text

Page 1

Systems & Control Letters 58 (2009) 178–182
Contents lists available at ScienceDirect
Systems & Control Letters
journal homepage: www.elsevier.com/locate/sysconle
Performance bounds for linear stochastic control
Yang Wang∗, Stephen Boyd1
Room 243, Packard Electrical Engineering, 350 Serra Mall, Stanford, CA 94305-9505, United States
a r t i c l e i n f o
Article history:
Received 16 August 2008
Received in revised form
7 October 2008
Accepted 7 October 2008
Available online 22 November 2008
Keywords:
Stochastic control
Model predictive control
Linear matrix inequality
Convex optimization
a b s t r a c t
Wedevelopcomputationalboundsonperformanceforcausalstatefeedbackstochasticcontrolwithlinear
dynamics, arbitrary noise distribution, and arbitrary input constraint set. This can be very useful as a
comparison with the performance of suboptimal control policies, which we can evaluate using Monte
Carlo simulation. Our method involves solving a semidefinite program (a linear optimization problem
with linear matrix inequality constraints), a convex optimization problem which can be efficiently
solved. Numerical experiments show that the lower bound obtained by our method is often close to the
performance achieved by several widely-used suboptimal control policies, which shows that both are
nearly optimal. As a by-product, our performance bound yields approximate value functions that can be
used as control Lyapunov functions for suboptimal control policies.
© 2008 Elsevier B.V. All rights reserved.
1. Linear stochastic control
We consider a discrete-time linear time-invariant system (or
plant), with dynamics
x(t + 1) = Ax(t) + Bu(t) + w(t),
where x(t) ∈ Rnis the state, u(t) ∈ Rmis the control input,
w(t) ∈ Rnis the process noise or exogenous input, A ∈ Rn×nis the
dynamics matrix, and B ∈ Rn×mis the input matrix. We assume
that w(t), for different values of t, are zero mean IID. We will also
assume that x(0) is random, and independent of all w(t).
We consider causal state feedback control policies, where the
current input u(t) is determined from the current and previous
states x(0),...,x(t), i.e.,
u(t) = φt(x(0),...,x(t)),
where φt: R(t+1)n→ Rm. The collection of functions φ0,φ1,... is
called the control policy. For the problem we will consider, it can
be shown that there is an optimal policy that is time-invariant and
depends only on the current state, i.e., has the form
u(t) = φ(x(t)),
where φ : Rn→ Rm. We will refer to φ as the state feedback
function, or the control policy. For fixed state feedback function
t = 0,1,...,
(1)
t = 0,1,...,
t = 0,1,...,
(2)
∗Corresponding author.
E-mail addresses: yw224@stanford.edu (Y. Wang), boyd@stanford.edu
(S. Boyd).
1This material is based upon work supported by the Precourt Institute on Energy
Efficiency, by NSF award 0529426, by NASA award NNX07AEIIA, by AFOSR award
FA9550-06-1-0514, and by AFOSR award FA9550-06-1-0312.
φ, the Eqs. (1) and (2) determine the state and control input
trajectories as functions of x(0) and the process noise trajectory.
Thus,forfixedchoiceofstatefeedbackfunction,thestateandinput
trajectories become stochastic processes.
Wenowintroducetheobjectivefunction,whichweassumehas
the form
J = limsup
T→∞
where ?x: Rn→ R is the state stage cost function, and ?u: Rm→
Ristheinputstatecostfunction.(Weassumethattheexpectations
exist.) The objective J is the average stage cost. Finally, we impose
the control input constraint
1
TE
T−1
?
t=0
(?x(x(t)) + ?u(u(t))),
(3)
u(t) ∈ U
whereU ⊆ Rmis a nonempty constraint set with 0 ∈ U. The stage
cost functions ?xand ?u, and the input constraint set U, need not
be convex.
We can now describe the stochastic control problem. The
problem data are A, B, the distribution of w(t), the stage
cost functions ?x and ?u, and the input constraint set U; the
optimization variable is the state feedback function φ.
stochastic control problem is to find the state feedback function φ
that minimizes the objective J, among those that satisfy the input
constraint (4). We will let J?denote the optimal value of J, and we
let φ?denote an optimal state feedback function.
For more on the formulation of the linear stochastic control
problem, including technical details (e.g., finiteness of J?, existence
and uniqueness of an optimal state feedback function), see, e.g.,
[5,6,26,2,13,27].
(a.s.) ,t = 0,1,...,
(4)
The
0167-6911/$ – see front matter © 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.sysconle.2008.10.004

Page 2

Y. Wang, S. Boyd / Systems & Control Letters 58 (2009) 178–182
179
The stochastic control problem can be effectively solved in
only a few special cases. The most famous example (described
in Section 2 in more detail) is when U = Rm(i.e., there are
no constraints on the input) and ?xand ?uare convex quadratic
functions [14]. In this case the optimal state feedback function is
linear, i.e., u(t) = Kx(t), where K ∈ Rm×ncan be effectively
computed from the problem data.
1.1. Suboptimal control policies
Many methods can be used to find a suboptimal state feedback
function, i.e., one with (one hopes) a small value of J. We describe
three methods in this section; many others can be found in the
literature.
1.1.1. Projected linear state feedback
Perhaps the simplest form is a projected linear state feedback,
φplsf(z) = P(Kplsfz),
where Kplsf ∈ Rm×nis a gain matrix (to be chosen), and P is
projection onto U. When U is a box, i.e.,
U = {u | ?u?∞≤ Umax},
projection is the same as entry-wise saturation, so the projected
linear state feedback policy has the form
φplsf(z) = Umaxsat((1/Umax)Kplsfz),
where the sat function is defined for scalar argument as
?
−1
and extended to vectors by acting entry-wise. (Projected linear
state feedback is sometimes called saturated linear state feedback
in this case.)
(5)
sat(a) =
a
1
|a| ≤ 1
a > 1
a < −1,
1.1.2. Control-Lyapunov feedback
A more sophisticated state feedback function is given by
φclf(z) = argmin
v∈U
where Vclf : Rn→ R (which is to be chosen) is called a control-
Lyapunovfunction[11,22,12,23].(Theoptimalcontrolhasthisform,
for a particular choice of Vclf, called the value function or Bellman
function for the problem.) When Vclf is quadratic, the control-
Lyapunov policy (6) can be simplified to
φclf(z) = argmin
v∈U
(?u(v) + EVclf(Az + Bv + w(t))),
(6)
(?u(v) + Vclf(Az + Bv)).
(7)
1.1.3. Certainty-equivalent model predictive control
An even more sophisticated feedback control function is given
by certainty-equivalent model predictive control (MPC) [16,21,10,15,
4,19], in which φ(z) is found by solving the optimization problem
minimize
Vmpc(˜ x(T)) +
˜ x(τ + 1) = A˜ x(τ) + Bv(τ),
v(τ) ∈ U,
˜ x(0) = z,
with variables v(0),...,v(T − 1), ˜ x(0),..., ˜ x(T). The function
Vmpc
:
T is the horizon (also to be chosen). Let v?(0),...,v?(T − 1),
˜ x?(0),..., ˜ x?(T) be a solution of this problem. The MPC policy is
φmpc(z) = v?(0), which is a (complicated) function of z through
the optimization problem (8). As the horizon T becomes larger,
the choice of Vmpcbecomes less and less important. When the
horizon is T = 1, the MPC policy reduces to the control-Lyapunov
policy (7), with Vmpc= Vclf.
T−1
?
τ = 0,...,T − 1
τ=0
??x(˜ x(τ)) + ?u(v(τ))?
subject to
τ = 0,...,T − 1
(8)
Rn
→
R is the terminal cost (to be chosen), and
1.1.4. Parameters in suboptimal control policies
The art in finding a good suboptimal control policy is in
choosing good values for the parameters that appear in them. For
projected state feedback, the gain matrix Kplsfmust be chosen; for
a control-Lyapunov policy, Vclfmust be chosen; and for MPC, the
terminal cost function Vmpc(and horizon T) must be chosen. A
common choice for Vclfor Vmpcis the (quadratic) value function for
a related linear stochastic control problem with no constraints and
quadratic stage cost; Kplsfcan be chosen as the associated optimal
gain matrix.
These methods (as well as many others) can give suboptimal
state feedback functions that give good performance, i.e., a low
value for J. (The objective J is typically evaluated by stochastic
simulation,e.g.,MonteCarlo.)Anaturalquestionthatarisesis:how
close to optimal are these suboptimal control policies? In other
words, how much larger than the optimal performance J?is J, the
performanceobtainedwithasuboptimalcontrolpolicy?Toanswer
thisquestion,weneedtocomputealowerboundonJ?,i.e.,abound
on achievable performance over all feasible state feedback control
functions.
1.2. Performance bounds
In this paper we show how a numerical lower bound on J?
can be effectively computed, using convex optimization, from the
problem data. Our bound is not a generic one, that depends only
on the problem dimensions and general assumptions about ?x, ?u,
and U; instead, it is computed for each specific problem instance.
We cannot (at this time) guarantee that the bound will be close
to J?. But in a large number of numerical simulations, we have
found that our bound is often not too far from the performance
achieved by a suboptimal control policy. It is very valuable
knowledge in practice to know that a proposed suboptimal control
policy attains a specific cost J (found by Monte Carlo simulation),
and that the optimal value of the stochastic control problem must
exceed a known lower bound Jlb(found by the method described
in this paper). If the gap between the two is small, we can be
certain that our suboptimal control policy is nearly optimal (and
that our bound is nearly tight) for this problem instance. The gap
can be large, of course, for two reasons: our suboptimal controller
is substantially suboptimal, or, our lower bound is poor (for this
problem instance).
2. Linear quadratic control
It is well known that the linear stochastic control problem can
be effectively solved when U = Rm(i.e., there are no constraints
on the input) and the stage cost functions have the form
?x(z) = zTQz,
where Q ? 0, R ? 0 (meaning, they are symmetric positive
semidefinite).Inthissectionwe(briefly)reviewtheseresults,since
ourboundreliesonthem.Formoredetaileddiscussionofthelinear
quadratic stochastic control problem, see, e.g., [5,6,26].
The optimal cost is
?u(v) = vTRv,
J?= Tr(P?W),
and the optimal state feedback function is
φ?(z) = K?z,
where
K?= −(R + BTP?B)−1BTP?Az.
The matrix W is the covariance of w(t), W = Ew(t)w(t)T. The
symmetric matrix P?can be effectively found by several methods.

Page 3

180
Y. Wang, S. Boyd / Systems & Control Letters 58 (2009) 178–182
The traditional characterization (and computation) of P?is via the
algebraic Riccati equation (ARE),
P?= Q + ATP?A − ATP?B(R + BTP?B)−1BTP?A,
along with P?? 0. (For a more detailed discussion of the ARE
and its relation to the linear quadratic stochastic control problem,
see, e.g., [5, Section 4.1] [7]) The control policy φ?(z) = K?z
is called the linear quadratic regulator (LQR) [5, Section 4.1] [8,
Section 10.8] [14].
It will be more convenient for us to use a characterization of
P?and J?in terms of a convex optimization problem. Consider the
optimization problem
maximize
Tr(PW)
?
P ? 0,
subject to
R + BTPB
ATPB
BTPA
Q + ATPA − P
?
? 0 (9)
with variable P. The optimal point is P = P?, and the optimal value
of this problem is J?[8,1,20,3,7,29].
For future use we make some important comments about this
problem.
• The problem (9) is a convex optimization problem, more
specifically, a semidefinite program, and can be effectively
solved [24,9,18,17,28].
• The block matrix inequality appearing as a constraint is called a
linear matrix inequality (LMI) [8].
• The solution P?does not depend on W, as long as W ? 0.
• Since the objective and constraints are also convex (in fact,
affine) jointly in the variables (P,Q,R), it follows that the
optimal value J?is a concave function of (Q,R). (See [9,
Section 5.6.1].)
In the sequel we will refer to the optimal cost J?(feedback gain
K?, value function matrix P?) for the linear quadratic stochastic
control problem with the subscript ‘lq’ (for ‘linear quadratic’), and
explicitly show the dependence on the stage cost data Q and R:
J?
lq(Q,R).
(The value also depends on the other problem data, specifically, A,
B, and W.) We have seen that J?
lqis concave in (Q,R).
3. Performance bound
3.1. Basic bound
Wereturnnowtothegenerallinearstochasticcontrolproblem,
with input constraint setU and general stage cost functions?xand
?u, with optimal value J?. Suppose Q ? 0, R ? 0, and s satisfy the
condition
zTQz + vTRv + s ≤ ?x(z) + ?u(v),
which can be expressed as
?
Then we have the lower bound
for all z ∈ Rn, v ∈ U,
(10)
sup
z
zTQz − ?x(z)?+ sup
v∈U
?vTRv − ?u(v)?+ s ≤ 0.
J?
lq(Q,R) + s ≤ J?.
The lefthand side can be effectively computed, as described in
Section 2. The challenge will be in verifying the condition (10).
We now justify the lower bound (11). Assume that (10) holds.
Let φ?be an optimal state feedback function (for the general
stochastic problem), and let x and u be the associated (stochastic)
(11)
trajectories with control policy φ?. We have u(t) ∈ U a.s., so
by (10), we have
Jlq= lim
T→∞
1
TE
T−1
?
T−1
?
t=0
?
(?x(x(t)) + ?u(u(t)))
x(t)TQx(t) + u(t)TRu(t) + s?
≤ lim
T→∞
1
TE
t=0
= J?.
Here Jlq is the objective of the optimal policy (for the general
problem), evaluated with the quadratic objective. The bound (11)
follows from J?
evaluated with any feedback function.
We mention one simple case. When the stage costs are
quadratic,
lq≤ Jlq, which holds when the righthand side is
?x(z) = zTQ0z,
where Q0 ? 0, R0 ? 0, the simple choice Q = Q0, R = R0, and
s = 0 satisfies (10). The corresponding lower bound on J?is just
J?
constraints on the input, is larger than the optimal average stage
cost, with no constraints on the input (which we can effectively
compute).
?u(v) = vTR0v,
(12)
lq(Q0,R0). This is obvious: the optimal average stage cost, with
3.2. Optimizing the bound
We can optimize the lower bound (11), over the parameters Q,
R, and s, by solving the optimization problem
maximize
subject to (10),
J?
lq(Q,R) + s
Q ? 0,
R ? 0,
(13)
with variables Q, R, and s. This is a convex optimization problem,
since the objective is concave, and the condition (10) is convex
(since it is convex for each z and v). In the general case the
constraint (10) is a semi-infinite constraint, since it is really a
family of constraints, parametrized by the (infinite) set z ∈ Rn,
u ∈ U.
The idea behind (13) is similar to the basic idea in Lagrangian
duality. In Lagrangian duality, we ignore the constraints, but add
to the objective an augmenting function that is nonpositive on
the feasible set. Minimizing this composite function gives a lower
bound on the optimal value of the original problem; optimizing
over the parameters that parametrize the augmenting function we
obtain the best lower bound obtainable using this technique. We
use the same technique here, relying on our ability to solve the
stochastic control problem in the special case when the stage costs
are quadratic and there are no input constraints.
In a few special cases we can solve the problem (13) exactly. In
other cases, we can replace the condition (10) with a conservative
approximation, which still yields a lower bound on J?. We give
more specific examples of each of these cases below.
3.3. Quadratic stage cost and finite input constraint set
We assume the stage costs are quadratic, with the form given in
(12), and the input constraint set is finite, U = {u1,...,uK}. The
condition (10) is then
zTQz + uT
which is the same as Q ? Q0, and
uT
iRui+ s ≤ zTQ0z + uT
iR0ui,
for all z ∈ Rn, i=1,...,K,
iRui+ s ≤ uT
iR0ui,
i = 1,...,K,

Page 4

Y. Wang, S. Boyd / Systems & Control Letters 58 (2009) 178–182
181
a set of linear inequalities on R and s. The problem (13) can be
expressed as the SDP
maximize
Tr(PW) + s
?
P ? 0,
uT
∗subject to
R + BTPB
ATPB
BTPA
Q + ATPA − P
Q ? 0,
iR0ui,
?
? 0
R ? 0,
i = 1,...,K,
Q ? Q0
iRui+ s ≤ uT
with variables P, Q, R, and s. The LMI is (matrix) monotone in Q, so
the optimal value of Q is Q = Q0. So we can just as well drop the
variable Q, and replace it with Q0, to obtain the problem
maximize
Tr(PW) + s
?
P ? 0,
uT
subject to
R + BTPB
ATPB
BTPA
Q0+ ATPA − P
R ? 0
iR0ui,
?
? 0
iRui+ s ≤ uT
i = 1,...,K,
(14)
with variables P, R, and s.
3.4. S-procedure relaxation
We suppose again that the stage costs are quadratic, with the
form (12). Suppose we can find R1,...,RM and s1,...,sM for
which
U ⊆ ˜ U = {v | vTRiv + si≤ 0, i = 1,...,M}.
A sufficient condition for (10) to hold is
zTQz + vTRv + s ≤ zTQ0z + vTR0v,
which holds if and only if Q ? Q0and
vTRiv + si≤ 0, i = 1,...,M ?⇒ vTRv + s ≤ vTR0v.
A sufficient condition for this to hold is (by the so-called S-
procedure; see, e.g., [8, Section 2.6.3]) the existence of nonnegative
λ1,...,λMwhich satisfy
for all z ∈ Rn, v ∈ ˜ U,
R − R0?
M
?
i=1
λiRi,
s ≤
M
?
i=1
λisi.
We can optimize over the lower bound by solving the SDP
maximize
Tr(PW) + s0
?
P ? 0,
R − R0?
λi≥ 0,
∗subject to
R + BTPB
ATPB
BTPA
Q + ATPA − P
Q ? 0,
M
?
i = 1,...,M,
?
? 0
R ? 0,
s0≤
Q ? Q0
λisi
i=1
λiRi,
M
?
i=1
with variables P, Q, R, λ1,...,λM, and s0,...,sM. As before, we
can replace Q with Q0; we can also take s0=?M
maximize
?
P ? 0,
R − R0?
i=1
with variables P, R, and λ ∈ RM, where we use ? between vectors
to mean componentwise inequality. The optimal value of this SDP
gives a lower bound on J?.
i=1λisi. This yields
the SDP
Tr(PW) + sTλ
R + BTPB
ATPB
subject to
BTPA
Q0+ ATPA − P
R ? 0,
M
?
?
? 0
λ ? 0
λiRi,
(15)
3.4.1. Box constraints
As a more specific example, consider the case where U is a box,
{v ∈ Rm| ?v?∞ ≤ Umax}. Since we have v2
quadratic inequalities
vT(eieT
i = 1,...,m,
for v ∈ U, where eiis the ith unit vector. If we use only these
inequalities to define ˜ U, we have M = m. The objective in the
problem (15) becomes
Tr(PW) − (Umax)21Tλ,
where 1 is the vector with all entries one, and the last inequality in
the problem becomes
R − R0? diag(λ).
i≤ 1, we have the
i)v − (Umax)2≤ 0,
3.5. Suboptimal control policies
When we solve the problem (13), or a restriction of it obtained
by replacing the condition (10) with some stronger set of tractable
inequalities, we obtain Plband Rlb(where ‘lb’ stands for ‘lower
bound’). Very roughly speaking, we can interpret our method as
findinganunconstrainedquadraticproblemthatapproximatesour
original problem. This suggests that
Vlb(z) = zTPz,
and
Klb= −(Rlb+ BTPlbB)−1BTPlbAz
would be good candidates for use in synthesizing suboptimal
control policies, as described in Section 1.1. Examples show that
this is the case.
4. Numerical examples
We will illustrate our bound, and compare it with the
performance of several suboptimal control policies, on three
problem instances. (We have used the method on many other
problem instances, with similar results. All the data and code
required to replicate the results reported here are available on-
line.)
The first instance is a small problem, with n =
m = 2 inputs. The data are generated randomly: the entries of the
matrices A and B are drawn from a standard normal distribution,
after which A is scaled so that its spectral radius is one, i.e., so
the open-loop system is marginally stable. The stage costs are
quadratic with R0 = I and Q0 = I. The process disturbance
w(t)hasdistributionN(0,0.25I).Theinputconstraintsetisfinite:
U = {−0.2,0,0.2}2. At each time step, each of the two inputs can
only have three possible values, so we refer to a controller for this
system as trilevel.
The second instance is generated in the same way, but is larger,
with n = 30 states, m = 10 inputs. The stage costs are quadratic,
withR0= I andQ0= I,andw(t) ∼ N(0,0.25I).Forthisexample,
the input constraint set U is a box with Umax= 0.1. For the S-
procedure, we use Ri = eieT
si = −(Umax)2, so the inequality vTRiv − si ≤ 0 is equivalent to
|vi| ≤ Umax.
Our third problem instance is a discretized mechanical control
system, consisting of 6 masses, connected by springs, with three
input forces that can be applied between pairs of the masses. This
isthesameexampleasdescribedin[25].Forthisproblem,wehave
n = 12, m = 3, quadratic stage costs with Q0= I, R0= I; each
entry of w(t) is uniformly distributed on the interval [−0.5,0.5].
The input constraint set is a box with Umax= 0.1. We use the same
S-procedure relaxation as in the second example.
For each example, we evaluate the performance for several
suboptimal controllers (via Monte Carlo simulation), as well as
several lower bounds. The suboptimal controllers are as follows.
• Projected linear state feedback (PLSF) with K?
respectively.
8 states,
i, where eiis the ith unit vector, and
lqand with Klb,

Page 5

182
Y. Wang, S. Boyd / Systems & Control Letters 58 (2009) 178–182
Table 1
Performance of suboptimal control policies (top half) and lower bounds (bottom
half) for three examples.
Small trilevelLarge randomMasses
PLSF, K?
PLSF, Klb
CLF, P?
CLF, Plb
MPC
Jlb
LQR
Prescient
lq
12.91
11.15
11.12
10.76
10.87
9.12
7.47
6.67
31.27
27.13
27.23
25.60
25.70
23.83
16.81
17.30
269.84
70.91
68.01
61.06
58.87
43.15
4.66
11.78
lq
• Control-Lyapunov function (CLF) with Vclf(z) = zTP?
Vclf(z) = zTPlbz, respectively.
• Model predictive control (MPC) with horizon T = 30.
We do not carry out exact MPC for the trilevel example,
since this requires solving a mixed-integer quadratic program at
each step. Instead we solve the convex relaxation, with u(t) ∈
[−0.2,0.2], and round the value obtained to {−0.2,0,0.2}.
These suboptimal performance values are compared with
several lower bounds, described below.
• The prescient bound, which is the value of J obtained when the
controlu(t)iscomputedwithknowledgeofall(pastandfuture)
disturbances. This is found by Monte Carlo: in each step, we
generate a realization of w, and solve the resulting quadratic
program over a long horizon. For the trilevel example, we do
not solve prescient optimal control problem exactly, since it
is a mixed-integer quadratic program. Instead we solve the
relaxation, which gives a lower bound.
• The LQR cost, which gives a simple lower bound.
• The lower bound Jlbfound by our method. For the first example,
with finite U, we use the optimal value of (14) as our lower
bound; for the second and third examples, with box U, we use
the optimal value of (15) as our lower bound.
The results are shown in Table 1. We can see that, in general,
projected linear state feedback with the gain matrix found as a
by-product of our bound computation does better than the simple
LQR gain matrix. Control-Lyapunov control policies do better still,
againbetterwiththecontrol-Laypunovfunctionthatarisesasaby-
product of our bound computation. Finally, MPC does very well.
On the lower bound side, we can see that Jlbis often much
better than the prescient bound, and the simple bound from LQR.
In two of the examples, our lower bound is quite close to the
performance achieved by the best suboptimal control policies. For
the masses example, the gap is larger, but it is still interesting
and useful to know that the MPC control policy is no more than
100(58.87 − 43.15)/43.15 = 36% suboptimal. This is not at all
obvious.
lqz and
5. Conclusions
We have shown how to effectively compute performance
bounds for linear stochastic control problems using convex
optimization. Our bounds are often close to the performance
attained by suboptimal controllers based on control-Lyapunov
functions and MPC. In these cases, our method shows that these
controllers are close to optimal.
Acknowledgement
Yang Wang is supported by a Rambus Corporation Stanford
Graduate Fellowship.
References
[1] M. Ait-Rami, L. El Ghaoui, Solving non-standard Riccati equations using LMI
optimization, in: Proceedings of the 34th Conference on Decision and Control,
1995.
[2] K.J. Åström, Introduction to Stochastic Control Theory, Academic Press, 1970.
[3] F.A. Badawi, On a quadratic matrix inequality and the corresponding algebraic
Riccati equation, International Journal of Control 36 (1982) 313–322.
[4] A. Bemporad, M. Morari, V. Dua, E.N. Pistikopoulos, The explicit linear
quadratic regulator for constrained systems, Automatica 38 (1) (2002)
3–20.
[5] D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 1, Athena
Scientific, 2005.
[6] D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 2, Athena
Scientific, 2007.
[7] S. Bittanti, A.J. Laub, J.C. Willems, The Riccati Equation, Springer-Verlag, 1991.
[8] S. Boyd, L. El Ghaoui, E. Feron, V. Balakrishnan, Linear Matrix Inequalities in
Systems and Control Theory, SIAM Books, Philadelphia, 1994.
[9] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press,
2004.
[10] E.F. Camacho, C. Bordons, Model Predictive Control, Springer-Verlag, 2004.
[11] M. Corless, G. Leitmann, Controller design for uncertain system via Lyapunov
functions, in: Proceedings of the American Control Conference, vol. 3, 1988.
[12] R.A. Freeman, J.A. Primbs, Control Lyapunov functions, new ideas from an old
source, in: Proceedings of the 35th IEEE Conference on Decision and Control,
vol. 4, 1996.
[13] O. Hernàndez-Lerma, J.B. Lassere, Discrete-Time Markov Control Processes:
Basic Optimality Criteria, Springer, 1997.
[14] R.E. Kalman, When is a linear control system optimal? Journal of Basic
Engineering 86 (1) (1964) 1–10.
[15] W.H. Kwon, S. Han, Receding Horizon Control, Springer-Verlag, 2005.
[16] J.M. Maciejowski, Predictive Control with Constraints, Prentice-Hall, 2002.
[17] J. Nocedal, S.J. Wright, Numerical Optimization, Springer, 1999.
[18] F.A. Potra, S.J. Wright, Interior-point methods, Journal of Computational and
Applied Mathematics 124 (1–2) (2000) 281–302.
[19] S.J. Qin, T.A. Badgwell, A survey of industrial model predictive control
technology, Control Engineering Practice 11 (7) (2003) 733–764.
[20] M.A. Rami, X.Y. Zhou, Linear matrix inequalities, Riccati equations and
indefinitestochasticlinearquadraticcontrols,IEEETransactionsonAutomatic
Control 45 (6) (2000) 1131–1143.
[21] R. Soeterboek, Predictive Control: A Unified Approach, Prentice Hall, 1992.
[22] E.D. Sontag, A Lyapunov-like characterization of asymptotic controllability,
SIAM Journal on Control and Optimization 21 (3) (1983) 462–471.
[23] M. Sznaier, R. Suarez, J. Cloutier, Suboptimal control of constrained nonlinear
systems via receding horizon constrained control Lyapunov functions,
International Journal on Robust and Nonlinear Control 13 (3–4) (2003)
247–259.
[24] L. Vandenberghe, S. Boyd, Semidefinite programming, SIAM Review 38 (1)
(1996) 49–95.
[25] Y. Wang, S. Boyd, Fast model predictive control using online optimization, in:
Proceedings of the 17th IFAC World Congress, 2008.
[26] P. Whittle, Optimization Over Time, John Wiley & Sons, Inc., New York, NY,
USA, 1982.
[27] H.S. Witsenhausen, On performance bounds for uncertain systems, SIAM
Journal on Control 8 (1) (1970) 55–89.
[28] S.J. Wright, Primal-Dual Interior-Point Methods, SIAM, 1997.
[29] K. Zhou, J.C. Doyle, K. Glover, Robust and Optimal Control, Prentice Hall,
1995.