-
[show abstract]
[hide abstract]
ABSTRACT: This article is concerned with stability and performance of controlled
stochastic processes under receding horizon policies. We carry out a systematic
study of methods to guarantee stability under receding horizon policies via
appropriate selections of cost functions in the underlying finite-horizon
optimal control problem. We also obtain quantitative bounds on the performance
of the system under receding horizon policies as measured by the long-run
expected average cost. The results are illustrated with the help of several
simple examples.
04/2013;
-
[show abstract]
[hide abstract]
ABSTRACT: This article deals with stability of continuous-time switched linear systems
under constrained switching. Given a family of linear systems, possibly
containing unstable dynamics, we characterize a new class of switching signals
under which the switched linear system generated by it and the family of
systems is globally asymptotically stable. Our characterization of such
stabilizing switching signals involves the asymptotic frequency of switching,
the asymptotic fraction of activation of the constituent systems, and the
asymptotic densities of admissible transitions among them. Our techniques
employ multiple Lyapunov-like functions, and extend preceding results both in
scope and applicability.
03/2013;
-
[show abstract]
[hide abstract]
ABSTRACT: We study stochastic motion planning problems which involve a controlled
process, with possibly discontinuous sample paths, visiting certain subsets of
the state-space while avoiding others in a sequential fashion. For this
purpose, we first introduce two basic notions of motion planning, and then
establish a connection to a class of stochastic optimal control problems
concerned with sequential stopping-times. A weak dynamic programming principle
(DPP) is then proposed, which characterizes the set of initial states that
admit the existence of a policy enabling the process to execute the desired
maneuver with probability at least as much as some pre-specified value. The
proposed DPP consists of some auxiliary value functions defined in terms of
discontinuous payoff functions. An application of the DPP is demonstrated in
the context of controlled diffusion processes thereafter. It turns out that the
aforementioned set of initial states can be characterized as the level set of a
discontinuous viscosity solution to a sequence of partial differential
equations, for which the first one has a known boundary condition, while the
boundary conditions of the subsequent ones are determined by the solutions to
the preceding steps. Finally, the generality and flexibility of the theoretical
results are illustrated with the aid of an example involving biological
switches.
11/2012;
-
[show abstract]
[hide abstract]
ABSTRACT: We develop a framework for formulating a class of stochastic reachability
problems with state constraints as a stochastic optimal control problem.
Previous approaches to solving these problems are either confined to the
deterministic setting or address almost-sure stochastic notions. In contrast,
we propose a new methodology to tackle probabilistic specifications that are
less stringent than almost sure requirements. To this end, we first establish a
connection between two stochastic reach-avoid problems and two classes of
different stochastic optimal control problems for diffusions with discontinuous
payoff functions. Subsequently, we focus on solutions to one of the classes of
stochastic optimal control problems---the exit-time problem, which solves both
the reach-avoid problems mentioned above. We then derive a weak version of a
dynamic programming principle (DPP) for the corresponding value function; in
this direction our contribution compared to the existing literature is to allow
for discontinuous payoff functions. Moreover, based on our DPP, we give an
alternative characterization of the value function as a solution to a partial
differential equation in the sense of discontinuous viscosity solutions, along
with boundary conditions both in Dirichlet and viscosity senses. Theoretical
justifications are discussed so as to employ off-the-shelf PDE solvers for
numerical computations. Finally, we validate the performance of the proposed
framework on the stochastic Zermelo navigation problem.
02/2012;
-
[show abstract]
[hide abstract]
ABSTRACT: We present a new matrix-valued isospectral ordinary differential equation
that asymptotically block-diagonalizes $n\times n$ zero-diagonal Jacobi
matrices employed as its initial condition. This o.d.e.\ features a right-hand
side with a nested commutator of matrices, and structurally resembles the
double-bracket o.d.e.\ studied by R.W.\ Brockett in 1991. We prove that its
solutions converge asymptotically, that the limit is block-diagonal, and above
all, that the limit matrix is defined uniquely as follows: For $n$ even, a
block-diagonal matrix containing $2\times 2$ blocks, such that the
super-diagonal entries are sorted by strictly increasing absolute value.
Furthermore, the off-diagonal entries in these $2\times 2$ blocks have the same
sign as the respective entries in the matrix employed as initial condition. For
$n$ odd, there is one additional $1\times 1$ block containing a zero that is
the top left entry of the limit matrix. The results presented here extend some
early work of Kac and van Moerbeke.
02/2012;
-
Automatica. 01/2012; 48:77-88.
-
Systems & Control Letters. 01/2012; 61:375-380.
-
[show abstract]
[hide abstract]
ABSTRACT: We establish conditions for uniform $r$-th moment bound of certain
$\R^d$-valued functions of a discrete-time stochastic process taking values in
a general metric space. The conditions include an appropriate negative drift
together with a uniform $L_p$ bound on the jumps of the process for $p > r +
1$. Applications of the result are given in connection to iterated function
systems and biochemical reaction networks.
07/2011;
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a procedure to design a state-quantizer with finitely many bins
for a marginally stable stochastic linear system evolving in $\R^d$, and a
bounded policy based on the resulting quantized state measurements to ensure
bounded second moment in closed-loop.
03/2011;
-
Automatica. 01/2011; 47:2082-2087.
-
IEEE Trans. Automat. Contr. 01/2011; 56:2704-2710.
-
[show abstract]
[hide abstract]
ABSTRACT: We consider the problem of controlling marginally stable linear systems using bounded control inputs for networked control settings in which the communication channel between the remote controller and the system is unreliable. We assume that the states are perfectly observed, but the control inputs are transmitted over a noisy communication channel. Under mild hypotheses on the noise introduced by the control communication channel and large enough control authority, we construct a control policy that renders the state of the closed-loop system mean-square bounded. Comment: 11 pages, 1 figure
04/2010;
-
[show abstract]
[hide abstract]
ABSTRACT: We provide a solution to the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and imperfect state measurements. For a suitable choice of control policies, we show that the finite-horizon optimization problem to be solved on-line is convex and successively feasible. Due to the inherent nonlinearity of the feedback loop, a slight extension of the Kalman filter is exploited to estimate the state optimally in mean-square sense. We show that the receding horizon implementation of the resulting control policies renders the state of the overall system mean-square bounded under mild assumptions. Finally, we discuss how some of the quantities required by the finite-horizon optimization problem can be computed off-line, reducing the on-line computation, and present some numerical examples. Comment: 25 pages, 4 figures
01/2010;
-
[show abstract]
[hide abstract]
ABSTRACT: Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which
has become a standard for expressing temporal properties of finite-state Markov
chains in the context of automated model checking. In this paper, we give a
definition of PCTL for noncountable-space Markov chains, and we show that there
is a substantial affinity between certain of its operators and problems of
Dynamic Programming. After proving some uniqueness properties of the solutions
to the latter, we conclude the paper with two examples to show that some
recovery strategies in practical applications, which are naturally stated as
reach-avoid problems, can be actually viewed as particular cases of PCTL
formulas.
10/2009;
-
[show abstract]
[hide abstract]
ABSTRACT: We construct control policies that ensure bounded variance of a noisy marginally stable linear system in closed-loop. It is assumed that the noise sequence is a mutually independent sequence of random vectors, enters the dynamics affinely, and has bounded fourth moment. The magnitude of the control is required to be of the order of the first moment of the noise, and the policies we obtain are simple and computable. Comment: 10 pages
IEEE Transactions on Automatic Control 07/2009; · 2.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We investigate constrained optimal control problems for linear stochastic
dynamical systems evolving in discrete time. We consider minimization of an
expected value cost over a finite horizon. Hard constraints are introduced
first, and then reformulated in terms of probabilistic constraints. It is shown
that, for a suitable parametrization of the control policy, a wide class of the
resulting optimization problems are convex, or admit reasonable convex
approximations.
05/2009;
-
[show abstract]
[hide abstract]
ABSTRACT: We present a dynamic programming-based solution to the problem of maximizing
the probability of attaining a target set before hitting a cemetery set for a
discrete-time Markov control process. Under mild hypotheses we establish that
there exists a deterministic stationary policy that achieves the maximum value
of this probability. We demonstrate how the maximization of this probability
can be computed through the maximization of an expected total reward until the
first hitting time to either the target or the cemetery set. Martingale
characterizations of thrifty, equalizing, and optimal policies in the context
of our problem are also established.
04/2009;
-
[show abstract]
[hide abstract]
ABSTRACT: We design receding horizon control strategies for stochastic discrete-time
linear systems with additive (possibly) unbounded disturbances, while obeying
hard bounds on the control inputs. We pose the problem of selecting an
appropriate optimal controller on vector spaces of functions and show that the
resulting optimization problem has a tractable convex solution. Under the
assumption that the zero-input and zero-noise system is asymptotically stable,
we show that the variance of the state is bounded when enforcing hard bounds on
the control inputs, for any receding horizon implementation. Throughout the
article we provide several examples that illustrate how quantities needed in
the formulation of the resulting optimization problems can be calculated
off-line, as well as comparative examples that illustrate the effectiveness of
our control strategies.
03/2009;
-
[show abstract]
[hide abstract]
ABSTRACT: This paper is concerned with the problem of Model Predictive Control and Rolling Horizon Control of discrete-time systems subject to possibly unbounded random noise inputs, while satisfying hard bounds on the control inputs. We use a nonlinear feedback policy with respect to noise measurements and show that the resulting mathematical program has a tractable convex solution in both cases. Moreover, under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state, under the resulting Model Predictive Control and Rolling Horizon Control policies, is bounded. Finally, we provide some numerical examples on how certain matrices in the underlying mathematical program can be calculated off-line. Comment: 8 pages
02/2009;
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact target set while simultaneously minimizing an expected discounted cost. We then provide a rolling-horizon strategy for approximating the optimal policy, together with quantitative characterization of its sub-optimality with respect to the optimal policy.
Proceedings of the 48th IEEE Conference on Decision and Control, CDC 2009, combined withe the 28th Chinese Control Conference, December 16-18, 2009, Shanghai, China; 01/2009
