No file available
To read the file of this research,
you can request a copy directly from the authors.
An algorithmic guide for finite-dimensional optimal control problems
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
We survey the main numerical techniques for finite-dimensional nonlinear optimal control. The chapter is written as a guide to practitioners who wish to get rapidly acquainted with the main numerical methods used to efficiently solve an optimal control problem. We consider two classical examples, simple but significant enough to be enriched and generalized to other settings: Zermelo and Goddard problems. We provide sample of the codes used to solve them and make these codes available online. We discuss direct and indirect methods, Hamilton-Jacobi approach, ending with optimistic planning. The examples illustrate the pros and cons of each method, and we show how these approaches can be combined into powerful tools for the numerical solution of optimal control problems for ordinary differential equations.
... The discretization of the HJB equation is carried out using the form of (31), which enables convenient computation. The partial differentials are discretized using the monotone and consistent upwind method (e.g., [71][72][73] ...
Environmental management and restoration should be designed such that the risk and uncertainty owing to nonlinear stochastic systems can be successfully addressed. We apply the robustified dynamic Orlicz risk to the modeling and analysis of environmental management and restoration to consider both the risk and uncertainty within a unified theory. We focus on the control of a jump-driven hybrid stochastic system that represents macrophyte dynamics. The dynamic programming equation based on the Orlicz risk is first obtained heuristically, from which the associated Hamilton-Jacobi-Bellman (HJB) equation is derived. In the proposed Orlicz risk, the risk aversion of the decision-maker is represented by a power coefficient that resembles a certainty equivalence, whereas the uncertainty aversion is represented by the Kullback-Leibler divergence, in which the risk and uncertainty are handled consistently and separately. The HJB equation includes a new state-dependent discount factor that arises from the uncertainty aversion, which leads to a unique, nonlinear, and nonlocal term. The link between the proposed and classical stochastic control problems is discussed with a focus on control-dependent discount rates. We propose a finite difference method for computing the HJB equation. Finally, the proposed model is applied to an optimal harvesting problem for macrophytes in a brackish lake that contains both growing and drifting populations.
This paper is concerned with the relationship between the maximum principle and dynamic programming for a large class of optimal control problems with maximum running cost. Inspired by a technique introduced by Vinter in the 1980s, we are able to obtain jointly a global and a partial sensitivity relation that link the coextremal with the value function of the problem at hand. One of the main contributions of this work is that these relations are derived by using a single perturbed problem, and therefore, both sensitivity relations hold, at the same time, for the same coextremal. As a by-product, and thanks to the level-set approach, we obtain a new set of sensitivity relations for Mayer problems with state constraints. One important feature of this last result is that it holds under mild assumptions, without the need of imposing strong compatibility assumptions between the dynamics and the state constraints set.
In this work, we study optimistic planning methods to solve some state-constrained optimal control problems in finite horizon. While classical methods for calculating the value function are generally based on a discretization in the state space, optimistic planning algorithms have the advantage of using adaptive discretization in the control space. These approaches are therefore very suitable for control problems where the dimension of the control variable is low and allow to deal with problems where the dimension of the state space can be very high. Our algorithms also have the advantage of providing, for given computing resources, the best control strategy whose performance is as close as possible to optimality while its corresponding trajectory comply with the state constraints up to a given accuracy.
In this paper, we consider a class of optimal control problems governed by a differential system. We analyse the sensitivity relations satisfied by the co-state arc of the Pontryagin maximum principle and the value function that associates the optimal value of the control problem to the initial time and state. Such a relationship has been already investigated for state-constrained problems under some controllability assumptions to guarantee Lipschitz regularity property of the value function. Here, we consider the case with intermediate and final state constraints, without any controllability assumption on the system, and without Lipschitz regularity of the value function. Because of this lack of regularity, the sensitivity relations cannot be expressed with the sub-differentials of the value function. This work shows that the constrained problem can be reformulated with an auxiliary value function which is more regular and suitable to express the sensitivity of the adjoint arc of the original state-constrained control problem along an optimal trajectory. Furthermore, our analysis covers the case of normal optimal solutions, and abnormal solutions as well.
This work presents a convex-optimization-based framework for analysis and control of nonlinear partial differential equations. The approach uses a particular weak embedding of the nonlinear PDE, resulting in a linear equation in the space of Borel measures. This equation is then used as a constraint of an infinite-dimensional linear programming problem (LP). This LP is then approximated by a hierarchy of convex, finite-dimensional, semidefinite programming problems (SDPs). In the case of analysis of uncontrolled PDEs, the solutions to these SDPs provide bounds on a specified, possibly nonlinear, functional of the solutions to the PDE; in the case of PDE control, the solutions to these SDPs provide bounds on the optimal value of a given optimal control problem as well as suboptimal feedback controllers. The entire approach is based purely on convex optimization and does not rely on spatio-temporal gridding, even though the PDE addressed can be fully nonlinear. The approach is applicable to a very broad class nonlinear PDEs with polynomial data. Computational complexity is analyzed and several complexity reduction procedures are described. Numerical examples demonstrate the approach.
The aim of this article is to study the Hamilton Jacobi Bellman (HJB) approach for state-constrained control problems with maximum cost. In particular, we are interested in the characterization of the value functions of such problems and the analysis of the associated optimal trajectories, without assuming any controllability assumption. The rigorous theoretical results lead to several trajectory reconstruction procedures for which convergence results are also investigated. An application to a five-state aircraft abort landing problem is then considered, for which several numerical simulations are performed to analyse the relevance of the theoretical approach.
We introduce a new class of "filtered" schemes for some first order nonlinear Hamilton-Jacobi equations. The work follows recent ideas of Froese and Oberman [SIAM J. Numer. Anal., 51 (2013), pp. 423-444] and Oberman and Salvador [J. Comput. Phys., 284 (2015), pp. 367-388] for steady equations. Here we mainly study the time-dependent setting and focus on fully explicit schemes. Furthermore, specific corrections to the filtering idea are also needed in order to obtain high-order accuracy. The proposed schemes are not monotone but still satisfy some ε-monotone property. A general convergence result together with a precise error estimate of order √Δx are given (Δx is the mesh size). The framework allows us to construct finite difference discretizations that are easy to implement and high-order in the domain where the solution is smooth. A novel error estimate is also given in the case of the approximation of steady equations. Numerical tests including evolutive convex and nonconvex Hamiltonians, and obstacle problems are presented to validate the approach. We show with several examples how the filter technique can be applied to stabilize an otherwise unstable high-order scheme.
An approach to solve finite time horizon suboptimal feedback control problems for partial differential equations is proposed by solving dynamic programming equations on adaptive sparse grids. The approach is illustrated for the wave equation and an extension to equations of Schrödinger type is indicated. A semi-discrete optimal control problem is introduced and the feedback control is derived from the corresponding value function.The value function can be characterized as the solution of an evolutionary Hamilton-Jacobi Bellman (HJB) equation which is defined over a state space whose dimension is equal to the dimension of the underlying semi-discrete system. Besides a low dimensional semi-discretization it is important to solve the HJB equation efficiently to address the curse of dimensionality.We propose to apply a semi-Lagrangian scheme using spatially adaptive sparse grids. Sparse grids allow the discretization of the value functions in (higher) space dimensions since the curse of dimensionality of full grid methods arises to a much smaller extent. For additional efficiency an adaptive grid refinement procedure is explored.We present several numerical examples studying the effect the parameters characterizing the sparse grid have on the accuracy of the value function and the optimal trajectory.
It is well known that symplectic Runge-Kutta and Partitioned Runge-Kutta
methods exactly preserve {\em quadratic} first integrals (invariants of motion)
of the system being integrated. While this property is often seen as a mere
curiosity (it does not hold for arbitrary first integrals), it plays an
important role in the computation of numerical sensitivities, optimal control
theory and Lagrangian mechanics, as described in this paper, which, together
with some new material, presents in a unified way a number of results now
scattered or implicit in the literature. Some widely used procedures, such as
the direct method in optimal control theory and the computation of
sensitivities via reverse accumulation imply "hidden" integrations with
symplectic Partitioned Runge-Kutta schemes.
This paper exposes a methodology to solve state and input constrained optimal control problems for nonlin-ear systems. In the presented 'interior penalty' approach, constraints are penalized in a way that guarantees the strict interiority of the approaching solutions. This property allows one to invoke simple (without con-straints) stationarity conditions to characterize the unknowns. A constructive choice for the penalty functions is exhibited. The property of interiority is established, and practical guidelines for implementation are given. A numerical benchmark example is given for illustration.
We present an abstract convergence result for the fixed point approximation of station-ary Hamilton–Jacobi equations. The basic assumptions on the discrete operator are invariance with respect to the addition of constants, ε-monotonicity and consistency. The result can be applied to various high-order approximation schemes which are illustrated in the paper. Several applications to Hamilton–Jacobi equations and numerical tests are presented.
The problem of the vertical flight of a rocket in a resisting medium with a constraint of isoperimetric type is studied. Only the case of maximum altitude for given flight time has been analyzed, but the methodology applies to other types of constraints as well, e.g., the minimum-time problem, due to Mayer reciprocity. Analysis shows that a one-parameter family of singular extremals is generated according to the value of time of flight. Three cases of switching structure have been found, varying with the specified duration of flight, with the most interesting case featuring appearance of a second full-thrust subarc at the departure from the singular subarc, owing to a low value of the upper bound on the thrust.
The paper deals with deterministic optimal control problems with state constraints and non-linear dynamics. It is known for such problems that the value function is in general discontinuous and its characterization by means of a Hamilton-Jacobi equation requires some controllability assumptions involving the dynamics and the set of state constraints. Here, we first adopt the viability point of view and look at the value function as its epigraph. Then, we prove that this epigraph can always be described by an auxiliary optimal control problem free of state constraints, and for which the value function is Lipschitz continuous and can be characterized, without any additional assumptions, as the unique viscosity solution of a Hamilton-Jacobi equation. The idea introduced in this paper bypasses the regularity issues on the value function of the constrained control problem and leads to a constructive way to compute its epigraph by a large panel of numerical schemes. Our approach can be extended to more general control problems. We study in this paper the extension to the infinite horizon problem as well as for the two-player game setting. Finally, an illustrative numerical example is given to show the relevance of the approach.
We propose a semi-Lagrangian scheme using a spatially adaptive sparse grid to deal with non-linear time-dependent Hamilton-Jacobi Bellman equations. We focus in particular on front propagation models in higher dimensions which are related to control problems. We test the numerical efficiency of the method on several benchmark problems up to space dimension d=8, and give evidence of convergence towards the exact viscosity solution. In addition, we study how the complexity and precision scale with the dimension of the problem.
. From a theoretical viewpoint, the GPM has developments and impact in var-ious area of Mathematics like algebra, Fourier analysis, functional analysis, operator theory, probabilityand statistics, to cite a few. In addition, and despite its rather simple and short formulation, the GPMhas a large number of important applications in various fields like optimization, probability, mathematicalfinance, optimal control, control and signal processing, chemistry, cristallography, tomography, quantumcomputing, etc.In its full generality, the GPM is untractable numerically. However when K is a compact basic semi-algebraic set, and the functions involved are polynomials (and in some cases piecewise polynomials orrational functions), then the situation is much nicer. Indeed, one can define a systematic numerical schemebased on a hierarchy of semidefinite programs, which provides a monotone sequence that converges tothe optimal value of the GPM. (A semidefinite program is a convex optimization problem which up toarbitrary fixed precision, can be solved in polynomial time.) Sometimes finite convergence may evenocccur.In the talk, we will present the semidefinite programming methodology to solve the GPM and describein detail several applications of the GPM (notably in optimization, probability, optimal control andmathematical finance).R´ef´erences[1] J.B. Lasserre, Moments, Positive Polynomials and their Applications, Imperial College Press, inpress.[2] J.B. Lasserre, A Semidefinite programming approach to the generalized problem of moments,Math. Prog. 112 (2008), pp. 65–92.
A general method for constructing high-order approximation schemes for
Hamilton-Jacobi-Bellman equations is given. The method is based on a
discrete version of the Dynamic Programming Principle. We prove a
general convergence result for this class of approximation schemes also
obtaining, under more restrictive assumptions, an estimate in
of the order of convergence and of the local truncation error. The
schemes can be applied, in particular, to the stationary linear first
order equation in . We present several
examples of schemes
belonging to this class and with fast convergence to the solution.
Optimal control of piecewise deterministic processes with state space constraint is studied. Under appropriate assumptions, it is shown that the optimal value function is the only viscosity solution on the open domain which is also a supersolution on the closed domain. Finally, the uniform continuity of the value function is obtained under a condition on the deterministic drift.
Hamilton-Jacobi (H-J) equations are frequently encountered in applications, e.g., in control theory and differential games. H-J equations are closely related to hyperbolic conservation laws - in one space dimension the former is simply the integrated version of the latter. Similarity also exists for the multi-dimensional case, and this is helpful in the design of difference approximations. High order essentially non-oscillatory (ENO) schemes for H-J equations are investigated which yield uniform high order accuracy in smooth regions and resolve discontinuities in the derivatives sharply. The ENO scheme construction procedure is adapted from that for hyperbolic conservation laws. The schemes are numerically tested on a variety of one-dimensional and two-dimensional problems, including a problem related to control-optimization. High order accuracy in smooth regions, good resolution of discontinuities in the derivatives, and convergence to viscosity solutions are observed.
The turnpike property refers to the phenomenon that in many optimal control problems, the solutions for different initial conditions and varying horizons approach a neighborhood of a specific steady state, then stay in this neighborhood for the major part of the time horizon, until they may finally depart. While early observations of the phenomenon can be traced back to works of Ramsey and von Neumann on problems in economics in 1928 and 1938, the turnpike property received continuous interest in economics since the 1960s and recent interest in systems and control. The present chapter provides an introductory overview of discrete-time and continuous-time results in finite and infinite-dimensions. We comment on dissipativity-based approaches and infinite-horizon results, which enable the exploitation of turnpike properties for the numerical solution of problems with long and infinite horizons. After drawing upon numerical examples, the chapter concludes with an outlook on time-varying, discounted, and open problems.
In this article, based on two case studies, we discuss the role of abnormal geodesics in planar Zermelo navigation problems. Such curves are limit curves of the accessibility set, in the domain where the current is strong. The problem is set in the frame of geometric time optimal control, where the control is the heading angle of the ship and in this context, abnormal curves are shown to separate time minimal curves from time maximal curves and are both small-time minimizing and maximizing. We describe the small-time minimal balls. For bigger time, a cusp singularity can occur in the abnormal direction, which corresponds to a conjugate point along the non-smooth image. It is interpreted in terms of the regularity property of the time minimal value function.
We devise new numerical algorithms, called PSC algorithms, for following fronts propagating with curvature-dependent speed. The speed may be an arbitrary function of curvature, and the front also can be passively advected by an underlying flow. These algorithms approximate the equations of motion, which resemble Hamilton-Jacobi equations with parabolic right-hand sides, by using techniques from hyperbolic conservation laws. Non-oscillatory schemes of various orders of accuracy are used to solve the equations, providing methods that accurately capture the formation of sharp gradients and cusps in the moving fronts. The algorithms handle topological merging and breaking naturally, work in any number of space dimensions, and do not require that the moving surface be written as a function. The methods can be also used for more general Hamilton-Jacobi-type problems. We demonstrate our algorithms by computing the solution to a variety of surface motion problems.
From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning covers several aspects of the "optimism in the face of uncertainty" principle for large scale optimization problems under finite numerical budget. The monograph's initial motivation came from the empirical success of the so-called "Monte-Carlo Tree Search" method popularized in Computer Go and further extended to many other games as well as optimization and planning problems. It lays out the theoretical foundations of the field by characterizing the complexity of the optimization problems and designing efficient algorithms with performance guarantees. The main direction followed in this monograph consists in decomposing a complex decision making problem (such as an optimization problem in a large search space) into a sequence of elementary decisions, where each decision of the sequence is solved using a stochastic "multi-armed bandit" (mathematical model for decision making in stochastic environments). This defines a hierarchical search which possesses the nice feature of starting the exploration by a quasi-uniform sampling of the space and then focusing, at different scales, on the most promising areas (using the optimistic principle) until eventually performing a local search around the global optima of the function. This monograph considers the problem of function optimization in general search spaces (such as metric spaces, structured spaces, trees, and graphs) as well as the problem of planning in Markov decision processes. Its main contribution is a class of hierarchical optimistic algorithms with different algorithmic instantiations depending on whether the evaluations are noisy or noiseless and whether some measure of the local "smoothness" of the function around the global maximum is known or unknown.
The classical dynamic programming (DP) approach to optimal control problems is based on the characterization of the value function as the unique viscosity solution of a Hamilton--Jacobi--Bellman equation. The DP scheme for the numerical approximation of viscosity solutions of Bellman equations is typically based on a time discretization which is projected on a fixed state-space grid. The time discretization can be done by a one-step scheme for the dynamics and the projection on the grid typically uses a local interpolation. Clearly the use of a grid is a limitation with respect to possible applications in high-dimensional problems due to the curse of dimensionality. Here, we present a new approach for finite horizon optimal control problems where the value function is computed using a DP algorithm with a tree structure algorithm constructed by the time discrete dynamics. In this way there is no need to build a fixed space triangulation and to project on it. The tree will guarantee a perfect matching with the discrete dynamics and drop off the cost of the space interpolation allowing for the solution of very high-dimensional problems. Numerical tests will show the effectiveness of the proposed method.
Read More: https://epubs.siam.org/doi/10.1137/18M1203900
We present CasADi, an open-source software framework for numerical optimization. CasADi is a general-purpose tool that can be used to model and solve optimization problems with a large degree of flexibility, larger than what is associated with popular algebraic modeling languages such as AMPL, GAMS, JuMP or Pyomo. Of special interest are problems constrained by differential equations, i.e. optimal control problems. CasADi is written in self-contained C++, but is most conveniently used via full-featured interfaces to Python, MATLAB or Octave. Since its inception in late 2009, it has been used successfully for academic teaching as well as in applications from multiple fields, including process control, robotics and aerospace. This article gives an up-to-date and accessible introduction to the CasADi framework, which has undergone numerous design improvements over the last 7 years.
We consider discrete-time, infinite-horizon optimal control problems with discounted rewards. The value function must be Lipschitz continuous over action (input) sequences, the actions are in a scalar interval, while the dynamics and rewards can be nonlinear/nonquadratic. Exploiting ideas from artificial intelligence, we propose two optimistic planning methods that perform an adaptive-horizon search over the infinite-dimensional space of action sequences. The first method optimistically refines regions with the largest upper bound on the optimal value, using the Lipschitz constant to find the bounds. The second method simultaneously refines all potentially optimistic regions, without explicitly using the bounds. Our analysis proves convergence rates to the global infinite-horizon optimum for both algorithms, as a function of computation invested and of a measure of problem complexity. It turns out that the second, simultaneous algorithm works nearly as well as the first, despite not needing to know the (usually difficult to find) Lipschitz constant. We provide simulations showing the algorithms are useful in practice, compare them with value iteration and model predictive control, and give a real-time example.
We present a simple, purely analytic method for proving the convergence of a wide class of approximation schemes to the solution of fully non linear second-order elliptic or parabolic PDE. Roughly speaking, we prove that any monotone, stable and consistent scheme converges to the correct solution provided that there exists a comparison principle for the limiting equation. This method is based on the notion of viscosity solution of Crandall and Lions and it gives completely new results concerning the convergence of numerical schemes for stochastic differential games.
Recently M. G. Crandall and P. L. Lions introduced the notion of “viscosity solutions” of scalar nonlinear first order partial differential equations. Viscosity solutions need not be differentiable anywhere and thus are not sensitive to the classical problem of the crossing of characteristics. The value of this concept is established by the fact that very general existence, uniqueness and continuous dependence results hold for viscosity solutions of many problems arising in fields of application. The notion of a “viscosity solution” admits several equivalent formulations. Here we look more closely at two of these equivalent criteria and exhibit their virtues by both proving several new facts and reproving various known results in a simpler manner. Moreover, by forsaking technical generality we hereby provide a more congenial introduction to this subject than the original paper.
We present a primal-dual interior-point algorithm with a lter line-search method for non- linear programming. Local and global convergence properties of this method were analyzed in previous work. Here we provide a comprehensive description of the algorithm, including the fea- sibility restoration phase for the lter method, second-order corrections, and inertia correction of the KKT matrix. Heuristics are also considered that allow faster performance. This method has been implemented in the IPOPT code, which we demonstrate in a detailed numerical study based on 954 problems from the CUTEr test set. An evaluation is made of several line-search options, and a comparison is provided with two state-of-the-art interior-point codes for nonlin- ear programming.
Mécanique céleste et contrôle de systèmes spatiaux
This book may be regarded as consisting of two parts. In Chapters I-IV we pre sent what we regard as essential topics in an introduction to deterministic optimal control theory. This material has been used by the authors for one semester graduate-level courses at Brown University and the University of Kentucky. The simplest problem in calculus of variations is taken as the point of departure, in Chapter I. Chapters II, III, and IV deal with necessary conditions for an opti mum, existence and regularity theorems for optimal controls, and the method of dynamic programming. The beginning reader may find it useful first to learn the main results, corollaries, and examples. These tend to be found in the earlier parts of each chapter. We have deliberately postponed some difficult technical proofs to later parts of these chapters. In the second part of the book we give an introduction to stochastic optimal control for Markov diffusion processes. Our treatment follows the dynamic pro gramming method, and depends on the intimate relationship between second order partial differential equations of parabolic type and stochastic differential equations. This relationship is reviewed in Chapter V, which may be read inde pendently of Chapters I-IV. Chapter VI is based to a considerable extent on the authors' work in stochastic control since 1961. It also includes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle.
Preface.- Basic notations.- Outline of the main ideas on a model problem.- Continuous viscosity solutions of Hamilton-Jacobi equations.- Optimal control problems with continuous value functions: unrestricted state space.- Optimal control problems with continuous value functions: restricted state space.- Discontinuous viscosity solutions and applications.- Approximation and perturbation problems.- Asymptotic problems.- Differential Games.- Numerical solution of Dynamic Programming.- Nonlinear H-infinity control by Pierpaolo Soravia.- Bibliography.- Index
Normed Spaces.- Convex sets and functions.- Weak topologies.- Convex analysis.- Banach spaces.- Lebesgue spaces.- Hilbert spaces.- Additional exercises for Part I.- Optimization and multipliers.- Generalized gradients.- Proximal analysis.- Invariance and monotonicity.- Additional exercises for Part II.- The classical theory.- Nonsmooth extremals.- Absolutely continuous solutions.- The multiplier rule.- Nonsmooth Lagrangians.- Hamilton-Jacobi methods.- Additional exercises for Part III.- Multiple integrals.- Necessary conditions.- Existence and regularity.- Inductive methods.- Differential inclusions.- Additional exercises for Part IV.
The aim of this paper is to give a survey on some known results concerning the regularity properties of the minimum-time map around an equilibrium point of a control system and to discuss the links of these properties with the viscosity solutions of the Hamilton Jacobi Gellman equation. For sake of simplicity let us consider a control system on Rn defined by:
where the fi’s are C∞ vector fields and the control map u = (u1,...,um) belongs to the class u of the integrable maps with values in the set
\Omega = \left\{ {({\omega _1}, \ldots ,{\omega _m}) \in {R^m}:\left| {{\omega _i}} \right| \leqslant 1,i = 1, \cdots ,m} \right\}.
A pseudospectral method for generating optimal trajectories of linear and nonlinear constrained dynamic systems is proposed. The method consists of representing the solution of the optimal control problem by an mth degree interpolating polynomial, using Chebyshev nodes, and then discretizing the problem using a cell-averaging technique. The optimal control problem is thereby transformed into an algebraic nonlinear programming problem. Due to its dynamic nature, the proposed method avoids many of the numerical difficulties typically encountered in solving standard optimal control problems. Furthermore, for discontinuous optimal control problems, we develop and implement a Chebyshev smoothing procedure which extracts the piecewise smooth solution from the oscillatory solution near the points of discontinuities. Numerical examples are provided, which confirm the convergence of the proposed method. Moreover, a comparison is made with optimal solutions obtained by closed-form analysis and/or other numerical methods in the literature.
Turnpike properties have been established long time ago in finite-dimensional
optimal control problems arising in econometry. They refer to the fact that,
under quite general assumptions, the optimal solutions of a given optimal
control problem settled in large time consist approximately of three pieces,
the first and the last of which being transient short-time arcs, and the middle
piece being a long-time arc staying exponentially close to the optimal
steady-state solution of an associated static optimal control problem. We
provide in this paper a general version of a turnpike theorem, valuable for
nonlinear dynamics without any specific assumption, and for very general
terminal conditions. Not only the optimal trajectory is shown to remain
exponentially close to a steady-state, but also the corresponding adjoint
vector of the Pontryagin maximum principle. The exponential closedness is
quantified with the use of appropriate normal forms of Riccati equations. We
show then how the property on the adjoint vector can be adequately used in
order to initialize successfully a numerical direct method, or a shooting
method. In particular, we provide an appropriate variant of the usual shooting
method in which we initialize the adjoint vector, not at the initial time, but
at the middle of the trajectory.
This largely self-contained book provides a unified framework of semi-Lagrangian strategy for the approximation of hyperbolic PDEs, with a special focus on Hamilton–Jacobi equations. The authors provide a rigorous discussion of the theory of viscosity solutions and the concepts underlying the construction and analysis of difference schemes; they then proceed to high-order semi-Lagrangian schemes and their applications to problems in fluid dynamics, front propagation, optimal control, and image processing. The developments covered in the text and the references come from a wide range of literature.
Audience
Semi-Lagrangian Approximation Schemes for Linear and Hamilton–Jacobi Equations is written for advanced undergraduate and graduate courses on numerical methods for PDEs and for researchers and practitioners whose work focuses on numerical analysis and numerical methods for nonlinear hyperbolic PDEs.
approx. x + 325, List Price 69.30 / Order Code: OT133
http://www.ec-securehost.com/SIAM/OT133.html
This article surveys the usual techniques of nonlinear optimal control such as the Pontryagin Maximum Principle and the conjugate point theory, and how they can be implemented numerically, with a special focus on applications to aerospace problems. In practice the knowledge resulting from the maximum principle is often insufficient for solving the problem, in particular because of the well-known problem of initializing adequately the shooting method. In this survey article it is explained how the usual tools of optimal control can be combined with other mathematical techniques to improve significantly their performances and widen their domain of application. The focus is put onto three important issues. The first is geometric optimal control, which is a theory that has emerged in the 1980s and is combining optimal control with various concepts of differential geometry, the ultimate objective being to derive optimal synthesis results for general classes of control systems. Its applicability and relevance is demonstrated on the problem of atmospheric reentry of a space shuttle. The second is the powerful continuation or homotopy method, consisting of deforming continuously a problem toward a simpler one and then of solving a series of parameterized problems to end up with the solution of the initial problem. After having recalled its mathematical foundations, it is shown how to combine successfully this method with the shooting method on several aerospace problems such as the orbit transfer problem. The third one consists of concepts of dynamical system theory, providing evidence of nice properties of the celestial dynamics that are of great interest for future mission design such as low-cost interplanetary space missions. The article ends with open problems and perspectives.
We consider an optimal control problem under state constraints and show that to every optimal solution corresponds an adjoint state satisfying the first order necessary optimality conditions in the form of a maximum principle and sensitivity relations involving the value function. Such sensitivity relations were recently investigated by P. Bettiol and R.B. Vinter for state constraints with smooth boundary. In the difference with their work, our setting concerns differential inclusions and nonsmooth state constraints. To obtain our result we derive neighboring feasible trajectory estimates using a novel generalization of the so-called inward pointing condition.
This paper deals with the optimal control problem of an ordinary differential equation with several pure state constraints, of arbitrary orders, as well as mixed control-state constraints. We assume (i) the control to be continuous and the strengthened Legendre–Clebsch condition to hold, and (ii) a linear independence condition of the active constraints at their respective order to hold. We give a complete analysis of the smoothness and junction conditions of the control and of the constraints multipliers. This allows us to obtain, when there are finitely many nontangential junction points, a theory of no-gap second-order optimality conditions and a characterization of the well-posedness of the shooting algorithm. These results generalize those obtained in the case of a scalar-valued state constraint and a scalar-valued control.
Résumé
Dans cet article on s'intéresse au problème de commande optimale d'une équation différentielle ordinaire avec plusieures contraintes pures sur l'état, d'ordres quelconques, et des contraintes mixtes sur la commande et sur l'état. On suppose que (i) la commande est continue et la condition forte de Legendre–Clebsch satisfaite, et (ii) une condition d'indépendance linéaire des contraintes actives est satisfaite. Des résultats de régularité des solutions et multiplicateurs et des conditions de jonction sont donnés. Lorsqu'il y a un nombre fini de points de jonction, on obtient des conditions d'optimalité du second-ordre nécessaires ou suffisantes, ainsi qu'une caractérisation du caractère bien posé de l'algorithme de tir. Ces résultats généralisent les résultats obtenus dans le cas d'une contrainte sur l'état et d'une commande scalaires.
Preface 1. Introduction to nonlinear programming 2. Large, sparse nonlinear programming 3. Optimal control preliminaries 4. The optimal control problem 5. Optimal control examples Appendix A. Software Bibliography, Index.
Let V(t,x) be the infimum cost of an optimal control problem, viewed as a function of the initial time and state (t,x). Dynamic programming is concerned with the properties of V and in particular with its characterization as a solution to the Hamilton-Jacobi-Bellman equation. Heuristic arguments have long been advanced relating the maximum principle to dynamic programming according to p(t) equals minus V//x(t,x//0(t)). Here x//0 is the minimizing state function under consideration and p is the costate function of the maximum principle. In this paper we examine the validity of such claims and find that this relationship, interpreted as a differential inclusion involving the generalized gradient, is indeed true, almost everywhere and at the endpoints, for a very large class of nonsmooth optimal control problems.
Equations of Hamilton-Jacobi type arise in many areas of application, including the calculus of variations, control theory and differential games. The associated initial-value problems almost never have global-time classical solutions, and one must deal with suitable generalized solutions. The correct class of generalized solutions has only recently been established by the authors. This article establishes the convergence of a class of difference approximations to these solutions by obtaining explicit error estimates. Analogous results are proved by similar means for the method of vanishing viscosity.
p(t) = @H @x ( (t);p(t);u(t)) and the "critical point condition" @H @u ( (t);p(t);u(t)) = 0.