Science topic

# Optimal Control - Science topic

Explore the latest questions and answers in Optimal Control, and find Optimal Control experts.
Questions related to Optimal Control
Question
I wonder if we can list direct methods to solve the optimal control problem. I would start with:
1. Steepest descent method
Can anyone add more? With appreciations.
I do not think that the answer is so simple.
Direct methods for optimal control are a very broad class of methods that require two ingredients: a transcription approach and an optimisation method.
Virtually all optimisation methods on the market would be applicable if properly paired with a transcription method.
Already among the trancription approaches there are plenty of options:
- Single shooting with polynomial collocation of the controls
- Multiple shooting with polynomial collocation of the controls
- Collocation with pseudo-spectral methods (there are at least 5 variants)
- Collocation with Finite Temporal Elements (see or )
- Transcription with multiple discrete impulses (see Sims and Flanagan)
- Transcription with finite trajectory elements (see https://www.sciencedirect.com/science/article/abs/pii/S0094576511002852)
This is just to list some. There are also direct methods that use neurocontrollers to model the control part.
You can then, mix and match optimiser with transcription method.
Then there are extensions to multi-objective optimal control problems (see ) that use multi-objective optimisers and extensions to problems with uncertainty ( )
Your question seems to be more, can we list all optimisation methods used to solve directly transcribed optimal control problems?
Question
My background is control engineering (I studied all ogata book) but It's my first time to read in optimal control and I want recommendations for optimal control books for a beginner. It will be nice to be with Matlab examples.
Question
Is there any procedure to find exact feedback optimal control for the fixed-end-point finite-horizon discrete-time LQ problem?
For finite-horizon LQ, you basically apply dynamic programming (back backpropagation) to find the optimal control policy. See Chapter 11 of the book: Digital Control System Analysis & Design 4th Edition.
For the infinite-horizon counterpart, it reduces to the H2 optimization problem where you can use LMI or algebraic Ricatti equation to solve the optimal steady-state control gain.
Question
Hi,
I was reading a paper with the following title, (If you want the paper please let me know):
"Adaptive optimal control for continuous-time linear systems based on policy iteration"
and I was trying to develop a MATLAB code for it. In section 3, I was confused that how I can generate the X and Y matrices, as I did not understand the following sentence:
"Evaluating the right hand side of (21) at N ≥ n(n+1)/2 (the number of independent elements in the matrix Pi) points \bar{x}i in the state space, over the same time interval T"
The necessary parts are attached to this question alongside the code that I have written.
The main problem is that the estimated P matrix is not Positive Definite (PD), which I think that I am generating the X and Y matrices in the wrong way.
Thank you for taking your time for reading this question,
I will be thankful if you can post your opinion here.
A. Marashian.
Update:
I wrote another code and I got the following results (check the attached files).
I don't know if it is correct or not, please check it and let me know.
At p3.png I showed the P elements:
blue: P11
green: P23
red: P24
cyan: P44
the star one is the optimal value.
At P4.png I showed the state trajectories.
Question
I need recent mathematical modelling constrained optimal control of covid-19 as application of my work on optimal control problems.
Many thanks.
Question
I am trying to analyze an optimal control problem where the control system is a mathematical model for HIV transmission. The objective function involved in minimizing of the number of HIV infected (unaware infected and aware infected) as well as the cost for applying control strategies (education, screening and therapy).
Hello , I am not sure, but check this video link, it may help you find a better answer.
Question
Which is the best software for finding Optimal Control for System of Fractional Order Differential Equations?
Thank You for your response sir, surely i read those ref papers you mentioned, i will learn and use FOMCON toolbox. @ Kibru Teka Nida
Question
A MATLAB sample code for optimal control.. I also appreciate a private message for explanation... Thank you professionals
Matlab central file exchange should yield some info.
Question
I am looking for an equivalent formula for the right Riemann-Liouville fractional derivative by Volterra fractional integral equation to use it in Forward-Backward Sweep Euler method to solve the fractional optimal control problem, or anyone can help me to find a code to solve the fractional optimal control numerically.
Dear Mohamed
I suggest you see the following link:
Question
Introduce an example from a book or article on the optimal control problem that has a constraint and the control function in the constraint has a first-order or second-order derivative. This case problem can also have an initial condition for the state function.
Thanks to all the professors who answered my question, after reviewing the references introduced, unfortunately I still could not find the problem of optimal control that has a constraint and the control function in the constraint has a first-order or second-order derivative.
Question
Dear researchers
As you know, nowadays optimal control is one of the important criteria in studying the qualitative properties of a given fractional dynamical system. What is the best source and paper for introducing optimal control analysis for fractional dynamical models?
Dear Dr. @Muhammad Ali
Thank you very much.
Question
If the aerial vehicle like blimp or quadrotor is required to track the 3D position coordinates then how we will generate the reference trajectory for the vehicle? What will be the reference roll angle, pitch angle, and yaw angle for tracking the desired spatial position coordinates?
For optimal control, the problem formulation is simple like you can select the position states as your outputs and the rest of the states will act as an internal state. Your system will be three outputs and multi inputs. While for SMC you will need the desired position, attitude, and their successive derivatives.
Your suggestions are valuable to me...
Theory:
1-Path planning problem: We can use a global path planner algorithm such as RRT, RRT* or A*. Input of path planner module: Initial position and final position of the vehicle.Output: list of reference waypoints in the form of (x,y,z) triples.
2-Trajectory generation problem: This aims to generate a dynamically feasible smooth trajectory that passes through the previously obtaiNed waypoints. Input: waypoints. Output: continuous polynomial representing the trajectory (time-parametrised path). You can use for instance B-spline algorithm.
3-Control problem (flight control system, trajectory tracking control): Let consider a quad rotor aircraft for instance. We should design a position tracking controller and an attitude stabilization controller as follows:
3.1-Position tracking controller (besides velocity controller): input: generated trajectory. Output: reference attitude angles.
3.2-Attitude stabilization controller (besides angular rate controller): input: reference attitude angles. Output: motor control signals (low level control).
Practice:
A commercially available autopilot such as PIXHAWK can be be connected to another companion computer like RASPBERRY PI or JETSON NANO. Path planning, trajectory generator, position controller can be implemented in ROS framework in the companion controller. PIXHAWK autopilot can be used for the low level control including attitude stabilization.
Question
Plz suggest me how I make simulink model for this inverted pendulum..because I am not finding any reference regarding model of inverted pendulum..plz guide me..
Thank you
Question
Hi,
I am working on writing my own optimal control software for my master's thesis, which will focus on comparing the mesh refinement processes for difficult singular optimal control problems.
I don't understand how is it possible to implement the continuity constraints for endpoints of each phase when using the LGR collocation scheme. To ensure that one endpoint of each phase (which is the noncollocated point because of the LGR collocation) is the same as the first point of the other phase, what should I do? Should I just use the integration operator to approximate the endpoint of each phase and then add it as an additional constraint?
In other words, when using LGR collocation, how can i make sure that continuity condition is satisfied for the states?
What about finding the approximate control and the costate at the endpoint?
If I am going to find the endpoint control by extrapolating, how can i make sure that it is the optimal one?
Also, for the fLGR scheme, how can i calculate the control and estimate the costate at the beginning of the first phase?
Question
Hi All,
I am working a project with cost function as "minimum time" by numerical optimal control method. Now, I am struggling with one thing:
How to formulate a "time-optimal" control problem with pseudospectral method?
thanks
Jie
Dear Jie Yao,
I suggest you to see links and attached files on topic.
Best regards
Question
Control Lyapunov function (CLF) methodology tries to minimize a quadratic cost-function as; V(x)=x^2, through a point-wise setup. However, optimal control minimizes a quadratic cost-function through a time-interval as; J(x)=int(x^2.dt,0..infinity), through the time axis.
Therefore, we may conclude CLF is equivalent to proportional-control, while optimal control is an integral-control methodology. The robustness of optimal-control is attributable to its integral format.
Technical CFL is the basis of the main loop design methods such as Bellman's dynamic programming in optimal control. We can consider a CFL as an optimal tracking control (finite or infinite) with more robustness by the introduction of LMI.
For more details and information about this subject i suggest you see links and attached file on topic.
Best regards
Question
By introducing certain constraints, It is common that in some of the existing literature that solves the American style options, the early exercise feature and the Greeks are either not computed accurately or unavailable.
I am seeking insight and would love to inquire about various numerical, analytical, and/or analytical approximation techniques for computing the early exercise feature in a high-dimensional American options pricing problem.
Thank you, J. Rafiee, Lilian Mboya and Paul A Agbodza for the great suggestion and insight. I will go through all your suggestions accordingly.
Question
In "optimal control, linear quadratic methods" by B.O. Anderson and J. Moore, in chapter 2 there is a problem which asks the reader to show that it is not possible to solve finite horizon LQ with x(tf)=0 in feedback form u=Kx.
After doing some calculations, I found out that K->inf as t->tf due to the exponential state transition matrix (xdot=Ax+Bu; if u=Kx then xdot=(A+BK)x). So the feedback form u=Kx is not valid optimal control solution because K is unbounded.
Open loop solution can be found simply, but I was wondering if there is any way to solve this problem in feedback form (optimally or sub-optimally) when it is necessary to satisfy x(tf)=0 exactly. Do you have any recommendations?
Question
Dear all,
would you have some references about the link between a reference price effect and cyclical pricing in general (both theory and empiric)? Also, in particular, do you know some references from dynamic optimization (optimal control and the like)?
Many thanks for your help ;-)
Best,
Régis
Many thanks Mohamed-Mourad Lafifi for this detail answer. I appreciate very much your help ;)
Régis
Question
LQR based control system is offline because it is computed once before running of simulation or experiment.
As input-output data we use LQR based optimal control system applied to motor speed tracking application. We would like to estimate K optimal gain based Moorse-Penrose pseudo -inverse derivation. So, this control system is not based on the model, therefore A,B matrices are unknown. Model is black box.
Except the Neural Networks based control system, I would like to know whether it possible to implement online control system when estimated K optimal gain matrix will be updated each instant (each cycle).
For a RG discussion on the meaning of the 'online' (or 'on-line') terminology at this forum:
Question
Dear All,
When I do my project, I need add a constraint to the trajectory at a certain time within nonlinear optimal control design. For example, there are two dimensional system , x and y. I would like to set x(3) = 5 and y(3) = 6 (the time interval is [0, 5]s), which means I want the trajectory travel (5, 6) at 3 second.
Is there any way to formulate that ?
If there are some papers or materials can be recommends , that would be great.
thanks
Jie
Hello, do you mean the optimal local points?
Question
Hi All
I am doing a nonlinear optimal control design project, typically a practical engineering problem.
It is easy to solve that with direct method. However, I am wondering is there any way to prove that the optimal solution exists ? Further, since the direct method is employed, is there anyway to prove that the numerical solution will converge to that optimal solution?
Could anyone offer me some reference paper about that ?
thanks
Jie
The answer to the first question depends on the type of the nonlinear system, i.e., ODE or PDE, and the order. For example, see
“Slemrod, Marshall. "Existence of optimal controls for control systems governed by nonlinear partial differential equations." Annali della Scuola Normale Superiore di Pisa-Classe di Scienze 1.3-4 (1974): 229-246.”
“Bors, Dorota, Andrzej Skowron, and Stanislaw Walczak. "On Existence of Solutions to Nonlinear Optimal Control Systems." Dynamic Systems and Applications 21.2 (2012): 441.”
“The existence and uniqueness of solutions to differential equations”
Moreover, the answer also depends on the smoothness or non-smoothness solution of the problem. As we know, determination of the optimal feedback law for nonlinear optimal control problems leads to the Hamilton Jacobi-Bellman (HJB) partial differential equations, and in general, the solutions may not be smooth. The question of existence in the class of unique non-smooth solutions (called the viscosity solutions) is studied in
“M. Crandall and P. Lions, “Viscosity solutions of Hamilton-Jacobi equations,” Transactions of the American Mathematical Society, vol. 277, no. 1, pp. 1–42, 1983.”
“M. Bardi and I. Dolcetta, Optimal control and viscosity solutions of Hamilton Jacobi-Bellman equations. Springer, 1997.”
To answer the second question, note that to find the optimal solution, one has to solve an HJB equation. This HJB equation is generally difficult or impossible to solve analytically and therefore, in order to approximately solve the HJB partial differential equations, numerical methods such as Approximate dynamic programming (ADP), which is an efficient and forwarded in time RL method, can be used to generate approximate optimal control policy (near-optimal solution). However, in general, HJB partial differential equations can be solved numerically for very low state dimensions, and moreover, numerical methods generally give up the optimality of the solution in favor of reduced computational complexity, and other implementation-related factors. This implies that the optimal numerical solutions are near-optimal, and the guarantees are mainly provided in the sense of ultimately uniformly boundedness of the solution. Moreover, Banach's fixed-point theorem (i.e., Contraction Mapping Theorem) plays as a key tool to provide guarantees for the existence and uniqueness of fixed points (optimal solutions) of nonlinear optimal control problems, and also provides a practical and powerful method to find those solutions. See the below for more details
Question
How to minimize an Hamiltonian to find the optimal control?
I need to solve an optimal control problem using Pontryagin's Minimum Principle.
To find the u* I should minimize the Hamiltonian function. But the Hamiltonian's minimization required to know the optimal state x* and the optimal co-state p*, that
I can know only solving the state and co-state ODEs x*_dot=f(x*,u*) and p*_dot=-Hx.
So, I need to know the optimal state and costate to minimize the Hamiltonian and find the optimal input u*, but I need to know the optimal input u* to solve the ODEs and finding the optimal state x* and costate p*.
How can I get out of this loop? Or is this reasoning wrong?
Dear Carmine, only a few, simple optimal control problems are solvable in explicit form. The PMP system should be seen as a set of simultaneous conditions to be satisfied, but there exists no "algorithmic" way to solve it, although I would start by expressing the optimal control as a function of state and co-state via the minimum condition. One should also keep in mind that the optimal triple might not be unique.
What kind of optimal control problem do you have in mind?
Question
Dear All,
I would like to solve the nonlinear optimal control problem with the nonlinear programming techniques. But I can not find some proper references to state this procedure. Could you recommend some references or papers about that ? Thanks.
Thanks
Jie
Dear Jie Yao,
Also I suggest you to see links and attached files on topic.
Question
I can't find many real world applications of reinforcement learning. Also I wonder if there are clear criterias under which reinforcement learning outperforms optimal control regarding performance, stability, costs, etc.
Classical methods for control of dynamical systems require complete and exact knowledge of the system dynamics. However, most real-world dynamical systems are uncertain and their exact knowledge is not available. Adaptive control theory consists of tools for designing stabilizing controllers which can adapt online to modeling uncertainty of dynamical systems and has been applied for years in process control, industry, aerospace systems, vehicle systems, and elsewhere. However, classical adaptive control methods are generally far from optimal. On the other hand, optimal control theory is a branch of mathematics developed to find the optimal way to control a dynamical system. Reinforcement learning is actually closely tied theoretically to both adaptive control and optimal control. One can see RL methods as a direct approach to adaptive optimal control of dynamic systems. See the below link for more details:
R. S. Sutton, A. G. Barto and R. J. Williams, "Reinforcement learning is direct adaptive optimal control," in IEEE Control Systems Magazine.
This one provides an overview of the reinforcement learning and optimal adaptive control literature and its application to robotics:
Khan, S. G., Herrmann, G., Lewis, F. L., Pipe, T., & Melhuish, C. (2012). Reinforcement learning and optimal adaptive control: An overview and implementation examples. Annual Reviews in Control, 36(1), 42–59.
Question
In the layout problem, I would like to determine not only the optimal layout design to optimise performance, but also determine/generate the number of components to include in the system and which type of components, as there are a number of different options that will affect the optimal solution in different ways. Would it be incorrect to formulate this as an optimal control problem rather than a programming problem/optimisation problem? The control would be generated once before system operation, and then used throughout the time frame, but I do need to generate the number and types of components and not just their optimal layout.
welocme,
I think one has to define the different terms precisely. Then one has to define the problem clearly. Afterwards one proposes the possible solution methods and solutions.
You can apply the idea of reconfigurable system in which you can vary the system by using other system parameters.
This is applied in the reconfigurable communications transceiver where one can vary the modulation techniques, the modulation order, the channel coding techniques the carrier frequencies etc.
Also it is used in pv controllers such as the paper in the link:
There are also the term software defined systems which is used to alter the system according to the new standards.
Best wishes.
Question
Most of the hybrid automata papers target formal verification of the systems. I am looking for papers that are solving optimal response problems that propose optimal control in power systems under the cyber compromise.
TABOR: A Graphical Model-based Approach for Anomaly Detection in Industrial Control Systems
Question
Hi,
As I have understood trajectory generation in optimum time is related to solving an optimal control problem with constraints on states defined as position, velocity and acceleration and input as jerk. One way to decrease the position error in this problem is solving the discrete system which I couldn't find any toolbox for it (would be nice to mention if you know any), but I was wondering if could involve time and constraint of minimum time in boundary value problem solvers like bvp4c in matlab.
Thanks
Hi, I think in your case it is easier to implement your problem yourself in MATLAB for example by using the fmincon optimizer. As a matter of fact, your dynamics equations can be integrated analytically. Best
Question
Hi,
I am having trouble with a problem, in the field of Optimal Control and the generation of optimal time-series.
Let's consider a system, whose dynamics are represented by dx/dt = f(t,x(t),u(t),p(t)), x and u being respectively the state and control vectors for the system. p is a vector of parameters which have a direct influence on a system's dynamics.
An example illustrating this would be considering a drone, going from point A to point B, in minimum time, but subject to a windy environment (the wind being represented by the time-dependent variable p(t)).
I have generated, by solving an Optimal Control Problem, optimal time-series for x(t) and u(t), for several values of p=p(t)=constant.
I would now like to interpolate, for any given value of p(t) at time t, the "nearly-optimal" control u(t) to be applied to the system between time t and time t+1, based on the OCP results previously computed.
Would you know if this is even possible ? I have not really been able to find published work on this topic, if you had any suggestions I would be grateful.
Thanks,
Hi,
Thanks for your reply. Sadly, the disturbance is does not appear linearly in the state equation, it directly influences the system dynamics.
Question
I am going to treat in a lecture the application of deep reinforcement learning for the optimal control of physical processes. Therefore, I am looking for the best references. Besides the classics on reinforcement learning (like Sutton and Barto), I am using for example the book by Bertsekas "Reinforcement Learning and Optimal Control". What other recommended references are there?
Are there any recommended publications that discuss the advantages and disadvantages of deep reinforcement learning and learning-based model predictive control in this field?
Question
What is the elaboration of various control laws in comparison to each other. It means, in the level of comparison, when to take Adaptive control? when to adopt Robust control? when for Neuro-fuzzy control? when for Optimal control? when for model predictive control?
Well, that is not a "unique response" answer. The controller's design strongly depends on the requirements, knowledge, and resources of the whole system. But, considering some mild conditions, in my opinion, a good way of star answering that hard question is understanding the "The Internal Model Principle" (See https://www.control.utoronto.ca/~wonham/W.M.Wonham_IMP_20180617.pdf)
In this sense, a controller is good as it can "capture" the system's dynamics. A common practice for the controller design is using a system model. The parts of the system that are not included in the model as external disturbances, unmodeled dynamics, noise, discretization effects, and other possible variations, are assumed as fulfilling some hypotheses.
Then, for this case, those hypotheses on the unknown effects are used for performing a stability analysis according to the proposed framework. An example is assuming bounded uncertainty, then looking for an ultimate bound for the system's trajectories. Or, another example, assuming the terms of the systems are all Lipschitz, then the systems and the uncertainties can be "dominated" linear control terms, hence the design reduces to solving an LMI.
Therefore, taking care of the details, the better the model, the better the controller. However, no model is perfect. There will always be external disturbances, unmodeled dynamics, and other possible variations that the model does not include. So, a controller is good as it can incorporate the dynamics of the system while attenuating the effect of the unmodeled dynamics and other sources of uncertainty,
Question
I am working on a paper that looks into the dynamics of the spread of Wolbachia and its potential impact on dengue transmission, particularly in the Philippines.
Interesting research, although not in my field of work. I have heard about Wolbachia infections in other Diptera possibly causing melanistic non-fertile females. Is this the main reason for the slowing of the spread of Dengue? Is there any research on how Wolbachia will affect other insects?
Best wishes,
Jeroen
Question
Any decision-making problem when precisely formulated within the framework of mathematics is posed as an optimization problem. There are so many ways, in fact, I think infinitely many ways one can partition the set of all possible optimization problems into classes of problems.
1. I often hear people label meta-heuristic and heuristic algorithms as general algorithms (I understand what they mean) but I'm thinking about some things, can we apply these algorithms to any arbitrary optimization problems from any class or more precisely can we adjust/re-model any optimization problem in a way that permits us to attack those problems by the algorithms in question?
2. Then I thought well if we assumed that the answer to 1 is yes then by extending the argument I think also we can re-formulate any given problem to be attacked by any algorithm we desire (of-course with a cost) then it is just a useless tautology.
I'm looking foe different insights :)
Thanks.
The change propagation models may give a great idea
Question
What's the most fundamental difference?
The optimal output feedback controller is essentially just a matrix multiplication, while the model predictive control has some fundamental advantages when there are constraints, which are explicitly taken into consideration in the computation of the optimal control trajectory. Optimal output-feedback control cannot consider constraints, they have to be added in a separate step. Also note that standard MPC requires to solve an optimization problem at each sample.
best regards
Question
Dear Colleagues,
I am dealing with optimal control problem, my query is about how do we relate state and co-state variables, please add your valuable knowledge it.
Thank You in advance.
I have attached a PDF with regard to your query. Please notice it.
Question
Hi,
Have you ever try to solve the Optimal Control Problem (OCP) with differential-algebraic equations (DAEs) constraints by MUSCOD-II?
Please let me know with an example of an article/paper?
Thank you for your help.
With best regards,
Duy
Thank you so much. It looks helpful to me.
Best wishes,
Duy
Question
I have read some papers about the Model Predictive Control. As I know, MPC mainly update the optimal solutions based on the updated initial condition, i.e. repeated optimal control. As real-time optimal control also do the repeated optimal control, what is the difference between MPC and Real-time optimal control?
Dear Jingwen Zhang
real-time optimal control is an open-loop control, and MPC is a closed-loop control.
Regards
Question
Hi everyone,
I hope you are well today,
OK, let me go straight forward, I really like to implement the new control techniques, but I'm having to much problem in the concepts, it will be really helpful if I can see some codes about RL in Control Theory.
So I will be really thankful if you can share me some codes, whether you wrote it or you see them somewhere else.
Thanks a lot,
Sincerely,
Arash.
Question
I am working in Fractional Optimal Control problem of Epidemic models, I am trying to solve the problem numerically but I can not do it, as there is no toolbox and any sample MATLAB code is available to solve fractional optimal control problem. Please share the MATLAB code to solve Fractional Optimal Control Problem. Please help me.
Thank You in advance.
Thank you very much Sir.
Question
Hello,
I am a bit familiar already with MPC and optimization routine by defining the cost function.
My question is, generally we see MPC has cost function (if we talk about linear MPC) of the form (see the picture attached.)
Is there any possibility to add or alter this cost function.
For example, I am solving an optimal control problem where the time of arrival of vehicles at a particular location is also important. How can I change or modify this to include time as well in my optimization routine?.
Can anybody help me with this?
I see your point. I think even I came to the same conclusion after reading some papers. I think this problem can be solved (may be) by using MILP or dynamic programming. However, even I think the time is one the main factor which discards MPC in my application.
For now, I think an analytical approach using Pontriagin's maximum principle might be suitable. Of-course drawback is, I cannot directly incorporate the constraint's on the states or inputs.
Therefore, I am also still looking for alternatives. Thank you for this discussion.
Question
Dual control in Stochastic optimal control suggests that control input has probing action that would generate with non zero probability the uncertainty in measurement of state.
It means we have a Covariance error matrix at each state.
The same matrix is also deployed by Kalman filter so how it is different from Dual Control
Dear Sandeep,
These are the dual objectives to be achieved in particular, a major difficulty consists in resolving the Exploration / Exploitation (E / E) compromise.
Best regards
Question
Stochastic Modelling with Optimal Control
I suggest Stochastic Differential Equations - An Introduction with Applications, by Bernt Øksendal, https://www.springer.com/gp/book/9783540047582
Question
Considering the powered descent guidance problem for a spacecraft or launch vehicle - Early publications by Acikmese, et.al. used lossless convexification to solve the landing guidance problem using point mass dynamics.
Literature that dealt with the powered descent guidance problem using 6 DoF dynamics, also by Acikmese, et.al. proposed the use of successive convexification, with the reasons being that lossless convexification could not handle non-convex state and some classes of non-convex control constraints.
I have the following questions with respect to the above
1) Why is it that Lossless convexification cannot handle some classes of non-convex control constraints ? and what are those classes ? (e.g. the glideslope constraint)
2) What is it about lossless convexification that makes it unsuitable to operate on non-convex state constraints ,as opposed to successive convexification ?
2) Are there specific rotational formalisms ,e.g. quaternions, DCMs, Dual quaternions, MRPs, etc. that are inherently convex ? if so, how does one go about showing that they are convex ?
I would be much obliged if someone could answer these questions or atleast provide references that would aid in my understanding of the topic of convexification.
Thank you.
A successive convexification algorithm designed to solve non-convex constrained optimal control problems with global convergence and it can handle nonlinear dynamics and non-convex state and control constraints. However lossless convexification cannot handle nonlinear dynamics. For the third question the answer is affirmative.
For more details and information about your questions I suggest you to see links and attached files on topic.
Best regards
Question
Project Details :
The project involves the development of a novel navigational model and a DSS based on multi-criteria approach for providing support on e-navigation and collision / obstacle avoidance for operations including conventional and marine autonomous surface vessels (MASV). The developed system will be used for both improving the vessels propulsion system design and control as well as for ensuring safe navigation. The project commences with the investigation of the current and future propulsion plant alternatives for both vessels types and proceed to the selection of the ship types to be investigated along with their hull and propulsion system characteristics. Fidel models for adequately representing the investigated vessels subsystems/components as well as the ships control systems (engine and steering control) modelling will be developed. Based on these models, the identification of the propulsion system limitations will be carried out. The next project phase includes the integration of the developed models to form a vessel digital twin for representing the behaviour and response of the investigated vessels propulsion and manoeuvring. The uncertainty involved in the complete chain (sensors, models, conditions, actuators, course, decision made) will be thoroughly investigated and its influence on the proposed actions will be quantified. Appropriate risk metrics including the navigation practices, minimum distances as well as reliability and failures of the propulsion system and its components will be introduced for characterising the safety of the ship’s navigation. A number of counter scenarios including the number or the involved vessels (and their types), the navigation area details, the prevailing environmental conditions as well as  the quantified level of uncertainty and the human  response, will be evaluated for accessing the associated safety metrics and characterising the collision risk.  Based on the examined scenarios results, optimal control systems will be proposed and decision guidelines will be suggested. All the previous steps will allow the development of the novel structured DSS.
Question
We have a stochastic dynamic model: Xk+1 =f(Xk,uk,wk ). We can design a cost function to be optimized using dynamic programming algorithm. How do we design a cost function for this dynamic system to ensure stability?
In Chapter 4 of Ref. [a] for a quadratic cost function and a linear system (Xk+1 =AXk+Buk+wk), a proposition shows that under a few assumptions, the quadratic cost function results in a stable fixed state feedback. However, I think about how we can consider stability issue in the designation of the cost function as a whole when we are going to define the optimal control problem for a nonlinear system generally. Can we use the meaning of stability to design the cost function? Please share me your ideas.
[a] Bertsekas, Dimitri P., et al. Dynamic programming and optimal control. Vol. 1. No. 2. Belmont, MA: Athena scientific, 1995.
Unfortunately, the attached article
" [a] Bertsekas, Dimitri P., et al. Dynamic programming and optimal control. Vol. 1. No. 2. Belmont, MA: Athena scientific, 1995."
is full of typing errors.
General talking, we consider the linearization of the nonlinear system. Next, we study the stability of the equilibrium state of the new linear system, which indicates the nature of the stability of the nonlinear system in some neighborhoods.
The signs of the real parts of the eigenvalues of the Jacobian matrix decide which approach we should follow. We have the direct and indirect Lyapunov methods to study the stability based on the eigenvalues.
Best regards
Question
I'm a final semester student in BS Mathematics and my research interest is in Mathematical Biology. Would you like to provide me the best SEIQR ODEs model for stability and optimal control? I want to do stability and optimal control for our province's real data. So, please recommend the paper.
Thank you so much.
Question
Despite the time I have spent on GPOPS, I could not understand the application of static parameters. What are they? Can anybody present an example?
A general-purpose MATLAB software program called GPOPS is described for solving multiple-phase optimal control problems using variable-order Gaussian quadrature collocation methods. The software em- ploys a Legendre-Gauss-Radau quadrature orthogonal collocation method where the continuous-time opti- mal control problem is transcribed to a large sparse nonlinear programming problem (NLP).
hops it help. regards.
Question
Hi All
Are there any real engineering applications that are required to design the controller by using the finite-time optimal control technique?
thanks
Jie
Please have a look at this paper for an in-depth understanding.
Aganovic, Zijad, and Zoran Gajic. "The successive approximation procedure for finite-time optimal control of bilinear systems." IEEE Transactions on Automatic Control 39, no. 9 (1994): 1932-1935.
Question
I need a more updated method for solving optimal control models, since the Pontryagin's principle seemed old
I want to get to know you better than what has been done on this!
Question
Dear Colleagues,
I am struggling to formulate fractional order optimal control problem, I am considering constant delay here. Please help me how to proceed and please share some MATLAB codes that can deal with the aforesaid problem.
Thank You
Regards,
Ramashis Banerjee
Question
Brachistochrone problem in optimal control/calculus of variations, is a well-known problem in physics and mathematics.
Through numerous sources in the existing literature (as in the URL provided below), the elementary solution to the brachistochrone problem, is an ordinary differential equation (ode) as: y(1+y' ^2)=k^2. Hence, y should be always positive, no matter if y' (derivative) is positive or negative.
This is what seems to be a philosophical glitch, as no solution to a physics problem, could impose the differential variable (as y) to be always positive, with regard to a free coordinate system.
Let me know how do you think?.
The reflected brachistochrone problem does not retain any longer the same mathematical formulation, nor the same physical meaning if y changes sign.
Best regards, R.F.
Question
The brachistochrone is a well-known problem in calculus of variations and optimal control. I ask if there is any explicit solution to the problem?. In the existing texts, x and y are usually parameterized in dummy variables as t, phi or theta. I want to know whether any explicit solution is available or not? I mean y(x) in terms of x, not any other intermediary variable.
In the pdf, attached to this question, there is some hint on the explicit solution (but it seems it is computational and numerical).
It seems somewhat explicit analytic solution is presented in this book:
Introduction to the calculus of variations, by William Elwood Byerly.
Equation (2) of chapter II.
Question
I want to learn how to imlement any optimization algorithm that is not in the Matlab Toolbox in the S-Function block in Simulink.I want to do this with matlab codes, not in languages like C, C ++. (So I want to do it with the first type S-function).
Question
Hi all,
I am solving a lineair constrained MPC problem, which could be seen in the attached image.
In this optimal control problem Q and R are both symmetric positive definite matrices.
When I rewrite this optimal control problem to a constrained QP problem and solve this with a QP solver (e.g. Quadprog or Gurobi) I get a negative objective function value. Is this correct? I would expect a positive value, because Q and R are both positive definite and the terms in the objective function are quadratic.
Indeed the objective function can only be positive in the case of an evolution of a normal process.
Regards
Question
HI ALL :
I encounter a technical question in optimal control,
I have to add some constraints conditions on the system for a real application. So the first idea is to take inverse optimal control techniques to design an optimal control which can satisfy a certain cost performance index.
My question is that are there other methods that can incorporate the constraints into the Q or R matrices other than inverse optimal control technique?
thanks
Jie
You probably mean the weighting matrices, in the quadratic cost functional (or performance criterion). To start with, in the case of the continuous-time control system, matrix M, which refers to the final time, regarding the state trajectory, and matrix Q which refers to all other intermediate times, should be symmetric and semi-positive. On the other hand, matrix R, which refers to the control sequence should be symmetric and positive definite.
Considering a discrete-time optimal control problem and N time-periods, after augmenting the state vector to incorporate the control vector, then we have N weighting matrices, usually denoted by QN and Qj (j=1,2,…N-1). The deviations from the desired path are to be minimized. This implies that the desired state and control sequence should be predefined. The weighting matrices should be symmetric. This is not restrictive, since the quadratic form x΄Ax, where A is not initially symmetric, can be transform to an equivalent symmetric B, i.e. A=1/2[A+Α΄], so that x΄Ax = x΄Bx (prime denotes transposition).
Regarding the elements of the weighting matrices, this mainly depends upon the capacity of your system. If you don’t have such limitations, then with repeated applications, you can find such a set of weights, so that the deviations of the optimal path from the desired path are acceptable. Another point that deserves particular attention is the method of solution you will finally choose to solve your optimal control problem. I think that you will find very useful my book: “Dynamic Systems in Management Science. Design, Estimation and Control” Palgrave Macmillan, 2015 and particularly Chapter 9 (Optimal Control of Linear Dynamic Systems).
Good luck
Question
I found the process of detection of model structure in reduced models very trial and error based. I think we always prefer to have linear models like ARX(Autoregressive Exogenous) model instead of nonlinear models like NARX( Nonlinear Autoregressive Exogenous) model whether our main model is no-linear.
I need some help and guidance in the procedure of detection of the model structure. Is there any MATLAB toolbox or software for this purpose?
no such tool
Question
As we are known, the Falsification of CPS already has made some progress. But there is a problem: how to select input trace efficiently for finding the bad output trace. It can divide into two parts:
1. What is the input space and how to describe it
2. What optimization algorithms can help select input trace for the falsification of CPS
Can you introduce some ideas about the two parts and which next direction do you think?
Zeashan H. Khan .Thank you for your replay. I already have read those papers, what do you think of the role of machine learning or reinforcement learning? Are they effective for counter-example search?
Question
Dear RG-community,
In an MPC-implementation, we defined a cost function
C1 = f(x(t_1), x(t_2), ..., x(t_N))
which is function of all N temporal samples of a specified predicted state x.
Thus, to our interpretation it neither can be classified as stage cost
C2 = integral f(x(t)) dt
nor as terminal cost
C3 = f( x(t_N) ).
I would be very happy, if you could direct me to any theoretical classifications of C1-type cost functions.
As well, I would be very interested in publications/applications of C1-type cost functions.
Thank you very much in advance!
Best regards,
Stefan Loew
Hi Timm,
E[.] is the expected value indeed. I guess I should have been more precise with what I was trying to describe. The formulation above allows one to write C1 as a terminal cost, of course, at the price of creating a N-step state/input space. To my best understanding, if you try to develop an optimal policy (e.g. maximising C1) from that reformulation, you will end up with a "U = pi(Z)" that is actually trivial because static (supposing that the cost is concerned solely with x_0,...,x_N, then Z+ is not relevant for us). The PoO and Dynamic Programming is then trivially applicable (trivially because the dynamics in the Z space are irrelevant) only on that N-step state/input space. I view this as a Pyrrhic victory since we do not recover the properties of optimal control problems that I find fundamentally useful (PoO and DP).
Btw, I have the impression that the N-step approach above is generally applicable to C1 functions, so that all these C1 costs are "simply" terminal costs.
Best
Sebastien
Question
Dear Colleagues,
Could anyone suggest me the name of the SCI/SCIE journals publish work in the field of control system with high acceptance rate? I am working in fractional order optimal control.
Thank you in advance.
@Zeashan Khan Thank you very much sir.
Question
Dear Colleagues,
I am dealing with Fractional order optimal control problem, I am trying to solve my problem using Multi Step Differential Transformation Method in MATLAB/Mathematica. I am looking for the code.
Please share the code(MATLAB/Mathematica) for solving my problem using Multi Step Differential Transformation Method.
Thank you in advance.
Question
Let us consider an optimal control problem, where the final time is a free parameter. The objective function to minimize is J1. Since the final time is a free variable, the optimal control solution can be obtained in a fixed final time, optimal control framework by considering the final time as a design parameter.
Now, can we choose a different cost function J2 and select the final time (design parameter)
such that the final time will minimize J2?
Dear Avijit,
I think the answer depends on the application of the problem. In many applications, the final time has to be chosen by actual constraints and physical conditions. So, the designer is not free to choose it according to the cost function minimization.
But in other applications, cost reduction is a top priority. Therefor you can increase the end time to further reduction of the cost function. In this case, you can select the final time as design parameter based on amount of cost function .
Best regards
Question
It seems the state evolution ode; xdot=f(x,u), is known to satisfy f(0,0)=0, hence f(0,u(0))=0, then can we conclude at any control system, always u(0)=u(x=0)=0, and get the result that limit(u(x)) is zero when x goes to zero.
I think in classical linear/nonlinear control system this is trivial. How do you think?
But my focus is on optimal control. I want to know whether this assumption similarly holds also for optimal control problems (in case of regulation or trajectory optimization or so on)?. I want to know whether it is possible to assume in a typical optimal control problem; u(x=0)=u(0)=0?
Yew-Chung Chak Thank you. I missed to mention steady state phase in my question. By limit, I meant when x approaches zero with convergence notion and in a certain interval (not abruptly crossing the time axis). However, you are right with your objection, as your counterexample is clarifying. Moreover, I said my focus is optimal control, and as I know, in optimal control solution, state variables critically damp down to zero, and for non-zero initial conditions, state variable becomes zero only at steady-state (when t gets infinite), and therefore, u becomes zero at only such a steady-state zero state variable. Hence, I guess at optimal solution, limit (u(x)) is zero when x goes to zero, hence, probably we can conclude u(0)=0, as is lucid in plots (u versus x) for optimal regulation in existing benchmarks.
But my concern, is to know whether this assumption is valid, when optimal control is used for trajectory optimization (when final state and possibly its rate are not zero). However, I think for the case of trajectory optimization as lunar landing, we can still assume u(x=0)=u(0)=0, whereas this happens at the initiation of optimal control operation, whilst initial state is probably zero. But I am still looking to have more evidence for such optimal control problems and other free end-time (tf; free), and whether the assumption u(x=0)=0, holds or not? Notice that in optimal control, usually control variable is a function of time u=u(t), and not a feedback-law as u=u(x).
Question
Real-time optimal control is widely adopted in central heating, ventilation and air-conditioning systems to improve the building energy management efficiency by adjusting system controllable variables (e.g. chilled water supply temperature) according to system operating conditions. In literature, there are generally two types of methods to trigger the HVAC optimal control: (mostly is) time-driven optimal control and (some is) event-driven optimal control.
In time-driven optimal control, optimal control actions are triggered periodically, e.g every 1 hour trigger an optimal control. In event-driven optimal control, optimal control actions are
triggered by “events” instead of a “clock”, which creates non-periodic control.
I was wondering what are the triggering methods of HVAC optimal control used in engineering projects? As reported in the literature, most practical projects adopted time-driven optimal control. Some engineering projects may use the so-called rule-based method to trigger optimal control (indeed rule-based method belongs to event-driven method). The rule can be pre-defined in building automation system or sometimes defined by human operators to decide a experience-based rule and operate the optimal control manually.
This is what I have learnt from the literature. I am not sure whether this is true in actual engineering projects. Pls leave a comment.
Thanks for your comment.
Question
Hi,
What are the books and/or papaers/reports that are recommended in order to understand fullly the mathematical stuff in the following book:
Robust and Optimal Control by Kemin Zhou et al.
i.e. how the thorems are constructed and the assumptions are made and ultimately how they are proved. It will be helpful to understand the historical devlopment as well.
Thanks & regards
Dear Colleagues,
can one tell how Theorem 16.4 that appears in Robust Optimal Control by Kemin Zhou is constructed?
Thanks & regards
Question
Hi dear researchers,
I am currently implementing direct collocation NMPC,
However, I have a doubt on the control value for the first control u_0
that should be applied to the system once the control trajectory is found as collocation downsamples the system dynamics before solving the dynamic NLP problem.
will the control value u_0 be (u*(0)+u*(1))/2?
x_k+1 = f(x_k,u_0);
Should this be the case, will that not have a detrimental effect in the accuracy of the true control?
Kind regards