Science topic

# Optimal Control - Science topic

Explore the latest questions and answers in Optimal Control, and find Optimal Control experts.

Questions related to Optimal Control

I wonder if we can list direct methods to solve the optimal control problem. I would start with:

1. Steepest descent method

Can anyone add more? With appreciations.

My background is control engineering (I studied all ogata book) but It's my first time to read in optimal control and I want recommendations for optimal control books for a beginner. It will be nice to be with Matlab examples.

Is there any procedure to find exact feedback optimal control for the fixed-end-point finite-horizon discrete-time LQ problem?

Hi,

I was reading a paper with the following title, (If you want the paper please let me know):

**"Adaptive optimal control for continuous-time linear systems based on policy iteration"**

and I was trying to develop a MATLAB code for it. In section 3, I was confused that how I can generate the X and Y matrices, as I did not understand the following sentence:

*"Evaluating the right hand side of (21) at N ≥ n(n+1)/2 (the number of independent elements in the matrix Pi) points \bar{x}*

^{i}in the state space, over the same time interval T"The necessary parts are

**attached**to this question alongside the**code**that I have written.**The main problem is that the estimated P matrix is not Positive Definite (PD), which I think that I am generating the X and Y matrices in the wrong way.**

Thank you for taking your time for reading this question,

I will be thankful if you can post your opinion here.

A. Marashian.

I need recent mathematical modelling constrained optimal control of covid-19 as application of my work on optimal control problems.

I am trying to analyze an optimal control problem where the control system is a mathematical model for HIV transmission. The objective function involved in minimizing of the number of HIV infected (unaware infected and aware infected) as well as the cost for applying control strategies (education, screening and therapy).

Which is the best software for finding Optimal Control for System of Fractional Order Differential Equations?

A MATLAB sample code for optimal control.. I also appreciate a private message for explanation... Thank you professionals

I am looking for an equivalent formula for the right Riemann-Liouville fractional derivative by Volterra fractional integral equation to use it in Forward-Backward Sweep Euler method to solve the fractional optimal control problem, or anyone can help me to find a code to solve the fractional optimal control numerically.

Introduce an example from a book or article on the optimal control problem that has a constraint and the control function in the constraint has a first-order or second-order derivative. This case problem can also have an initial condition for the state function.

Dear researchers

As you know, nowadays optimal control is one of the important criteria in studying the qualitative properties of a given fractional dynamical system. What is the best source and paper for introducing optimal control analysis for fractional dynamical models?

If the aerial vehicle like blimp or quadrotor is required to track the 3D position coordinates then how we will generate the reference trajectory for the vehicle? What will be the reference roll angle, pitch angle, and yaw angle for tracking the desired spatial position coordinates?

For optimal control, the problem formulation is simple like you can select the position states as your outputs and the rest of the states will act as an internal state. Your system will be three outputs and multi inputs. While for SMC you will need the desired position, attitude, and their successive derivatives.

Your suggestions are valuable to me...

Plz suggest me how I make simulink model for this inverted pendulum..because I am not finding any reference regarding model of inverted pendulum..plz guide me..

Hi,

I am working on writing my own optimal control software for my master's thesis, which will focus on comparing the mesh refinement processes for difficult singular optimal control problems.

I don't understand how is it possible to implement the continuity constraints for endpoints of each phase when using the LGR collocation scheme. To ensure that one endpoint of each phase (which is the noncollocated point because of the LGR collocation) is the same as the first point of the other phase, what should I do? Should I just use the integration operator to approximate the endpoint of each phase and then add it as an additional constraint?

In other words, when using LGR collocation, how can i make sure that continuity condition is satisfied for the states?

What about finding the approximate control and the costate at the endpoint?

If I am going to find the endpoint control by extrapolating, how can i make sure that it is the optimal one?

Also, for the fLGR scheme, how can i calculate the control and estimate the costate at the beginning of the first phase?

Thanks in advance...

Hi All,

I am working a project with cost function as "minimum time" by numerical optimal control method. Now, I am struggling with one thing:

How to formulate a "time-optimal" control problem with pseudospectral method?

thanks

Jie

Anyone expert in optimal

Control Lyapunov function (CLF) methodology tries to minimize a quadratic cost-function as; V(x)=x^2, through a point-wise setup. However, optimal control minimizes a quadratic cost-function through a time-interval as; J(x)=int(x^2.dt,0..infinity), through the time axis.

Therefore, we may conclude CLF is equivalent to proportional-control, while optimal control is an integral-control methodology. The robustness of optimal-control is attributable to its integral format.

By introducing certain constraints, It is common that in some of the existing literature that solves the American style options, the early exercise feature and the Greeks are either not computed accurately or unavailable.

I am seeking insight and would love to inquire about various numerical, analytical, and/or analytical approximation techniques for computing the early exercise feature in a high-dimensional American options pricing problem.

In "optimal control, linear quadratic methods" by B.O. Anderson and J. Moore, in chapter 2 there is a problem which asks the reader to show that it is not possible to solve finite horizon LQ with x(tf)=0 in feedback form u=Kx.

After doing some calculations, I found out that K->inf as t->tf due to the exponential state transition matrix (xdot=Ax+Bu; if u=Kx then xdot=(A+BK)x). So the feedback form u=Kx is not valid optimal control solution because K is unbounded.

Open loop solution can be found simply, but I was wondering if there is any way to solve this problem in feedback form (optimally or sub-optimally) when it is necessary to

**satisfy x(tf)=0 exactly.**Do you have any recommendations?Dear all,

would you have some references about the link between a reference price effect and cyclical pricing in general (both theory and empiric)? Also, in particular, do you know some references from dynamic optimization (optimal control and the like)?

Many thanks for your help ;-)

Best,

Régis

LQR based control system is offline because it is computed once before running of simulation or experiment.

As input-output data we use LQR based optimal control system applied to motor speed tracking application. We would like to estimate K optimal gain based Moorse-Penrose pseudo -inverse derivation. So, this control system is not based on the model, therefore A,B matrices are unknown. Model is black box.

Except the Neural Networks based control system, I would like to know whether it possible to implement online control system when estimated K optimal gain matrix will be updated each instant (each cycle).

Dear All,

When I do my project, I need add a constraint to the trajectory at a certain time within nonlinear optimal control design. For example, there are two dimensional system , x and y. I would like to set x(3) = 5 and y(3) = 6 (the time interval is [0, 5]s), which means I want the trajectory travel (5, 6) at 3 second.

Is there any way to formulate that ?

If there are some papers or materials can be recommends , that would be great.

thanks

Jie

Hi All

I am doing a nonlinear optimal control design project, typically a practical engineering problem.

It is easy to solve that with direct method. However, I am wondering is there any way to prove that the optimal solution exists ? Further, since the direct method is employed, is there anyway to prove that the numerical solution will converge to that optimal solution?

Could anyone offer me some reference paper about that ?

thanks

Jie

How to minimize an Hamiltonian to find the optimal control?

I need to solve an optimal control problem using Pontryagin's Minimum Principle.

To find the u* I should minimize the Hamiltonian function. But the Hamiltonian's minimization required to know the optimal state x* and the optimal co-state p*, that

I can know only solving the state and co-state ODEs x*_dot=f(x*,u*) and p*_dot=-Hx.

So, I need to know the optimal state and costate to minimize the Hamiltonian and find the optimal input u*, but I need to know the optimal input u* to solve the ODEs and finding the optimal state x* and costate p*.

How can I get out of this loop? Or is this reasoning wrong?

Dear All,

I would like to solve the nonlinear optimal control problem with the nonlinear programming techniques. But I can not find some proper references to state this procedure. Could you recommend some references or papers about that ? Thanks.

Thanks

Jie

I can't find many real world applications of reinforcement learning. Also I wonder if there are clear criterias under which reinforcement learning outperforms optimal control regarding performance, stability, costs, etc.

In the layout problem, I would like to determine not only the optimal layout design to optimise performance, but also determine/generate the number of components to include in the system and which type of components, as there are a number of different options that will affect the optimal solution in different ways. Would it be incorrect to formulate this as an optimal control problem rather than a programming problem/optimisation problem? The control would be generated once before system operation, and then used throughout the time frame, but I do need to generate the number and types of components and not just their optimal layout.

Most of the hybrid automata papers target formal verification of the systems. I am looking for papers that are solving optimal response problems that propose optimal control in power systems under the cyber compromise.

Hi,

As I have understood trajectory generation in optimum time is related to solving an optimal control problem with constraints on states defined as position, velocity and acceleration and input as jerk. One way to decrease the position error in this problem is solving the discrete system which I couldn't find any toolbox for it (would be nice to mention if you know any), but I was wondering if could involve time and constraint of minimum time in boundary value problem solvers like bvp4c in matlab.

Thanks

Hi,

I am having trouble with a problem, in the field of Optimal Control and the generation of optimal time-series.

Let's consider a system, whose dynamics are represented by dx/dt = f(t,x(t),u(t),p(t)), x and u being respectively the state and control vectors for the system. p is a vector of parameters which have a direct influence on a system's dynamics.

An example illustrating this would be considering a drone, going from point A to point B, in minimum time, but subject to a windy environment (the wind being represented by the time-dependent variable p(t)).

I have generated, by solving an Optimal Control Problem, optimal time-series for x(t) and u(t), for several values of p=p(t)=constant.

I would now like to interpolate, for any given value of p(t) at time t, the "nearly-optimal" control u(t) to be applied to the system between time t and time t+1, based on the OCP results previously computed.

Would you know if this is even possible ? I have not really been able to find published work on this topic, if you had any suggestions I would be grateful.

Thanks,

I am going to treat in a lecture the application of deep reinforcement learning for the optimal control of physical processes. Therefore, I am looking for the best references. Besides the classics on reinforcement learning (like Sutton and Barto), I am using for example the book by Bertsekas "Reinforcement Learning and Optimal Control". What other recommended references are there?

Are there any recommended publications that discuss the advantages and disadvantages of deep reinforcement learning and learning-based model predictive control in this field?

What is the elaboration of various control laws in comparison to each other. It means, in the level of comparison, when to take Adaptive control? when to adopt Robust control? when for Neuro-fuzzy control? when for Optimal control? when for model predictive control?

I am working on a paper that looks into the dynamics of the spread of Wolbachia and its potential impact on dengue transmission, particularly in the Philippines.

Any decision-making problem when precisely formulated within the framework of mathematics is posed as an optimization problem. There are so many ways, in fact, I think infinitely many ways one can partition the set of all possible optimization problems into classes of problems.

1. I often hear people label meta-heuristic and heuristic algorithms as general algorithms (I understand what they mean) but I'm thinking about some things, can we apply these algorithms to any arbitrary optimization problems from any class or more precisely can we adjust/re-model any optimization problem in a way that permits us to attack those problems by the algorithms in question?

2. Then I thought well

*then by extending the argument I think also we can re-formulate any given problem to be attacked by any algorithm we desire (of-course with a cost) then it is just a useless tautology.***if we assumed that the answer to 1 is yes**I'm looking foe different insights :)

Thanks.

What's the most fundamental difference?

Dear Colleagues,

I am dealing with optimal control problem, my query is about how do we relate state and co-state variables, please add your valuable knowledge it.

Thank You in advance.

Hi,

Have you ever try to solve the Optimal Control Problem (OCP) with differential-algebraic equations (DAEs) constraints by MUSCOD-II?

Please let me know with an example of an article/paper?

Thank you for your help.

With best regards,

Duy

I have read some papers about the

**Model Predictive Control**. As I know, MPC mainly update the optimal solutions based on the updated initial condition, i.e. repeated optimal control. As real-time optimal control also do the repeated optimal control, what is the difference between MPC and Real-time optimal control?Please help me if you know something. Thanks.

Hi everyone,

I hope you are well today,

OK, let me go straight forward, I really like to implement the new control techniques, but I'm having to much problem in the concepts,

**it will be really helpful if I can see some codes about RL in Control Theory.**So I will be really thankful if you can share me some

**codes**, whether you wrote it or you see them somewhere else.Thanks a lot,

Sincerely,

*Arash*.

I am working in Fractional Optimal Control problem of Epidemic models, I am trying to solve the problem numerically but I can not do it, as there is no toolbox and any sample MATLAB code is available to solve fractional optimal control problem. Please share the MATLAB code to solve Fractional Optimal Control Problem. Please help me.

Reach me at ramashisbanerjee@gmail.com

Thank You in advance.

Hello,

I am a bit familiar already with MPC and optimization routine by defining the cost function.

My question is, generally we see MPC has cost function (if we talk about linear MPC) of the form (see the picture attached.)

Is there any possibility to add or alter this cost function.

For example,

**I am solving an optimal control problem where the time of arrival of vehicles at a particular location is also important. How can I change or modify this to include time as well in my optimization routine?.**Can anybody help me with this?

Dual control in Stochastic optimal control suggests that control input has probing action that would generate with non zero probability the uncertainty in measurement of state.

It means we have a Covariance error matrix at each state.

The same matrix is also deployed by Kalman filter so how it is different from Dual Control

Stochastic Modelling with Optimal Control

Considering the powered descent guidance problem for a spacecraft or launch vehicle - Early publications by Acikmese, et.al. used lossless convexification to solve the landing guidance problem using point mass dynamics.

Literature that dealt with the powered descent guidance problem using 6 DoF dynamics, also by Acikmese, et.al. proposed the use of successive convexification, with the reasons being that lossless convexification could not handle non-convex state and some classes of non-convex control constraints.

I have the following questions with respect to the above

1) Why is it that Lossless convexification cannot handle some classes of non-convex control constraints ? and what are those classes ? (e.g. the glideslope constraint)

2) What is it about lossless convexification that makes it unsuitable to operate on non-convex state constraints ,as opposed to successive convexification ?

2) Are there specific rotational formalisms ,e.g. quaternions, DCMs, Dual quaternions, MRPs, etc. that are inherently convex ? if so, how does one go about showing that they are convex ?

I would be much obliged if someone could answer these questions or atleast provide references that would aid in my understanding of the topic of convexification.

Thank you.

**Project Details**:

The project involves the development of a novel navigational model and a DSS based on multi-criteria approach for providing support on e-navigation and collision / obstacle avoidance for operations including conventional and marine autonomous surface vessels (MASV). The developed system will be used for both improving the vessels propulsion system design and control as well as for ensuring safe navigation. The project commences with the investigation of the current and future propulsion plant alternatives for both vessels types and proceed to the selection of the ship types to be investigated along with their hull and propulsion system characteristics. Fidel models for adequately representing the investigated vessels subsystems/components as well as the ships control systems (engine and steering control) modelling will be developed. Based on these models, the identification of the propulsion system limitations will be carried out. The next project phase includes the integration of the developed models to form a vessel digital twin for representing the behaviour and response of the investigated vessels propulsion and manoeuvring. The uncertainty involved in the complete chain (sensors, models, conditions, actuators, course, decision made) will be thoroughly investigated and its influence on the proposed actions will be quantified. Appropriate risk metrics including the navigation practices, minimum distances as well as reliability and failures of the propulsion system and its components will be introduced for characterising the safety of the ship’s navigation. A number of counter scenarios including the number or the involved vessels (and their types), the navigation area details, the prevailing environmental conditions as well as the quantified level of uncertainty and the human response, will be evaluated for accessing the associated safety metrics and characterising the collision risk. Based on the examined scenarios results, optimal control systems will be proposed and decision guidelines will be suggested. All the previous steps will allow the development of the novel structured DSS.

We have a stochastic dynamic model: X

_{k+1}=f(X_{k,}u_{k,}w_{k }). We can design a cost function to be optimized using dynamic programming algorithm. How do we design a cost function for this dynamic system to ensure stability?In Chapter 4 of Ref. [a] for a quadratic cost function and a linear system (X

_{k+1}=AX_{k}+Bu_{k}+w_{k}), a proposition shows that under a few assumptions, the quadratic cost function results in a stable fixed state feedback. However, I think about how we can consider stability issue in the designation of the cost function as a whole when we are going to define the optimal control problem for a nonlinear system generally. Can we use the meaning of stability to design the cost function? Please share me your ideas.[a] Bertsekas, Dimitri P., et al.

*Dynamic programming and optimal control*. Vol. 1. No. 2. Belmont, MA: Athena scientific, 1995.Dear sir/madam!

I'm a final semester student in BS Mathematics and my research interest is in Mathematical Biology. Would you like to provide me the best SEIQR ODEs model for stability and optimal control? I want to do stability and optimal control for our province's real data. So, please recommend the paper.

Thank you so much.

Despite the time I have spent on GPOPS, I could not understand the application of static parameters. What are they? Can anybody present an example?

Hi All

Are there any real engineering applications that are required to design the controller by using the finite-time optimal control technique?

thanks

Jie

I need a more updated method for solving optimal control models, since the Pontryagin's principle seemed old

Dear Colleagues,

I am struggling to formulate fractional order optimal control problem, I am considering constant delay here. Please help me how to proceed and please share some MATLAB codes that can deal with the aforesaid problem.

Thank You

Regards,

Ramashis Banerjee

Brachistochrone problem in optimal control/calculus of variations, is a well-known problem in physics and mathematics.

Through numerous sources in the existing literature (as in the URL provided below), the elementary solution to the brachistochrone problem, is an ordinary differential equation (ode) as: y(1+y' ^2)=k^2. Hence, y should be always positive, no matter if y' (derivative) is positive or negative.

This is what seems to be a philosophical glitch, as no solution to a physics problem, could impose the differential variable (as y) to be always positive, with regard to a free coordinate system.

Let me know how do you think?.

The brachistochrone is a well-known problem in calculus of variations and optimal control. I ask if there is any explicit solution to the problem?. In the existing texts, x and y are usually parameterized in dummy variables as t, phi or theta. I want to know whether any explicit solution is available or not? I mean y(x) in terms of x, not any other intermediary variable.

In the pdf, attached to this question, there is some hint on the explicit solution (but it seems it is computational and numerical).

I want to learn how to imlement any optimization algorithm that is not in the Matlab Toolbox in the S-Function block in Simulink.I want to do this with matlab codes, not in languages like C, C ++. (So I want to do it with the first type S-function).

Hi all,

I am solving a lineair constrained MPC problem, which could be seen in the attached image.

In this optimal control problem Q and R are both symmetric positive definite matrices.

When I rewrite this optimal control problem to a constrained QP problem and solve this with a QP solver (e.g. Quadprog or Gurobi) I get a negative objective function value. Is this correct? I would expect a positive value, because Q and R are both positive definite and the terms in the objective function are quadratic.

HI ALL :

I encounter a technical question in optimal control,

I have to add some constraints conditions on the system for a real application. So the first idea is to take

**inverse optimal control**techniques to design an optimal control which can satisfy a certain cost performance index.My question is that are there other methods that can incorporate the constraints into the Q or R matrices other than inverse optimal control technique?

thanks

Jie

I found the process of detection of model structure in reduced models very trial and error based. I think we always prefer to have linear models like ARX(Autoregressive Exogenous

**)**model instead of nonlinear models like NARX( Nonlinear Autoregressive Exogenous) model whether our main model is no-linear.I need some help and guidance in the procedure of detection of the model structure. Is there any MATLAB toolbox or software for this purpose?

As we are known, the Falsification of CPS already has made some progress. But there is a problem: how to select input trace efficiently for finding the bad output trace. It can divide into two parts:

1. What is the input space and how to describe it

2. What optimization algorithms can help select input trace for the falsification of CPS

Can you introduce some ideas about the two parts and which next direction do you think?

Dear RG-community,

In an MPC-implementation, we defined a cost function

**C1 = f(x(t_1), x(t_2), ..., x(t_N))**

which is

**function of all N temporal samples**of a specified predicted state x.Thus, to our interpretation it neither can be classified as stage cost

C2 = integral f(x(t)) dt

nor as terminal cost

C3 = f( x(t_N) ).

I would be very happy, if you could direct me to any

**theoretical classifications**of C1-type cost functions.As well, I would be very interested in

**publications/applications**of C1-type cost functions.Thank you very much in advance!

Best regards,

Stefan Loew

Dear Colleagues,

Could anyone suggest me the name of the SCI/SCIE journals publish work in the field of control system with high acceptance rate? I am working in fractional order optimal control.

Please suggest me.

Thank you in advance.

Dear Colleagues,

I am dealing with Fractional order optimal control problem, I am trying to solve my problem using Multi Step Differential Transformation Method in MATLAB/Mathematica. I am looking for the code.

Please share the code(MATLAB/Mathematica) for solving my problem using Multi Step Differential Transformation Method.

Thank you in advance.

Let us consider an optimal control problem, where the final time is a free parameter. The objective function to minimize is J1. Since the final time is a free variable, the optimal control solution can be obtained in a fixed final time, optimal control framework by considering the final time as a design parameter.

Now, can we choose a different cost function J2 and select the final time (design parameter)

such that the final time will minimize J2?

It seems the state evolution ode; xdot=f(x,u), is known to satisfy f(0,0)=0, hence f(0,u(0))=0, then can we conclude at any control system, always u(0)=u(x=0)=0, and get the result that limit(u(x)) is zero when x goes to zero.

I think in classical linear/nonlinear control system this is trivial. How do you think?

But my focus is on optimal control. I want to know whether this assumption similarly holds also for optimal control problems (in case of regulation or trajectory optimization or so on)?. I want to know whether it is possible to assume in a typical optimal control problem; u(x=0)=u(0)=0?

Real-time optimal control is widely adopted in central heating, ventilation and air-conditioning systems to improve the building energy management efficiency by adjusting system controllable variables (e.g. chilled water supply temperature) according to system operating conditions. In literature, there are generally two types of methods to trigger the HVAC optimal control: (mostly is) time-driven optimal control and (some is) event-driven optimal control.

In time-driven optimal control, optimal control actions are triggered periodically, e.g every 1 hour trigger an optimal control. In event-driven optimal control, optimal control actions are

triggered by “events” instead of a “clock”, which creates non-periodic control.

I was wondering what are the triggering methods of HVAC optimal control used in engineering projects? As reported in the literature, most practical projects adopted time-driven optimal control. Some engineering projects may use the so-called rule-based method to trigger optimal control (indeed rule-based method belongs to event-driven method). The rule can be pre-defined in building automation system or sometimes defined by human operators to decide a experience-based rule and operate the optimal control manually.

This is what I have learnt from the literature. I am not sure whether this is true in actual engineering projects. Pls leave a comment.

Hi,

What are the books and/or papaers/reports that are recommended in order to understand

**fullly**the mathematical stuff in the following book:Robust and Optimal Control by Kemin Zhou et al.

i.e. how the thorems are constructed and the assumptions are made and ultimately how they are proved. It will be helpful to understand the historical devlopment as well.

Thanks & regards

Hi dear researchers,

I am currently implementing

**direct collocation NMPC**,However, I have a doubt on the control value for the first control u_0

that should be applied to the system once the control trajectory is found as collocation downsamples the system dynamics before solving the dynamic NLP problem.

will the control value u_0 be (u*(0)+u*(1))/2?

x_k+1 = f(x_k,u_0);

Should this be the case, will that not have a detrimental effect in the accuracy of the true control?

Kind regards