Science topic

# Nonlinear Optimization - Science topic

Nonlinear, convex, fractional optimization, numerical methods and related statistical methods, its mathematical foundations, applications

Questions related to Nonlinear Optimization

In robust optimization, random variables are modeled as uncertain parameters belonging to a convex uncertainty set and the decision-maker protects the system against the worst case within that set.

In the context of nonlinear multi-stage max-min robust optimization problems:

What are the best robustness models such as Strict robustness, Cardinality constrained robustness, Adjustable robustness, Light robustness, Regret robustness, and Recoverable robustness?

How to solve max-min robust optimization problems without linearization/approximations efficiently? Algorithms?

How to approach nested robust optimization problems?

For example, the problem can be security-constrained AC optimal power flow.

Hello everyone!

I am trying to write a long objective function (below) in MATLAB but I could not write the summations and the squared term (x_ptsk, which is a decision variable).

Is writing a function necessary int his case or can I completely write the objective function in my main code?

Also, the a and b values in the objective function are lower and upper limits for a uniform distribution and have three indices (p,s,q) which I pre-defined the values in another function. How can I implement the indices in the function?

Thank you so much in advance and best regards.

Any decision-making problem when precisely formulated within the framework of mathematics is posed as an optimization problem. There are so many ways, in fact, I think infinitely many ways one can partition the set of all possible optimization problems into classes of problems.

1. I often hear people label meta-heuristic and heuristic algorithms as general algorithms (I understand what they mean) but I'm thinking about some things, can we apply these algorithms to any arbitrary optimization problems from any class or more precisely can we adjust/re-model any optimization problem in a way that permits us to attack those problems by the algorithms in question?

2. Then I thought well

*then by extending the argument I think also we can re-formulate any given problem to be attacked by any algorithm we desire (of-course with a cost) then it is just a useless tautology.***if we assumed that the answer to 1 is yes**I'm looking foe different insights :)

Thanks.

What are some of the well-written good references that discusses why finding the penalty parameter when solving a nonlinear constrained optimization problem is hard to find from the computational perspective. What are some of the computational methods done to find the parameter as I understand finding such a parameter is problem-dependant.

Any insights also would be very helpful.

Thanks

Hello, I currently solve the optimization problem (please see the attached figure).

Basically, this problem is equivalent to find the confidence interval for logistic regression. The objective function is

**linear**(no second derivative), meanwhile, the constraint is non-linear. Specifically, I used n = 1, alpha = 0.05, theta = logit of p where p = [0,1] (for detail, please see binomial distribution). Thus, I have a closed-form solution for the gradient and jacobian for objective and constraints respectively.In R, I first tried the alabama::auglag function which used augmented Lagrangian method with BFGS (as a default) and nloptr::auglag function which used augmented Lagrangian method with SLSQP (i.e. SLSQP as a local minimizer). Although they were able to find the (global) minimizer most time, sometimes they failed and produced a far-off solution. After all, I could obtain the best (most stable) result using SLSQP method (nloptr::nloptr with algorithm=NLOPT_LD_SLSQP).

Now, I have a question of why SLSQP produced a better result in this setting than the first two methods and why the first two methods (augmented Lagrangian with BFGS and SLSQP as a local optimizer) did not perform well. Another question is, considering my problem setting, what would be the best method to find the optimizer?

Any comments and suggestions would be much appreciated. Thanks.

Mathematics or finance masters or doctoral students can look at decomposing credit rating matrices from market prices as research project.

the project page is given below. Current research questions/ hypotheses are stated on the project page.

What is the best way to find the optimal solution of a non convex non linear optimization problem that really constitutes finding the optimal/ best combination of one member each from a number of groups or sub populations.

Each group of sub population has a number of members that are sequences - an array of elements.

The optimal solution is given by best matching a sequence or (as) member from each group or sub population.

It is similar to recombination of evolutionary algorithms (but not entirely identical to evolutionary algorithms).

Based on your expertise, what is the better optimizer tool between GAMS ( https://www.gams.com/ ) and Gurobi ( https://www.gurobi.com/ )?

Please also let me know your field of research.

Is it possible to specify upper bounds on the coefficients other than an universal or global constraint?

I am using excel solver evolutionary algorithm at present. I do not know whether this is the same for other evolutionary algorithms for nonconvex nonlinear optimization problems.

the evolutionary algorithm requires both a lower and upper bound on the optimization problem coefficients.

If i dont specify: all coefficients <= x, but specify an upper bound per coefficient instead: c1 <= x1, c2 <= x2, c3 <= x3, it complains and refuses to run. So i implement and use both.

but specifying individual upper bounds on individual coefficients imply a (significantly) smaller solution space.

I also dont want the algorithm to search outside these individual coefficient upper boundaries. Yet i get the impression that it does. Can someone confirm or refute this? Also, if this does occur, i am of the impression that it can impact the solution i get.

Is there a way to specify individual coefficient (upper) boundaries and ensure that the algorithm only searches within these?

How do you measure, gauge, etc the general non-linearity of a/ your non convex non linear optimization problem?

(How) can one quantify (%, number, measurement, etc) or qualify (scale, category, etc) it?

with evolutionary algorithms, i understand the concept of recombination and mutation, and that offspring may essentially constitute or approximate - be seen as - sub-populations.

but, what if your optimization problem really constitutes, and the initial population is best comprised out, of a combination of sub-populations?

creating a number of sub-populations, and combining them.

i suppose one can see this notion of sub-populations as a characteristic one would recombine and mutate on/ according to.

From a combinatorics perspective, what if what you are requiring of the optimization algorithm is too scattered?

In other words, from a combinatorics perspective, what if what you are requiring in terms of the optimization problem specification, also referring to the constraints, including equality constraints, implies too scattered and sparse coefficient (variable cell) ranges.

Is this possible? The algorithm may then struggle to locate the feasible regions or ranges.

I am particularly referring to a nonconvex nonlinear optimization problem.

What is the best/ next step if you suspect the evolutionary algorithm recombines and mutates in an inferior way, for your nonvonvex nonlinear optimization problem?

Write your own heuristic?

How do you implement it?

How good are evolutionary algorithms with equality constraints?

I must add the objective function is specified as the sum of absolutes.

I need/ want to use equality constraints to guide the solution.

Some of the optimization coefficients (variable cells), when summed, must equal to a certain value.

With a non convex nonlinear optimization problem, would specifying the objective function as a sum of absolutes, rather than a sum of differences, make the optimization problem (significantly) more complex?

With my problem, i think working with absolutes rather than sum of squares would be better, if this does not imply a more complex optimization problem

Min X = sum(abs(x1-y1) + abs(x2-y2) + abs(x3-y3)...)

Vs

Min X = sum((x1-y1)^2 + (x2-y2)^2 + (x3-y3)^2...)

With a non-convex non-linear optimization problem, how can you increase or amplify the remaining residual to get a better answer?

The objective function or optimization problem is a minimization.

The residual is already very low - of the order of 1e-6 or 1e-10. But the solution is still far away from the known optimal solution, given that it is a lab case.

It is a sum of differences squared. So small differences are made even smaller.

How can one amplify the remaining residuals to search them still?

I thought of multiplying the residual with a large amount, like 1e6 etc to inflate it again.

Any suggestions welcome

Up To what resolution can non-linear optimization problem algorithms work and solve nonlinear optimization problems?

I have a simulation as test case, so i know the global solution, and the coefficient values thereof.

So, theoretically, the optimization is 0.

Yet, What would be the practical residual value the optimization algorithm will and can yield? Can it be lower than 1e-15, or even lower? What about 1e-20, or so?

I am already working with initial values with residual of 1e-9.

I run a nested or multi level optimization problem, because the nonlinearity is simply too much for the evolutionary algorithm.

I use optimization to search an initial solution, to further search and interrogate at the second level.

It then essentially helps to cut up the solution space, to help the algorithm.

yet, In a sense, at the 2nd level of optimization, the evolutionary algorithm still descents too quickly. It then causes it to miss the better solution.

I have seen something similar with ordinary nonlinear programming/ optimization based on 2nd order partial derivatives, and newtons step etc.

Is there a way to prevent the evolutionary algorithm from descending too quickly?

It can rather descent slower and recalculate at that point.

What are the implications when this phenomenon is possible with nonlinear optimization and evolutionary algorithms? Still too much/ fine nonlinearity?

With a nonlinear optimization problem, would limiting the full coefficient set of x coefficients to y coefficients, with y < x, constitute searching in specific sub regions?

The remaining coefficients are set/ rendered as constants

BFGS algorithm is known to be an iterative method for solving unconstrained nonlinear optimization problems. Will different presentations of objective functions affect the applicability/feasibility/convergence of the BFGS algorithm?

On the other hand, SQP is known to be an iterative method for solving constrained nonlinear optimization problems. Will different presentations of objective functions and/or constraints affect the applicability/feasibility/convergence of the SQP?

What are the rationales behind?

In converting an optimization problem with nonlinear obj function with linear constraints into an LP or a MIP. What standard techniques are out there for doing such, Is a first orderTaylor series expansion sufficient? if yes, at what point do you linearize? Do we randomly sample the obj func over search space and fit with a hyperplane?

Dear All,

I am looking for an open source method doing dynamic constrained optimization with nonlinear constraints and upper / lower boundaries in Java. I would appreciate your advice and thoughts on it.

More specifically,

1. I am translating some MATLAB codes that include fmincon(...) function into Java. So, I am looking for a function in Java to work similar to fmincon function in MATLAB.

2. One of the nonlinear constraints is the norm of eigenvalues of a matrix ( this matrix is built by some of the decision variables) that should be less than 1 (norm(eig())<1).

Thank you,

Iman

We want to compare the optimization method as applied in Lu et al., 1999 [1] with a number of alternative optimization strategies. Ideally, we would make use of an existing academic or commercial application of this method. The algorithm is named "simplex method with restricted basis entry rule" and is used to solve an approximating linear program (ALP) which approximates the actual nonlinear problem. Any suggestion on how this algorithm can be obtained, apart for implementing it ourselves, is welcome. Ideally, we would use this algorithm as a subroutine within Matlab/Octave.

[1] IEEE Transactions on Neural Networks, 10, 6, 1999, 1271-2290.

I have studied and understood the Moment-SOS hierarchy proposed by Lasserre where a sequence of semi-definite programs are required to be solved and a rank condition is invoked in order to check if the global solution is found. I was not able to find such conditions for its dual viewpoint ( also known as the Putinar's Positivstellensatz). Alternatively, is there a similar rank condition for Parrilo's sum-of-squares relaxation?

I want to solve some nonlinear optimisation problems like minimising/maximizing f(x)=x^2+sinx*y + 1/xy under the solution space 3<=x<=7, 4<=y<=11 using artificial neural network. Is it possible to solve it?

I have formulated optimization problem for building, where cost concerns with energy consumption and constraints are related to hardware limits and model of building. To solve this formulation, I need to know if problem is convex or non-convex, to select appropriate tool to solve the same.

Thanks a million in advance.

I have a nonlinear dynamical model that is described by ordinary differential equation in the form of

X'(t)= F(X,U); where X is the states and U is the control input.

- I have constraints on the states and the controls

- I would like to use optimal control theory to find the optimal control actions given those constraints.

- I am using the Pontryagin Minimun Principle

- I defined my cost function, Hamiltonian, and the co-state dynamical equations

- They are complex/coupled that they cannot be solved analytically

- I have Initial and final conditions on the states, X.

- I don't have final time (i.e. free-terminal-time)

So, I am looking for numerical algorithms that can be implemented, for example,, in MATLAB or C++ in order to find the optimal control, efficiently.

Could someone please help me with this,

I am trying to minimize a 3 variable problem, and, keep getting this error in return,

"Not enough input arguments"

Number of variables = 3

Upper bounds = [12 3 5]

Lower bounds = [6 1 1]

Function used,

function y = pallab(x)

y = -0.59567+0.008793*x(1)+0.1025*x(2)+0.40967*x(3)-0.000266667*(x(1)*x(2))+0.0000*(x(1)*x(3))-0.106*(x(3)^2);

end

Thanks in advance

I am trying non linear analysis of shear wall. I am getting results, but I am unable to plot moment vs Curvature plot.

Given a minimization problem, i want to compute the lower bounds and upper bounds for the optimal solution to lie in. i have a scalar valued function f(x1,x2,... , xn). Then suppose i use GA, PSO etc. to minimize it. The algorithm outputs me some answer it thinks is good. i want to compute best possible lower and upper bounds but mostly lower bounds to make sanity check on the solution generated. What are the available methods to compute such bounds for the optimal solution on minimization problems? Assume both cases when f is differentiable and not. Second is of more importance to me. please suggest some seminal papers that do a good job at this.

I am using accelerated bender decomposition algorithm which lower and upper bound reach convergence before second iteration ,is it normal such as case?

,as I have attached file,it is shown after iteration 1 lower and upper bound convergence,what happened ?,in fact when I use classic bender decomposition in iter num 2 lb and ub are convergence even with large size,I try to increase size of my problem however classic BDA is convergence in second iter so in this case is it worth to use accelerated BDA?

I'm going to use an utility function in my nonlinear optimization model. (for asset allocation problem)

It can be logarithmic function like : log(cTx) (C transpose X)

Please introduce me some other popular utility functions except log.

HI,

Does anyone know what are the free or commercial solvers used to solve multi-objective nonlinear optimization problems, and are they based on MOEA or they follow another approach?

Any help is appreciated

Best regards

More concrete, I would like to generate realizations of Gaussian fields having

1 The mean function : a realization of a Gaussian field (generated with SGS, with 0 mean, Gaussian type covariance and marginally standard normal)

2. The covariance type let's say Gaussian with variance 1;

I would like to know if there is a software that does that because I have tried with SGS with local varying mean implemented in SGeMS, but does not worked.

My function is nonlinear with respect to a scalar \alpha .

However, the calculation of objective function is very time consuming, making optimization also very time consuming. Also, I have to do it for 1/2 millon voxels (3d equivalent of pixels). I plan to do it using “lsqnonlin” of matlab.

Rather than optimizing over all possible real values, I plan to search over preselected 60 values. My variable \alpha (or flip angle error)

**could be anything between 0-35%;**but, I want to pass only**linearly spaced points as candidates (i.e. 0:005:0.35)**. In other words, I want lsqnonlin to choose possible solution only from (0:005:0.35). Since I can pre-calculate objective values for these, it would be very fast. In other words, I need to restrict search space.Here, I am talking about single voxel; though I performs lsqnonlin over multivoxel and corresponding \alpha is mapped accordingly to a column vector.

I can not do grid search over preselected value as I plan to perform spatial smoothing in 3D. Some guidance would be highly appreciated.

Regards, Dushyant

How can I reduce the gap in NLP problem by GAMS software?

I have an optimization problem where the objective function is nonlinear and has 4 nonlinear constraints that can be translated in linear matrix inequality form, and decision variables are matrix. How can solve this problem with MATLAB?

For linear CG method the search directions must satisfy the conjugacy condition

d_i'Hd_j=0, i not equal to j.

It is a fact that for a nonlinear CG method the above equation does not hold, since the Hessian changes at different iterations.

With this in mind, I am asking whether there is any modified conjugacy condition for the directions of nonlinear CG methods. If there is no such condition, then why are we using the word "conjugate" for nonlinear CG methods.

Is there any

**simple**way for**obtaining global optimum**when you have nonlinear conditions for optimization problem.(Toolbox of Matlab is useful however, it has some disadvantages)How to minimize the following non linear optimization problem:

**find x***that minimizes the H-infinity-norm:

**|| G(s, x) ||**

_{oo}Subject to :

**xmin < x < xmax**Where:

**s=j*w**and

**w**=

**wmin to wmax { for example: w = logspace (-3,+3,50)}**

**G**: denotes the transfer matrix that given (for example as):

**G(s,x)=[(s^x1)+3/s -9+x2*s+x3; 1+x1^x2 s/x3]**

**x=[x1;x2;x3]**denotes the design parameter vector to be determined by an optimization method

I am doing structural optimization using a response surface method. My objective function is to reduce weight using the LS-OPT tool. I have done a total of 16 experiments. For each experiment, it displayed a computed value and a predicted value for weight. Which one should I choose as a optimum final answer? The compound or the predicted?

Hi there,

I do some research on approximation algorithms for quadratic programming. I try to optimize a quadratic function with a polytope as feasible set (a QP in standard form, to define it briefly). The matrix of the quadratic term would be indefinite in the general case.

I already know Vavasis' algorithm [1] to approximate global minima of such QP's is polynomial time (provided that the number of negative eigenvalues of the quadratic term is a fixed constant). Recently, I found an algorithm by Ye [2], which yields a guaranteed 4/7-approximation of the solution of a quadratically constrained QP. Ye developed his algorithm starting from a positive semi-definite relaxation of the original problem.

I wonder if there are PSDP relaxations of linearly constrained QP's that lead to similar approximation guarantees. Does anyone know at least one paper in which such a technique is posed?

[1] S. A. Vavasis, Approximation algorithms for indefinite quadratic programming, Math. Prog. 57 (1992), pp. 279-311.

[2] Y. Ye, Interior point algorithms: theory and analysis, Wiley-Interscience (1997), pp. 325-332.

Can I approximate over the total range of operation a synchronise machine with nonlinear magnetisation characteristic as a PT2-element with variable parameters?

I found a large number of libraries for solvong nonlinear problems.

Some are listed here on Wikipedia: http://en.wikipedia.org/wiki/List_of_optimization_software

Is it possible to tell which one is to be used preferrably by means of

* time-efficient problem solving

* wide range of available optimization algorithms

* wide range of possible parameterizations

* reasonably complex API / good documentation

?

If not, are there any libraries which you prefer?

And which are your reasons for preferring those?

Best.

During my research of study about how to reduce the complexity of ML problem, I presented many algorithms which are used to solve this problem. I need to compare my results with the early new results

Notice that S is pointed if and only if S \cap -S = {0}

I model a structure in Ansys and I want to make optimization and I think I can use Levenberg-Marquardt for optimization purpose. But I don't know how and what is the theory behind it.

I have a non-linear integer optimization problem which I found a closed form solution for. It is more accurate in some cases than typical solver solutions. I have never used CONOPT but I heard it is a powerful optimization tool which may question my closed form algorithm.

Consider solving the following system for unknowns A1, A2, B, D, G:

w = 2 B^2 + D^2;

z = A1 B + A2 B + G D;

y = 2 A1 A2 + G^2;

x = A1^2 + A2^2 + G^2;

a2 A1 + a2 A2 + a G = x2 B + x4 B + x3 D;

Where w, z, y, x, a2, x2, x4, x3 are known quantities.

Any ideas ?

***** EDIT: This problem is solved. See comment below. *******

What weight functions you are using for IRLS and for what kind of tasks?

Results could be wildly different with different weight choice. Simulating L1 is quite polular, especially for sparce recovery.

what about L_eps eps<1 ? Simulating L0 ? "negative" L norm - E^2/E_previous^(2+eps) ?