Optimization (Mathematical Programming) - Science topic

To tractably reformulate robust nonlinear constraints, you can use the Fenchel duality scheme proposed by Ben Tal, Hertog and Vial in

"Deriving Robust Counterparts of Nonlinear Uncertain Inequalities"

Also, you can use Affine Decision Rules to deal with the multi-stage decision making structure. Check for example: "Optimality of Affine Policies in Multistage Robust Optimization" by Bertsimas, Iancu and Parrilo.

Hi Masar Alrabeeah,

Did you code the epsilon constraint method using OPL? May I ask how did you code this? I tried but could not get the right results.

Thanks a lot.

Ondřej Benedikt Michael Patriksson Alexandre Frias Faria Adam N. Letchford Thanks all for your valuable insights. I have played more with my instances and find out that it's more related to the structure of my model formulation.

My model needs to determine both routes of vehicles & time-windows of customers given a set of scenarios each of which contains parameters like traveling time. So, it's quite harder than classic VRPTW where time-windows are known.

It's hard for some specific problem size. If I simply ignore these "bad instances", then the solving time indeed increases when problem size grows.

In the Matlab page of fmincon mentioned that a parameter to speed up as follows:

UseParallel

"When true, fmincon estimates gradients in parallel. Disable by setting to the default, false. trust-region-reflective requires a gradient in the objective, so UseParallel does not apply. See Parallel Computing. "

Yes, OpEMCSS comes with my Wiley textbook “Simulation-Based Engineering of Complex Systems” that has a Classifier block that can do Reinforcement Learning (RL). RL does the same thing as Dynamic Programming without storing all the paths. RL learns the rules to create a plan (a sequence of rules) that solves a problem. I covered both of theses techniques in my CSUF, EE585 class, Mathematical Optimization.”. How to set up a program to do RL is explained in my textbook.

Please refer to the following link for the population-based method Black-Hole Optimization Algorithm:

Bat algorithm (BA) is a bio-inspired algorithm developed by Xin-She Yang in 2010 and BA has been found to be very efficient .

The method for solving this problem depends on the properties of functions F, f, g (convex, concave, other properties).

If properties are not known - the only method is looking through all values of variables.

Dear Lars

I agreed 100 % with you in that a Solver must identify unfeasibility, but my question is: How many solvers or MCDM methods do you know that have that capacity?

Only one: Linear Programming

The procedure is very simple: It compares criteria independent values, and if a solution satisfies these values. If only one criterion is not satisfied the project is unfeasible.

Nowadays, problems are 'solved' assuming that a problem is feasible, not taking into account that t hat circumstance may not exist.

I have read hundreds of comments and papers from our colleagues. How many of them posed this problem?

Nobody.

I wrote in RG almost a year ago about this problem, you can see it in my profile under the number 304, and again, in May 2020 under the number 318. Both have had some moderate reading but nobody came forward to acknowledge and discuss it. You are the only person that addresses that issue.

Regarding LocalSolver I know what it, is but my experience on it is null.

I suggest referring section seven of the following paper:

The 7^th section, namely "Results, Data Analysis, and Comparison", of the following current-state-of-the-art research paper have a sufficient answer for this question:

Dear Lars,

I suggest you to see links and attached files on topic.

Best regards

Hi my dear friend

Good day!

I think the below book will help you a lot to provide relevant codes.

Messac, A. (2015). Optimization in practice with MATLAB®: for engineering students and professionals. Cambridge University Press.

best regards.

Saeed Rezaeian-Marjani

Good!

Hi, Imran and Farah

Before questions, you must read the Linus textbook that is free from LINO HP.

This

https://docs.mosek.com/modeling-cookbook/index.html

also provide a lot of information for how to convert a convex problem to conic form e.g. second order cone form. This should be your best option as mentioned by Adam Letchford. Mosek is of course one among several optimizers for SOCPs.

Hello Amid,

very sorry to see only today that you asked me a question two years ago !

My answer :

1. What's the practical implication of CICA? and please mention some CICA for worker's hand cut off senario due to conveyor belt stiking in

MERMOS is built at the level of a the failure of the working team, then I do no know how to consider that event without more information. Our assumption is that failure happen because of a rational teamwork behavior that is no more adequate. The anayst describe with cica the behaviour of the team that are at the center of the failure story that is quantified. here it could be the CICA 1 the team (or the worker if no team) wants to fix the conveyor 2 the team doesnt want to spare time by avoding to switch off conveyor power source . that two elements are enough to exmplain the story.

2. After multiplying the results of 3 parameters (No reconfiguration probability -sf -cica) and obtaining a probability number , how the obtained probability number is interpreted?

the three elements (situation, CICAs, non reconfiguration) are the required elements of the failing scenario to exist (situation generating the CICAs, CICAS lasting too long without reconfiguration). they have to be conditional : the situtation features probability is the conditional probabilities that that particular situation occurs given the general context of the analysis, the CICAs occurs with a given probability given that situation, the non reconfiguration depends on the both. The has to try to built a plausible scenrio by having the probability of the cicas to 1. tha mean that he have to describe precisely the situation features (more it is precise, more the probability os low). the non reconfiguration takes in account the recovery induced by the MMI and the organsiaation (redundant roles, verification by procedures ...).

If these explanations are still useful but not enough do not hesitate to ask me again.

@ Peter Breuer,

Do you have a reference to a geometry book that one can cite for the pdf of 2r/R^2??

I do not believe researchers can screen out the most important factors that influence processes. It makes no sense. A good foundational knowledge of what to do, when to do, why do and how to do is strongly recommended. You can do a good or decent screening with the wealth of information you have acquired about the nature of research you embark on.

The easy methos employ a momento factor which is a stechastic mechanism to avoid get trapped in local minima

There are really more than 50-60 algorithms. It looks like everyone is trying to change a few things in an algorithm and come up with a new algorithm. And of course everyone is promoting their own algorithms. I am checking papers which compare the performance of these algorithms, yet everyone choose to compare different sets of algorithms. At least, a few algorithms seem to appear more in these comparison papers. I just need multiobjective optimization for a FEM updating problem that I am encountering. But what I have seen that, multiobjective optimization research community needs to sit down and to do more surveys about which algorithms are most suitable for what kind of problems. In a few weeks, I can tell you which algorithms that I will choose and why. I am working on it too.

My graph also came the same way as that of Achour Hadjar . Do we have to add constraints in Genetic Algorithm (gamultiobj in MATLAB). Because if non-linear constraints are ignored then Parento front is being plotted in the correct way but on adding constraints its showing only 1 point. what might be the problem?

In principle, when modeling a problem with different modeling languages (such as GAMS, AMPL), you can expect that a global solver, such as CPlex, will return the same objective function value, no matter what modeling language you used. But...

The solution process within Cplex is different, when you present the variables and constraints in a different ordering. And you have no control about the ordering of those (i.e. the interface between the modeling language and the solver), and even if you would have that kind of control, there's no general strategy to find a good ordering (for example, and ordering that leads to minimum runtime).

What does that mean for you? In case you have an easy problem with a unique optimum, you would not feel much difference from using different modeling languages or different MIP solvers.

In case you have multiple optima, you might get different solutions from different modeling languages (although, of course, they have the same objective function value).

In case your problem is too difficult to find a global optimal solution in finite time, and you terminate the solution process prematurely, then you will easily end up with very different solutions and very different objective function values, depending on the modeling language you have used.

Nice links from Nicolas Dupin. The articles are interesting and straight to the point. Am sure Alex will find them useful.

 IEEE CEC 2011 Real World Problems

In you case, M must be grater than the largest difference between the reorder point and the current inventory position. Say, M=(The largest applicable inventory position - The lowest/smallest applicable reorder point).

Dear Tejaswinee Darure,

The following link can help you to find the convexity of the objective function and if the objective function and the constraints are convex you can find the following papers useful to solve the optimization problem:

https://www.researchgate.net/publication/236899619_A_differential_inclusion-based_approach_for_solving_nonsmooth_convex_optimization_problems 

This publication and relevant publications of the authors can help you.

Hi Radhia,

I think the main difficulty comes from the fact that nonlinear constraints are inherently much more difficult to satisfy than linear ones.

Evolutionary algorithms can usually deal reasonably well with linear constraints, but when it comes to nonlinear ones, the usually employed strategy of augmenting the objective function(s) with a penalty term can perform quite poorly depending on the cases.

What can happen is that you try to solve a problem with a different objective function, and the addition of the penalty term can drive the evolutionary algorithm "astray". In addition, if you employ a penalty approach, you need to balance between solutions that have a good feasibility but poor objective values and solutions with poor feasibility but good objective values.

You can submit and run your GAMS file through the NEOS server.  There is no limit on model size or CPU time.  See https://neos-server.org/neos/.  The list of MINLP solvers is provided at https://neos-server.org/neos/solvers/index.html#minco and it includes the global solvers BARON, Couenne, LindoGlobal and Scip under GAMS.

We discovered rounding, etc, was not the best approach for PSOs and discrete sets of integers. There is an adapted PSO addressing this for discretes. We were able to combine this with the other types of PSOs. There is a "Discrete Particle Swarm Optimization" algorithm. Below is a reference that might help.

Dear MARUTI PATIL, please check the following link  http://yarpiz.com/

You will find what you want, or check this link

Supposing that you can write/change the programming code of your computer program or computer program that you are using:

You can firstly test, improve your algorithm and adapt its parameters for a little example where you can find deterministically the optimal Pareto Front, for example by an enumeration model (probably exponential) if the space domain is discrete. For larger problems you can end the run in a similar way to what is normally done for a conventional optimization algorithm with a single objective function, for example when there is no improvement (punctual or in average) of the objective function value, with the necessary adaptations given that all objective functions must be considered simultaneously now.

Thank you for your responses.

Ronald's suggestion gave me a basic vision to produce the vector. Also attached the function is more appropriate than using "if" & "for". 

Sadegh

Yes. In fact there are some tutorials on YouTube. If R2014a features optimization toolbox, you can easily optimize your functions with its help.

Yes, you must just to add to the fitness function a penalty one for the non fulfillment of the restrictions,

Best wishes,

As Alan suggests, you need to install it form github:

> require(devtools)

Loading required package: devtools

devtools::install_github("rCarto/osrm")

> devtools::install_github("rCarto/osrm")

> library(osrm)

Data (c) OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright

Routes: OSRM. http://project-osrm.org/

If you plan to use the OSRM public API, read the OSRM API Usage Policy:

> data("com")

> distCom <- osrmTable(loc = com[1:50, c("name","lon","lat")])

> distCom$duration[1:5,1:5]

Bethune Annezin Denderleeuw Haaltert Locon

Bethune 0.0 6.2 108.0 104.4 8.0

Annezin 5.5 0.0 110.0 106.3 7.2

Denderleeuw 111.2 114.2 0.0 12.7 107.1

Haaltert 107.7 110.8 12.8 0.0 103.7

Locon 7.3 7.0 102.8 99.2 0.0

Dear Kishore Shaurya,

your description of the problem under consideration makes clear, that your function f is neither differentiable nor convex (as long as it is not constant). So the idea of a bundle method is not applicable - and I got no idea how to determine a valid lower bound for it.

A domain decomposition approach like that of BARON for global optimization requires the ability to solve local minimization - but without further knowledge about the function f I see no way of doing that.

With such kind of problem you are really just left with search methods without any guarantee for the solution quality.

If n is small you could just try to evaluate f on a grid over the unit cube to get an idea how f behaves.

Even the search algorithm DIRECT

needs Lipschitz continuity, which is not fulfilled by an integer-valued function f with jumps.

Best regards

Ralf

Dear Carvalho, thanks much for the hints...

Hi,

you can use some (free) graphics tools, as Gnuplot or R (libreary scatterplot3d), both libraries are of easy use.

In the other hand maybe you are interested in some attainment surface tool, you can use the next tool in C language and Gnuplot which is easy to implement.

http://www.cs.bham.ac.uk/~jdk/plot_attainments/ 

There are others graph representations like:

scatterplots, bar chart, star coordinates

More information in the Deb book: chapter 8 "Salient Issues of Multi-Objective Evolutionary Algorithms"

Regards,

X(u) is a norm function (total variation norm). dL/du = 0 mean one must find the image u that minimizes X(u)+\lambda*||u-g||^2.

||u-g|| is the Frobenius norm.

The system is solved numerically, so, as Aparna pointed out, is not exact. But I get a close behavior of \lambda* to -dX*/dc, except from a scaling factor, which should not happen (at least if the theoretical derivation is correct).

Thanks 

Hi, you can refer for example to this complete page:

.

here is a link to a limited empirical research in the firefly parameter tuning

.

Hello, the reason is that it is a secondary criterion that is used if there is a surplus of solutions based on the non-domination rank, starting from the first/best front, and the last front considered has more solutions than are required in the next generation. More solutions tend to belong to the same non-domination rank as optimization proceeds.

Dear Soumitra K Mallick

Thanks for your answer. Well, my purpose is to specify a predefined controlled step for the variables at each GA iteration. In fact, this shall decrease the computation time which is something I look to.

Kind regards.

Moussa KAFAL 

Hello Morteza,

Thank you for providing me with the sources. 

Best regards

Redouane

Dear Hamid,

To check the statistical significance of pairwise differences among solutions, the Wilcoxon test is fine and the Chi-square test also would work. To perform these tests, the null hypothesis is that there is no significant difference between the two solutions at a significant level (e.g., 5 %). The p value and z value are used to assess the significance of differences between the solutions. When the p value is less than the significant level (0.05) and the z value exceeds the critical values of z (−1.96 and +1.96), the null hypothesis is rejected, meaning that the performance of the algorithms is significantly different.

Good luck with your study.

Thanks for the answer. Is it possible to reduce the computational complexity by adopting any other methods under specialized cases such as uni -modality and so on?

Amirsaman and Michael,

Thank you so much for your inputs.

Hi,

one of the solutions to escape local optimum is to perturb your new GA generation by adding some random chromosomes at the end of each new generation. Note that the randomness that you add must be well chosen and small compared to the rest of the genetic operators otherwise, you will have a random search instead of guided GA search.

Another solution is to reconsider the mutation rate (which induces some randomness to your population), be careful though with the chosen rate otherwise, your GA will be too perturbed to find a good solution. You can also consider allowing some bad solution to pass to the next generation, the same remarque here, the number of bad individuals that you allow to survive must not be big just a little few otherwise your GA will not converge to the hopeful results

I hope that this answer is helpful 

Good Luck

No i am using a sigmoid function, that generates the binary matrix for the next iteration. using the simplest binary code available in literature similar to what is attached.

Hi, could you please give us the formulation  of your mathematical  programming  problem? Normally, the total number of unknowns, variables and multipliers is ewual to the number if equations (contraints plus necessary conditions)

You can evaluate those optimizers according to several benchmark problems provided by IEEE Congress on Evolutionary Computation (CEC). There are some details that I think researchers in the field of optimization should be aware of them. Here, I list them again (I also have mentioned them for other researchers) because it can be very useful for you and other researchers. 

CEC'05: Evolutionary Real Parameter single objective optimization

CEC'06 : Evolutionary Constrained Real Parameter single objective optimization

CEC'07: Real-parameter MOEAs

CEC'08: large scale single objective global optimization with bound constraints

CEC'09: Dynamic Optimization

CEC09: real-parameter MOEAs

CEC10: large-scale single objective global optimization with bound constraints

CEC10: Evolutionary Constrained Real Parameter single objective optimization

CEC10: Niching Introduces novel scalable test problems

CEC11:  Real-world Numerical Optimization Problems

CEC2013: Real Parameter Single Objective Optimization

CEC2014: Real Parameter Single Objective Optimization (incorporates expensive function optimization).

CEC2014: Dynamic MOEA

CEC2015: Real Parameter Single Objective Optimization (incorporates 3 scenarios)

CEC2016: Real Parameter Single Objective Optimization (incorporates 4 scenarios)

CEC2017: Real Parameter Single Objective Optimization (incorporates 3 scenarios)

I think that the following points about real-world problems can help all interested researchers:

             CEC11:  Real-world Numerical Optimization Problems

In addition to these benchmark problems, several engineering benchmark problems are also used by researchers.They use these benchmarks to demonstrate the efficiency of their algorithms for some engineering applications. I summarized some of the most utilized engineering benchmark problems: 

Thank you very much for your answer!

Our library has this book, I will try to get it this afternoon. I read about the MatLAB codes, but tried to solve the problem solely with Abaqus to reduce complexity. As this does not work, I guess MatLAB should be the easiest solution then, I will check it out. Thank you again for your input!

Best regards

Rene Moitroux

There are many methods for choosing the "best compromise" solution in a Pareto front. Two of the classical methods are the Laplace's criterion and the Wald's criterion.

Assume you have the objectives J1 and J2. Assume we have to minimize both of them, so we solve the optimization problem and we obtain the Pareto front composed by n Pareto-optimal solutions Si composed by two objectives values for each of them: J1i and J2i (i=1...n)

The Laplace's criterion consists in the minimization of the objectives mean values for each solution, i.e. finding the solution i that solves the problem:

min(E(J1i,J2i)), where E states for "expected value". For each solution you keep the mean value and you find among them the minimum one.

The Wald's criterion consists in choosing the "least worse" case i.e. finding the solution that solves the problem: min(max(J1i,J2i)). For each solution you keep the maximum objective value and you choose among them the minimum one.

Little tip: think about two J1 and J2 objectives that belong to the set [0,10] and we want to minimize both of them (0 means excellent, 10 means awful)

The Laplace's criterion tells us that these two solutions are at the same level:

S1: {J1 = 10 , J2 = 0}

S2: {J1 = 5 , J2 = 5}

which means that even if the first objective is very bad with respect to the second in the solution S1, we consider that both S1 and S2 are a good compromise between the two stakeholders.

The Wald's criterion is quite more "pessimistic": we choose to make the best of the worse.

The two solutions:

S1: {J1 = 8 , J2 = 0}

S2: {J1 = 8 , J2 = 7}

are at the same level, because in both the solutions we are making the best we can to avoid J1 being more (worse) than 8.

These two methods both have lacks, so be careful to use them in a "rational" way.

I hope this will help you; I suggest you these keywords for a simple web research:

best compromise solution, decision theory, choosing among Pareto alternatives.

Ales:  I remember my first algorithms course at my university's computer science department - Hill climbing and A* were the most popular then, at least teaching-wise. I wonder if that still is the case. We then learnt heuristic branch-and-bound and rollout like techniques to code a chess program. (In LISP!) That was quite nice. Gosh - more than 30 years has passed since then!

thank you very much, i will read them and tell you.

You can utilize the following codes if you use MATLAB as follows:

before merging the population at the end of algorithm, please type "Stop time;"

and save the initial population in another Pop, e.g. Pop2.

Why don't you try to evolve the sequence of operators. If you are interested  contacted me.

Your results of course depend on the solver you used. There is nothing wrong with GAMS as far as I see - have a look at the attached files, where I solved your model with BARON as the MINLP solver framework, giving it an options file to have a smaller relative gap than the default 0.01.

You'll see in the lst file that the binary variables take binary values in the solution.

So what solver did you use? Maybe the solver is buggy...these are done by humans, too. To err is human ;-)

Dear Saad,

There is a mathematical theorem linking some quantities: Little's law for queuing systems isL=λW: the average queue length equals the average arrival rate times the average waiting time in the system.

It means you should not diverge too much from that without clearly violating at least one hypothesis. Usual deviations are:

Then, I can discuss more about interpreting results, because cooperative intersection is a topic I have highly investigated. But before going to theory (optimization criteria, mathematical tools, hypotheses...), check the simple issues I have mentioned.

Best regards,

Arnaud

There are several possibilities. The simpler one would be to minimize the normalized Tchebyshev distance from the ideal solution to the existence space using any numerical Non lineal Programming method. Did you understood me?

Best wishes,

Then, I think that, in general, just looking at the efficient solutions is not sufficient.  Suppose you have just two continuous variables, x and y, and that your feasible region is defined by the following two quadratic constraints:

x^2  + y^2 <= 1, (x+1)^2 + y^2 >= 1.

This feasible region is non-convex, but if your objectives are max x and max y, then the weighted sum and epsilon-constraint methods will give the same efficient frontier, namely, the region:

(x,y): 0 <= x <= 1, 0 <= y <= 1, x^2 + y^2 = 1.

Hi, if you have two different objective functions to minimize at the same time, you fall into the framework of multi-objective optimization. You can fpr example have a look to the following survey on this topic:

Dear professor Patriksson

In this paper:  ' A new cross decomposition method for stochastic mixed-integer linear programming Emmanuel Ogbe, Xiang Li' said that in mixed-integer linear programming:

" The DWD restricted master problem DWRMP provides a rigorous upper bound for Problem (P) , while the restricted Lagrangian dual RLD does not. Actually, according to Van Roy (1983) , RLD is a dual of DWRMP. On the other hand, DWPP is similar to LS and either one can provide a cut to BRMP (according to the discussion in the previous section). Therefore, using DWD instead of Lagrangian decomposition in the cross decomposition framework is likely to achieve better convergence rate."

Would you say always the upper bound obtained in Dantzig Wolfe decomposition is better than upper bound in lagrangian relaxation method and lower bound conversely?

Best Regards

it is absolutely possible, you can check this book on your question:

1- Metaheuristics for Bi-level Optimization

2- Sinha, A., Malo, P., & Deb, K. (2017). Evolutionary Bilevel Optimization: An Introduction and Recent Advances. In Recent Advances in Evolutionary Multi-objective Optimization (pp. 71-103). Springer International Publishing.

for GA, these papers have answered your question:

1- Genetic algorithm based approach to bi-level linear programming

2- A hybrid of genetic algorithm and particle swarm optimization for solving bi-level linear programming problem–A case study on supply chain model

3- I also recommend this work:

A parameterised complexity analysis of bi-level optimisation with evolutionary algorithms, MIT, 2016

Corus, D., Lehre, P. K., Neumann, F., & Pourhassan, M. (2016). A parameterised complexity analysis of bi-level optimisation with evolutionary algorithms. Evolutionary computation, 24(1), 183-203.

A remark might be added to this: if you have a convex problem, as you state, WHY do you use heuristics?? If you know that the problem is convex, you probably have an expression on the objective function and the constraints (if any), so you should be able to solve it accurately by convex optimization methods, such as by utilising (sub-)gradients.

Trying to understand your question.  Since the possibilities/models are infinite in number, parametrization cannot truly represent a state of nature but only help model a certain phenomenon (or part of it) and the loss function (in supervised methods) tests the goodness of fit of that model. The model of nature that one puts together can perform well or poorly.  The parameters of the model need to be viewed accordingly.  

The goal of all statistical methods (Bayesian or Frequentist) is to find a model that can adequately represent a phenomenon (represented by a set of available data -- real or simulated) and use that to make logical predictions.  

My sense here is that you're asking if that data can be used for bump hunting.  All unsupervised methods try to answer that very question and there are a vast number of techniques available for that, some better at certain things than others.  In these methods the term loss function is at a loss since the error functions used need to be seen more as frames of judgment for the fit. 

But even for supervised methods, one can use non-parametric methods that use the available data in different ways to establish the probability distribution, which might be the other answer that you're looking for.  

I am answering this question with respect to linear optimization.

Multi objective functions will take more than one objective functions. Let us say we want to maximize contribution (to profit) and maximize volume (in no of units) for a multiple product company. So it is necessary to use something like giving goal so that I can know how much weights we want to give each goal. So in a multi- objective framework it is necessary to assign right weights to the goals.

The slope of some of the variables, I understand that it is the coefficients will be different.

Let us say objective function is = w1(f1(X) + w2(f2(X)  

= w1(C11X1+ C12 X2+...+C1nXn) + w2 (C21X1+....+C2nXn) 

Given this , only requirement is how to assign the w1 and w2 and not C vector for 

Here we must understand that 

1) Should we solve for one goal or more than one goal . Optimal Solution for profit maximization will not be same for solution for utilization  maximization  

2) Optimal Solution for profit maximization will be different from that of cost minimization.

In my opinion finding the weights (W1 and W2) are more important that finding the

3) The coefficients are generally provided are  parameters. 

I hope it helps 

Goutam Dutta

Sorry, I don't get what actually you mean by 'I am not sure about the coding publicly shared by Mr. Zhifei Li under TSP_QPSO title (whether it is the Quantum-PSO or not).' I am not talking about his codes. Am I?

Greeting Maruti,

It is definitely possible. For me this happened on a real-world problem, where for some inputs the solution was a single point on the Pareto. This happens when the objectives araren't independant.

Best Regards,

Miha