Multiobjective Optimization - Science topic

Hello Sofia,

It is indeed possible that for some instances of your problem, the objectives are not conflicting. For example, in a shortest path problem, the path with the shortest distance may also be the fastest in terms of travel time, resulting in no trade-off between the objectives.

However, the lack of a diverse Pareto front in some instances does not always imply non-conflicting objectives. It could also be due to factors such as:

If you describe your problem, we can help you more easily and as Roy already stated, you can verify whether your objectives are conflicting by calculating the correlation coefficient.

There was a similar question already asked here:

I hope this helps!

I recommend studying the following systematic review.

The development of descent methods for convex multiobjective optimization problems may still be useful, even in cases where the Pareto critical points can be obtained using weighted scalarization.

One reason for this is that descent methods can often be implemented more efficiently and with lower computational complexity than weighted scalarization. This can make them more practical to use in cases where the optimization problem is large or the available computational resources are limited.

Additionally, descent methods may be more flexible than weighted scalarization, as they can be applied to a broader range of optimization problems and can be adapted to different scenarios and objectives. This can make them more versatile and useful in a wider range of applications.

Overall, while weighted scalarization can be a useful tool for solving convex multiobjective optimization problems, the development of descent methods for these problems is still an active area of research and can provide valuable insights and solutions for addressing these types of optimization challenges.

Hi Mohammadhassan Eftekhar

You can stick with Mamdani Fuzzy System if it is comfortable for you to design a "workable" FIS. You probably don't need optimization if your FIS works satisfactorily. From the "workable" FIS to the "satisfactory" FIS, you probably just need make minor adjustment to the design parameters. That's the Spirit of Engineering Design.

See platEMO:

dear Ahmed Tijani Salawudeen

dear Roy de Winter

thank you for your suggestion.

You most certainly can. In fact, there may be an advantage in employing such a method since we know that some algorithms are better suited for some problems over others (NFL theorem). Just be sure to properly select the correct MH for any given objective function. This may require some experimentation on your part, but whatever you do, don't let the algorithm selection process be randomized. A methodological approach needs to be adopted here.

Personally, I would advise using Pareto-based techniques instead of conventional non-Pareto-based approaches like Lexicographic or sum weighted optimization. Nonetheless, good luck!

If there is no layout available, nexttile creates one. Create a Pareto chart by passing the axes to the pareto function as the first argument. Call the nexttile function to add a second axes object to the layout. Then create a second Pareto chart.

Try implementing your methodology in structural optimization tasks. A quick literature search will get you up-to-date on all the details required.

Good Luck

this is a natural thing in mathematics there is no perfect solution especially the no linear problem, and most metaH utilized in the no linear problem, so you need always to expect and accept that fact, and keep improving the study of the metaH

You did not specify the type of problem you are dealing with and why you have to resort to heuristics instead of exact optimization algorithms. That's why nobody will be able to give valuable advice.

Best regards

MOSFP: https://github.com/sh7adeh1990/MOSFP

Thank you Prof. Adam.

Same problem. I've already tried to change the optimization parameters (Pareto percentage, population stability, and others) but the objective function still does not converge.

Two functions fitted simultaneously to one dataset? Do I understand correctly? What is the objective function?

Dear Jose

As I see it, to do what you say, you need to get a physical experiment done, and that is impossible in most MCDM scenarios

I am a bit muddled up with respect to tournament selection..

Should the population set size be the same as the size of the offspring?

If that is the case, why do we need a tournament selection strategy to form a mating pool- we might as well as directly use the entire population as a whole?

The disadvantages of epsilon-constraint method are difficulty in choice of perfect value for epsilon vector, none equality-spaced Pareto solutions and requiring a large number of iterations like weighting method.

Iago Augusto Carvalho sorry for the late response. I saw this approach in some papers. I can't remember all of them, but I found at least one where they used this method:

I suggest to read

Thank you Mr. Rasel Ahmed for you response. I'm agree with you. The best way to read a Pareto Front is to take 2 by 2 objectives curves.

Based on our research, I come to conclusion that GD, IGD and HVI are truly distinct performance indicators.

As it is known that HVI is unsuitable for many objective optimization. I recommend to use IGD indicator for many objective optimization.

For further details, you can check the article:

Suggest to read my papers. For example, the codes of MOSEO (Multi-objective Social Engineering Optimizer) is one the best and outperforms the aforementioned metaheuristics. See the codes:

I also suggest to read my other papers as I applied several multi-objective optimization algorithms from my studies.

Regards,

Amir

No.

I don't think brute or exhaustive search is convenient to solve a complicate problem such as multi-objective optimisation. You can use the brute search on small scale to examine the design variables and explore the search domain.

There are different ways to check the correctness and accuracy of the implemented code. The following approaches to check the optimization can be addressed briefly as follows:

A) Comparison with classical approaches: In some problems, it can be possible to run both the classical approach and multi-variate GA to make sure the results are correct and accurate. However, in most cases, because of mathematical complexity using a classical approach for comparison is not possible.

B) Using Well-known Benchmarks: The correctness of the code can be also proven though existing benchmarks. The used algorithm can be validated via well-known benchmarks [1-5].

C) Using Expert Idea: An expert of the system can check the results and their rationality. They should all agree that the final results are logical.

D) Convergence: The convergence of the implemented multi-objective genetic algorithm can be easily observed through the plot of cost vs. iterations. There are also other factors such as roundoff errors and local extrema that have the same perspective.

E) Domination Factor: The NSGA-II optimization algorithm and other recent versions have the function to separate dominant results from non-dominant one. So, all final results should be dominant and with a proper initial population, it is very rare to be trapped by a local minimum. For more information regarding the domination please check [6].

[1] Yao, X., Liu, Y., & Lin, G. (1999). Evolutionary programming made faster. IEEE Transactions on Evolutionary Computation, 3(2), 82-102.

[2] Digalakis, J. G., & Margaritis, K. G. (2001). On benchmarking functions for genetic algorithms. International journal of computer mathematics, 77(4), 481-506.

[3] Molga, M., & Smutnicki, C. (2005). Test functions for optimization needs. Test functions for optimization needs, 101.

[4] Yang, X. S. (2010). Test problems in optimization. arXiv preprint arXiv:1008.0549.

[5] Mirjalili, S., & Lewis, A. (2013). S-shaped versus V-shaped transfer functions for binary particle swarm optimization. Swarm and Evolutionary Computation, 9, 1-14.

[6] Elarbi, M., Bechikh, S., Gupta, A., Said, L. B., & Ong, Y. S. (2017). A new decomposition-based NSGA-II for many-objective optimization. IEEE transactions on systems, man, and cybernetics: systems, 48(7), 1191-1210.

There is at least one bug: to detect the Pareto front you have to use exact optimisation tools. GA is not going to provide you with guaranteed optimal solutions, hence the "front" that you receive is very likely not the front.

Good!

Dear Mohamed,

Quadprog and Gurobi are given me the same objective function value and optimized values x , but GPAD gives me the same optimized values x, but an objective function value, which is a factor 10 bigger compared to quadprog and Gurobi. How is this possible?

GAMS itself is no solver, it is a modeling system with interfaces to a number of solvers - Gurobi being one of them.

So there is no point in trying to compare GAMS with any solver.

Gurobi is a solver for linear and (convex) quadratic mixed-integer problems.

GAMS has interfaces to solvers for different problem classes, too (like more general nonlinear optimization problems).

But if your problem is a MILP, you might compare the modeling via GAMS with the modeling via the Gurobi Python API.

I have worked with both (and with other modeling systems like AMPL, OPL and other solvers, especially CPLEX) in research on discrete stochastic optimization. The choice of the modeling interface depends simply on your preferences and the tasks to be performed (like modifying the problem according to the solution and solving a series of problems). If you are used to the GAMS language you might prefer it to others. GAMS and AMPL are not free software and you have to pay for many of the solvers for them, too.

Best regards

Ralf

I've been using the emoa package. You can get the nondominated points then use the results in the plot function directly.

Ideal Cone method is an appropriate way to find complete Pareto front.

Of course, there is a marked difference between the one-objective, two-objective, and three-objective problems, and so on, in general. And then, there is an additional complication associated with how heterogeneous the objectives are. To cut it short - it's a mess. :-)

See the book

https://www.amazon.com/Multi-Objective-Optimization-Using-Evolutionary-Algorithms/dp/0470743611

Here, there are many usefull examples

The discontinuity looks like it arises due to a concave section of the objective front, leading to regions of the objective front that are dominated. Concavities in the objective front are quite common on many real engineering problems, leading to discontinuities in the Pareto front. The other possible reason is that the objective front forms two distinct feasible regions with a gap in between; from the way the top section is curving just before the break, two separated regions could be the cause. For clarity: the objective front is the outer boundary of the objective space region that contains feasible solutions, the Pareto front is the subset of the objective front with non-dominated solutions. There has been work over the years trying to develop optimisers that locate the objective front so that the reason for Pareto discontinuities can be explored more easily, but designing algorithms to find the objective front is really not easy (especially for problems with more than two objectives). Evan

Dear David:

You could change in your model the multiobjective function by a penalty function for the non-fulfillment of restrictions, to generate a very big solutions population calculating, for each solution, every one of your real optimization criteria. So, you will obtain probalistic distributions for each criteria and you could to compare the results of your optimization problem with the best results of your auxiliary model (using penalty function). Do you understood?

Exact methods: those of mathematical programming, depending on the nature of the equations and variables, i.e. Simplex, Branch & Bound, Non-linear methods, etc.

The multiobjective problem is transformed into a single-objective problem using diverse methods. The most used are weights method and epsilon-constraint method. I recommend you to read:

Ehrgott, M. (2005). Multicriteria optimization (Vol. 491). Springer Science & Business Media.

I am one of Dr. Deb's PhD student. Our Python framework contains a working and benchmarked NSGA-II and NSGA-III implementation.

Please have a look if you are still interested.

The problema seem to be small. Under such conditions clasical approaches are appropiated.

The MOEA approximated the set of efficient thinning regimes (with a discontinuous Pareto front) by employing a ranking scheme developed by Fonseca and Fleming (1993), which was a Pareto-based ranking (a.k.a Multiobjective Genetic Algorithm—MOGA). In this paper we solve the same problem using an improved version of a fitness sharing Pareto ranking algorithm (a.k.a Nondominated Sorting Genetic Algorithm—NSGA II) originally developed by Srinivas and Deb (1994) and examine the results. Our findings indicate that NSGA II approximates the entire Pareto front whereas MOGA only determines a subdomain of the Pareto points.

If you have only two criteria, then the trade-off curve approach is fairly practical You simply construct a curve, the so-called efficient frontier, that shows how you can trade off one criterion for another. You want the risk to be low, and the cost also low. Each point on the curve risk-cost is Pareto optimal, i.e. for any point on the curve there is no other point with lower cost and lower risk.

You can start with with some total cost limit C value, and solve the 1-criterion risk problem for that cost limit. Then you change that cost limit to various values, solve the 1-criterion risk problem again for that new cost limit constraint, and plot the curve cost-risk. You will get a trade-off curve, or efficient frontier. From that curve you then pick the cost for the acceptable level of risk, or the risk that will give you the best possible cost.

The simplest way to deal with it is to swap signs of criteria values. Example: a given solution has score in three objectives of (4, 5, 6) with the 3rd a non-benefit (i.e. cost). Just change it to (4, 5, -6).

There are really more than 50-60 algorithms. It looks like everyone is trying to change a few things in an algorithm and come up with a new algorithm. And of course everyone is promoting their own algorithms. I am checking papers which compare the performance of these algorithms, yet everyone choose to compare different sets of algorithms. At least, a few algorithms seem to appear more in these comparison papers. I just need multiobjective optimization for a FEM updating problem that I am encountering. But what I have seen that, multiobjective optimization research community needs to sit down and to do more surveys about which algorithms are most suitable for what kind of problems. In a few weeks, I can tell you which algorithms that I will choose and why. I am working on it too.

Dear Thomas,

In case if any objective is of maximization type, convert it into minimization type by multiplying it with -1 and then optimize the problem as usual. This wouldn't conflict with aim of maximizing the hypervolume.

Dear Mohamed-Mourad Lafifi , thank you for your answer. The references provide information about the methods that work based on the reference-point and sampling reference-points to be used with performance indicators in the evaluation stage. However, there is no information about how does the DM determine the reference points, for a particular problem, to be used with reference point-based (or preference-based) optimization algorithms.

Curve number values never above 100. That may be the case.

Thanks a lot Mr. Temitayo Bankole !

You surely use some heuristic to find an approximate Pareto front. You will not able to find the Pareto front. To approach better to it, in the supposition that you generate a solutions population, you should to simulate the real behavior of your process, for each solution in the population to recalculate the the multiple objective function value and to reorder the found solutions. You won't reach exactly the Pareto front but you will approach enough and to obtain very good close to the Pareto front solutions

I think you can use global algorithm that observe all of equations

Well, first you should find an equation or a function in order to find the value of two variables

then you can use GA algorithm or PSO algorithm

please check the following links

My graph also came the same way as that of Achour Hadjar . Do we have to add constraints in Genetic Algorithm (gamultiobj in MATLAB). Because if non-linear constraints are ignored then Parento front is being plotted in the correct way but on adding constraints its showing only 1 point. what might be the problem?

"The IGD measure is calculated as the average distance from each reference point to the nearest solution in the solution set, which can be viewed as an approximate distance from the Pareto front to the solution set in the objective space."

Mohamed Azab

Would you please elaborate what do you mean by replace mean function with another one. Thank you

I think you cannot use the original IGD to evaluate the performance of a preference-based EMO algorithm. We have developed systematic way for this purpose. Please find more details from the following reference:

K. Li, K. Deb, X. Yao, “R-Metric: Evaluating the Performance of Preference-Based Evolutionary Multi-Objective Optimization Using Reference Points”, IEEE Trans. Evolutionary Computation (TEVC), in press.

You should test your algorithm on different and independent runs. Parameters such as population size, number of generations, mutation probability start with lower values. Crossover starts from higher values.

Example:

- population size 50, 100, 200, ...

- number of generations 100, 200, 500, 1000, ...

- mutation probability 0.1, 0.2, ..., 1.0

- crossover rate 0.9, 0.8, ...

Kindly follow:

Hi, if you do a weighted sum of the objectives, then you will only find a single solution to the optimisation problem for each run of the optimiser; a multi-objective GA will return many results that all are (hopefully) on the Pareto front. The Pareto front is the trade-off surface of the optimisation problem, so if one objective is more important than the other, the solutions that lie more to one end of the Pareto front will be the ones that interest you.

For many optimisation problems, the weighted sum approach is fine, however there are limitations; in particular if the shape of the Pareto front is concave (i.e. bends backwards on itself). Solutions in the concavity region are still optimal choices, but there are no combinations of weights for the weighted sum that will allow you to find them. Therefore if your ideal solution lies within a Pareto concavity, the weighted sum method will never find it, and the optimiser will report a result that is from the closest non-concave Pareto section it can reach.

If you have a situation where with one set of weights you find a solution, and then changing the weights slightly keeps giving you the same solution, up to a point when the optimiser output suddenly jumps to another solution; you have a concavity in that situation and should not be using a weighted sum.

Sometimes a good approach is to use a quick run (i.e. small population or a few generations only) of a multi-objective optimiser to give you an approximate Pareto front, allowing you to see the rough shape of the front (i.e. seeing if there is a concavity in the front shape), and also then allowing you to gauge what weightings may work to give you the solutions you need. You can then do a much more detailed run with a weighted sum optimiser (not necessarily even a GA if the problem allows it) to get a high precision single optimum.

If you are, for example, designing automobile you might have many objectives. E.g. you want for automobile to be big (spacious) from inside, but also to have smaller dimensions from the outside so it is easier to park the automobile. Obviously inside size and outside size are very dependent in a way that you might can not considerably increase inside space without increasing outer size. So there you have conflicting objectives.

So if improvement in one objective lead to worsening of the other, then this two objectives are conflicted.

Hi Teodoro,

Recently, I have developed an improvement version of MOEA/D algorithm for bi-objectives, and its Matlab codes are available for users to download. If you are interested in it, you can download at the link below. For some other studies and their Matlab codes, you can refer to my researchgate.

Good luck with your research,

Vinh

Yes, it's suitable for whatever number of objectives as long as you have a way to visualise it. In case of 3 objectives you can use scatter3!

Dear Pavitdeep,

You can find all mathematical equation in book Multi-objective evolutionary optimization for product design and manufacturing, written by Lihui Wang • Amos H. C. Ng • Kalyanmoy Deb

Regards,

Robert

Hi Leonardo,

I understand that you might be busy.

Please find time for answer.

Waiting for favourable response.

Thanks in advance.

Dear Ali,

The answer provided by Dr. Lafifi is recommendable. Please follow.

following

You may recognize some objective as "more important" and some as "less important" but it doesn't mean that there's any objective method to determine this importance. Instead, it may depend on a real-world application standing behind the optimization problem. You can assign weights to particular objectives or solve the problem in many stages.

Or, if you mean by "crucial" large influence of a particular objective on the "overall performance", you may consider normalizing objectives to [0,1] range of output values and maximizing the sum of them subject to the constraints of the problem. The larger is the value of a particular objective for the optimal solution, the more "crucial" this objective can be considered.

Just a quick idea, not based on any literature. :)

There is an efficient multi-objective decision making approach known as “technique for order of preference by similarity to ideal solution” (TOPSIS) which is presented in the following paper.

http://ieeexplore.ieee.org/document/8259734/

Hello

If I understand correctly, you are worried that MOPSO won't work because it requires that all objectives have to be minimized. One simple solution is to multiply the result of the maximizing objective function with -1. This will make it a minimization objective function. When the algorithm outputs the Pareto you take the absolute value of that objective to switch it back to normal.

Dear Victor,

Judging by your question, you possibly have a situation with ordered criteria: you are looking for a solution that is optimal with respect to the first criterion and amongst these solutions, you are looking for a solution that is optimal with respect to the second criterion.

If my description is valid, then the terminology question has the following answer. The above approach to solving multicriteria problems is called an ordered criteria approach or a lexicographic approach.

Please, be careful: you should guarantee that the found solution is really optimal with respect to the second criterion amongst all solutions that are optimal with respect to the first criterion. To guarantee this, you should prove, for example, that IN EACH STEP, your ALGORITHM GENERATES ALL POSSIBLE ALTERNATIVES for the first criterion. Otherwise, the final solution may not be optimal with respect to the second criterion.

More than that, even if you generate all alternatives, you should guarantee, that the local choice in each step gives a global optimum with respect to the second criterion. If you have no such guarantees, you cannot speak about solving a multicriteria problem and your approach has no any special name.

Thank u so much Mr. Mahesh.

my regards,

Hi . We recently published new optimization algorithm entitled "WCC" in informatics in medicine unlocked journal. also we have introduced some recent optimization algorithms in this category.

please follow below address: http://www.sciencedirect.com/science/article/pii/S2352914816300077

Supposing that you can write/change the programming code of your computer program or computer program that you are using:

You can firstly test, improve your algorithm and adapt its parameters for a little example where you can find deterministically the optimal Pareto Front, for example by an enumeration model (probably exponential) if the space domain is discrete. For larger problems you can end the run in a similar way to what is normally done for a conventional optimization algorithm with a single objective function, for example when there is no improvement (punctual or in average) of the objective function value, with the necessary adaptations given that all objective functions must be considered simultaneously now.

Hello, the reason is that it is a secondary criterion that is used if there is a surplus of solutions based on the non-domination rank, starting from the first/best front, and the last front considered has more solutions than are required in the next generation. More solutions tend to belong to the same non-domination rank as optimization proceeds.

Thank you for all your assistance. I really appreciate your help in resolving the problem. I currently apply Vortex algorithm, and the speed of convergence is quite good. I will study all of your recommended algorithms and try using it in coming.

Hello Morteza,

Thank you for providing me with the sources. 

Best regards

Redouane