Applied Optimization - Science topic

Well 😕, the work looks very academic if not "school-like". Every professional C++ programmer should know all the optimization techniques given and use them practically in everyday coding.

It would be useful to at least compare how much time each technique reduces when used together, and not just compare it to the obvious worst recursive case.

The function calculating the Fibonacci sequence does not give too many possibilities to show real differences, especially in terms of memory usage. These few necessary variables easily fit in the processor registers, which in itself means that any non-recursive algorithm will be many orders of magnitude faster than a recursive one.

I would suggest using some more demanding and really useful algorithm, for example from the field of neural networks.

Assessing the global behavior of a model and obtaining equilibrium points while analyzing their stability through simulation series typically involves the following steps:

Sure, the focus study helps to find many special points of Strength in the language.

No, a Pareto front produced by an evolutionary algorithm does not necessarily include both global and local optima. The Pareto front represents the set of non-dominated solutions in multi-objective optimization problems. These solutions are not dominated by any other solution in terms of all the objective functions simultaneously.

In a multi-objective optimization problem, there can be multiple optimal solutions, known as Pareto optimal solutions, that represent trade-offs between conflicting objectives. These solutions lie on the Pareto front and are considered efficient solutions because improving one objective would require sacrificing performance in another objective.

The Pareto front typically contains a mixture of global and local optima. Global optima are solutions that provide the best performance across all objectives in the entire search space. Local optima, on the other hand, are solutions that are optimal within a specific region of the search space but may not be globally optimal.

The evolutionary algorithm aims to explore the search space and find a diverse set of Pareto optimal solutions across the entire front, which may include both global and local optima. However, the algorithm's ability to discover global optima depends on its exploration and exploitation capabilities, the problem complexity, and the specific settings and parameters of the algorithm.

It's important to note that the distribution and representation of global and local optima on the Pareto front can vary depending on the problem and algorithm used. Analyzing the Pareto front and its solutions can provide valuable insights into the trade-offs and optimal solutions available in multi-objective optimization problems.

Dear Ruta

In my opinion, the correct name is 'criteria'. Attibutes define the elements that make a criteria, and performance values are the numebrs with whch each alternative contributes to each criterion.

Since an atribute defines a criterion, they tell us about the characteristics for each one, like type of performance factors , i.e., negatives, positives, integer, decimal or their dispersion. The action maximize or minimze a criterion, defines its puropse, normally benefit, costs or equal.

In addition, each criterion is an objective to whcih each alternative must comply, and in this sense, they need a goal, for instance, in criterion ' CO2 contamination', which of course must be minimzed, we need to put a nuneric value that indicates the maximum amount of contamination allowed

In this article a highly complex optimization problem such as optimal power flow is computed through various metaheuristics with varied performance.

udemy. com/course/optimization-in-python/?couponCode= 36C6F6B228A087695AD9

Baraa Mohandes Hi Barra! How we can change the objective functions? I don't want to use the existing objective of generation cost of generators.

For example, my objective functions is

obj. min(QP CC ), max(QP CC )

s.t. sumPinji = 0

sumQinji = 0

Umini < Ui < Umaxi ,

Qminj < Qj < Qmaxj ,

∀i ∈ Bus, ∀j ∈ Gen

How can I modify the existing objective function?

Please get it.

The numbers of lines and columns of the matrix representing the image you want to obtain must be integers and their product must equal the length of the vector you want to convert. In your example the length of the vector is n=55929. You want to obtain a squared matrix with square(n) lines and columns, but in your example m=square(n)=2.364931288642442e+02, which is not an integer. If we chose a number of 3 columns and a number of 18643 lines we will obtain the following Matlab code which works.

clear variables

load('Exe42022.mat');

n = length(b);

m=sqrt(n);

c=reshape(b,18643,3);

figure,

imagesc(c),

colormap(gray),

axis off;

Please try this code.

Siddhartha Pany, then, perhaps, you may be willing to look for something like

"marginal contribution to risk ... in portfolio theory"

"... the rate of change in risk (objective function) ... with respect to a small percentage change in the size of a portfolio allocation weight",

however, this is a very specific example from portfolio theory.

Similarly, the notion of elasticity might be another example (for bias) from economics in attachment. The magnitude of the elasticity (ratio) could of any size, so a bound on elasticity should be imposed as a constraint, I guess.

Good luck.

Armin Hajighasem Kashani Non-linear data may be simply handled and processed using a neural network that is otherwise difficult in perceptron and sigmoid neurons. In neural networks, the agonizing decision boundary problem is reduced.

However, the downsides include the loss of neighborhood knowledge, the addition of more parameters to optimize, and the lack of translation invariance.

Dear Murana Awad,

Application of PSO-BP Neural Network in GPS Height Fitting

these articles can be useful:

Dear R. M. Fouad,

Also i suggest you to see links on topic.

Best regards

MOSFP: https://github.com/sh7adeh1990/MOSFP

Hi Masar Alrabeeah,

Did you code the epsilon constraint method using OPL? May I ask how did you code this? I tried but could not get the right results.

Thanks a lot.

Dear Saeed Vasebi

These links might be an asset, have a look:

1. https://www.ibm.com/support/pages/themes/ibm_carbon/carbon-resources/additional_carbon.bundle.js.gz?qyuy97

2. https://www.ibm.com/support/pages/themes/ibm_carbon/carbon-resources/main_carbon.bundle.js.gz?qyuy97

3. https://mdpi-res.com/d_attachment/ai/ai-02-00010/article_deploy/ai-02-00010.pdf

4. https://www.ibm.com/support/pages/cplex-strategies-finding-good-feasible-solutions-quickly-when-solving-mips

5. https://www.ibm.com/docs/hu/icos/12.7.1.0?topic=cplex-list-parameters

Kind Regards

In general, you should combine two methods: imaginary frequency analysis of transition state (TS) structure and the calculation of IRC, respectively. On the other hand, you could check doubly by the calculation of QST2 and QST3. Best regards.

Dear Sajad Safari Farrokhad,

I suggest you to see links and attched files on topic.

The last few decades have seen the introduction of a large number of "novel" metaheuristics inspired by different natural and social phenomena. While metaphors have been useful inspirations, I believe this development has taken the field a step backwards, rather than forwards. When the metaphors are stripped away, are these algorithms different in their behaviour? Instead of more new methods, we need more critical evaluation of established methods to reveal their underlying mechanics.

You must to begin studying discrete optimization methods. in general. After that yout could to study models anf methids for choicing options. I am the author of the Selection of Proposals and of the Integration of Variables methods devoted to the options selection, that you can find in my researchgte profile, including applications.

You can solve this problem as optimization problem using Genetic algorithm, Ant colony algorithm, Simulated Annealing algorithm, PSO, etc.

Z. Nedelkov\'a, C.\ Cromvik, P.\ Lindroth, M.\ Patriksson, \and A.-B.\ Str\"omberg}, {\em A splitting algorithm for simulation-based optimization problems with categorical variables}, {\sf Engineering Optimization}, vol.~51 (2019), pp.~815--831.

It might help!

The below paper might be helpful:

Dear Charlie Vanaret,

maxtime Maximum CPU seconds allowed (default 500); -1 means no limit.

Please take a look at following links :

Best regards

Dear Dr. Lafifi, Mohamed-Mourad Lafifi

Your help is appreciated!

Best wishes,

Hussein

Dear Sajad Saraygord Afshari ,

Optimization algorithm in neural network

The process of minimizing (or maximizing) any mathematical expression is calledoptimization. Optimizers are algorithms or methods used to change the attributes of theneural network such as weights and learning rate to reduce the losses. Optimizers are used to solve optimization problems by minimizing the function.

Regards,

Shafagat

It might interest you that there is a possibility to use complexity measures to assess the state of the observed complex system and make decision about arrhythmias.

An example how to do it can be bound in our paper on prediction of TdP arrhythmias from ECG recordings. Everything is explained in the paper in detail. The final version will contain rewritten entropy section and substantially improved methods, intro, etc.

Back to your question. Complexity measures when applied wisely enable us to substantially reduce the complexity of complex systems under the observation. This includes biosignals along with ECGs, EEGs, etc.

Hopefully this will enable you to orientate yourself in this exciting, yet quite complicated area of research.

The change propagation models may give a great idea

Are you working with a specific dataset?

Bat algorithm (BA) is a bio-inspired algorithm developed by Xin-She Yang in 2010 and BA has been found to be very efficient .

The method for solving this problem depends on the properties of functions F, f, g (convex, concave, other properties).

If properties are not known - the only method is looking through all values of variables.

Dear Nils Wagner,

I suggest you to see links and attached files on topic.

Best regards

N. K. T. El-Omari, "Sea Lion Optimization Algorithm for Solving the Maximum Flow Problem", International Journal of Computer Science and Network Security (IJCSNS), e-ISSN: 1738-7906, DOI: 10.22937/IJCSNS.2020.20.08.5, 20(8):30-68, 2020.

I have many advanced books and can send you by e-mail, if you need.

Some penalty methods are exact, in the sense that a finite penalty is enough to identify an optimum. Typically the penalty function then needs to be non-differentiable, which might be a bummer. With differentiable penalty functions, such as the augmented Lagrangian, you typically need the penalty to grow indefinitely, but there are those that can be exact without a growing penalty parameter. It however has a cost, in the sense that some constraint qualification holds - which is hard to check.

Here is a paper about just that:

Exact augmented Lagrangian functions for nonlinear semidefinite programming, by Ellen H. Fukuda & Bruno F. Lourenço

Computational Optimization and Applications volume 71, pages 457–482 (2018)

Hope it helps! :-)

It is an interesting project. The first step should be collecting data. Open more or less systematically taps and valves and record angles, numbers of rotations, pressures, flow. With that data you can train a neural network. In the end the neural network is a model of your pipe system. You can then do simulations by varying inputs and try to achieve the desired outputs.

In hydrology, fluorescein sodium is used to find out the actual waterways. I would be surprised if there weren't any microbiology algorithms to infer pathways. Network optimization is often done with Ford Fulkerson.

Regards,

Joachim

This is a challenging problem, specially for manufacturing companies, needless to say this problem is undergoing lots of research and practically there are no viable examples of companies that have achieved success, best cases have hit a 90% mark (but the metrics of such proclaimed achievements have been highly debatable). As inventory management is being driven by close to real time demand and supply data(analytics) plugged to AI and Machine Learning tools, the potential to reach close to 99% efficiency in CLSCM might become a reality -- but how do we identify and embed external disruptions like COVID19 into this model? and to what extent will these external disruptions impact CLSCM based inventory dynamics?

Rahul Dewani Thank you very much for your contribution

The color reduction is very high. I am going from 16M colors to 12 basic colors. It is expected the information lost to be very high.

Are you using colors or you are plotting black and white?

And if the model is not stochastic?

Check out Leon Lasdon's book on large-scale optimisation, called

Optimization theory for large systems

Dear Mohammad Reza Salehizadeh

Unfortunately, the attached article

" [a] Bertsekas, Dimitri P., et al. Dynamic programming and optimal control. Vol. 1. No. 2. Belmont, MA: Athena scientific, 1995."

is full of typing errors.

General talking, we consider the linearization of the nonlinear system. Next, we study the stability of the equilibrium state of the new linear system, which indicates the nature of the stability of the nonlinear system in some neighborhoods.

The signs of the real parts of the eigenvalues of the Jacobian matrix decide which approach we should follow. We have the direct and indirect Lyapunov methods to study the stability based on the eigenvalues.

Best regards

Hi Morteza,

I am a new researcher in this field and I am working on similar problems. I hope you will find following paper useful:

Cheers,

Zaid

On trial and error principle I have decided to use fitness function like this:

What can be done is to create a representation of the objective space, by determining several (very many!!) points on the function surface, and from that create an explicit function that interpolates the points you have found. That function can hopefully be optimised by global optimisation software, and by evaluating new points in regions that may be interesting, you can refine the search towards the global optimum - at least if the function is not too weird. :-)

You need of course be aware that it may take some time, as the interpolated function needs to be re-optimized many times, perhaps, at several vectors.

Aicha Ghedamsi , you need to call myfitness with X as the input.

Better to follow: https://web.stanford.edu/~boyd/cvxbook/ may be more helpful.

you can use scipy.optimize.minimize function with method='SLSQP'.

To obtain an exact mathematical solution is necessary and sufficient.

1. Firstly, it is necessary to establish what quality of movement is being studied: translational, parabolic, elliptic, hyperbolic, stochastic.

2. Secondly - determine: how much data you want to use:

- so, for a stochastic movement one point is enough;

- for translational motion, two points are sufficient;

- for parabolic, elliptical and hyperbolic movements, three points are sufficient.

3. Thirdly, the generalized finite difference method is used to equation the model of established motion. The method gives a function of the dependence of the amplitude on a given interval from frequency. Each new frequency is a new derivative. Choose the right one for the direct task. Select the amplitude and find the frequency - for the inverse problem.

To obtain physical information from an exact mathematical solution, it is necessary and sufficient:

1. Choose the general quality of movement - stochastic.

2. Use the function of determinate probability.

3. Find the desired derivative and its value. You solve the direct or inverse problem by initially determining the total, kinetic and potential energies of the process at the measurement point.

This question is similar to another one that I have seen. My response to that one was basically this:

1. I don't know any mathematicians who are prejudiced against heuristics per se. Many of them (myself included) use them regularly.

2. I do know a lot of mathematicians who are fed up with people claiming to invent dozens of "new" meta-heuristics every year (like harmony search or the bat algorithm), when really they are just old ideas expressed in fancy new words. Actually, many people in the heuristics community are unhappy with it as well.

3. I also know people working in combinatorial and/or global optimisation who are fed up with people saying "problem X is NP-hard, and therefore one must use a heuristic". This shows a breathtaking ignorance of the (vast) literature on exact methods (and approximation algorithms) for NP-hard problems.

(By the way, I agree with Michael's comment about "matheuristics" being a very interesting research direction.)

Because some countries have a management system for their resources, and others do not have a management system

Please see the following links

Best Regards H. Naderpour

You can use active divider composed of emitter follower or source follower transistors connected in parallel. Every emitter follower has matching function to transform 50 ohm output resistance to 50 x16=800 ohm.

This will be a very wide band divider consisting of 16 emitter followers connected in parallel. The input will be capacitively coupled.

Best wishes

You can also use Hossein Karimi formulation although i wouldnt prefer a MIP as this would grow in O(e^N) where N is the row dimension of your matrix A. Here is CPLEX OPL formulation

Thank you Katarzyna

I believe that we have a misunderstanding.

I agree with your definition of variables, but I DON’T CALL as ‘criterion’ an objective function. NOT in LP, although YES in SIMUS, because in this method criteria and objective functions are interchangeable.

But in general, in MCDM, my definition of criterion or constraint is exactly the same as yours, as well as your definition of performance values. However, if it is true that criteria ‘constrain’ the alternatives, I prefer to use the term ‘constrain’ for the values which are limits or thresholds of criteria, that is, the RHSs, because they constrain the range of validity of criteria. In reality, criteria ‘force’ the alternatives to respect some conditions, which in turn may be high or low, and bounded by the RHSs.

You mentioned before that you had 2500 variables or alternatives, and you say that there are 3 production units; does it mean that each production unit may have many alternatives?

Can a particular alternative, be replicated in any number of production units? For instance, alternative 789 can be in units A and C, or even in A, B and C, while other alternatives may only be in one of them?

When you say that other variables are placed, are you referring to artificial and slack variables? If this is so, it is strange, since the solver adds them automatically, or maybe I did not understand your statement.

I assume that you want to know how production of unit B changes when you modify the RHS of some constants, for instance # 1.

What do you mean by ‘I display’ constraint #1 up, constraint 1, and constrain # 1 down? I understand that you want to say is that you can put low and high limits for constraint # 1. Is this correct?

I have two questions for this:

1. Why you consider a constrain 1, when if you have low and high limits defined by ≥ and ≤ operators respectively, constraint 1 is included in the range? In my opinion that arrangement may cause you to get an infeasible solution, because if the model select a value between the two ends, how does it manage to also comply with constraint 1, which I imagine has the ‘=’ operator?

2. How do you know that constraint 1 or any other is the one you have to work with? It could very well be that said constraint does not have any influence in the alternatives selected. That is, you have to work with the criteria that are responsible for the selection, maybe one or several. Once determined, you will have the certainty that these will change the solution found. If you work with constraint that are not relevant, you can increment and decrement by changing their RHSs, and you will see that there will be no change in the selection.

This information, as you know, is in the dual, or you can see directly in the Solver screen for sensitivity analysis (SA). Of course, you know all of this, but my comment derives because you don’t explain why criterion 1 is selected for SA.

No, I was not referring to the objective function Z coefficients. I am referring to changes in the RHSs.

To your question, my answer is that yes, you can. Each time you modify the RHS of a binding constraint and use the Simplex, you can immediately see the new values of the alternatives.

It is extremely useful to work at the same time with the dual, and once you get the new values for the alternatives go to ‘sensitivity’ (in Solver you have to run the Solver twice for each RHS change, the first gives you the new alternative values, and then you have to run it again in order to get the sensitivity analysis screen), that gives a lot of information, such as the shadow prices, the relevant criteria, the validity range for each shadow price, as well as the reduced costs.

For my software, don’t worry, you can have it any time you want.

I understand that there is a mistake when you say that the slope is determined by the reduced cost value.

The slope is given by the shadow prices. The reduced cost tells you how much you have to modify a coefficient in the objective function in order that the corresponding alternative enters in the solution. Observe that the reduced cost for selected variables is zero.

Outside the range nothing happens, because the criteria that you were using is no longer binding, and the alternative keeps it last value.

In your next paragraph, you say: ‘But if I change RHS to some value outside of the range, I can see that indeed, basis have been changed as well, but the change is related to, for example, a variable indicating the production in unit A jumped from 50 to 0’.

Yes, you are right, if the base changes, there will be another variable instead of A, and this one will be zero.

The jump from 50 to 0, means that when the upper/lower limit of the range is exceeded, that variable no longer belongs to the solution, and since Z is equal to de sum of the products between the aijs and the solution found, and A is no longer in the solution, its value is zero, but the variable keeps it last value..

I am attaching a worksheet for an example that I did years ago, with two alternatives A1 and A2, subject two five criteria and which objective was to maximize production.

The first optimal result indicates that A3=0.16 and A6 = 1.22. This result is showing a Z=52,000 and that the corresponding criteria are C2 and C3.

Observe that:

· C2 has as shadow price (λ2) = 0.07 and C3 a shadow price of (λ3) = 17.95.

· Solver also shows that maximum increment for λ3 is 429, while the maximum increment for λ2 is 48.

Consequently, the upper limit for C3 = 335 (Its RHS) + 429 = 764 and for C2= 559 (Its RHS) + 48 =607.

· If we consider only C3 and we increase its RHS, as shown in the Excel spreadsheet (we can increase in any appropriate amount even if not equal). Observe how the A3 value progressively rises up to 2.74, which corresponds to the upper limit of C3, that is 764.

· Both λ3 and λ2 keep constant along the whole procedure

· Simultaneously the value or score of A6 progressively decreases from 1.22 up to 0, for RHS3 = 764.

· The original upper limit of RHS3 decreases from 420 to 0, for RHS3 = 764.

Correspondingly the original upper limit of RHS2 increases from 48 to 825, for RHS3= 764.

· If the value of 764 is surpassed, say we put 765, alternative A3 keeps its value, since C3 does not have influence on it, however A6 = 0, meaning that this alternative it is no longer a solution, however, notice that λ3= 0 while λ2= 24.61, after the 764 limit is reached.

In fact, we should change RHS3 and RHS2 simultaneously, each one with its own increments/or decrements, and then the performance curve will be the result of both acting at the same time, as in real-world scenarios.

The graphic shows Z performance curve with the λ3. It appears as a broken straight line because I was using different amounts of increments for RHS3, since

I believe that this elemental example shows you how the range is diminishing in the selected alternative

Now, if you want to keep the original production of B unchanged, why don’t you express it as a new constraint, using the binary format, that is in this criterion all alternatives are zero, except B which is 1.

If you have a value in mind for B, say for instance 820,you put it as the RHS, and indicate with the ≥, ≤, or = operators if you want to get a result for B larger than 820, or lower than 820, or equal to 820, respectively.

However, I would stay away from the equality, because it imposes very hard restrictions to the problem.

I sincerely hope that my comments help you, and indeed, I am very interested if you can keep me informed me if it works

we can chose linear or dual dependent to conditional constrants

See the book

https://www.amazon.com/Multi-Objective-Optimization-Using-Evolutionary-Algorithms/dp/0470743611

Here, there are many usefull examples

The discontinuity looks like it arises due to a concave section of the objective front, leading to regions of the objective front that are dominated. Concavities in the objective front are quite common on many real engineering problems, leading to discontinuities in the Pareto front. The other possible reason is that the objective front forms two distinct feasible regions with a gap in between; from the way the top section is curving just before the break, two separated regions could be the cause. For clarity: the objective front is the outer boundary of the objective space region that contains feasible solutions, the Pareto front is the subset of the objective front with non-dominated solutions. There has been work over the years trying to develop optimisers that locate the objective front so that the reason for Pareto discontinuities can be explored more easily, but designing algorithms to find the objective front is really not easy (especially for problems with more than two objectives). Evan

you can review the sample of another one to know the instructions

good luck

Hi Farzad Torabi I think you need at first an idea of your optimization problem. What do you want to optimize (only mass or more e.g. multiple objectives?), what are your variables and how many (thickness, topology, ..., continuous or discrete) and what are the constraints (number and linear or nonlinear) you want to consider.Than roughly select an optimization method (gradient based, evolutionary,...). Than find a optimization toolbox providing a method you want (nlopt, Pyopt, pyoptsparse, scipy, midaco ...).In general you have to provide a function returning your objective and the constraints (e.g. max stresses, ...) based on a given variable vector. Sometimes objective and constraints are divided in two functions, depending on the toolbox.Now, in the function providing the objective and constraints you have to set up the FE run with the new variables (e.g. change the input file) and then solve the system running FEM. After finished FEM, you have to read the results back into Python (e.g. reading an ASCII-File) and provide the desired data as return values of your function.Starting FEM from Python is possible via "subprocess" module which is a build in module. Via e.g. subprocess.run(["path/to/executable", "input_file.xyz"]) you run your executable with a given input_file depending on your FEM solver.I hope this answers your question. More details depending on your concrete optimization problem.Kind regards Sascha

All are approaches that exploits the computational intelligence paradigm. Machine learning is refered to data analitics. Evolutionary computation deal with optimization problems.

Differential evolution is a good method. You can try with this method

Greetings,

The easiest way is to calculate the hypervolume of each solution and the one with the highest value is usually in the knee. But first, you have to flip the first goal, so that all goals are minimized. Just multiply the result of the first target by -1. Before you calculate the hypervolume, do not forget to normalize all the goals.

As a reference, you can check out "A Knee Point Driven Evolutionary Algorithm for Many-Objective Optimization" and "Finding Knees in Multi-Object Optimization".

Regards,

Miha