Page 1

Statistical Analysis of Parameter Setting in

Real-Coded Evolutionary Algorithms

Maria I. Garc´ ıa Arenas, Pedro´Angel Castillo Valdivieso,

Antonio M. Mora Garc´ ıa, Juan J. Merelo Guerv´ os,

Juan L. Jim´ enez Laredo, and Pablo Garc´ ıa-S´ anchez

Department of Architecture and Computer Technology

University of Granada, Spain

mgarenas@atc.ugr.es

Abstract. When evolutionary algorithm (EA) applications are being

developed it is very important to know which parameters have the great-

est influence on the behavior and performance of the algorithm. This pa-

per proposes using the ANOVA (ANalysis Of the VAriance) method to

carry out an exhaustive analysis of an EA method and the different pa-

rameters it requires, such as those related to the number of generations,

population size, operators application and selection type. When under-

taking a detailed statistical analysis of the influence of each parameter,

the designer should pay attention mostly to the parameter presenting

values that are statistically most significant. Following this idea, the sig-

nificance and relative importance of the parameters with respect to the

obtained results, as well as suitable values for each of these, were obtained

using ANOVA on four well known function optimization problems.

1Introduction

When using search heuristics such as evolutionary algorithms (EAs), simulated

annealing and local search algorithms, components such as genetic operators,

selection and replacement mechanisms, and the initial population, must first be

chosen. The parameters used to apply some of these elements determine the way

they operate and influence the results obtained. Obtaining suitable values for

them is an expensive, time-consuming and laborious task.

One of the most common ways of setting these parameters is by hand, af-

ter intensive experimentation with different values [21]. As Eiben states, the

straightforward approach is to generate and test [8,28]. An alternative is to use

a meta-algorithm to optimise the parameters [15], that is, to run a higher level

algorithm that searches for an optimal and general set of parameters to solve a

wide range of optimisation problems.

However, as some authors remark, solving specific problems requires specific

parameter value sets [3,18,10] and, as Harik [17] claims, nobody knows the “op-

timal” parameter settings for an arbitrary real-world problem. Therefore, estab-

lishing the optimal set of parameters for a sufficiently general case is a difficult

problem.

R. Schaefer et al. (Eds.): PPSN XI, Part II, LNCS 6239, pp. 452–461, 2010.

c ? Springer-Verlag Berlin Heidelberg 2010

Page 2

Statistical Analysis of Parameter Setting in Real-Coded EAs453

Current best practices are based on intensive test, ad-hoc choices and con-

ventions [29,8,28,9], that is why new practices, based on solid tuning methods

(i.e. robust mathematical methods), are needed. Such a methodology is what we

intend to present in this paper.

Genetic algorithm users adjust their main design parameters (crossover prob-

ability, mutation probability, population size, number of generations, selection

rate) by hand [6,19]. The decision as to which values are best is usually made

in terms of the most common values or experimental formulae given in the bib-

liography, or by trial and error [15,22].

However, other researchers have proposed determining a good set of evolu-

tionary algorithm parameters by analogy, undertaking a theoretical analysis

[2,13,14,16,25,31]. Establishing parameters by analogy means using suitable sets

of parameters to solve similar problems. However they do not explain how to

measure the similarity between problems. Also, a clear protocol has not been

proposed for situations when the similarity between problems implies that the

most suitable sets of parameters are also similar [18,10]. Weyland has described

a theoretical analysis of both an evolutionary algorithm [20] and simulated an-

nealing algorithm [33] to search for the optimal parameter setting to solve the

longest common subsequence problem. However, Weyland does not carry out

this approach to practice.

Some authors have proposed practical approaches that eliminate the need for

a parameter search in genetic algorithms [17]. In these works, a set of parameters

is found, but instead of finding them by means of intense experimentation, the

parameter settings are backed up with theoretical work - meaning that these

settings are robust.

New approaches to the problem of establishing parameter values [23] have

been proposed. Several proposals are based on setting parameter values on-line

(during the run) instead of testing and comparing different values before the

run (parameter tuning). In this sense, some authors propose self-adaptation of

parameters (coding those parameters in the individual’s genome); others pro-

pose non-static parameter settings techniques (controlled by feedback from the

search and optimization process) [9,28,30]. However, control strategies also have

parameters and there are indicators that good tuning works better than control

[8].

Finally, authors proposed in [4] using the ANOVA (ANalysis Of the VAriance)

[12] statistical method to analyze the main parameters involved in the design of

a neuro-genetic algorithm.

It is very important to know which parameter values involved in the design

of an optimization method have the greatest influence on its behaviour and per-

formance and to obtain accurate values for those parameters. In any case, after

performing a detailed statistical analysis of the influence of each parameter, the

designer should pay attention mostly to the parameter providing the values that

are statistically most significant. In this paper, we propose using the ANOVA

statistical method as a powerful tool to analyze a real-coded EA to solve function

approximation problems.

Page 3

454M.I.G. Arenas et al.

The ANOVA method allows us to determine whether a change in the results

(responses) is due to a change in a parameter (variable or factor) or due to

a random effect. Thus it is possible to determine the variables that have the

greatest effect on the method that is being evaluated.

The theory and methodology of ANOVA was mainly developed by R.A. Fisher

during the 1920s [12]. ANOVA examines the effects of one, two or more quan-

titative or qualitative variables (called factors) on one quantitative response.

ANOVA is useful in a range of disciplines when it is suspected that one or more

factors might affect a response. ANOVA is essentially a method used to anal-

yse the variance to which a response is subjected, dividing it into the various

components corresponding to the sources of variation, which can be identified.

With ANOVA, we test a null hypothesis that all the population means are

equal against the alternative hypothesis that there is at least one mean that is

not equal to the others. We find the sample mean and variance for each level

(value) of the main factor. Using these values, we obtain a significance value

(Sig. Level). If this level is lower than 0.05, then the influence of the factor is

statistically significant at the confidence level of 95%.

After applying ANOVA (to determine if means are different), another tests

must be used to determine which are different; that will give information about

which parameter values are more accurate. In this sense, either TukeyHDS

(Tukey’s Honestly Significant Difference) [7] or Bonferroni [24] tests can be used.

In this paper, ANOVA will be used to determine the most important param-

eters of an EA (in terms of their influence on the results), and to establish the

most suitable values for such parameters (thus obtaining an optimal operation).

The rest of this paper is structured as follows: Section 2 describes the EA and

the parameters we propose to be evaluated. This section contains an exhaustive

analysis of the method, showing how the parameters are determined. Section 3

details the experimental setup and the statistical study using ANOVA. Obtained

results are analysed in order to establish the most suitable values. Finally, a brief

conclusion and future work is presented in section 4.

2The EA Method and the Experimental Setup

The purpose of this study is to analyze the dynamics of a typical EA [3,11],

to determine which parameters influence the obtained fitness and to find an

adequate value for the following parameters:

– Generations (G): number of generations.

– Population Size (P): number of individuals in the population.

– Selector (S): Selection operator to generate the offspring. In this paper a

roulette wheel selector, a random selector and a selector based on always

taking the best individual in the popultation are proposed.

– Operators Combination (O): This parameter refers to the percentage of

offspring generated using either an uniform mutator or a BLX-α crossover

operator.

Page 4

Statistical Analysis of Parameter Setting in Real-Coded EAs455

Table1showsthedifferentlevelsusedtoevaluatetheseparametersusingANOVA.

The response variable used to perform the statistical analysis is the fitness in the

last generation. The changes in the response variable are produced when a new

combination of parameters is considered.

Table 1. Parameters (factors) and the abbreviation used as reference later that deter-

mine the EA behaviour and values used to apply ANOVA

Generations (G) Population Operators

Size (P)

160100

320 200

640 400

1280800

25601600

51203200

Selector (S)

Combination (O)

- Crossover only (C)

- Mutation only (M)

- Roulette Wheel

- Random

- Crossover (80%) and

Mut. (20%) (C8M2)- Always the best

individual

- Crossover (90%) and

Mut. (10%) (C9M1)

Thus, 12960 runs were carried out for each problem (30 times * 6 levels for

G * 6 levels for P * 4 levels for O * 3 levels for S, that represent the possible

combinations) to obtain the fitness for each combination.

The application of ANOVA consisted in running an EA using these parameter

combinations to obtain the best fitness. Then R1was used to obtain the ANOVA

table. In this paper a simplified table is shown, including for each factor, the sum

of squares (Sum Sq), the value of the statistical F (F value) and its significance

level (Sig. Level). As previously stated, if the latter is smaller than 0.05, then

the factor effect is statistically significant at a 95% confidence level (what means

that some means are different for these parameter values).

In order to evaluate the EA and its parameters, four function approximation

problems are used:

– The Griewangk function [1] is a continuous multimodal function with a high

number of local optima. Its global optimum is located at (0,...,0) [5]. This

problem has been used with vectors of 100 real numbers in the interval

[−512,512].

– The Rastrigin function [32] is a multimodal real function optimisation prob-

lem, whose global optimum is located at point 0 and whose minimum value

is 0. This problem has been addressed with vectors of 100 real numbers in

the interval [−512,512].

– The NormalizedSchwefel function [26] is a multimodal separablerealfunction

optimisation problem, whose global optimum is located at point (x0,...,xd)

withxi= 420.96,andwhoseminimumvalueis(y0,...,yd)withyi= −418.9828.

Vectors of 100 real numbers in the interval [−512,512] were used.

– The Shekel function [27] is a multimodal real function optimisation problem,

whose optimum for dimension 5 is located at point x1 = 8.02, x2 = 9.15,

1http://www.r-project.org

Page 5

456M.I.G. Arenas et al.

x3= 5.11, x4= 7.62, x5= 4.56 and whose minimum value is −10.4056. This

problem has been addressed with vectors of 5 real numbers in the interval

[−10,10].

In all cases, the fitness of an individual is calculated as the distance to the

optimum for that function (the optimum is known).

3 Statistical Study and Results Obtained

In this section, the ANOVA statistical tool is applied to determine whether the

influence on parameter values (factors) is significant in the obtained fitness, (to

obtain an optimal operation). The set of tests carried out to apply the ANOVA

method and thus to determine the most suitable parameter values, is described

in detail. In all cases, the goal is to obtain the smallest fitness for the optimised

function.

Table 2. ANOVA tables for the fitness (response) with the EA parameters as factors.

Those parameters with a significance level over 95% are in bold. Although the full

ANOVA table includes combinations of 2, 3 and 4 parameters, only results related to

single parameters are shown.

Param. Sum Sq F value Pr(> F)

G71278

O4540135 154.4671 <2e−16

P29353

S 2781525 94.6346 <2e−16

2.4251 0.1202

0.99870.3182

Param. Sum Sq F value Pr(> F)

G1e+12

O7.4e+13154.4393 <2e−16

P5.3e+11

S4.6e+1394.9568 <2e−16

2.1689 0.1416

1.10180.2945

Griewangk Rastrigin

Param. Sum Sq F value Pr(> F)

G6

O 52393 135.3265 <2.2e−16

P38579.9611

S 36652 94.6682 <2.2e−16

0.01620.898701

0.002

Param. Sum Sq F value Pr(> F)

G 9.79

O18.73

P 79.09 18.7111 1.9e−05

S118.79 28.1017 1.9e−07

2.3170 0.12873

4.43130.036

Normalized Schwefel Shekel

We will determine if the influence on a parameter value is significant in the

value of the approximation function (fitness).

Table 2 shows the result of applying ANOVA on proposed approximation

function problems. Parameters with a signification level over 95% are highlighted

in boldface. The ANOVA analysis shows that O and S parameters influence the

obtained fitness, which indicates that changes in these parameters influence the

results significantly. However, this influence is not as important for the rest of

the parameters in all the cases (problems).

Page 6

Statistical Analysis of Parameter Setting in Real-Coded EAs457

Table 3. Griewangk function: obtained error for the different parameter levels. This

table shows the effect each level has on the fitness.

Param.

G

Means

160 320 640 1280 2560 5120

129.5 95.3 81.6 74.3 74.9 70.2

CM C8M2

289.0 57.8 1.7

100 200 400 800 1600 3200

104.5 93.1 91.1 83.1 79.1 75.0

rouletterandom

21.723.0

OC9M1

2.1

P

S best

218.3

Table 4. Rastrigin function: obtained error for the different parameter levels. This

table shows the effect each level has on the fitness.

Param.

G

Means

640160 320128025605120

525797 387416 321216 296155 288133 295020

CMC8M2

1167593 2287965560

100200400

419618 382480 359795 337218 316751 297876

roulette random

8527290843

OC9M1

7211

P80016003200

Sbest

880754

In Normalized Schwefel and Shekel, the P parameter is significant too. This

fact shows how for each problem different set of parameters can influence results

in a different manner.

Once the parameters with greater influence on the results are determined,

accurate parameter values should be established in order to obtain an optimal

operation. To do so, tables of means are calculated to show the effect each level

has on the approximation error.

The obtained error for the Griewangk, Rastrigin, Normalized Schwefel and

Shekel functions and the different parameter levels are shown in Tables 3, 4, 5

and 6.

Lets examine each one of the parameters in turn:

– Focusing attention to the operator combinations (O), using either only mu-

tation or only crossover leads to worse fitness results, which was only to be

expected. According to the tables, using a low crossover and high mutation

probabilities to generate offspring in each generation is the more accurate.

However, as the mutation role is to generate diversity, reducing too much

the mutation probability leads to premature convergence of the population.

Page 7

458M.I.G. Arenas et al.

Table 5. Normalized Schwefel function: obtained error for the different parameter

levels. This table shows the effect each level has on the fitness.

Param.

G

Means

160320640 128025605120

8.146208 7.848331 8.151095 9.062385 8.641955 7.645236

CMC8M2

3.3e+01

2.2e−01

8.1e−04

100200400

14.911.0 8.0

roulette random

0.80.7

O C9M1

6.6e−03

P 800

6.8

1600

5.1

3200

3.8

S best

23.3

Table 6. Shekel function: obtained error for the different parameter levels. This table

shows the effect each level has on the fitness.

Param.

G

Means

160 320 640 1280 2560 5120

7.32 6.90 6.78 6.70 6.64 6.63

CMC8M2

8.10 4.14 7.49

100 200 400 800 1600 3200

7.57 7.30 6.94 6.65 6.34 6.19

roulette random

6.37 6.48

O C9M1

7.60

P

S best

7.65

On the other hand, applying too much mutation affects exploration, leading

to random search.

– Both P and G have not been reported as significant according to the ANOVA

table in all problems. However, as it can be observed, using the higher values

yields better fitness values, although those values lead to a higher number of

evaluations and time needed to run the algorithm. Logically, the greater the

number of generations or population size, the more possibilities there are of

achieving a good individual from the current population, as there exists a

greater variety of individuals.

– Taking into account the S parameter, using the roulette wheel selector yields

much better results than using a random selector or always taking the best

individual. The reason for this is that the roulette selector produces the

greatest diversification in the EA solutions. The third selector is the most

elitist; obtained results using this one selector (taking the best individual)

are much worse, which indicates that too much selective presion is not appro-

priate (the population must be diverse, otherwise a premature convergence

of the algorithm might occur).

Page 8

Statistical Analysis of Parameter Setting in Real-Coded EAs459

These conclusions have been confirmed by applying the Bonferroni statistical

test [24]. As can be seen, these results are in agreement with those available in

bibliography. In any case, in this paper we have verified the adequacy of these

parameter values through a rigorous statistical study.

4Conclusions and Work in Progress

A statistical study of the different parameters involved in the design of an EA

has been carried out by using ANOVA, which consists of a set of statistical

techniques that analyze and compare experiments by describing the interactions

and interrelations between the variables (factors) of the system. The motivation

of the present statistical study lies in the great variety of alternatives that a

designer has to take into account when designing an EA.

Proposed methodology has been applied to four well known function approxi-

mation problems, widely used by practitioners, having determined which param-

eters have a higher influence on obtained results (a change on those parameters

will affect the fitness). Likewise, the more accurate value has been determined

for those parameters in order to obtain an optimal operation.

Accurate results are obtained using the higher value tested for population size

and number of generations, although this increases the number of evaluations

and time needed to run the algorithm. Obviously, as the number of generations

or individuals in the population increases, there is a greater probability that the

fitness of the best individual will be higher.

Paying attention to the selector operator, a roulette wheel selector yields much

better results than using a random selector or always taking the best individual,

due to the fact that the roulette selector produces the greatest diversification in

the EA solutions.

As far as the genetic operator application rates are concerned, reducing too

much the mutation probability leads to premature convergenceof the population,

while applying too much mutation is like a random search. According to the

tables, using a low mutation and high crossover probabilities to generate offspring

in each generation is the more accurate.

This methodology based on ANOVA and Bonferroni statistical tests could be

helpful for practitioners in analyzing and adjusting parameters of any optimisa-

tion method.

Our work in progress includes the analysis of modified EA considering other

selection schemes, new genetic operators and other meta-heuristics. As future

work, the implementation of a parameter control method would be of interest,

as proposed by Eiben et al. [9]. In this case, ANOVA could be used to anal-

yse not only the optimisation method parameters but also the control strategy

parameters.

Acknowledgments

This work has been supported in part by the Spanish MICYT project NoHNES

(Spanish Ministerio de Educacion y Ciencia - TIN2007-68083), the Junta de

Page 9

460 M.I.G. Arenas et al.

Andaluc´ ıa P06-TIC-02025,P07-TIC-03044,P08-TIC03928projects and the Ja´ en

University UJA-08-16-30 project.

References

1. Griewank, A.O.: Generalized descent for global optimization. Journal of Optimiza-

tion Theory and Applications 34(1), 11–39 (1981)

2. B¨ ack, T.: Optimal mutation rates in genetic search. In: Forrest, S. (ed.) Proceed-

ings of the 5th International Conference on Genetic Algorithms, pp. 2–8. Morgan

Kaufmann, San Francisco (1993)

3. B¨ ack, T., Fogel, D., Michalewicz, Z.: Handbook of Evolutionary Computation.

Institute of Physics Publishing Ltd. Bristol and Oxford University Press, NY (1997)

4. Castillo, P.A., Merelo, J.J., Romero, G., Prieto, A., Rojas, I.: Statistical Analysis

of the Parameters of a Neuro-Genetic Algorithm. IEEE Trans. on Neural Net-

works 13(6), 1374–1394 (2002) ISSN:1045-9227

5. Cho, H., Olivera, F., Guikema, S.D.: A derivation of the number of minima of the

griewank function. Applied Mathematics and Computation 204(2), 694–701 (2008)

6. Davis, L.: Handbook of genetic algorithms. Van Nostrpand Reinhold, NY (1991)

7. Dickinson, P., Chow, B.: Some properties of the Tukey test to Duckworth’s spec-

ification by Peter Dickinson, Bryant Chow. Office of Institutional Research, Uni-

versity of Southwestern Louisiana, Lafayette, Louisiana, USA (1971)

8. Eiben, A.E.: Principled Approaches to tuning EA parameters. In: Tutorials - IEEE

Congress on Evolutionary Computation (CEC 2009) (2009), http://www.few.vu.

nl/~gusz/papers/eiben-cec-2009-tutorial-corrected.pdf

9. Eiben, A.E., Michalewicz, Z., Schoenauer, M., Smith, J.E.: Parameter control in

evolutionary algorithms. In: Parameter Setting in Evolutionary Algorithms. Stud-

ies in Computational Intelligence, vol. 54, pp. 19–46. Springer, Heidelberg (2007)

ISBN 978-3-540-69431-1, ISSN 1860-949X

10. Eiben, A.E., Hinterding, R., Michalewicz, Z.: Parameter Control in Evolution-

ary Algorithms. IEEE Transactions on Evolutionary Computation 3(2), 124–141

(1999), doi:10.1109/4235.771166, ISSN: 1089-778X

11. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Hei-

delberg (2003) ISBN 3-540-40184-9

12. Fisher, R.A.: Theory of Statistical Estimation. In: Proceedings of the Cambridge

Philosophical Society, vol. 22, pp. 700–725 (1925)

13. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learn-

ing. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)

14. Goldberg, D.E., Deb, K., Theirens, D.: Toward a better understanding of mixing in

genetic algorithms. In: Belew, R.K., Booker, L.B. (eds.) Proc. of the 4th Int. Conf.

on Genetic Algorithms, pp. 190–195. Morgan Kaufmann, San Francisco (1991)

15. Grefenstette, J.J.: Optimization of control parameters for genetic algorithms. IEEE

Trans. Systems, Man, and Cybernetics, SMC- 16(1), 122–128 (1986)

16. Harik, G., Cant´ u-Paz, E., Goldberg, D.E., Miller, B.L.: The gambler’s ruin prob-

lem, genetic algorithms, and the sizing of populations. In: Proceedings of the 4th

IEEE Conference on Evolutionary Computation, pp. 7–12. IEEE Press, Los Alami-

tos (1997)

17. Harik, G.R., Lobo, F.G.: A parameter-less genetic algorithm. In: Banzhaf, W.,

Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.)

Proceedings of the Genetic and Evolutionary Computation Conference, Orlando,

Florida, USA, 13-17, vol. 1, pp. 258–265. Morgan Kaufmann, San Francisco (1999)

Page 10

Statistical Analysis of Parameter Setting in Real-Coded EAs 461

18. Hinterding, R., Michalewicz, Z., Eiben, A.E.: Adaptation in Evolutionary Com-

putation: A Survey. In: Proceedings of the 4th IEEE Conference on Evolutionary

Computation, pp. 65–69. IEEE Press, Los Alamitos (1997)

19. Jagielska, I., Matthews, C., Whitfort, T.: An investigation into the application

of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated

knowledge acquisition problems. Neurocomputing 24(1-3), 37–54 (1999)

20. Jansen, T., Weyland, D.: Analysis of evolutionary algorithms for the longest com-

mon subsequence problem. In: GECCO 2007: Proceedings of the 9th Annual Con-

ference on Genetic and Evolutionary Computation, pp. 939–946. ACM, New York

(2007)

21. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems.

Ph.D. thesis, University of Michigan, Ann Arbor (1975)

22. Kim, D., Kim, C.: Forecasting time series with genetic fuzzy predictor ensemble.

IEEE Transactions on Fuzzy Systems 5(4), 523–535 (1997)

23. Lobo, F.G., Lima, C.F., Michalewicz, Z.: Parameter Setting in Evolutionary Algo-

rithms. Springer Publishing Company, Incorporated, Heidelberg (2007)

24. Savin, N.E.: The bonferroni and the scheff multiple comparison procedures. Review

of Economic Studies (XLVII), pp. 255–273 (1980)

25. Schaffer, J.D., Morishima, A.: An adaptive crossover distribution mechanism for

genetic algorithms. In: Grefenstette, J.J. (ed.) Proceedings of the 2nd International

Conference on Genetic Algorithms and Their Applications, pp. 36–40. Lawrence

Erlbaum Associates, Mahwah (1987)

26. Schwefel, H.-P.: Numerical Optimization of Computer Models. John Wiley & Sons,

Inc., New York (1981)

27. Shekel, J.: Test functions for multimodal search techniques. In: Fifth Annual

Princeton Conference on Information Science and Systems, pp. 354–359 (1971)

28. Smit, S.K., Eiben, A.E.: Comparing parameter tuning methods for evolutionary

algorithms. In: Haddow, P., et al. (eds.) CEC 2009: Proc. of the Eleventh conference

on Congress on Evolutionary Computation, Piscataway, NJ, USA, pp. 399–406.

IEEE Press, Los Alamitos (2009)

29. Smit, S.K., Eiben, A.E.: Using Entropy for Parameter Analysis of Evolutionary

Algorithms. In: Beielstein, B., et al. (eds.) Empirical Methods for the Analysis of

Optimization Algorithms. Natural Computing Series. Springer, Heidelberg (2009)

30. Smit, S.K., Eiben, A.E.: Parameter tuning of evolutionary algorithms: Generalist

vs. specialist. In: Di Chio, C., et al. (eds.) EvoApplicatons 2010. LNCS, vol. 6024,

pp. 542–551. Springer, Heidelberg (2010)

31. Thierens, D.: Dimensional analysis of allele-wise mixing revisited. In: Ebeling, W.,

Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141,

pp. 255–265. Springer, Heidelberg (1996)

32. Torn, A., Zilinskas, A.: Global Optimization. LNCS, vol. 350, p. 124. Springer,

Heidelberg (1989)

33. Weyland, D.: Simulated annealing, its parameter settings and the longest common

subsequence problem. In: GECCO 2008: Proceedings of the 10th Annual Confer-

ence on Genetic and Evolutionary Computation, pp. 803–810. ACM, New York

(2008)