Publications (60)34.48 Total impact

Article: A Theoretical Assessment of Solution Quality in Evolutionary Algorithms for the Knapsack Problem
[show abstract] [hide abstract]
ABSTRACT: Evolutionary algorithms are well suited for solving the knapsack problem. Some empirical studies claim that evolutionary algorithms can produce good solutions to the 01 knapsack problem. Nonetheless, few rigorous investigations address the quality of solutions that evolutionary algorithms may produce for the knapsack problem. The current paper focuses on a theoretical investigation of three types of (N+1) evolutionary algorithms that exploit bitwise mutation, truncation selection, plus different repair methods for the 01 knapsack problem. It assesses the solution quality in terms of the approximation ratio. Our work indicates that the solution produced by pure strategy and mixed strategy evolutionary algorithms is arbitrarily bad. Nevertheless, the evolutionary algorithm using helper objectives may produce 1/2approximation solutions to the 01 knapsack problem.04/2014;  [show abstract] [hide abstract]
ABSTRACT: The 01 knapsack problem is a wellknown combinatorial optimisation problem. Approximation algorithms have been designed for solving it and they return provably good solutions within polynomial time. On the other hand, genetic algorithms are well suited for solving the knapsack problem and they find reasonably good solutions quickly. A naturally arising question is whether genetic algorithms are able to find solutions as good as approximation algorithms do. This paper presents a novel multiobjective optimisation genetic algorithm for solving the 01 knapsack problem. Experiment results show that the new algorithm outperforms its rivals, the greedy algorithm, mixed strategy genetic algorithm, and greedy algorithm + mixed strategy genetic algorithm.04/2014;  [show abstract] [hide abstract]
ABSTRACT: The convergence, convergence rate and expected hitting time play fundamental roles in the analysis of randomised search heuristics. This paper presents a unified Markov chain approach to studying them. Using the approach, the sufficient and necessary conditions of convergence in distribution are established. Then the average convergence rate is introduced to randomised search heuristics and its lower and upper bounds are derived. Finally, novel average drift analysis and backward drift analysis are proposed for bounding the expected hitting time. A computational study is also conducted to investigate the convergence, convergence rate and expected hitting time. The theoretical study belongs to a prior and general study while the computational study belongs to a posterior and case study.12/2013;  [show abstract] [hide abstract]
ABSTRACT: In the current work we introduce a novel estimation of distribution algorithm to tackle a hard combinatorial optimization problem, namely the singlemachine scheduling problem, with uncertain delivery times. The majority of the existing research coping with optimization problems in uncertain environment aims at finding a single sufficiently robust solution so that random noise and unpredictable circumstances would have the least possible detrimental effect on the quality of the solution. The measures of robustness are usually based on various kinds of empirically designed averaging techniques. In contrast to the previous work, our algorithm aims at finding a collection of robust schedules that allow for a more informative decision making. The notion of robustness is measured quantitatively in terms of the classical mathematical notion of a norm on a vector space. We provide a theoretical insight into the relationship between the properties of the probability distribution over the uncertain delivery times and the robustness quality of the schedules produced by the algorithm after a polynomial runtime in terms of approximation ratios.12/2013;  [show abstract] [hide abstract]
ABSTRACT: Drift analysis is a useful tool for estimating upper and lower bounds on the runtime of evolutionary algorithms. A new representation of drift analysis, called average drift analysis, is introduced in the paper. It takes a weaker requirement than pointwise drift analysis does. Pointwise drift theorems are a corollary of average drift theorems. Therefore average drift analysis is more powerful than pointwise drift analysis. To demonstrate the advantage of average drift analysis, we choose (1+N) evolutionary algorithms for linearlike functions as case study. Using average drift analysis, an exact bound on the runtime has been drawn for the algorithm and then the cutoff point of population scalability is derived.08/2013;  [show abstract] [hide abstract]
ABSTRACT: Hybrid and mixed strategy EAs have become rather popular for tackling various complex and NPhard optimization problems. While empirical evidence suggests that such algorithms are successful in practice, rather little theoretical support for their success is available, not mentioning a solid mathematical foundation that would provide guidance towards an efficient design of this type of EAs. In the current paper we develop a rigorous mathematical framework that suggests such designs based on generalized schema theory, fitness levels and drift analysis. An exampleapplication for tackling one of the classical NPhard problems, the "singlemachine scheduling problem" is presented.05/2013;  [show abstract] [hide abstract]
ABSTRACT: The classical Geiringer theorem addresses the limiting frequency of occurrence of various alleles after repeated application of crossover. It has been adopted to the setting of evolutionary algorithms and, a lot more recently, reinforcement learning and MonteCarlo tree search methodology to cope with a rather challenging question of action evaluation at the chance nodes. The theorem motivates novel dynamic parallel algorithms that are explicitly described in the current paper for the first time. The algorithms involve independent agents traversing a dynamically constructed directed graph that possibly has loops. A rather elegant and profound categorytheoretic model of cognition in biological neural networks developed by a wellknown French mathematician, professor Andree Ehresmann jointly with a neurosurgeon, Jan Paul Vanbremeersch over the last thirty years provides a hint at the connection between such algorithms and Hebbian learning.05/2013;  [show abstract] [hide abstract]
ABSTRACT: A popular current research trend deals with expanding the MonteCarlo tree search sampling methodologies to the environments with uncertainty and incomplete information. Recently a finite population version of Geiringer theorem with nonhomologous recombination has been adopted to the setting of MonteCarlo tree search to cope with randomness and incomplete information by exploiting the entrinsic similarities within the state space of the problem. The only limitation of the new theorem is that the similarity relation was assumed to be an equivalence relation on the set of states. In the current paper we lift this "curtain of limitation" by allowing the similarity relation to be modeled in terms of an arbitrary set cover of the set of stateaction pairs.05/2013;  [show abstract] [hide abstract]
ABSTRACT: In pure strategy metaheuristics, only one search strategy is applied for all time. In mixed strategy metaheuristics, each time one search strategy is chosen from a strategy pool with a probability and then is applied. An example is classical genetic algorithms, where either a mutation or crossover operator is chosen with a probability each time. The aim of this paper is to compare the performance between mixed strategy and pure strategy metaheuristic algorithms. First an experimental study is implemented and results demonstrate that mixed strategy evolutionary algorithms may outperform pure strategy evolutionary algorithms on the 01 knapsack problem in up to 77.8% instances. Then Complementary Strategy Theorem is rigorously proven for applying mixed strategy at the population level. The theorem asserts that given two metaheuristic algorithms where one uses pure strategy 1 and another uses pure strategy 2, the condition of pure strategy 2 being complementary to pure strategy 1 is sufficient and necessary if there exists a mixed strategy metaheuristics derived from these two pure strategies and its expected number of generations to find an optimal solution is no more than that of using pure strategy 1 for any initial population, and less than that of using pure strategy 1 for some initial population.03/2013;  [show abstract] [hide abstract]
ABSTRACT: The hardness of fitness functions is an important research issue in evolutionary computation. In theory, the study of the hardness of fitness functions can help understand the ability of evolutionary algorithms (EAs). In practice, the study may provide a guideline to the design of benchmarks. The aim of this paper is to answer the question: what are the easiest and hardest fitness functions with respect to an EA and how will such functions be constructed? In the paper, the easiest fitness (and hardest) fitness functions have been constructed to any given elitist (1+1) EA for maximising any class of fitness functions with the same optima. In terms of the timefitness landscape, the unimodal functions are the easiest and deceptive functions are the hardest. The paper also reveals that a fitness function, that is easiest to one EA, may become the hardest to another EA, and vice versa.03/2012;  [show abstract] [hide abstract]
ABSTRACT: Nowadays hybrid evolutionary algorithms, i.e, heuristic search algorithms combining several mutation operators some of which are meant to implement stochastically a well known technique designed for the specific problem in question while some others playing the role of random search, have become rather popular for tackling various NPhard optimization problems. While empirical studies demonstrate that hybrid evolutionary algorithms are frequently successful at finding solutions having fitness sufficiently close to the optimal, many fewer articles address the computational complexity in a mathematically rigorous fashion. This paper is devoted to a mathematically motivated design and analysis of a parameterized family of evolutionary algorithms which provides a polynomial time approximation scheme for one of the wellknown NPhard combinatorial optimization problems, namely the "single machine scheduling problem without precedence constraints". The authors hope that the techniques and ideas developed in this article may be applied in many other situations.02/2012; 
Article: Pure Strategy or Mixed Strategy?
[show abstract] [hide abstract]
ABSTRACT: Mixed strategy EAs aim to integrate several mutation operators into a single algorithm. However few theoretical analysis has been made to answer the question whether and when the performance of mixed strategy EAs is better than that of pure strategy EAs. In theory, the performance of EAs can be measured by asymptotic convergence rate and asymptotic hitting time. In this paper, it is proven that given a mixed strategy (1+1) EAs consisting of several mutation operators, its performance (asymptotic convergence rate and asymptotic hitting time)is not worse than that of the worst pure strategy (1+1) EA using one mutation operator; if these mutation operators are mutually complementary, then it is possible to design a mixed strategy (1+1) EA whose performance is better than that of any pure strategy (1+1) EA using one mutation operator.12/2011;  [show abstract] [hide abstract]
ABSTRACT: Populationbased evolutionary algorithms (EAs) have been widely applied to solve various optimization problems. The question of how the performance of a populationbased EA depends on the population size arises naturally. The performance of an EA may be evaluated by different measures, such as the average convergence rate to the optimal set per generation or the expected number of generations to encounter an optimal solution for the first time. Population scalability is the performance ratio between a benchmark EA and another EA using identical genetic operators but a larger population size. Although intuitively the performance of an EA may improve if its population size increases, currently there exist only a few case studies for simple fitness functions. This paper aims at providing a general study for discrete optimisation. A novel approach is introduced to analyse population scalability using the fundamental matrix. The following two contributions summarize the major results of the current article. (1) We demonstrate rigorously that for elitist EAs with identical global mutation, using a lager population size always increases the average rate of convergence to the optimal set; and yet, sometimes, the expected number of generations needed to find an optimal solution (measured by either the maximal value or the average value) may increase, rather than decrease. (2) We establish sufficient and/or necessary conditions for the superlinear scalability, that is, when the average convergence rate of a $(\mu+\mu)$ EA (where $\mu\ge2$) is bigger than $\mu$ times that of a $(1+1)$ EA.08/2011; 
Article: Population Scalability Analysis of Abstract Populationbased Random Search: Spectral Radius
[show abstract] [hide abstract]
ABSTRACT: Populationbased Random Search (RS) algorithms, such as Evolutionary Algorithms (EAs), Ant Colony Optimization (ACO), Artificial Immune Systems (AIS) and Particle Swarm Optimization (PSO), have been widely applied to solving discrete optimization problems. A common belief in this area is that the performance of a populationbased RS algorithm may improve if increasing its population size. The term of population scalability is used to describe the relationship between the performance of RS algorithms and their population size. Although understanding population scalability is important to design efficient RS algorithms, there exist few theoretical results about population scalability so far. Among those limited results, most of them belong to case studies, e.g. simple RS algorithms for simple problems. Different from them, the paper aims at providing a general study. A large family of RS algorithms, called ARS, has been investigated in the paper. The main contribution of this paper is to introduce a novel approach based on the fundamental matrix for analyzing population scalability. The performance of ARS is measured by a new index: spectral radius of the fundamental matrix. Through analyzing fundamental matrix associated with ARS, several general results have been proven: (1) increasing population size may increase population scalability; (2) no super linear scalability is available on any regular monotonic fitness landscape; (3) potential super linear scalability may exist on deceptive fitness landscapes; (4) "bridgeable point" and "diversity preservation" are two necessary conditions for super linear scalability on all fitness landscapes; and (5) "road through bridges" is a sufficient condition for super linear scalability.01/2011;  [show abstract] [hide abstract]
ABSTRACT: To exploit an evolutionary algorithmâ€™s performance to the full extent, the selection scheme should be chosen carefully. Empirically, it is commonly acknowledged that low selection pressure can prevent an evolutionary algorithm from premature convergence, and is thereby more suitable for widegap problems. However, there are few theoretical time complexity studies that actually give the conditions under which a high or a low selection pressure is better. In this paper, we provide a rigorous time complexity analysis showing that low selection pressure is better for the widegap problems with two optima.Theoretical Computer Science. 01/2010; 
Article: Approximating covering problems by randomized search heuristics using multiobjective models.
[show abstract] [hide abstract]
ABSTRACT: The main aim of randomized search heuristics is to produce good approximations of optimal solutions within a small amount of time. In contrast to numerous experimental results, there are only a few theoretical explorations on this subject. We consider the approximation ability of randomized search heuristics for the class of covering problems and compare singleobjective and multiobjective models for such problems. For the VertexCover problem, we point out situations where the multiobjective model leads to a fast construction of optimal solutions while in the singleobjective case, no good approximation can be achieved within the expected polynomial time. Examining the more general SetCover problem, we show that optimal solutions can be approximated within a logarithmic factor of the size of the ground set, using the multiobjective approach, while the approximation quality obtainable by the singleobjective approach in expected polynomial time may be arbitrarily bad.Evolutionary Computation 01/2010; 18(4):61733. · 2.11 Impact Factor  [show abstract] [hide abstract]
ABSTRACT: Vertex cover is one of the best known NPhard combinatorial optimization problems. Experimental work has claimed that evolutionary algorithms (EAs) perform fairly well for the problem and can compete with problemspecific ones. A theoretical analysis that explains these empirical results is presented concerning the random local search algorithm and the (1+1)EA. Since it is not expected that an algorithm can solve the vertex cover problem in polynomial time, a worst case approximation analysis is carried out for the two considered algorithms and comparisons with the best known problemspecific ones are presented. By studying instance classes of the problem, general results are derived. Although arbitrarily bad approximation ratios of the (1+1)EA can be proved for a bipartite instance class, the same algorithm can quickly find the minimum cover of the graph when a restart strategy is used. Instance classes where multiple runs cannot considerably improve the performance of the (1+1)EA are considered and the characteristics of the graphs that make the optimization task hard for the algorithm are investigated and highlighted. An instance class is designed to prove that the (1+1)EA cannot guarantee better solutions than the stateoftheart algorithm for vertex cover if worst cases are considered. In particular, a lower bound for the worst case approximation ratio, slightly less than two, is proved. Nevertheless, there are subclasses of the vertex cover problem for which the (1+1)EA is efficient. It is proved that if the vertex degree is at most two, then the algorithm can solve the problem in polynomial time.IEEE Transactions on Evolutionary Computation 11/2009; · 4.81 Impact Factor  [show abstract] [hide abstract]
ABSTRACT: In the past decades, many theoretical results related to the time complexity of evolutionary algorithms (EAs) on different problems are obtained. However, there is not any general and easytoapply approach designed particularly for populationbased EAs on unimodal problems. In this paper, we first generalize the concept of the takeover time to EAs with mutation, then we utilize the generalized takeover time to obtain the mean first hitting time of EAs and, thus, propose a general approach for analyzing EAs on unimodal problems. As examples, we consider the socalled (N + N) EAs and we show that, on two wellknown unimodal problems, leadingones and onemax , the EAs with the bitwise mutation and two commonly used selection schemes both need O(n ln n + n(2)/N) and O(n ln ln n + n ln n/N) generations to find the global optimum, respectively. Except for the new results above, our approach can also be applied directly for obtaining results for some populationbased EAs on some other unimodal problems. Moreover, we also discuss when the general approach is valid to provide us tight bounds of the mean first hitting times and when our approach should be combined with problemspecific knowledge to get the tight bounds. It is the first time a general idea for analyzing populationbased EAs on unimodal problems is discussed theoretically.IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics: a publication of the IEEE Systems, Man, and Cybernetics Society 04/2009; 39(5):1092106. · 3.01 Impact Factor  [show abstract] [hide abstract]
ABSTRACT: The satisfiability problem is a basic core NPcomplete problem. In recent years, a lot of heuristic algorithms have been developed to solve this problem, and many experiments have evaluated and compared the performance of different heuristic algorithms. However, rigorous theoretical analysis and comparison are rare. This paper analyzes and compares the expected runtime of three basic heuristic algorithms: RandomWalk, (1+1) EA, and hybrid algorithm. The runtime analysis of these heuristic algorithms on two 2SAT instances shows that the expected runtime of these heuristic algorithms can be exponential time or polynomial time. Furthermore, these heuristic algorithms have their own advantages and disadvantages in solving different SAT instances. It also demonstrates that the expected runtime upper bound of RandomWalk on arbitrary kSAT(k >/= 3) is O((k  1)(n)), and presents a kSAT instance that has Theta((k  1)(n)) expected runtime bound.Artificial Intelligence 02/2009; 173(2):240257. · 2.19 Impact Factor  [show abstract] [hide abstract]
ABSTRACT: Hybrid methods are very popular for solving problems from combinatorial optimization. In contrast, the theoretical understanding of the interplay of different optimization methods is rare. In this paper, we make a first step into the rigorous analysis of such combinations for combinatorial optimization problems. The subject of our analyses is the vertex cover problem for which several approximation algorithms have been proposed. We point out specific instances where solutions can (or cannot) be improved by the search process of a simple evolutionary algorithm in expected polynomial time.Evolutionary Computation 02/2009; 17(1):319. · 2.11 Impact Factor
Publication Stats
636  Citations  
316  Downloads  
3k  Views  
34.48  Total Impact Points  
Top Journals
Institutions

2011–2013

Aberystwyth University
 Department of Computer Science
Aberystwyth, Wales, United Kingdom


2009

University of Science and Technology of China
 School of Computer Science and Technology
Hefei, Anhui Sheng, China 
University of Wales
 Department of Computer Science
Cardiff, WLS, United Kingdom


2002–2009

University of Birmingham
 • Centre of Excellence for Research in Computational Intelligence and Applications (CERCIA)
 • School of Computer Science
Birmingham, ENG, United Kingdom


2004–2007

South China University of Technology
 School of Computer Science and Engineering
Shengcheng, Guangdong, China 
Wuhan University
 State Key Lab of Software Engineering
Wuhanshih, Hubei, China


1999–2001

Beijing Jiaotong University
 Department of Computer Science
Peping, Beijing, China
