Yasser Gonzalez-Fernandez’s research while affiliated with York University and other places

Restarts are a popular remedy to address (premature) convergence in metaheuristics. In Particle Swarm Optimization , it has been observed that swarms often "stall" as opposed to "converge". A stall occurs when all of the forward progress that could occur is instead rejected as failed exploration. Since the swarm is in a good region of the search space with the potential to make more progress, a (random) restart could be counter productive. We instead introduce a method to address the stall mechanism. The introduction of perturbations to the pbest positions leads to significant improvements in the performance of standard Particle Swarm Optimization.

As software systems grow in size and complexity, the process of configuring them to meet individual needs becomes more and more challenging. Users, especially those that are new to a system, are faced with an ever increasing number of configuration possibilities, making the task of choosing the right one more and more daunting. However, users are rarely alone in using a software system. Crowds of other users or the designers themselves can provide with examples and rules as to what constitutes a meaningful configuration. We introduce a technique for designing optimal interactive configuration elicitation dialogs, aimed at utilizing crowd and expert information to reduce the amount of manual configuration effort. A repository of existing user configurations supplies us with information about popular ways to complete an existing partial configuration. Designers augment this information with their own constraints. A Markov decision process (MDP) model is then created to encode configuration elicitation dialogs that maximize the automatic configuration decisions based on the crowd and the designers’ information. A genetic algorithm is employed to solve the MDP when problem sizes prevent use of common exact techniques. In our evaluation with various configuration models we show that the technique is feasible, saves configuration effort and scales for real problem sizes of a few hundreds of features.

Results for an implementation of Leaders and Followers on the CEC2013 Real-Parameter Optimization Benchmark Functions. Source code for Leaders and Followers: https://www.researchgate.net/publication/281827268_Leaders_and_Followers_%28Matlab_code%29?ev=prf_pub

Finding good solutions on multi-modal optimization problems depends mainly on the efficacy of exploration. However, many search techniques applied to multi-modal problems were initially conceptualized with unimodal functions in mind, prioritizing exploitation over exploration. In this paper, we perform a study on the efficacy of exploration under random sampling, which leads to the identification of an important comparison bias that occurs when a solution which has benefited from local search is compared to the first (random) solution in a new search area. With the goal of eliminating this bias and improving the efficacy of exploration, we have developed a new search technique explicitly designed for multi-modal search spaces. “Leaders and Followers” aims to eliminate the negative effects of information accumulation and at the same time use the information from the best solutions in a way that they have controlled influence over the newly-sampled solutions. The proposed metaheuristic outperforms both Particle Swarm Optimization and Differential Evolution across a broad range of multi-modal optimization problems.

A multi-modal search space can be defined as having multiple attraction basins – each basin has a single local optimum which is reached from all points in that basin when greedy local search is used. Optimization in multi-modal search spaces can then be viewed as a two-phase process. The first phase is exploration in which the most promising attraction basin is identified. The second phase is exploitation in which the best solution (i.e. the local optimum) within the previously identified attraction basin is attained. The goal of thresheld convergence is to improve the performance of search techniques during the first phase of exploration. The effectiveness of thresheld convergence has been demonstrated through applications to existing metaheuristics such as particle swarm optimization and differential evolution, and through the development of novel metaheuristics such as minimum population search and leaders and followers.

C-code implementation of Minimum Population Search for the CEC-2013 benchmark.

The aim of this work is studying the use of copulas and vines in numerical optimization with Estimation of Distribution Algorithms (EDAs). Two EDAs built around the multivariate product and normal copulas, and other two based on pair-copula decomposition of vine models are studied. We analyze empirically the effect of both marginal distributions and dependence structure in order to show that both aspects play a crucial role in the success of the optimization process. The results show that the use of copulas and vines opens new opportunities to a more appropriate modeling of search distributions in EDAs.

Multi-modal optimization involves two distinct tasks: identifying promising attraction basins and finding the local optima in these basins. Unfortunately, the second task can interfere with the first task if they are performed simultaneously. Specifically, the promise of an attraction basin is often estimated by the fitness of a single sample solution, so an attraction basin represented by a random sample solution can appear to be less promising than an attraction basin represented by its local optimum. The goal of thresheld convergence is to prevent these biased comparisons by disallowing local search while global search is still in progress. Ideally, thresheld convergence achieves this goal by using a distance threshold that is correlated to the size of the attraction basins in the search space. In this paper, a clustering-based method is developed to identify the scale of the search space which thresheld convergence can then exploit. The proposed method employed in the context of a multi-start particle swarm optimization algorithm has led to large improvements across a broad range of multi-modal problems.

Optimisation in multimodal landscapes involves two distinct tasks: identifying promising regions and location of the (local) optimum within each region. Progress towards the second task can interfere with the first by providing a misleading estimate of a region's value. Thresheld convergence is a generally applicable "meta"-heuristic designed to control an algorithm's rate of convergence and hence which mode of search it is using at a given time. Previous applications of thresheld convergence in differential evolution (DE) have shown considerable promise, but the question of which threshold values to use for a given (unknown) function landscape remains open. This work explores the use of clustering-based method to infer the distances between local optima in order to set a series of decreasing thresholds in a multi-start DE algorithm. Results indicate that on those problems where normal DE converges, the proposed strategy can lead to sizable improvements.