Boris Naujoks

Technische Hochschule Köln | FH Köln · Faculty of Computer Science and Engineering Science

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design o...

The evaluation and analysis of optimisation algorithms through benchmarks is an important aspect of research in evolutionary computation. This is especially true in the context of many-objective optimisation, where the complexity of the problems usually makes theoretical analysis difficult. However, the availability of suitable benchmarking problem...

Many-objective optimization problems (MaOPs) are problems that feature four or more objectives, criteria or attributes that must be considered simultaneously. MaOPs often arise in real-world situations and the development of algorithms for solving MaOPs has become one of the hot topics in the field of evolutionary multi-criteria optimization (EMO)....

The term Procedural Content Generation (PCG) refers to the (semi-)automatic generation of game content by algorithmic means, and its methods are becoming increasingly popular in game-oriented research and industry. A special class of these methods, which is commonly known as search-based PCG, treats the given task as an optimisation problem. Such p...

Optimization problems with multiple objectives and many input variables inherit challenges from both large-scale optimization and multi-objective optimization. To solve the problems, decomposition and transformation methods are frequently used. In this study, an improved control variable analysis is proposed based on dominance and diversity in Pare...

Automated model selection is often proposed to users to choose which machine learning model (or method) to apply to a given regression task. In this paper, we show that combining different regression models can yield better results than selecting a single ('best') regression model, and outline an efficient method that obtains optimally weighted con...

Speedrunning in general means to play a video game fast, i.e. using all means at one’s disposal to achieve a given goal in the least amount of time possible. To do so, a speedrun must be planned in advance, or routed, as referred to by the community. This paper focuses on discovering challenges and defining models needed when trying to approach the...

Speedrunning in general means to play a video game fast, i.e. using all means at one's disposal to achieve a given goal in the least amount of time possible. To do so, a speedrun must be planned in advance, or routed, as it is referred to by the community. This paper focuses on discovering challenges and defining models needed when trying to approa...

This chapter describes the differences between single-objective, multi-objective, and many-objective optimization problems. In multi- and many-objective optimization, often the objectives are conflicting; hence there is no single best point, and a trade-off between the objectives must be considered. Many-objective optimization problems can be more...

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design o...

The cooperative coevolution framework has been used extensively to solve large scale global optimization problems. Recently, the framework is used in CC-RDG3 where it uses recursive differential grouping and covariance matrix adaptation evolution strategies (CMA-ES). It was shown that the algorithm performs well on the CEC2013-LSGO benchmark functi...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm’s performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm’s performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well-specified problems, suitable algorithms, adequ...

While games have been used extensively as milestones to evaluate game-playing AI, there exists no standardised framework for reporting the obtained observations. As a result, it remains difficult to draw general conclusions about the strengths and weaknesses of different game-playing AI algorithms. In this paper, we propose reporting guidelines for...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm's performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

Available at ArXiv: https://arxiv.org/abs/2001.02957 -- Surrogate-based optimization relies on so-called infill criteria (acquisition functions) to decide which point to evaluate next. When Kriging is used as the surrogate model of choice (also called Bayesian optimization), then one of the most frequently chosen criteria is expected improvement. Y...

Surrogate-assisted optimization was developed for handling complex and costly problems, which arise from real-world applications. The main idea behind surrogate-assisted optimization is to optimally exhaust the available information to lower the amount of required expensive function evaluations thus saving time, resources and the related costs. Thi...

Despite a large interest in real-world problems from the research field of evolutionary optimisation, established benchmarks in the field are mostly artificial. We propose to use game optimisation problems in order to form a benchmark and implement function suites designed to work with the established COCO benchmarking framework. Game optimisation...

In this position paper, we discuss the need for systematic benchmarking of surrogate-assisted evolutionary algorithms and give an overview of existing suitable function suites. Based on the findings, we hope to encourage more comparative studies in this field supported by benchmarks and outline how a concerted effort of the community could create b...

It is a common technique in global optimization with expensive black-box functions, to learn a regression model (or surrogate-model) of the response function from past evaluations and to use this model to decide on the location of future evaluations. In surrogate model assisted optimization it can be difficult to select the right modeling technique...

Uncertainty propagation is a technique to incorporate individuals with uncertain fitness estimates in evolutionary algorithms. The Surrogate-Assisted Partial Order-Based Evolutionary Optimisation Algorithm (SAPEO) uses uncertainty propagation of fitness predictions from a Kriging model to reduce the number of function evaluations. The fitness predi...

When designing or developing optimization algorithms, test functions are crucial to evaluate performance. Often, test functions are not sufficiently difficult, diverse, flexible or relevant to real-world applications. Previously, test functions with real-world relevance were generated by training a machine learning model based on real-world data. T...

In this paper, we propose a novel approach (SAPEO) to support the survival selection process in evolutionary multi-objective algorithms with surrogate models. The approach dynamically chooses individuals to evaluate exactly based on the model uncertainty and the distinctness of the population. We introduce multiple SAPEO variants that differ in ter...

In practical situations, complex systems are often composed of subsystems or subproblems with single or multiple objectives. These subsystems focus on different aspects of the overall system, but they often have strong interactions with each other and they are usually not sequentially ordered or obviously decomposable. Thus, the individual solution...

PPSN 2016 hosts a total number of 16 tutorials covering a broad range of current research in evolutionary computation. The tutorials range from introductory to advanced and specialized but can all be attended without prior requirements. All PPSN attendees are cordially invited to take this opportunity to learn about ongoing research activities in o...

Game balancing is an important part of the (computer) game design process, in which designers adapt a game prototype so that the resulting gameplay is as entertaining as possible. In industry, the evaluation of a game is often based on costly playtests with human players. It suggests itself to automate this process using surrogate models for the pr...

The influence of non-constant population sizes in evolutionary multi-objective optimization algorithms is investigated. In contrast to evolutionary single-objective optimization algorithms an increasing population size is considered beneficial when approaching the Pareto-front. Firstly, different deterministic schedules are tested, featuring differ...

Cyclone separators are filtration devices frequently used in industry, e.g., to filter particles from flue gas. Optimizing the cyclone geometry is a demanding task. Accurate simulations of cyclone separators are based on time consuming computational fluid dynamics simulations. Thus, the need for exploiting cheap information from analytical, approxi...

Real-world optimization problems may require time consuming and expensive measurements or simulations. Recently, the application of surrogate model-based approaches was extended from continuous to combinatorial spaces. This extension is based on the utilization of suitable distance measures such as Hamming or Swap Distance. In this work, such an ex...

Several methods were developed to solve cost-extensive multi-criteria optimization problems by reducing the number of function evaluations by means of surrogate optimization. In this study, we apply different multi-criteria surrogate optimization methods to improve (tune) an event-detection software for water-quality monitoring. For tuning two impo...

The progression of the dominated hypervolume in the course of the optimization process, with respect to a global reference point, is thought to be monotonically increasing. This intuition is based on the observation that in each iteration, the solution that contributes the least to the dominated hypervolume is eliminated. Derived from results of mu...

This paper proposes an information sharing model of artificial bee colony for locating multiple peaks in dynamic environments. The concept of niching is implemented by using a hybridized approach that combines a modified variant of the fitness sharing ...

This work provides a preliminary study on applying state-of-the-art time-series forecasting methods to electrical energy consumption data recorded by smart metering equipment. We compare a custom-build commercial baseline method to modern ensemble-based methods from statistical time-series analysis and to a modern commercial GP system. Our prelimin...

Decreases in dominated hypervolume w.r.t a fixed reference point for the (μ + 1)-SMS-EMOA are able to appear. We examine the impact of these decreases and different reference point handling techniques by providing four different algorithmic variants for selection. In addition, we show that yet further decreases can occur due to numerical instabilit...

Formerly, multi-criteria optimization algorithms were often tested using tens of thousands function evaluations. In many real-world settings function evaluations are very costly or the available budget is very limited. Several methods were developed to solve these cost-extensive multi-criteria optimization problems by reducing the number of functio...

Development and deployment of interactive evolutionary multiobjective optimization algorithms (EMOAs) have recently gained broad interest. In this study, first steps towards a theory of interactive EMOAs are made by deriving bounds on the expected number of function evaluations and queries to a decision maker. We analyze randomized local search and...

Energy systems are not only real-world systems; they are also one of the most important foundations of the modern world. Especially with the upcoming required changes to make more efficient use of energy and to shift towards a global use of sustainable, ...

Many relevant industrial optimization tasks feature more than just one quality criterion. State-of-the-art multi-criteria optimization algorithms require a relatively large number of function evaluations (usually more than 10^5) to approximate Pareto fronts. Due to high cost or time consumption this large amount of function evaluations is not alway...

It is possible for the (μ + 1)-SMS-EMOA to decrease in dominated hypervolume w.r.t. a global reference point. We study the influence of SMS-EMOA parameter settings on number and amount of the observed decreases. We show that the number of decreases drop and the number of increases rise with a higher population size. In addition, a positive correlat...

Evolutionary (multi-objective optimization) algorithms (EMOAs) are widely accepted to be competitive optimization methods in industry today. However, normally only standard techniques are employed by the engineering experts. Here, it is shown how these standard techniques can be completed and improved with respect to interactivity to other tools, r...

Sequential parameter optimization (SPO) is a heuristic that combines classical and modern statistical techniques to improve the performance of search algorithms. In this study, SPO is directly used as an optimization method on different noisy mathematical test functions. SPO includes a broad variety of meta models, which can have significant impact...

The complex, often redundant and noisy data in real-world data mining (DM) applications frequently lead to inferior results when out-of-the-box DM models are applied. A tuning of parameters is essential to achieve high-quality results. In this work we aim at tuning parameters of the preprocessing and the modeling phase conjointly. The framework TDM...

In this cumulative thesis an approach to multiobjective evolutionary optimisation using the hypervolume or the S-metric, respectively for selection is presented. This algorithm is tested and compared to standard techniques on two-, three and more dimensional objective spaces. To decide upon the right time when to stop a stochastic optimisation run,...

Typically, the variation operators deployed in evolutionary multiobjective optimization algorithms (EMOA) are either simulated binary crossover with polynomial mutation or differential evolution operators. This empirical study aims at the development of a sound method how to assess which of these variation operators perform best in the multiobjecti...

Choosing and tuning an optimization procedure for a given class of nonlinear optimization problems is not an easy task. One way to proceed is to consider this as a tournament, where each procedure will compete in different `disciplines'. Here, disciplines could either be different functions, which we want to optimize, or specific performance measur...

Optimizing an algorithm's parameter set for evolutionary multi-objective optimization (EMO) algorithms is not performed regularly until now. However, it could have been learned from single-objective optimization that doing so yields remarkable improvements in algorithm's performance. Here, the sequential parameter optimization (SPO) framework is ex...

Players of real-time strategy (RTS) games are often annoyed by the inability of the game AI to select and move teams of units in a natural way. Units travel and battle separately, resulting in huge losses and the AI looking unintelligent, as can the choice of units sent to counteract the opponents. Players are affected as well as computer commanded...

Evolutionary algorithms are non-deterministic and highly parameterizable optimization methods. Therefore, the setting of parameters greatly influences their performance and methods for parameter tuning became more and more popular in recent years. However, obtained parameter settings are usually valid only for the tackled combination of algorithm,...

Choosing and tuning an optimization procedure for a given class of nonlinear optimization problems is not an easy task. One way to proceed is to consider this as a tournament, where each procedure will compete in different ‘disciplines’. Here, disciplines could either be different functions, which we want to optimize, or specific performance measur...

In recent years, new approaches for multi-modal and multiobjective stochastic optimisation have been developed. It is a rather normal process that these experimental fields develop independently from other scientific areas. However, the connection between stochastic optimisation and statistics is obvious and highly appreciated. Recent works, such a...