Boris Naujoks

• TH Köln - University of Applied Sciences

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design o...

The evaluation and analysis of optimisation algorithms through benchmarks is an important aspect of research in evolutionary computation. This is especially true in the context of many-objective optimisation, where the complexity of the problems usually makes theoretical analysis difficult. However, the availability of suitable benchmarking problem...

Many-objective optimization problems (MaOPs) are problems that feature four or more objectives, criteria or attributes that must be considered simultaneously. MaOPs often arise in real-world situations and the development of algorithms for solving MaOPs has become one of the hot topics in the field of evolutionary multi-criteria optimization (EMO)....

The term Procedural Content Generation (PCG) refers to the (semi-)automatic generation of game content by algorithmic means, and its methods are becoming increasingly popular in game-oriented research and industry. A special class of these methods, which is commonly known as search-based PCG, treats the given task as an optimisation problem. Such p...

Optimization problems with multiple objectives and many input variables inherit challenges from both large-scale optimization and multi-objective optimization. To solve the problems, decomposition and transformation methods are frequently used. In this study, an improved control variable analysis is proposed based on dominance and diversity in Pare...

Automated model selection is often proposed to users to choose which machine learning model (or method) to apply to a given regression task. In this paper, we show that combining different regression models can yield better results than selecting a single ('best') regression model, and outline an efficient method that obtains optimally weighted con...

Speedrunning in general means to play a video game fast, i.e. using all means at one’s disposal to achieve a given goal in the least amount of time possible. To do so, a speedrun must be planned in advance, or routed, as referred to by the community. This paper focuses on discovering challenges and defining models needed when trying to approach the...

Speedrunning in general means to play a video game fast, i.e. using all means at one's disposal to achieve a given goal in the least amount of time possible. To do so, a speedrun must be planned in advance, or routed, as it is referred to by the community. This paper focuses on discovering challenges and defining models needed when trying to approa...

This chapter describes the differences between single-objective, multi-objective, and many-objective optimization problems. In multi- and many-objective optimization, often the objectives are conflicting; hence there is no single best point, and a trade-off between the objectives must be considered. Many-objective optimization problems can be more...

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design o...

The cooperative coevolution framework has been used extensively to solve large scale global optimization problems. Recently, the framework is used in CC-RDG3 where it uses recursive differential grouping and covariance matrix adaptation evolution strategies (CMA-ES). It was shown that the algorithm performs well on the CEC2013-LSGO benchmark functi...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm’s performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm’s performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well-specified problems, suitable algorithms, adequ...

While games have been used extensively as milestones to evaluate game-playing AI, there exists no standardised framework for reporting the obtained observations. As a result, it remains difficult to draw general conclusions about the strengths and weaknesses of different game-playing AI algorithms. In this paper, we propose reporting guidelines for...

Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm's performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify propertie...

Available at ArXiv: https://arxiv.org/abs/2001.02957 -- Surrogate-based optimization relies on so-called infill criteria (acquisition functions) to decide which point to evaluate next. When Kriging is used as the surrogate model of choice (also called Bayesian optimization), then one of the most frequently chosen criteria is expected improvement. Y...

Surrogate-assisted optimization was developed for handling complex and costly problems, which arise from real-world applications. The main idea behind surrogate-assisted optimization is to optimally exhaust the available information to lower the amount of required expensive function evaluations thus saving time, resources and the related costs. Thi...

Despite a large interest in real-world problems from the research field of evolutionary optimisation, established benchmarks in the field are mostly artificial. We propose to use game optimisation problems in order to form a benchmark and implement function suites designed to work with the established COCO benchmarking framework. Game optimisation...

In this position paper, we discuss the need for systematic benchmarking of surrogate-assisted evolutionary algorithms and give an overview of existing suitable function suites. Based on the findings, we hope to encourage more comparative studies in this field supported by benchmarks and outline how a concerted effort of the community could create b...

It is a common technique in global optimization with expensive black-box functions, to learn a regression model (or surrogate-model) of the response function from past evaluations and to use this model to decide on the location of future evaluations. In surrogate model assisted optimization it can be difficult to select the right modeling technique...

Uncertainty propagation is a technique to incorporate individuals with uncertain fitness estimates in evolutionary algorithms. The Surrogate-Assisted Partial Order-Based Evolutionary Optimisation Algorithm (SAPEO) uses uncertainty propagation of fitness predictions from a Kriging model to reduce the number of function evaluations. The fitness predi...

When designing or developing optimization algorithms, test functions are crucial to evaluate performance. Often, test functions are not sufficiently difficult, diverse, flexible or relevant to real-world applications. Previously, test functions with real-world relevance were generated by training a machine learning model based on real-world data. T...

In this paper, we propose a novel approach (SAPEO) to support the survival selection process in evolutionary multi-objective algorithms with surrogate models. The approach dynamically chooses individuals to evaluate exactly based on the model uncertainty and the distinctness of the population. We introduce multiple SAPEO variants that differ in ter...

In practical situations, complex systems are often composed of subsystems or subproblems with single or multiple objectives. These subsystems focus on different aspects of the overall system, but they often have strong interactions with each other and they are usually not sequentially ordered or obviously decomposable. Thus, the individual solution...

In this paper, we propose a novel approach (SAPEO) to support the survival selection process in multi-objective evolutionary algorithms with surrogate models - it dynamically chooses individuals to evaluate exactly based on the model uncertainty and the distinctness of the population. We introduce variants that differ in terms of the risk they allo...

PPSN 2016 hosts a total number of 16 tutorials covering a broad range of current research in evolutionary computation. The tutorials range from introductory to advanced and specialized but can all be attended without prior requirements. All PPSN attendees are cordially invited to take this opportunity to learn about ongoing research activities in o...

Game balancing is an important part of the (computer) game design process, in which designers adapt a game prototype so that the resulting gameplay is as entertaining as possible. In industry, the evaluation of a game is often based on costly playtests with human players. It suggests itself to automate this process using surrogate models for the pr...

Game balancing is an important part of the (computer) game design process, in which designers adapt a game prototype so that the resulting gameplay is as entertaining as possible. In industry, the evaluation of a game is often based on costly playtests with human players. It suggests itself to automate this process using surrogate models for the pr...

The influence of non-constant population sizes in evolutionary multi-objective optimization algorithms is investigated. In contrast to evolutionary single-objective optimization algorithms an increasing population size is considered beneficial when approaching the Pareto-front. Firstly, different deterministic schedules are tested, featuring differ...

Cyclone separators are filtration devices frequently used in industry, e.g., to filter particles from flue gas. Optimizing the cyclone geometry is a demanding task. Accurate simulations of cyclone separators are based on time consuming computational fluid dynamics simulations. Thus, the need for exploiting cheap information from analytical, approxi...

Real-world optimization problems may require time consuming and expensive measurements or simulations. Recently, the application of surrogate model-based approaches was extended from continuous to combinatorial spaces. This extension is based on the utilization of suitable distance measures such as Hamming or Swap Distance. In this work, such an ex...

Several methods were developed to solve cost-extensive multi-criteria optimization problems by reducing the number of function evaluations by means of surrogate optimization. In this study, we apply different multi-criteria surrogate optimization methods to improve (tune) an event-detection software for water-quality monitoring. For tuning two impo...

The progression of the dominated hypervolume in the course of the optimization process, with respect to a global reference point, is thought to be monotonically increasing. This intuition is based on the observation that in each iteration, the solution that contributes the least to the dominated hypervolume is eliminated. Derived from results of mu...

This paper proposes an information sharing model of artificial bee colony for locating multiple peaks in dynamic environments. The concept of niching is implemented by using a hybridized approach that combines a modified variant of the fitness sharing ...

This work provides a preliminary study on applying state-of-the-art time-series forecasting methods to electrical energy consumption data recorded by smart metering equipment. We compare a custom-build commercial baseline method to modern ensemble-based methods from statistical time-series analysis and to a modern commercial GP system. Our prelimin...

Decreases in dominated hypervolume w.r.t a fixed reference point for the (μ + 1)-SMS-EMOA are able to appear. We examine the impact of these decreases and different reference point handling techniques by providing four different algorithmic variants for selection. In addition, we show that yet further decreases can occur due to numerical instabilit...

Formerly, multi-criteria optimization algorithms were often tested using tens of thousands function evaluations. In many real-world settings function evaluations are very costly or the available budget is very limited. Several methods were developed to solve these cost-extensive multi-criteria optimization problems by reducing the number of functio...

Development and deployment of interactive evolutionary multiobjective optimization algorithms (EMOAs) have recently gained broad interest. In this study, first steps towards a theory of interactive EMOAs are made by deriving bounds on the expected number of function evaluations and queries to a decision maker. We analyze randomized local search and...

Energy systems are not only real-world systems; they are also one of the most important foundations of the modern world. Especially with the upcoming required changes to make more efficient use of energy and to shift towards a global use of sustainable, ...

Many relevant industrial optimization tasks feature more than just one quality criterion. State-of-the-art multi-criteria optimization algorithms require a relatively large number of function evaluations (usually more than 10^5) to approximate Pareto fronts. Due to high cost or time consumption this large amount of function evaluations is not alway...

It is possible for the (μ + 1)-SMS-EMOA to decrease in dominated hypervolume w.r.t. a global reference point. We study the influence of SMS-EMOA parameter settings on number and amount of the observed decreases. We show that the number of decreases drop and the number of increases rise with a higher population size. In addition, a positive correlat...

Evolutionary (multi-objective optimization) algorithms (EMOAs) are widely accepted to be competitive optimization methods in industry today. However, normally only standard techniques are employed by the engineering experts. Here, it is shown how these standard techniques can be completed and improved with respect to interactivity to other tools, r...

Sequential parameter optimization (SPO) is a heuristic that combines classical and modern statistical techniques to improve the performance of search algorithms. In this study, SPO is directly used as an optimization method on different noisy mathematical test functions. SPO includes a broad variety of meta models, which can have significant impact...

The complex, often redundant and noisy data in real-world data mining (DM) applications frequently lead to inferior results when out-of-the-box DM models are applied. A tuning of parameters is essential to achieve high-quality results. In this work we aim at tuning parameters of the preprocessing and the modeling phase conjointly. The framework TDM...

In this cumulative thesis an approach to multiobjective evolutionary optimisation using the hypervolume or the S-metric, respectively for selection is presented. This algorithm is tested and compared to standard techniques on two-, three and more dimensional objective spaces. To decide upon the right time when to stop a stochastic optimisation run,...

Typically, the variation operators deployed in evolutionary multiobjective optimization algorithms (EMOA) are either simulated binary crossover with polynomial mutation or differential evolution operators. This empirical study aims at the development of a sound method how to assess which of these variation operators perform best in the multiobjecti...

Choosing and tuning an optimization procedure for a given class of nonlinear optimization problems is not an easy task. One way to proceed is to consider this as a tournament, where each procedure will compete in different `disciplines'. Here, disciplines could either be different functions, which we want to optimize, or specific performance measur...

Optimizing an algorithm's parameter set for evolutionary multi-objective optimization (EMO) algorithms is not performed regularly until now. However, it could have been learned from single-objective optimization that doing so yields remarkable improvements in algorithm's performance. Here, the sequential parameter optimization (SPO) framework is ex...

Players of real-time strategy (RTS) games are often annoyed by the inability of the game AI to select and move teams of units in a natural way. Units travel and battle separately, resulting in huge losses and the AI looking unintelligent, as can the choice of units sent to counteract the opponents. Players are affected as well as computer commanded...

Evolutionary algorithms are non-deterministic and highly parameterizable optimization methods. Therefore, the setting of parameters greatly influences their performance and methods for parameter tuning became more and more popular in recent years. However, obtained parameter settings are usually valid only for the tackled combination of algorithm,...

Choosing and tuning an optimization procedure for a given class of nonlinear optimization problems is not an easy task. One way to proceed is to consider this as a tournament, where each procedure will compete in different ‘disciplines’. Here, disciplines could either be different functions, which we want to optimize, or specific performance measur...

In recent years, new approaches for multi-modal and multiobjective stochastic optimisation have been developed. It is a rather normal process that these experimental fields develop independently from other scientific areas. However, the connection between stochastic optimisation and statistics is obvious and highly appreciated. Recent works, such a...

In this paper, two approaches for estimating the generation in which a multi-objective evolutionary algorithm (MOEA) shows statistically significant signs of convergence are introduced. A set-based perspective is taken where convergence is measured by performance indicators. The proposed techniques fulfill the requirements of proper statistical ass...

Industry applications of multiobjective optimization problems mostly are characterized by the demand for high quality solutions on the one hand. On the other hand an optimization result is desired which at any rate meets the time constraints for the evolutionary multiobjective algorithms (EMOA). The handling of this trade-off is a frequently discus...

Noisy environments are a challenging task for multiobjective evolutionary algorithms. The algorithms may be trapped in local optima or even become a random search in the decision and objective space. In the course of the paper the classical definition of Pareto-dominance is enhanced subject to noisy objective functions in order to make the evolutio...

In less-than-truckload (LTL) terminals, arriving trucks have to be assigned to inbound doors and to suitable time slots for unloading. Simultaneously, waiting trucks have to be allocated to outbound doors. During a couple of hours, shipments from all incoming trucks are unloaded, sorted according to their relation, transported to the right outbound...

Over the last decades, evolutionary algorithms (EA) have proven their applicability to hard and complex industrial optimization problems in many cases. However, especially in cases with high computational demands for fitness evaluations (FE), the number of required FE is often seen as a drawback of these techniques. This is partly due to lacking ro...

In multi-criterion optimization, Pareto-optimal solutions that appear very similar in the objective space may have very different pre-images. In many practical applications the decision makers, who select a solution or preferred region on the Pareto-front, may want to know different pre-images of the selected solutions. Especially, this will be the...

Evolutionary multi-objective algorithms (EMOA) using performance indicators for the selection of individuals have turned out to be a successful technique for multi-objective problems. Especially, the selection based on the S\mathcal{S}-metric, as implemented in the SMS-EMOA, seems to be effective. A special feature of this EMOA is the greedy (μ + 1...

This paper investigates the intelligent moving and path-finding of groups in real-time strategy (RTS) games exemplified by the open source game Glest. We utilize the technique of flocking for achieving a smooth and natural movement of a group of units and expect grouping to decrease the amount of unit losses in RTS games. Furthermore, we present a...

This group explored the use of EMO in an interactive manner to solve multiobjective optimization problems. @InProceedings{_et_al:DSP:2009:2004, author = {Fonseca Carlos and Xavier Gandibleux and Pekka Korhonen and Luis Marti and Boris Naujoks and Lothar Thiele and Wallenius Jyrki and Eckart Zitzler}, title = {09041 Working Group on EMO for Interact...

Assembling suitable groups of fighting units to combat incoming enemy groups is a tactical necessity in real-time strategy (RTS) games. Furthermore it heavily influences future strategic decisions like unit building. Here, we demonstrate how to efficiently (offline) solve the problem of finding matches for the current enemy group(s) based on self-o...

Movement of groups in realtime strategy games is often a nuisance: Units travel and battle separately, resulting in huge losses and the AI looking dumb. This applies to computer as well as human commanded factions. We suggest to tackle that by using flocking improved by influence-map based pathfinding which leads to a much more natural and intellig...

Popular games often have a high-quality graphic design but quite simple-minded non player characters (NPC). Recently, Computational Intelligence (CI) methods have been discovered as suitable methods to revive NPC, making games more interesting, challenging, and funny. We present a fairly large study of human players on the simple arcade game Pac-Ma...

The creation of interesting opponents for human players in computer games is an interesting and challenging task. In contrast to up-to-date computer games, e.g. real time strategy games, learning of non-player-character strategies for older games seems to be easier and not that time-consuming. This way, older games, like the famous arcade game Pac-...

In multiobjective optimization there are conflicting demands for which a set of compromise solution is searched which approximates best possible solutions. Evolutionary algorithms established as effective and robust methodologies for this task. The quality of an approximation may be evaluated by its hypervolume dominated in the objective space, cal...

The concept of a linear jet yields the potential to play a decisive role in modern ship propulsion systems technology. A linear jet consists of multiple components, namely a rotor, a stator, and a nozzle. The optimization of a complete linear jet is a complex and challenging task, which requires a couple of comprehensive simulation tools to work to...

Research within the area of Evolutionary Multi-objective Optimization (EMO) focused on two- and three-dimensional objective functions, so far. Most algorithms have been developed for and tested on this limited application area. To broaden the insight in the behavior of EMO algorithms (EMOA) in higher dimensional objective spaces, a comprehensive be...