Thomas Gabel

• Prof. Dr. rer. nat., Dipl.-Inf.
• Professor of Computer Science and Mathematics at Frankfurt University of Applied Sciences

The goal of the presented approach is to improve the stability of our RoboCup team code by providing an improved continuous integration software engineering process. As big and even small changes in our code base cannot be judged by just a couple of games, roughly 1000 games were run each night to have a good feeling whether changes were for the be...

The ability to correctly anticipate an opponent’s next action in real-time adversarial environments depends on both, the amount of collected observations of that agent’s behavior as well as on the capability to incorporate new knowledge into the opponent model easily. We present a novel approach to instance-based action prediction that utilizes gra...

Instance-based and case-based learning algorithms learn by remembering instances. When scaling such approaches to datasets of sizes that are typically faced in today’s data-rich and data-driven decade, basic approaches to case retrieval and case learning quickly come to their limits. In this paper, we introduce a novel scalable algorithm for both,...

The parameters of support vector machines (SVMs) such as kernel parameters and the penalty parameter have a great influence on the accuracy and complexity of the classification models. In the past, different evolutionary optimization algorithms were employed for optimizing SVMs; in this paper, we propose a social ski-driver (SSD) optimization algor...

Inter-agent communication has been playing an important role in soccer simulation 2D since its introduction. Its primary usage has been to communicate with teammates in order to share state observations to fill gaps in the players’ world models, to announce near future actions like passes or requesting passes, as well as for sharing and synchronizi...

In machine learning and numerical optimization, there has been an ongoing debate about properties of local optima and the impact of these properties on generalization. In this paper, we make a first attempt to address this question for case-based reasoning systems, more specifically for instance-based learning as it takes place in the retain phase....

Path planning algorithms have been used in different applications with the aim of finding a suitable collision-free path which satisfies some certain criteria such as the shortest path length and smoothness; thus, defining a suitable curve to describe path is essential. The main goal of these algorithms is to find the shortest and smooth path betwe...

Grasshopper Optimization Algorithm (GOA) was modified in this paper, to optimize multi-objective problems, and the modified version is called Multi-Objective Grasshopper Optimization Algorithm (MOGOA). An external archive is integrated with the GOA for saving the Pareto optimal solutions. The archive is then employed for defining the social behavio...

Fish identification is crucial for the survival of our threatened fish species. In this paper, a novel and robust biometric-based approach was proposed to identify fish species. The proposed approach consists of three phases. In the first phase, different features were extracted from fish images. In this phase, Weber's Local Descriptor (WLD) and co...

Support Vector Machine (SVM) parameters such as penalty and kernel parameters have a great influence on the complexity and accuracy of the classification model. In this paper, Dragonfly algorithm (DA) has been proposed to optimize the parameters of SVM; thus, the classification error can be decreased. To evaluate the proposed model (DA-SVM), the ex...

Measuring toxicity is an important step in drug development, and there is a high demand to develop computational models that can predict the drug toxicity risks. In this study, we used a dataset that consists of 553 drug samples that biotransformed in liver. The toxic effects were calculated for the current data are mutagenic, tumorigenic, irritant...

A remarkable feature of RoboCup’s soccer simulation leagues is their ability to quantify and prove the exact progress made over years. In this paper, we present and discuss the results of an extensive empirical study of the progress and the currently reached state of 2D soccer simulation. Our main finding is that the current decade has witnessed a...

We present a method for learning to interpret and understand foreign agent communication. Our approach is based on casting the contents of intercepted opponent agent communication to a bit-level representation and on training and employing deep convolutional neural networks for decoding the meaning of received messages. We empirically evaluate our...

The automatic acquisition of a similarity measure for a CBR system is appealing as it frees the system designer from the tedious task of defining it manually. However, acquiring similarity measures with some machine learning approach typically results in some black box representation of similarity whose magic-like combination of high precision and...

This paper presents a novel way to provide easy lecture recording. The actual recording is done on the hardware usually available anyhow: a laptop for the presentation and a smartphone. Voice and slides are recorded separately and displayed together in an HMTL5-based web application. This leads to minimal bandwidth requirements while still offering...

This paper focuses on an investigation of case-based opponent player modeling in the domain of simulated robotic soccer. While in previous and related work it has frequently been claimed that the prediction of low-level actions of an opponent agent in this application domain is infeasible, we show that – at least in certain settings – an online pre...

The optimization of traffic flow on roads and highways of modern industrialized countries is key to their economic growth and success. Besides, the reduction of traffic congestions and jams is also desirable from an ecological point of view as it yields a contribution to climate protection. In this article, we stick to a microscopic traffic simulat...

Batch reinforcement learning is a subfield of dynamic programming-based reinforcement learning. Originally defined as the task of learning the best possible policy from a fixed set of a priori-known transition samples, the (batch) algorithms developed in this field can be easily adapted to the classical online case, where the agent interacts with t...

We interpret job-shop scheduling problems as sequential decision problems that are handled by independent learning agents. These agents act completely decoupled from one another and employ probabilistic dispatching policies for which we propose a compact representation using a small set of real-valued parameters. During ongoing learning, the agents...

Neural batch reinforcement learning (RL) algorithms have recently shown to be a powerful tool for model-free reinforcement learning problems. In this paper, we present a novel learning benchmark from the realm of computer games and apply a variant of a neural batch RL algorithm in the scope of this benchmark. Defining the learning problem and appro...

The annual RoboCup competitions are certainly relevant to present times as they elect the best team of the current year. With re- spect to RoboCup's well-known 2050 vision, however, it is crucial to assess the general progress being made not just qualitatively, but also in a quantitative manner. Although this is dicult to accomplish in most leagues...

Abstract Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning fr...

We propose joint equilibrium policy search as a multi-agent learning algorithm for decentralized Markov decision processes with chang- ing action sets. In its basic form, it relies on stochastic agent-specific poli- cies parameterized by probability distributions defined for every state as well as on a heuristic that tells whether a joint equilibri...

Credible case-based inference (CCBI) is a new and theoretically sound inferencing mechanism for case-based systems. In this paper, we formally investigate the level of precision that CCBI-based retrieval results may yield. Building upon our theoretical findings, we derive a number of optimization criteria that can be utilized for learning such simi...

While a lot of papers on RoboCup’s robotic 2D soccer simulation have focused on the players’ offensive behavior, there are only a few papers that specifically address a team’s defense strategy. In this paper, we consider a defense scenario of crucial importance: We focus on situations where one of our players must interfere and disturb an opponent...

DEC-MDPs with changing action sets and partially ordered transition dependencies have recently been suggested as a sub-class of general DEC-MDPs that features provably lower complexity. In this pa- per, we investigate the usability of a coordinated batch-mode reinforce- ment learning algorithm for this class of distributed problems. Our agents acqu...

Decentralized Markov decision processes are frequently used to model cooperative multi-agent systems. In this paper, we identify a subclass of general DEC-MDPs that features regularities in the way agents interact with one another. This class is of high relevance for many real-world applications and features provably reduced complexity (NP-complete...

We identify two fundamental points of utilizing CBR for an adaptive agent that tries to learn on the basis of trial and error without a model of its environment. The first link concerns the utmost efficient exploitation of experience the agent has collected by interacting within its environment, while the second relates to the acquisition and repre...

Most approaches to tackle job-shop scheduling problems assume complete task knowledge and search for a centralized solution. In this work, we adopt an alternative view on scheduling problems where each resource is equipped with an adaptive agent that, independent of other agents, makes job dispatching decisions based on its local view on the plant...

RoboCup soccer simulation features the challenges of a fully distributed multi-agent domain with continuous state and action spaces, partial observability, as well as noisy perception and action execution. While the application of machine learning techniques in this domain represents a promising idea in itself, the competitive character of RoboCup...

In this paper, we suggest and analyze the use of approximate reinforcement learning techniques for a new category of challenging benchmark problems from the field of operations research. We demonstrate that interpreting and solving the task of job-shop scheduling as a multi-agent learning problem is beneficial for obtaining near-optimal solutions a...

Traditional approaches to solving job-shop scheduling problems assume full knowledge of the problem and search for a centralized solution for a single problem instance. Finding optimal solutions, however, re-quires an enormous computational effort, which becomes critical for large problem instance sizes and, in particular, in situations where fre-q...

In both research fields, Case-Based Reasoning and Reinforce- ment Learning, the system under consideration gains its expertise from experience. Utilizing this fundamental common ground as well as further characteristics and results of these two disciplines, in this paper we de- velop an approach that facilitates the distributed learning of behaviou...

Das “Brainstormers” Projekt wurde 1998 gestartet mit dem Ziel, lernfähige autonome Agenten in komplexen Umgebungen am Beispiel Roboterfußball zu erforschen. Dabei hat die Bearbeitung der vielfältigen Fragestellungen, die sich in dieser sehr dynamischen und verrauschten Umgebung ergeben, zu einer Vielzahl neuartiger Methoden und theoretischer Ergebn...

We focus on neuro-dynamic programming methods to learn state-action value functions and outline some of the inherent problems to be faced, when per- forming reinforcement learning in combination with function approximation. In an attempt to overcome some of these problems, we develop a reinforcement learning method that monitors the learning proces...

The definition of accurate similarity measures is a key issue of every Case-Based Reasoning application. Although some ap- proaches to optimize similarity measures automatically have already been applied, these approaches are not suited for all CBR application domains. On the one hand, they are re- stricted to classification tasks, on the other han...

CBR is one of the techniques that can be applied to the task of approximating a function over high-dimensional, continuous spaces. In Reinforcement Learning systems a learning agent is faced with the problem of assessing the desirability of the state it finds itself in. If the state space is very large and/or continuous the availability of a suitab...

A very recent topic in CBR research deals with the automated optimisation of similarity measures—a core component of each CBR application—by using machine learning techniques. In our previous work, a number of approaches to bias and guide the learning process have been proposed aiming at more stable learning results and less susceptibility to overf...

The Brainstormers have been participating in RoboCup's soccer simulation tournaments since 1998. Ever since a number of suc- cesses could be achieved, including multiple World Vice Champion titles and the World Champion title at RoboCup 2005 in Osaka. By now, the source code of our team will be made publicly available. This way, we hope to make a c...

The definition of similarity measures—one core component of every CBR application—leads to a serious knowledge acquisition prob- lem if domain and application specific requirements have to be con- sidered. To reduce the knowledge acquisition effort, different machine learning techniques have been developed in the past. In this paper, en- hancements...

Deliverable D3.1.1.a (WP3.1) This document is an informal deliverable provided to SEKT partners. The main aim of this document is to get partners quickly started with using the KAON Open Source ontology management infrastructure. KAON consists of a number of different modules providing a broad bandwidth of functionalities centered around creation,...

Deliverable D7.1.1.a (WP7.1) This informal deliverable aims to provide use cases for SEKT. The use cases are described in natural language and serve as a first input for detailed discussion with other partners. In particular, we want to clarify the interactions of technical work packages and to understand better the synergies to be expected from co...

Deliverable D12.1.1. (WP12.1) This document accompanies the SEKT web site and mailing lists. Important features of the web site, which consists of a public and a private part, as well as open issues are being described. A brief overview summarizes the existing mailing lists available for SEKT and some subscription details are presented. Both, web s...

The definition of similarity measures is one of the most cru- cial aspects when developing case-based applications. In particular, when employing similarity measures that contain a lot of specific knowledge about the addressed application domain, modelling similarity measures is a complex and time-consuming task. One common element of the sim- ilar...

The definition of similarity measures is one of the most crucial aspects when developing case-based applications. In particular, when employing similarity measures that contain a lot of specific knowledge about the addressed application domain, modelling similarity measures is a complex and time-consuming task. One common element of the similarity...

It is a vital behaviour pattern to humans, the most highly developed autonomous agents, to make use of experiences accumulated in the past and to solve new problems in analogy to solutions of old, yet similar problems. This report gives an outline of our work to apply that case-based approach to an artificial agent in the domain of Robotic Soccer s...

agents { to make use of experiences accumulated in the past and to solve new problems in analogy to solutions of old, yet similar problems. This thesis gives an outline of our work to apply that case-based approach to an arti cial agent in the domain of Robotic Soccer simulation. We capacitate the online coach of a robotic soccer team to determine...

EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D12.1.1. (WP12.1) This document accompanies the SEKT web site and mailing lists. Important features of the web site, which consists of a public and a private part, as well as open issues are being described. A brief overview summarizes the existing mailing lists available for SEKT and...

In this paper, we discuss the application of reinforcement learning for autonomous robots using the RoboCup domain as benchmark. The paper compares successful learning approaches in simulation with learning on real robots and develops methodologies to overcome the additional problems in real world.

EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D7.1.1.a (WP7.1) This informal deliverable aims to provide use cases for SEKT. The use cases are described in natural language and serve as a first input for detailed discussion with other partners. In particular, we want to clarify the interactions of technical work packages and to un...

EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D3.1.1.a (WP3.1) This document,is an informal deliverable provided to SEKT partners. The main aim of this document is to get partners quickly started with using the KAON Open Source ontology management infrastructure. KAON consists of a number,of different modules,providing a broad ban...