Franz Kummert

• Bielefeld University

Zusammenfassung This paper presents the development of a human-centred AI system for the classification of laundry according to washing categories such as color and type. The system aims to provide a solution that is both accurate and easy to use for individuals with varying levels of technical expertise. The development process involved a human-ce...

Introduction: Due to the changes in the indication range for cochlear implants and the demographic development towards an aging society, more and more people are in receipt of cochlear implants. An implantation requires a close-meshed audiological and logopedic aftercare. Hearing therapy rehabilitation currently requires great personnel effort and...

In this work, the creation of a dataset labeled in a pixel-wise manner for the uncommon domain of stain detection on patterned laundry is described. The unique properties of images in this dataset—stains are small and sometimes occur in large amounts—led to the creation of noisy labels. Indeed, the training of a fully convolutional neural network f...

Learning and matching a user’s preference is an essential aspect of achieving a productive collaboration in long-term Human–Robot Interaction (HRI). However, there are different techniques on how to match the behavior of a robot to a user’s preference. The robot can be adaptable so that a user can change the robot’s behavior to one’s need, or the r...

Preventing diseases of affluence is one of the major challenges for our future society. Researchers introduced robots as a tool to support people on dieting or rehabilitation tasks. However, deploying robots as exercising companions is cost-intensive. Therefore, in our current work, we are investigating how the embodiment of an exercising partner i...

Für die Weiterentwicklung der klassischen Wäschereitechnik zu intelligenten technischen Systemen ist ein strukturiertes Vorgehen unerlässlich. In dem Projekt wurden wissenschaftliche Methoden genutzt und auf die Wäschereitechnik adaptiert, um das Ziel einer ressourcenschonenden Wäscherei zu erreichen. Neben der modellbasierten Entwurfstechnik für i...

Preventing diseases of affluence is one of the major challenges for our future society. Researchers introduced robots as a tool to support people on dieting or rehabilitation tasks. However, deploying robots as exercising companions is cost-intensive. Therefore, in our current work, we are investigating how the embodiment of an exercising partner i...

Intelligent agents need to perceive and correctly interpret the social signals of their interaction partners. In order to support the development of these skills, we establish a process of long-term data acquisition, annotation and continuous model evaluation. We facilitate automatic recording and annotation of unconstrained, multicentric interacti...

Im Folgenden werden die Pilotprojekte des ReSerW-Projekts vorgestellt. Diese Pilotprojekte sollen die Fähigkeit der in den Querschnittsprojekten (siehe Abb. 4.1) erarbeiteten Methoden nachweisen und die Industrietauglichkeit der Lösungen sicherstellen. Aus diesem Grund fließen die Ergebnisse jedes einzelnen Querschnittsprojekts in die vier Pilotpro...

Even if only the acoustic channel is considered, human communication is highly multi-modal. Non-lexical cues provide a variety of information such as emotion or agreement. The ability to process such cues is highly relevant for spoken dialog systems, especially in assistance systems. In this paper we focus on the recognition of non-lexical confirma...

A lack of motivation is the most common obstacle for physical activity. Socially Assistive Robots (SAR) can be used to motivate people to workout regularly. However, the embodiment and companionship type can influence user's engagement. We investigated the effects of embodiment (co-present vs. remote-located) and companionship (instructor vs. compa...

This work explores what people do to inform their surroundings about who is the main addressee of their communicative acts in smart environments. A corpus of naive users, solving daily tasks in a smart home, which is additionally inhabited by a robot, is investigated. Evidence drawn from the corpus is used to create a first model for addressee reco...

Preventing diseases of affluence is one of the major challenges for our future society. Recently, robots have been introduced as support for people on dieting or rehabilitation tasks. In our current work, we are investigating how the companionship and acknowledgement of a socially assistive robot (SAR) can influence the user to persist longer on a...

The purpose of this Wizard-of-Oz study was to explore the intuitive verbal and non-verbal goal-directed behavior of naïve participants in an intelligent robotics apartment. Participants had to complete seven mundane tasks, for instance, they were asked to turn on the light. Participants were explicitly instructed to consider nonstandard ways of com...

The detection of lane layout in the surroundings of the ego-vehicle is a key issue for modern ADAS and autonomous driving. Most modern systems rely on annotated spatial maps to provide lane information. However, these maps are not available everywhere, and thus have to be often supported by direct detection systems (e.g. cameras, lasers). Such syst...

In this paper, we present our humanoid robot "Meka", participating in a multi party human robot dialogue scenario. Active arbitration of the robot's attention based on multi-modal stimuli is utilised to observe persons which are outside of the robots field of view. We investigate the impact of this attention management and addressee recognition on...

Studying long-term human-robot interactions in the context of playing games can help answer many questions about how humans perceive robots. This paper presents the results of a study where the robot Flobi [11] plays a game of pairs against a human player and employs a memory with information about past interactions. The study focuses on long-term...

Robots are increasingly tested in different socially assistive scenarios. Future applications range from dieting, coaching, tutoring to autism therapy. In such applications the success of the system is commonly evaluated by the ability to encourage the user to keep up with a task. Hence, one important requirement for supportive systems is to have a...

We present a long-term feedback model for socially assistive robots evaluated during an 18 day long-term indoor cycling training. Our feedback model captures different aspects from sport motivation theory applied to long-term use cases. Furthermore, we present our designed measurements to evaluate the robot's persuasiveness and user's compliance....

Detecting the road terrain ahead of the ego-vehicle is an important issue for modern driver assistance systems. In particular, vehicle motion planning in inner city environment requires the detection of road terrain up to 3 seconds in advance. State-of-the-art visual road terrain detection systems have a hard time fulfilling this task, due to their...

This paper presents results of a first study for a human-robot-interaction in which the robot Flobi autonomously plays the game 'pairs' against a human player. Integrated is a dialog system which can use different dynamic sentence structures for altering the verbal output during the interaction. The system is compared with a dialog system using sta...

In inner-city, most vehicle-pedestrian collisions occur when a pedestrian is crossing the road and the driver does not see or pay attention to him. Current ADAS warn the driver or apply the brakes shortly before the collision, but in some situations the collision cannot be fully avoided because most systems react only when the pedestrian is already...

We present a method which is able to adapt from a generic facial representation to a person-specific model of a face. It is referred to as Adaptive Constrained Polynomial Trees (ACPT). Especially in vehicle driving scenarios, special assumptions can be made. A generic facial representation which is able to handle many different persons can be speci...

For future driver assistance systems and autonomous vehicles, the road course, i.e., the width and shape of the driving path, is an important source of information. In this paper, we introduce a new hierarchical two-stage approach for learning the spatial layout of road scenes. In the first stage, base classifiers analyze the local visual propertie...

Before taking a decision, a driver anticipates the future behavior of other traffic participants. However, if a driver is inattentive or overloaded, he may fail to consider relevant information. This can lead to bad decisions and potentially result in an accident. A computational system that is designed to anticipate other traffic participants' beh...

The topic of motivation is a crucial issue for various human-robot interaction (HRI) scenarios. Interactional aspects of motivation can be studied in human-human interaction (HHI) and build the basis for modeling a robot's interactional conduct. Using an ethnographic approach we explored the factors relevant in the formation of motivation-relevant...

Understanding the effects of Socially Assistive Robots (SAR) on human's task performance is crucial for designing powerful assistive systems. A variety of interaction design questions have to be taken into account in order to implement SAR. We present the results of a case-control study (no robot present vs. robot giving generic motivational feedba...

Assigning the ego-vehicle to a lane is not only ben-eficial for navigation but will be an essential element in future Advanced Driver Assistance Systems. This paper describes an approach for ego-lane index estimation using only a monocular camera and no additional sensing equipment like, e.g., the typically employed GPS and Inertial Measurement Uni...

To drive safely, a good driver will observe his surroundings, anticipate the actions of other traffic participants and then decide for a maneuver. But if a driver is inattentive or overloaded he may fail to include some relevant information. This can than lead to wrong decisions and potentially result in an accident. In order to assist a driver in...

The recognition of traffic signs in many state-of-the-art driver assistance systems is performed by statistical pattern classification methods. Traffic signs in European countries share many similarities but also vary in colour, size, and depicted symbols, making it hard to obtain one general classifier with good performance in all countries. Train...

In order to support driver assistance systems in unconstrained environments, we propose to extend local appearance-based road classification with a spatial feature generation and classification. Therefore, a hierarchical approach consisting of multiple low level base classifiers, the novel spatial feature generation, as well as a final road terrain...

We present a system able to predict the future left lane change behavior of the ego-vehicle on highway, depending on detected information about the preceding vehicles. Our system learns the mapping between the current perceived scene (information about the ego-vehicle and the preceding vehicles) and the future driving behavior of the ego-vehicle. T...

Today, many vehicles are equipped with Advanced Driver Assistance Systems (ADAS) to warn the driver about the potential danger of a scene, but in some situations the warning is not early enough to avoid an accident. A solution for preparing the driver and giving him the time to react to such dangerous events is to predict the behavior of other traf...

Users draw on four sources to judge a robot's competence: (1) the robot's voice, (2) physical appearance of and (3) the interaction experience with the robot but also (4) the relationship between the robot's physical appearance and its conduct. Furthermore, most approaches in social robotics have an outcome-oriented focus and thus use questionnaire...

In this study we introduce a robust and accurate method for spatio-temporal 3D pose esti-mation and tracking of human body parts. The Shape Flow (SF) algorithm, a top-down spatio-temporal 3D pose estimation method, is integrated into a tracking system. Based on the example of tracking the human hand-forearm limb it is shown that the usage of two SF...

Socially assistive robots (SARs) contribute to task success by supporting humans through verbal information and guidance, while reducing task load through social mechanisms, e.g., human-intuitive mimics and gestures. In this paper, we present two novel interaction scenarios using such robots that could lead to assistance on long-term missions in sp...

Camera calibration is a process of optimizing the camera parameters. This paper describes an evaluation of different stochastic and heuristic estimators for cost function minimization used in camera calibration. The optimization algorithm is a standard gradient walk on the epipolar-constraint. The results show estimators work similar on the given s...

We present a system able to predict the future behavior of the ego-vehicle in an inner-city environment. Our system learns the mapping between the current perceived scene (information about the ego-vehicle and the preceding vehicle, as well as information about the possible traffic lights) and the future driving behavior of the ego-vehicle. We impr...

Traffic signs in Western European countries share many similarities but also can vary in colour, size, and depicted symbols. Statistical pattern classification methods are used for the automatic recognition of traffic signs in state-of-the-art driver assistance systems. Training a classifier separately for each country requires a huge amount of tra...

We present a flexible and scalable architecture that can learn to predict the future behavior of a vehicle in inner-city traffic. While behavior prediction studies have mainly been focusing on lane change events on highways, we apply our approach to a simple inner-city scenario: approaching a traffic light. Our system employs dynamic information ab...

In this paper a novel approach for road detection with a monocular camera is introduced. We propose a two step approach, combining a patch-based segmentation with additional boundary detection. We use Slow Feature Analysis (SFA) which leads to improved appearance descriptors for road and non-road parts on patch level. From the slow features a low o...

In this study we describe a method for 3D trajectory based recognition of and discrimination between different working actions in an industrial environment. A motion-attributed 3D point cloud represents the scene based on images of a small-baseline trinocular camera system. A two-stage mean-shift algorithm is used for detection and 3D tracking of a...

The recognition and prediction of intersection situations and an accompanying threat assessment are an indispensable skill of future driver assistance systems. This study focuses on the recognition of situations involving two vehicles at intersections. For each vehicle, a set of possible future motion trajectories is estimated and rated based on a...

It is still an open question how preliminary visual reflexes can be structured by auditory and visual modalities in order to recognize objects. Therefore, we propose a new method for a controlling strategy for an active vision system that learns to focus on relevant multi modal aspects of the environment. The method is bootstrapped by a bottom up v...

This study introduces an approach to model-based 3D pose estimation and instantaneous motion analysis of the human hand-forearm limb in the application context of safe human-robot interaction. 3D pose estimation is performed using two approaches: The Multiocular Contracting Curve Density (MOCCD) algorithm is a top-down technique based on pixel stat...

In this paper we describe an efficient but detailed new approach to analyze complex dynamic scenes directly in 3D. The arising information is important for mobile robots to solve tasks in the area of household robotics. In our work a mobile robot builds an articulated scene model by observing the environment in the visual field or rather in the so-...

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1-2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained...

In this paper we present a new system for a mobile robot to generate an articulated scene model by analyzing complex dynamic 3D scenes. The system extracts essential knowledge about the foreground, like moving persons, and the background, which consists of all visible static scene parts. In contrast to other 3D reconstruction approaches, we suggest...

The recognition and prediction of situations is an indispensable skill of future driver assistance systems. This study focuses on the recognition of situations involving two vehicles at intersections. For each vehicle, a set of possible future motion trajectories is estimated and rated based on a motion database for a time interval of 2-4 seconds a...

Since manual labelling of huge data sets is costly and time-consuming, we pro-pose a framework for iterative confidence-based self-learning of classifiers which autonomously extends its knowledge gained based on a small amount of initial, man-ually labelled training samples towards increasingly different representatives of the regarded pattern clas...

Imitating the facial expressions of another person is a meaningful signal within interpersonal communication. Providing a robot with the capability of imitating the face of an interactant marks a first step towards implementing a communication model of mimicry. In this paper, we present a novel approach to facial expression imitation which does not...

In this paper we introduce a method to combine bottom-up saliency, texture descriptors and top-down attention to direct the focus on arbitrary objects in a human-robot interaction scenario. In a usual bottom-up attention process interesting features are extracted out of the image to focus on salient regions. We expand this approach with a top-down...

This study introduces an approach to three-dimensional vehicle pose estimation using a stereo camera system. After computation of stereo and optical flow on the investigated scene, a four-dimensional clustering approach separates the static from the moving objects in the scene. The iterative closest point algorithm (ICP) estimates the vehicle pose...

Future driver assistance systems will have to cope with complex traffic situations, especially in the road crossing scenario. To detect potentially hazardous situations as early as possible, it is therefore desirable to know the position and motion of the ego-vehicle and vehicles around it for several seconds in advance. For this purpose, we propos...

Future driver assistance systems will have to cope with complex traffic situations, espe- cially at intersections. To detect potentially hazardous situations as early as possible, it is therefore desirable to know the position and motion of oncoming vehicles for several sec- onds in advance. For this purpose, we propose a combined approach that tra...

Traditional visitor guidance often suffers from the representational gap between 2D map representations and the real-world. Therefore, we propose a robotic information system that exploits its physical embodiment to present a readily interpretable interface for visitor guidance. Similar to human receptionists, it offers a familiar point of referenc...

The subcellular localisation of proteins in intact living cells is an important means for gaining information about protein functions. Even dynamic processes can be captured, which can barely be predicted based on amino acid sequences. Besides increasing our knowledge about intracellular processes, this information facilitates the development of in...

The automatic subcellular localisation of proteins in living cells is a critical step in determining their function. The evaluation of fluorescence images constitutes a common method of localising these proteins. For this, additional knowledge about the position of the considered cells within an image is required. In an automated system, it is adva...

In order to localise tagged proteins in living cells, the surrounding cells must be recognised first. Based on previous work regarding cell recognition in bright-field images, we propose an approach to the automated recognition of unstained live Drosophila cells, which are of high biological relevance. In order to achieve this goal, the original me...

The subcellular localisation of proteins in living cells is a crucial means for the determination of their function. We propose an approach to realise such a protein localisation based on microscope images. In order to reach this goal, appropriate features are selected. Then, the initial feature set is optimised by a genetic algorithm. The actual c...

The subcellular localisation of proteins in living cells is an important step to determine their function. A common method is the evaluation of fluorescence images. The position of marked proteins, visible as bright spots, enables conclusions concerning their function. In order to determine the subcellular localisation, it is crucial to know the ex...

The automatic subcellular localisation of proteins in living cells is a critical step to determine their function. The evaluation of fluorescence images constitutes a common method of localising these proteins. For this, additional knowledge about the position of the considered cells within an image is required. In an automated system, it is advant...

In diesem Beitrag werden drei Verfahren zur automatischen Segmentierung von ungefärbten, lebenden Zellen in Hellfeld-Bildern verglichen. Auf der Grundlage dieser Segmente soll in Zukunft eine Lokalisation subzellulärer Strukturen in parallel dazu aufgenommenen Fluoreszenzbildern stattfinden. Neben der Wasserscheidentransformation wurden zwei eigene...

A common approach to get clues about the potential function of proteins is the analysis of their (sub)cellulare location(s). Various appoaches to realise such localisations have been proposed recently. Such approaches require a recognition of single cells, which is performed either manually (1) or by using thresholds on images with specially staine...

Unlabelled: We present a method for automatic test case generation for protein-protein docking. A consensus-type approach is proposed processing the whole PDB and classifying protein structures into complexes and unbound proteins by combining information from three different approaches (current PDB-at-a-glance classification, search of complexes b...

In diesem Beitrag wird ein Verfahren zur automatischen Lokalisation und Segmentierung ungefärbter Zellen in Hellfeld-Mikroskopbildern vorgestellt. Hierfür erfolgt insbesondere eine Nutzung von aktiven Konturen und morphologischen Operatoren, da diese Techniken eine Einbeziehung von Vorwissen über den Zellaufbau und die Zellform zulassen und somit A...

IntroductionA docking system usually consists of three main parts, the representation of the molecules (e.g. proteins), conformationalspace sampling and last but not least ranking of potential solutions. Whereas the first two parts can be handled by severaldifferent methods the ranking of docking hypotheses is still not satisfactory.The main proble...

We describe a method to model pronunciation variation for ASR in a data-driven way, namely by use of automatically derived acoustic subword units. The inventory of units is designed so as to produce maximal separable pronunciation variants of words while at the same time only the most important variants for the particular application are trained. I...

Robust speech understanding is an important prerequisite for intelligent human-robotinteraction. This paper presents an approach for an adept analysis of speech in a multi-modal dialogue system. In this paper, we provide an analysis of spontaneous speech dialogues between humans and autonomous mobile service robots. Based on this analysis, we are d...

Although structural approaches to pattern recognition are not as popular anymore as they were some 30 years ago, they still provide reasonable solutions for certain recognition problems. This paper demonstrates that recognizing mechanical assemblies is among these problems. We will present and evaluate a framework for visual assembly recognition th...

The interaction of image and speech processing is a crucial property of multimedia systems. Classical systems using inferences on pure qualitative high level descriptions miss a lot of information when concerned with erroneous, vague, or incomplete data. We propose a new architecture that integrates various levels of processing by using multiple re...

This paper describes image processing methods for automatic spotted microarray image analysis. Automatic gridding is important to achieve constant data quality and is, therefore, especially interesting for large-scale experiments as well as for integration of microarray expression data from different sources. We propose a Markov random field (MRF)...

DNA microarray hybridisation is a popular high throughput technique in academic as well as industrial functional genomics research. In this paper we present a new approach to automatic grid segmentation of the raw fluorescence microarray images by Markov Random Field (MRF) techniques. The main objectives are applicability to various types of array...

Introduction Microarray hybridisation experiments are used to measure concentrations of many different nucleic acid sequences in biological samples in parallel. One of the most important applications is gene expression analysis which can give insight into an organisms metabolism and its regulation. In brief, a microarray gene expression experiment...

We give an overview of the ELMAR Docking System. Using a distributed modular and optionally parallel architecture results can be obtained within a few minutes. ELMAR incorporates protein flexibility obtained through statistics and force field calculation. Using a fast correlation technique steric clash penalties are weighted according to the possib...

Automatic understanding of multi-modal input is the central topic in modern human Computer interfaces. But the basic questions about how the interpretations provided by different modalities can be connected in a universal and robust manner is still an open problem. The most intuitive input modalities, speech perception and vision, can only be corre...

In automatic speech recognition complex state spaces are searched during the recognition process. By limiting these search spaces the computation time can be reduced, but unfortunately the recognition rate mostly decreases, too. However, especially for time-critical recognition tasks a search-space pruning is necessary. Therefore, we developed a dy...

Introduction A docking system usually consists of three main parts, the representation of the molecules (e.g. proteins), conformational space sampling and last but not least ranking of potential solutions. Whereas the first two parts can be handled by several different methods the ranking of docking hypotheses is still not satisfactory. The main pr...

This article presents the speech understanding and dialog system EVAR. All levels of linguistic knowledge are used both to control the analysis process and for the interpretation of an utterance. All kinds of knowledge are integrated in a homogeneous knowledge base.

Although structural approaches to pattern recognition are not as popular anymore as they were some 30 years ago, they still provide reasonable solutions for certain recognition problems. This paper demonstrates that recognizing mechanical assemblies is among these problems. We will present and evaluate a framework for visual assembly recognition th...

In recent years numerous approaches to automatic reasoning about mechanical assemblies have been published. CAD techniques, graph-based approaches and semantic methods to model assembled objects were proposed. However, all these methods are difficult to employ for visually supervising assembly processes since they lack the flexibility, generality o...

The representation of knowledge about assembled objects is a fundamental requirement, if computer systems are used to supervise construction tasks. Depending on the complexity of the task and the characteristics of the objects forming its basis the process of modeling may be burdensome. In this paper we introduce a method to model classes of assemb...

Using a computer vision system to supervise a construction task requires the reliable recognition of assembled objects. A particularly reliable object recognition is possible if a comprehensive representation of knowledge about assembled objects is provided. In this paper, the authors present a cyclic semantic network which models all possible asse...