Andreas Birk

• Dr. rer. nat.
• Professor at Constructor University

We present Spectral Registration with Multilayer Resampling (SRMR) as a 6 Degrees Of Freedom (DOF) registration method for noisy 3D data with partial overlap. The algorithm is based on decoupling 3D rotation from 3D translation by a corresponding resampling process of the spectral magnitude of a 3D Fast Fourier Transform (FFT) calculation on discre...

We present a robot-pose-registration algorithm, which is entirely based on large planar-surface patches extracted from point clouds sampled from a three-dimensional (3-D) sensor. This approach offers an alternative to the traditional point-to-point iterative-closest-point (ICP) algorithm, its point-to-plane variant, as well as newer grid-based algo...

A multi-robot system can be highly beneficial for exploration, which is a core robotics task. Application domains include, for example, surveillance, reconnaissance, planetary exploration or rescue missions. When using a team of robots, the overall performance can be much faster and more robust. In this article, an approach to multi-robot explorati...

Instead of establishing trust through defining compliancy-based standards like protocols augmented by cryptographic methods, it is shown that trust can emerge as a self-organizing phenomenon in a complex dynamical system. It is assumed that trust can be modeled on basis of an intrinsic property called trustworthyness in every individual i, which is...

We introduce SmileyNet, a novel neural network with psychic abilities. It is inspired by the fact that a positive mood can lead to improved cognitive capabilities including classification tasks. The network is hence presented in a first phase with smileys and an encouraging loss function is defined to bias it into a good mood. SmileyNet is then use...

An open-source software framework is presented that allows real-time underwater mapping with popular marine robotics components, namely a BlueRobotics BlueROV2 with its standard Ping360 Mechanical Scanning Sonar (MSS) and A50 Doppler Velocity Log (DVL), which are low-cost devices for their respective types - if not even the most affordable ones on...

Mechanical Scanning Sonars (MSS) are popular underwater sensors for Unmanned Underwater Vehicles (UUV) due to their low cost, small size, and low power consumption. But due to their simplicity, there are also many research challenges related to their usage. Unfortunately, there is also a lack of data with ground truth UUV localization. We provide M...

Results from the exploration of the flooded basement of the U-Boot Bunker Valentin are presented. The work is done in the context of the digitization of the bunker, which was constructed during World-War II using substantial amounts of forced labor and which is now a memorial. The presented research deals with walls in the basement for which it is...

Underwater mapping based on the registration of data from a mechanical scanning sonar (MSS) is presented. The specific use-case is the exploration of the flooded basement of the U-Boot Bunker Valentin in the context of the digitization of this World-War-II memorial. The visibility in the basement, which is inaccessible for divers due to safety reas...

In this paper, we introduce Fourier-SOFT 2D (FS2D) as a new robust registration method. FS2D operates in the frequency domain where it exploits the well-known decoupling of rotation and translation. The challenging part of determining the rotation parameter is solved here based on a projection of the Fourier magnitude on a sphere and the SO(3) Four...

Mechanical Scanning Sonars (MSS) are popular devices for Unmanned Underwater Vehicles (UUV), i.e., Remotely Operated Vehicles (ROV) and Autonomous Underwater Vehicles (AUV), as they function under low visibility conditions and over extended ranges. They are comparatively low-cost and easy to integrate. But they require motion-compensation due to th...

Purpose of Review This review provides an overview of the current state of the art in Underwater Human-Robot Interaction (U-HRI), which is an area that is quite different from standard Human-Robot Interaction (HRI). This is due to several reasons. First of all, there are the particular properties of water as a medium, e.g., the strong attenuation o...

This paper presents the curriculum development of the Robotics and Intelligent Systems Bachelor of Science program at Jacobs University, one of the very few bachelor robotics-focused programs in Germany. After highlighting the quality management measures and feedback received on the older Intelligent Mobile Systems, the new curriculum is highlighte...

Observations from the online teaching of a robotics class during the CoViD-19 pandemic caused by SARS-CoV-2 also known as the Corona virus are presented. The online teaching of the course lead to an unexpected increase in the grade performance of the students – despite the fact that the exams were at the same level of difficulty as in the years bef...

The COVID-19 pandemic had profound effects on almost all aspects of daily life, including education, where it led to a rapid and massive switch to online teaching. There is evidence that the responses to the pandemic will have lasting effects in higher education and that they have accelerated the trend that (elements of) online education will be th...

Signal processing of thermal images that are autonomously collected by a drone detects people in densely occluded forests.

We present a fully integrated approach to Underwater Human Robot Interaction (U-HRI) in form of a front-end for gesture recognition combined with a back-end with a full language interpreter. The gesture-based language is derived from the existing standard gestures for communication between human divers. It enables a diver to issue single commands a...

Conceptual knowledge about objects is essential for humans, as well as for animals, to interact with their environment. On this basis, the objects can be understood as tools, a selection process can be implemented and their usage can be planned in order to achieve a specific goal. The conceptual knowledge, in this case, is primarily concerned about...

Fourier-Mellin-SOFT (FMS) is a rigid 3D registration method, which allows the robust registration of 3 degrees-of-freedom (dof) rotation, 1-dof scale, and 3-dof translation between scans on discrete grids. FMS is based on a spectral decomposition of these 7-dof. This complete spectral representation of the input data enables an adaption to certain...

The article describes observations from the online teaching of a robotics class during the COVID-19 pandemic caused by SARS-CoV-2, also known as the coronavirus. The changes in the course structure and in the provided material lead to an unexpected increase in the grade performance of the students. The article provides a description and an analysis...

Sonars are essential for underwater sensing as they can operate over extended ranges and in poor visibility conditions. The use of a synthetic aperture is a popular approach to increase the resolution of sonars, i.e., the sonar with its N transducers is positioned at k places to generate a virtual sensor with kN transducers. The state of the art fo...

High-end laser range-finders provide accurate 3D data over long ranges. But their scans are inhomogeneous, i.e., the environment is non-uniformly sampled, as there is denser data in the near range than in the far range. Furthermore, the generation of a scan is time-consuming. Thus, it is desirable to cover an area by as few scans as possible, i.e.,...

Object shape is a key cue that contributes to the semantic understanding of objects. In this work we focus on the categorization of real-world object point clouds to particular shape types. Therein surface description and representation of object shape structure have significant influence on shape categorization accuracy, when dealing with real-wor...

Continuous System Integration and Validation is an increasingly important factor for an efficient system development process. In particular, for underwater projects involving semi- to fully-autonomous robotic systems since they progressively become more complex, need to perform under more challenging environmental conditions and execute more intric...

Underwater robot interventions require a high level of safety and reliability. A major challenge to address is a robust and accurate acquisition of localization estimates, as it is a prerequisite to enable more complex tasks, e.g. floating manipulation and mapping. State-of-the-art navigation in commercial operations, such as oil& gas production (O...

Tool-use applications in robotics require conceptual knowledge about objects for informed decision making and object interactions. State-of-the-art methods employ hand-crafted symbolic knowledge which is defined from a human perspective and grounded into sensory data afterwards. However, due to different sensing and acting capabilities of robots, t...

Underwater robot interventions require a high level of safety and reliability. A major challenge to address is a robust and accurate acquisition of localization estimates, as it is a prerequisite to enable more complex tasks, e.g. floating manipulation and mapping. State-of-the-art navigation in commercial operations, such as oil & gas production (...

In recent years, the number of maritime exploration and exploitation activities has rapidly increased, and with it the necessity to perform more complex tasks underwater, e.g., floating manipulation and mapping with Remote Operated Vehicles (ROVs). The first step to perform these activities in a reliable manner, is to obtain an accurate robot local...

A stereo vision system for deep-sea operations is presented. The system consists of cameras in pressure bottles, which are daisy-chained to a computer bottle. The system has substantial computation power for on-board stereo processing as well as for further computer vision methods to support autonomous intelligent functions, e.g., object recognitio...

Intervention missions, that is, underwater manipulation tasks, for example, in the context of oil-&-gas production, require a high amount of precise, robust navigation. In this article, we describe the use of an advanced vision system suited for deep-sea operations, which in combination with artificial markers on target structures like oil-&-gas pr...

In this article, we present a novel underwater dataset collected from several field trials within the EU FP7 project “Cognitive autonomous diving buddy (CADDY)”, where an Autonomous Underwater Vehicle (AUV) was used to interact with divers and monitor their activities. To our knowledge, this is one of the first efforts to collect a large public dat...

An unsupervised shape analysis is proposed to learn concepts reflecting shape commonalities. Our approach is two-fold: i) a spatial topology analysis of point cloud segment constellations within objects is used in which constellations are decomposed and described in a hierarchical and symbolic manner. ii) A topology analysis of the description spac...

Spectral methods such as the improved Fourier Mellin Invariant (iFMI) transform have proved faster, more robust and accurate than feature based methods on image registration. However, iFMI is restricted to work only when the camera moves in 2D space and has not been applied on omni-cameras images so far. In this work, we extend the iFMI method and...

We propose a robust gesture-based communication pipeline for divers to instruct an Autonomous Underwater Vehicle (AUV) to assist them in performing high-risk tasks and helping in case of emergency. A gesture communication language (CADDIAN) is developed, based on consolidated and standardized diver gestures, including an alphabet, syntax and semant...

Underwater manipulation is a challenging problem. The state-of-the-art technology is dominated by remotely operated vehicles (ROVs). ROV operations typically require an offshore crew consisting of, at minimum, an intendant (or supervisor), an operator, and a navigator. This crew must often be doubled or even tripled due to work shifts. In addition,...

Robots require knowledge about objects in order to efficiently perform various household tasks involving objects. The existing knowledge bases for robots acquire symbolic knowledge about objects from manually-coded external common sense knowledge bases such as ConceptNet, Word-Net etc. The problem with such approaches is the discrepancy between hum...

We propose a robust gesture-based communication pipeline for divers to instruct an Autonomous Underwater Vehicle (AUV) to assist them in performing high-risk tasks and helping in case of emergency. A gesture communication language (CADDIAN) is developed, based on consolidated and standardized diver gestures, including an alphabet, syntax and semant...

Deep-sea robot operations demand a high level of safety, efficiency and reliability. As a consequence, measures within the development stage have to be implemented to extensively evaluate and benchmark system components ranging from data acquisition, perception and localization to control. We present an approach based on high-fidelity simulation th...

A topological shape analysis is proposed and utilized to learn concepts that reflect shape commonalities. Our approach is two-fold: i) a spatial topology analysis of point cloud segment constellations within objects. Therein constellations are decomposed and described in an hierarchical manner – from single segments to segment groups until a single...

Robots require knowledge about objects in order to efficiently perform various household tasks involving objects. The existing knowledge bases for robots acquire symbolic knowledge about objects from manually-coded external common sense knowledge bases such as ConceptNet, WordNet etc. The problem with such approaches is the discrepancy between huma...

Deep-sea robotic operations require a high level of safety, efficiency and reliability. In the development stage of such systems, measures have to be taken into account to validate performance in order to assess the achievement of these requirements. In the context of continuous system integration, we proposed a simulation-in-the-loop framework foc...

In this article we present a novel underwater dataset collected from several field trials within the EU FP7 project "Cognitive autonomous diving buddy (CADDY)", where an Autonomous Underwater Vehicle (AUV) was used to interact with divers and monitor their activities. To our knowledge, this is one of the first efforts to collect a large dataset in...

Underwater images suffer from extremely unfavourable conditions. Light is heavily attenuated and scattered. Attenuation creates change in hue, scattering causes so called veiling light. General state of the art methods for enhancing image quality are either unreliable or cannot be easily used in underwater operations. On the other hand there is a w...

Fourier Mellin SOFT (FMS) as a novel method for global registration of 3D data is presented. It determines the seven degrees of freedom (7-DoF) transformation, i.e., the 6-DoF rigid motion parameters plus 1-DoF scale, between two scans, i.e., two noisy, only partially overlapping views on objects or scenes. It is based on a sequence of the 3D Fouri...

In this paper, an approach for the classification and localization of geometric shapes, e.g., man-made objects or different types of geological features, in sonar images is presented. It is applied to a concrete application case, namely the detection and classification of naval mines. The approach consists of three steps. In the first step, the son...

Object shape is a key cue that contributes to the semantic understanding of objects. In this work we focus on the categorization of real-world object point clouds to particular shape types. Therein surface description and representation of object shape structure have significant influence on shape categorization accuracy, when dealing with real-wor...

A topological shape analysis is proposed and utilized to learn concepts that reflect shape commonalities. Our approach is two-fold: i) a spatial topology analysis of point cloud segment constellations within objects. Therein constellations are decomposed and described in an hierarchical manner - from single segments to segment groups until a single...

Deep-sea robot operations demand a high level of safety, efficiency and reliability. As a consequence, measures within the development stage have to be implemented to extensively evaluate and benchmark system components ranging from data acquisition, perception and localization to control. We present an approach based on high-fidelity simulation th...

Superquadric surfaces play an important role in many different research fields due to their ability to model a variety of shapes with a small number of parameters. One of their core applications is the estimation of object shape characteristics from a set of discrete samples from the surface of an object. However, the corresponding optimization pro...

Scenario: A rescue team needs to cross a partially damaged bridge in a flooded area. It is unknown whether the construction is still able to carry a vehicle. Assessing the construction's integrity can be accomplished by the analysis of the bridge's eigenfrequencies. Rather than using proprietary expensive Vibration Measurement Systems (VMS) we prop...

A significant amount of cargo worldwide is transported in sacks and bags e.g. wheat, rice, coffee and cacao beans, etc. Despite being very strenuous and the health risks involved, the handling of sacks in logistics is predominantly done through manual labor. Hence, the automation of tasks such as cargo unloading from shipping containers is of high...

ROV operations are expensive and there is an urge to improve on their efficiency and speed. The EU project " Effective Dexterous ROV Operations in Presence of Communications Latencies (DexROV) " proposes a solution to those needs by developing a set of hardware and software tools to support the teleoperation of ROVs over a satellite link, i.e., fro...

Underwater images suffer from extremely unfavourable conditions. Light is heavily attenuated and scattered. Attenuation creates change in hue, scattering causes so called veiling light. General state of the art methods for enhancing image quality are either unreliable or cannot be easily used in underwater operations. On the other hand there is a w...

Vision methods are very commonly used in underwater robotics. However, the cameras must be enclosed in a sealed housing for which flat-pane windows are the most economical and hence most popular solution. But they cause non-trivial refraction-based distortions at the air-glass-water interface. Recently, a new model for underwater cameras calibratio...

Within the EU FP7 project " Cognitive autonomous diving buddy (CADDY) " , work has been made to assist and monitor divers through Autonomous Underwater Vehicles (AUVs) during their long underwater expeditions. To achieve this goal, one milestone is to give the AUV the capability to track the diver's whereabouts at all times. Inertial sensors are mo...

Deep-sea operations of remotely-operated vehicles (ROV) need robust testing and deployment strategies beyond the traditional pre-deployment validation on real hardware. Seamless integration of simulated components into the validation pipeline allows for rapid development of components and validation under controlled conditions. We describe the bene...

Bridges are fragile transport infrastructure elements which require special attention in evacuation planning within flood disaster scenarios. In this paper we present work on a bridge inspection procedure with a marine vehicle in which a bridge statics analysis expert assesses the condition of the bridge from a safe operating station. Given bridge...

The performance of an automatic target recognition (ATR) system in the context of naval mine detection is severely affected by the underwater environment. Especially in regions with the presence of sand ripples or mine-sized stones the number of false alarms can become unacceptable high, if the detection algorithm does not account for the type of s...

The calibration and refraction correction process for underwater cameras with flat-pane interfaces is presented that is very easy and convenient to use in real world applications while yielding very accurate results. The correction is derived from an analysis of the axial camera model for underwater cameras, which is among others computationally ha...

"CADDY - Cognitive Autonomous Diving Buddy" is an FP7 project that that is devoted to developing a cognitive underwater robotic system that will help divers during their activities in this hazardous environment. The envisioned resulting system will play a threefold role similar to those that a human buddy diver should have: buddy "observer", buddy...

This article gives an overview of simultaneous localization and mapping (SLAM), that is, probabilistic methods to generate a 2D or 3D map of unknown areas under imperfect localization. The article provides a survey of the theoretical basis of SLAM as well as the core background information about the underlying techniques for implementing actual SLA...

We present a damage-aware planning approach which determines the best sequence to manipulate a number of objects in a scene. This works on task-planning level, abstracts from motion planning and anticipates the dynamics of the scene using a physics simulation. Instead of avoiding interaction with the environment, we take unintended motion of other...

An extension based on attention and anticipation of a robot vision pipeline for object recognition in RGBD images from low-cost sensors like MS Kinect or ASUS Xtion is presented. This work originated in research on an industrial application scenario, namely shipping-container unloading, but it is applicable to advanced manipulation tasks in unstruc...

The operation of a ROV requires significant off-shore dedicated manpower to handle and operate the robotic platform. In order to reduce the burden of operations, DexROV proposes to work out more cost effective and time efficient ROV operations, where manned support is in a large extent delocalized onshore (i.e. from a ROV control center), possibly...

We present a framework to generate watertight mesh representations in an unsupervised manner from noisy point clouds of complex, heterogeneous objects with free-form surfaces. The resulting meshes are ready to use in applications like kinematics and dynamics simulation where watertightness and fast processing are the main quality criteria. This wor...

This article discusses the scientifically and industrially important problem of automating the process of unloading goods from standard shipping containers. We outline some of the challenges barring further adoption of robotic solutions to this problem: ranging from handling a vast variety of shapes, sizes, weights, appearance and packing arrangeme...

Divers operate in harsh and poorlymonitored environments, inwhich the slightest unexpected disturbance, technical malfunction, or lack of attention can have catastrophic consequences. Motivated by these considerations, the “CADDY—Cognitive Autonomous Diving Buddy” FP7 project sets forth the main goal of developing a cooperative autonomous underwate...

The MORPH project (FP 7, 2012–2016) is aimed at developing efficient methods and tools to map the underwater environment in situations that are not easily addressed by current technology. Namely, the missions that are of interest are those that involve underwater surveying and marine habitat mapping of rugged terrain and structures with full 3D com...

We present a hierarchical graph-based approach for unknown object discovery in RGB-D point clouds captured with a Kinect-like sensor from unstructured scenes. A two-step approach is proposed which first extracts meaningful regions from an input scene through over-segmentation. Secondly, a procedure is introduced to detect compositions of such regio...

Autonomous robots in unstructured and dynamically changing retail environments have to master complex perception, knowledgeprocessing, and manipulation tasks. To enable them to act competently, we propose a framework based on three core components: (o) a knowledge-enabled perception system, capable of combining diverse information sources to cope w...

Autonomous robots in unstructured and dynamically changing retail environments have to master complex perception, knowledge processing, and manipulation tasks. To enable them to act competently, we propose a framework based on three core components: (o) a knowledge-enabled perception system, capable of combining diverse information sources to cope...

Robust and accurate navigation of underwater vehicles is an essential cornerstone for mission success. Integrative techniques, e.g., using inertial navigation or doppler velocity logs (DVL), can show significant drift. Even absolute input, such as GNSS input or magnetometers, can either be biased by surrounding structures or very noisy. For good na...

Autonomous underwater vehicles (AUVs) often perform high-resolution survey missions. Such missions are often planned on low resolution bathymetry maps using offline coverage planning methods, e.g., using a standard lawnmower trajectory that is adapted to the coarse-resolution representation of the terrain. We present in this paper an approach to ad...

We present a classification framework to detect underwater terrain deformations commonly caused by marine species in real time during AUV surveys. The method uses two classifiers in a hierarchical manner to overcome the fact that there are no substantial datasets in the literature about underwater terrain and the creation of one would involve a gre...

We present a damage-aware planning approach which determines the best sequence to manipulate a number of objects in a scene. This works on task-planning level, abstracts from motion planning and anticipates the dynamics of the scene using a physics simulation. Instead of avoiding interaction with the environment, we take unintended motion of other...

We present a framework to generate watertight mesh representations in an unsupervised manner from noisy point clouds of complex, heterogeneous objects with free-form surfaces. The resulting meshes are ready to use in applications like kinematics and dynamics simulation where watertightness and fast processing are the main quality criteria. This wor...

"CADDY - Cognitive Autonomous Diving Buddy" is an FP7 project that started in January 2014. Seven partner institutions have joined their efforts towards developing a cognitive underwater robotic system that will help divers during their activities in this hazardous environment. The resulting system will play a threefold role similar to those that a...

The MORPH project (FP 7, 2012-2016) advances a novel concept of an underwater robotic system composed of a number of spatially separated mobile robot-modules, carrying complementary resources. Instead of being physically coupled, the modules are connected via communication links that allow a limited flow of information among them. Without rigid lin...

Underwater inspection and maintenance (e.g. in the oil & gas industry) are demanding and costly activities for which ROV based setups are often deployed in addition or in substitution to deep divers - contributing to operations risks and costs cutting. However the operation of a ROV requires significant off-shore dedicated manpower to handle and op...

For missions in Safety, Security, and Rescue Robotics (SSRR) maps are often a core deliverable. Hence it is of high interest to assess the quality of maps in a simple and efficient way. Since SSRR is mostly taking place in unstructured environments, 3D mapping has become more and more important. Here a method to evaluate the quality of 3D maps is p...

Mapping is an important task for mobile robots. The assessment of the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. Here, a new approach for the evaluation of 2D grid maps is presented. This structure-based method makes use of a topology graph, i.e., a topological representation that includes...

Research in Graph-based Simultaneous Localization and Mapping has experienced a recent trend towards robust methods. These methods take the combinatorial aspect of data association into account by allowing decisions of the graph topology to be made during optimization. The Generalized Graph Simultaneous Localization and Mapping framework presented...

This paper proposes a new method for segmenting and classifying seamines on Synthetic Aperture Sonar (SAS) side scan images. The method uses an active contours approach and superellipse a-priori knowledge to segment the image in object, object-shadow and background areas. In contrast to other methods using superellipse constraints, the shape prior...

One fundamental problem in Simultaneous Localization and Mapping is place recognition, used to create loop closures, necessary to improve map quality. Ideally, the sensor data alone should be used to detect loop closures, referred to as appearance-based methods. Such methods usually employ feature descriptors, especially popular in underwater resea...

Given that 3D scan matching is such a central part of the perception pipeline for robots, thorough and large-scale investigations of scan matching performance are still surprisingly few. A crucial part of the scientific method is to perform experiments that can be replicated by other researchers in order to compare different results. In light of th...