Content uploaded by Andreas Birk
Author content
All content in this area was uploaded by Andreas Birk on Dec 26, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Uncertainty estimation for a 6DoF spectral registration method as basis for sonar-based underwater 3D SLAM
Abstract and Figures
An uncertainty estimation method for 6 degree of freedom (6-DoF) spectral registration is introduced here. The underlying 6-DoF registration method based on Phase Only Matched Filtering (POMF) is capable of dealing with very noisy sensor data. It is hence well suited for 3D underwater mapping, where relatively inaccurate sonar imaging devices have to be employed. An uncertainty estimation method is required to use this registration method in a Simultaneous Localization and Mapping (SLAM) framework. To our knowledge, the first such method for 6-DoF spectral registration is presented here. This new uncertainty estimation method treats the POMF results as probability mass functions (PMF). Due to the decoupling in the underlying method, yaw is computed by a one-dimensional POMF leading hence to a 1D PMF; roll and pitch are simultaneously computed and hence encoded in a 2D PMF. Furthermore, a 3D PMF is generated for the translation as it is determined by a 3D POMF. A normal distribution is fitted on each of the PMF to get the uncertainty estimate. The method is experimentally evaluated with simulated as well as real world sonar data. It is shown that it indeed can be used for SLAM, which significantly improves the map quality.
Figures - uploaded by Andreas Birk
Author content
All figure content in this area was uploaded by Andreas Birk
Content may be subject to copyright.
... In [19,20], for example, it is shown that a registration and alignment of sonar scan-data is possible even with severe interference and partial overlap between individual scans. While this pairwise registration-which is the basis for the work presented in this article-is already very accurate, positional information over the entire aperture can even be further improved by Simultaneous Localization and Mapping (SLAM) [21]. ...
... As discussed in the previous sections, the coherent SAS processing of the separate array data requires the knowledge of the underlying spatial transformations within reconstructed sonar images. The idea of using registration of multiple scans from different unknown sensor positions is based on our previous work on registration of noisy data with partial overlap including especially sonar [19][20][21]. This includes especially spectral registration methods [38,39], which are capable of matching scans as an entire unit without a dependency on features within the scan representation. ...
Sonars are essential for underwater sensing as they can operate over extended ranges and in poor visibility conditions. The use of a synthetic aperture is a popular approach to increase the resolution of sonars, i.e., the sonar with its N transducers is positioned at k places to generate a virtual sensor with kN transducers. The state of the art for synthetic aperture sonar (SAS) is strongly coupled to constraints, especially with respect to the trajectory of the placements and the need for good navigation data. In this article, we introduce an approach to SAS using registration of scans from single arrays, i.e., at individual poses of arbitrary trajectories, hence avoiding the need for navigation data of conventional SAS systems. The approach is introduced here for the near field using the coherent phase information of sonar scans. A Delay and Sum (D&S) beamformer (BF) is used, which directly operates on pixel/voxel form on a Cartesian grid supporting the registration. It is shown that this pixel/voxel-based registration and the coherent processing of several scans forming a synthetic aperture yields substantial improvements of the image resolution. The experimental evaluation is done with an advanced simulation tool generating realistic 2D sonar array data, i.e., with simulations of a linear 1D antenna reconstructing 2D images. For the image registration of the raw sonar scans, a robust implementation of a spectral method is presented. Furthermore, analyses with respect to the trajectories of the sensor locations are provided to remedy possible grating lobes due to the gaping positions of the transmitter devices.
... In the underwater domain several applications of visual SLAM have been published [26,23,15,34,6,4,25], however mostly as post-processing and not in the AUV's control loop. This is because the underwater domain is difficult regarding visibility and features and hence the risk of malfunction is high. ...
... Utilizing an AUV to explore an underwater environment has gained popularity over the years. Sonar and stereo estimation for object modeling has been proposed in [31], [32]. Nornes et al. [33] acquired stereo images utilizing an ROV off the coast of Trondheim Harbour, Norway. ...
This paper addresses the robustness problem of visual-inertial state estimation for underwater operations. Underwater robots operating in a challenging environment are required to know their pose at all times. All vision-based localization schemes are prone to failure due to poor visibility conditions, color loss, and lack of features. The proposed approach utilizes a model of the robot's kinematics together with proprioceptive sensors to maintain the pose estimate during visual-inertial odometry (VIO) failures. Furthermore, the trajectories from successful VIO and the ones from the model-driven odometry are integrated in a coherent set that maintains a consistent pose at all times. Health-monitoring tracks the VIO process ensuring timely switches between the two estimators. Finally, loop closure is implemented on the overall trajectory. The resulting framework is a robust estimator switching between model-based and visual-inertial odometry (SM/VIO). Experimental results from numerous deployments of the Aqua2 vehicle demonstrate the robustness of our approach over coral reefs and a shipwreck.
... Singh et al. [26] have suggested how to improve the underwater imaging situation and during the last ten years many works improved 3D reconstruction and mapping using image data sets recorded underwater [21,25,11,6,30,5]. Recent systems consider also physically-motivated post-processing of underwater light propagation issues [4] and demonstrate location recognition on pure visual data sets recorded underwater [17] or for 3D underwater pointcloud registration [23]. ...
This paper addresses visual navigation of autonomous underwater vehicles (AUVs) with and without a given map, where the latter is called Simultaneous Localization and Mapping (SLAM). We summarize the challenges and opportunities in underwater environments that make visual navigation different from land navigation and also briefly survey the current state-of-the-art in this area. Then as a position paper we argue why many of these challenges could be met by a proper modeling of uncertainties in the SLAM representation. This would in particular allow the SLAM algorithm to thoroughly handle the ambiguity between “I see the same feature again.”, “I see a different but similar looking feature.” and “The environment has changed and the feature moved.”.
... In the underwater domain several applications of visual SLAM have been published [26,23,15,34,6,4,25], however mostly as post-processing and not in the AUV's control loop. This is because the underwater domain is difficult regarding visibility and features and hence the risk of malfunction is high. ...
This position paper presents initial thoughts on how some techniques from general robotics can help for autonomous underwater vehicle (AUV) navigation in confined spaces by exploiting in particular the spatial borders and considering information that is not available in open waters. There are natural confined spaces, e.g. caves, as well as artificial ones, e.g. tripods of off-shore wind turbines or underwater oil-and-gas facilities, which make this application interesting. We argue that the common AUV perceptual system with forward looking camera and/or sonar has deficits for measuring structures in the immediate surrounding of the AUV. This surrounding, however, is particularly important in confined spaces where the AUV cannot be seen as a “point in space” but its physical extension needs to be considered. Distant environment features that are observed in the remote sensors can be mapped, but later, when the AUV comes closer and the remote sensors cannot observe them anymore, they might not be directly usable for localization using these sensors. However, we still see the opportunity to make use of them and, moreover, to generate new features by other means. How this can be achieved is the central idea we want to convey here.
... Singh et al. [26] have suggested how to improve the underwater imaging situation and during the last ten years many works improved 3D reconstruction and mapping using image data sets recorded underwater [21,25,11,6,30,5]. Recent systems consider also physically-motivated post-processing of underwater light propagation issues [4] and demonstrate location recognition on pure visual data sets recorded underwater [17] or for 3D underwater pointcloud registration [23]. ...
This paper addresses visual navigation of autonomous underwater vehicles (AUVs) with and without a given map, where the latter is called Simultaneous Localization and Mapping (SLAM). We summarize the challenges and opportunities in underwater environments that make visual navigation different from land navigation and also briefly survey the current state-of-the-art in this area. Then as a position paper we argue why many of these challenges could be met by a proper modeling of uncertainties in the SLAM representation. This would in particular allow the SLAM algorithm to thoroughly handle the ambiguity between “I see the same feature again.”, “I see a different but similar looking feature.” and “The environment has changed and the feature moved.”.
... The perception system makes use of a stereo camera for the 3D data acquisition, online processing of the needed information and its transmission to the control center [20]. Furthermore the ROV is equipped with an AHRS (Attitude and Heading reference System), a DVL (Doppler velocity log) and a USBL (ultra-short baseline) that are concurrently used for its accurate pose estimation [21]. The cognitive engine is split in two parts: on the onshore side it recognizes the actions that the operator wants to perform learning from demonstrations; on the offshore side it reconstructs the motion primitive despite of the non homogeneous communication latency. ...
To enable cooperative task planning and coordination between the human operator and robot teams, new types of interfaces are needed. We present an interactive strategic mission management system (ISMMS) for underwater explorations performed by mixed teams of robots and human investigators that enables cooperative task planning and coordination between the human operator and robot teams. Main goals of the ISMMS are to enable robots to “explain” their intentions, problems, and situation fast and in an intuitive fashion to humans, to allow smooth blending between autonomous behavior and human control, to provide smart interfaces to mandatory external control and to enable adaptive task sharing while being optimized with respect to intuitive usage and interaction measured by behavioral and physiological human data.
In this chapter, a complete processing tool of use in the examination of 3D acoustic sub-bottom images has been described.
The processing chain allows one to separate — from raw data — the image regions representing natural or artificial objects
buried beneath the seafloor. In particular, the devised segmentation process is based on a semi-automatic “seeded” volume
growing approach. The voxel classification is guided by a statistical criterion by fitting current volume histograms with
an adequate probability density function. The segmented object is then analysed to extract measurements about its shape and
pose and to obtain a 3D wirtual representation by VRML modelling. In addition, a pre-processing noise reduction stage and
a multi-resolution data representation based on an octree approach can be applied, if necessary.
In this paper, we address the problem of creating an objective benchmark for evaluating SLAM approaches. We propose a framework
for analyzing the results of a SLAM approach based on a metric for measuring the error of the corrected trajectory. This metric
uses only relative relations between poses and does not rely on a global reference frame. This overcomes serious shortcomings
of approaches using a global reference frame to compute the error. Our method furthermore allows us to compare SLAM approaches
that use different estimation techniques or different sensor modalities since all computations are made based on the corrected
trajectory of the robot.
We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the
robotics community. The relations have been obtained by manually matching laser-range observations to avoid the errors caused
by matching algorithms. Our benchmark framework allows the user to easily analyze and objectively compare different SLAM approaches.
The aquatic realm is ideal for testing autonomous robotic technology. The challenges presented in this environment are numerous due to the highly dynamic nature of the medium. Applications for underwater robotics include the autonomous inspection of coral reef, ships, pipelines, and other environmental assessment programs. In this paper we present current results in using 6DOF entropy minimization SLAM (simultaneous localization and mapping) for creating dense 3D visual maps of underwater environments that are suitable for such applications. The proposed SLAM algorithm exploits dense information coming from a stereo system, and performs robust egomotion estimation and global-rectification following an optimization approach
A novel algorithm for stitching images, which is faster and more robust than standard approaches like the Scale Invariant Feature Transform (SIFT) is presented. The algorithm is particularly suited for Autonomous Underwater Vehicles (AUV), namely for the online generation of photo maps that can be the basis of intelligent onboard functionalities. The photo maps can be generated just based on registration, i.e., without any information about the vehicle's pose or its motion. A new method based on the Fourier-Mellin (FM) transform is presented. It is based on a 2D Fourier transform where the shift between two signals is detected by the phase information. In contrast to previous work using FM, a polar-logarithmic resampling of image information is used to turn rotation and scaling into corresponding phase shift and allow for a registration in one step.
In future developments of AUV systems 3D image processing techniques will gain in importance. The high resolution 3D-Sonar can be considered as the ldquoeyerdquo of the AUV which provides acoustical information utilized for the detection and classification of anomalies, e.g. at ships hulls. Therefore, the investigations presented include the simulation of underwater scenes, object shapes and AUV movements as well as the modelling of the acoustical system performance. The obtained beam signals provide the input to the 3D image formation and shape recognition algorithms. Beside conventional 3D interpolation and voxel image formation techniques we propose a novel image formation approach by exploiting the maximum of each beam signal. Moreover, efficient noise suppression algorithms are developed and validated by employing multi-beam/multi-aspect imaging. Finally, the proposed 3D image formation technique is computationally efficient, robust against noise and allows the application of conventional 2D image processing algorithms.
A visual SLAM system has been implemented and optimised for real-time deployment on an AUV equipped with calibrated stereo cameras. The system incorporates a novel approach to landmark description in which landmarks are local sub maps that consist of a cloud of 3D points and their associated SIFT/SURF descriptors. Landmarks are also sparsely distributed which simplifies and accelerates data association and map updates. In addition to landmark-based localisation the system utilises visual odometry to estimate the pose of the vehicle in 6 degrees of freedom by identifying temporal matches between consecutive local sub maps and computing the motion. Both the extended Kalman filter and unscented Kalman filter have been considered for filtering the observations. The output of the filter is also smoothed using the Rauch-Tung-Striebel (RTS) method to obtain a better alignment of the sequence of local sub maps and to deliver a large-scale 3D acquisition of the surveyed area. Synthetic experiments have been performed using a simulation environment in which ray tracing is used to generate synthetic images for the stereo system.
Surface-patches based 3D mapping in a real world underwater scenario is presented. It is based on a 6 degrees of freedom registration of sonar data. Planar surfaces are fitted into the sonar data and the subsequent registration method maximizes the overall geometric consistency within a search-space to determine correspondences between the planes. This approach has previously only been used on high quality range data from sensors on land robots like laser range finders. It is shown here that the algorithm is also applicable to very noisy, coarse sonar data. The 3D map presented is of a large underwater structure, namely the Lesumer Sperrwerk, a flood gate north of the city of Bremen, Germany. It is generated from 18 scans collected using a Tritech Eclipse sonar.
Environmental change is a growing international concern, calling for the regular monitoring, studying and preserving of detailed information about the evolution of underwater ecosystems. For example, fragile coral reefs are exposed to various sources of hazards and potential destruction, and need close observation. Computer vision offers promising technologies to build 3D models of an environment from two-dimensional images. The state of the art techniques have enabled high-quality digital reconstruction of large-scale structures, e.g., buildings and urban environments, but only sparse representations or dense reconstruction of small objects have been obtained from underwater video and still imagery. The application of standard 3D reconstruction methods to challenging underwater environments typically produces unsatisfactory results. Accurate, full camera trajectories are needed to serve as the basis for dense 3D reconstruction. A highly accurate sparse 3D reconstruction is the ideal foundation on which to base subsequent dense reconstruction algorithms. In our application the models are constructed from synchronized high definition videos collected using a wide baseline stereo rig. The rig can be hand-held, attached to a boat, or even to an autonomous underwater vehicle. We solve this problem by employing a smoothing and mapping toolkit developed in our lab specifically for this type of application. The result of our technique is a highly accurate sparse 3D reconstruction of underwater structures such as corals.
Mobile robots are dependent upon a model of the environment for many of their basic functions. Locally accurate maps are critical to collision avoidance, while large-scale maps (accurate both metrically and topologically) are necessary for efficient route planning. Solutions to these problems have immediate and important applications to autonomous vehicles, precision surveying, and domestic robots. Building accurate maps can be cast as an optimization problem: find the map that is most probable given the set of observations of the environment. However, the problem rapidly becomes difficult when dealing with large maps or large numbers of observations. Sensor noise and non-linearities make the problem even more difficult especially when using inexpensive (and therefore preferable) sensors. This thesis describes an optimization algorithm that can rapidly estimate the maximum likelihood map given a set of observations. The algorithm, which iteratively reduces map error by considering a single observation at a time, scales well to large environments with many observations. The approach is particularly robust to noise and non-linearities, quickly escaping local minima that trap current methods. Both batch and online versions of the algorithm are described. In order to build a map, however, a robot must first be able to recognize places that it has previously seen. Limitations in sensor processing algorithms, coupled with environmental ambiguity, make this difficult. Incorrect place recognitions can rapidly lead to divergence of the map. This thesis describes a place recognition algorithm that can robustly handle ambiguous data. We evaluate these algorithms on a number of challenging datasets and provide quantitative comparisons to other state-of-the-art methods, illustrating the advantages of our methods.
This work presents a system for 3D reconstruction from underwater images or video. Aside from a camera in an underwater housing, no special equipment is required. However, if navigational data is available, it is utilized in the algorithm. The algorithm is in essence a classical structure from motion approach, which is adapted to account for the special imaging conditions. Hence, there is no need for the camera to follow a specialized trajectory. Adaptions to the underwater imaging environment include a special filtering of background and floating particles, which allows a robust estimation of the camera poses and a sparse set of 3D points. Based on the estimated camera track, dense image correspondence computation enables building a detailed 3D surface model. Once the 3D surface model is completed, the colors of the texture are corrected by a physical model for underwater light propagation, allowing to view the model without the effects of scattering and attenuation or to simulate the effects of water on light in a 3D viewer.