Figure 1 - uploaded by Martin Magnusson
Content may be subject to copyright.
Source publication
![Figure 5: Overhead view of the Hannover data set [15]. The data set...](profile/Martin-Magnusson-2/publication/321532084/figure/fig4/AS:989600231198749@1612950489950/Overhead-view-of-the-Hannover-data-set-15-The-data-set-covers-an-area-of-approximately_Q320.jpg)
Matching and merging overlapping point clouds is a common procedure in many applications, including mobile robotics, three-dimensional mapping, and object visualization. However, fully automatic point-cloud matching, without manual verification, is still not possible because no matching algorithms exist today that can provide any certain methods fo...
Contexts in source publication
Context 1
... problem is visualized in Figures 1 and 2 where two point clouds that are aligned are shown in Figure 1 and two point clouds that are misaligned are shown in Figure 2. In this article we will present an investigation of methods that can be used for automatic detection of misaligned point clouds. ...
Context 2
... problem is visualized in Figures 1 and 2 where two point clouds that are aligned are shown in Figure 1 and two point clouds that are misaligned are shown in Figure 2. In this article we will present an investigation of methods that can be used for automatic detection of misaligned point clouds. ...
Context 3
... results Figures 10 and 11 shows the classifiers' accuracy on the Hannover datasets for all error types. ...
Context 4
... Figure 10 we see that most classifiers obtain more than 90% accuracy for large errors which can be considered as a good result. The NORM classifier manage well above a random result for large errors but fails to classify the point cloud pairs with medium and small errors, indicated by the accuracy of 50% which implies a random classification result. ...
Context 5
... we look at the dataset with errors of all three magnitudes, shown in Figure 11, we observe that the classifiers performance is roughly the mean value of the accuracy obtained over all error magnitudes in 10. This result suggests that the thresholds acquired during the training phase of the classification process is relatively stable for all classifiers even though error magnitudes vary. ...
Context 6
... results Figures 12 and 13 show the classifiers' accuracy on the Kjula dataset for all error types. ...
Context 7
... results from the Kjula set in Figure 12 show a more spread out result than the corresponding results in the Hannover set, indicating the higher difficulty of the less structured Kjula environment. There are still some classifiers that perform above 80% even on the small error magnitudes, but there are more classifiers that perform close to or at random than in the Hannover set. ...
Context 8
... boosted classifier, as a consequence, uses a suboptimal combination of the "weak" classifiers in this case. The effect of this on the AdaBoost classifier can be seen both in Figure 12 and Figure 13, as it performs worse than NDT3 but better than or equal to NDT4. This result emphasizes the risks of small training sets and also shows that AdaBoost is not immune to overfitting. ...
Context 9
... boosted classifier, as a consequence, uses a suboptimal combination of the "weak" classifiers in this case. The effect of this on the AdaBoost classifier can be seen both in Figure 12 and Figure 13, as it performs worse than NDT3 but better than or equal to NDT4. This result emphasizes the risks of small training sets and also shows that AdaBoost is not immune to overfitting. ...
Context 10
... evaluations are performed to investigate the classifiers sensitivity to the likeness between the training data and the evaluation data. Figures 14 and 15 show the accuracy for small, medium and large errors. In comparison to the evaluations with training and evaluation data from the same environment we see that many of the classifiers produce random or close to random results. ...
Context 11
... the effect of the method used to create the ground truth is not significant enough in our experiments to alter the outcome of the evaluations. This is shown in the lower two diagrams in Figure 16 where the two classifiers NDT3 (blue/dark) and RMS6 (red/light) is compared to each other with NDT-based ground truth to the left and ICP based ground truth to the right. The diagrams are nearly identical since the differences in accuracy for the classifiers is much larger than the effect of the method used to create the ground truth. ...
Context 12
... dataset In Figure 17 we show example ROC plots for the classifiers that showed the best performance on the Hannover dataset. Each plot shows ROC curves for each of the three difficulties on the combined dataset. ...
Context 13
... dataset In Figure 18 we see examples of ROC-plots for the Kjula set. The number of samples in the plots are lower because of the lower number of point clouds in the entire data set. ...
Context 14
... number of samples in the plots are lower because of the lower number of point clouds in the entire data set. Both classifiers (NDT3 to the left and RMS3 to the right) show an even distribution of samples along the curves, suggesting that the classifiers have a similar threshold sensitivity, which was also the case for the NDT3 and RMS6 classifiers in Figure 17. ...
Similar publications
Nowadays, multi-sensor fusion is a popular tool for feature recognition and object detection. Integrating various sensors allows us to obtain reliable information about the environment. This article proposes a 3D robot trajectory estimation based on a multimodal fusion of 2D features extracted from color images and 3D features from 3D point clouds....
3D feature descriptors provide essential information to find given models in captured scenes. In practical applications, these scenes often contain clutter. This imposes severe challenges on the 3D object recognition leading to feature mismatches between scenes and models. As such errors are not fully addressed by the existing methods, 3D feature m...
This paper presents a novel lidar SLAM system for localizing a mobile robot to build a map of the environment. To identify the unknown transform matrix, we design a new scan-matching approach, in which a point cloud segmentation algorithm is additionally integrated. Different from the traditional normal distribution transform algorithm for point cl...
When using point cloud technology to measure the dimension and geometric error of aircraft landing gear components, the point cloud data obtained after scanning may have certain differences because of the sophistication and diversity of the components that make up the landing gear. However, when using traditional point cloud registration algorithms...
Kernel point convolution (KPConv) can effectively represent the point features of point cloud data. However, KPConv-based methods just consider the local information of each point, which is very difficult to characterize the intrinsic properties of ALS point clouds for complex laser scanning conditions. Therefore, we rethink KPConv and propose a re...
Citations
... Alsayed et al. (2017) detects localization failures using extracted lines and curves. Almqvist et al. (2018) proposes a method based on NDT (Normal Distribution Transform) for detecting misaligned point clouds and predicting the degree of misalignment using machine learning. Yin et al. (2019a) proposes a statistical learning-based approach for localization failure detection by defining the problem as a binary classification task. ...
... Specifically, for a query point cloud and a point cloud in the prior map, certain spatial properties such as distances and angles are preserved between them. Such spatial invariances, i.e. spatially verifiable cues, can be used to assess the similarity of scenes (Vidanapathirana et al., 2023) and verify the reliability of the local pose estimation (Almqvist et al., 2018). For a point cloud sequence, the sequential motions and uncertainties (Wan et al., 2021), i.e. temporal verifiable cues, provided by the odometry can also verify the reliability of the current localization result. ...
Wearable laser scanning (WLS) system has the advantages of flexibility and portability. It can be used for determining the user's path within a prior map, which is a huge demand for applications in pedestrian navigation, collaborative mapping, augmented reality, and emergency rescue. However, existing LiDAR-based global localization methods suffer from insufficient robustness, especially in complex large-scale outdoor scenes with insufficient features and incomplete coverage of the prior map. To address such challenges, we propose LiDAR-based reliable global localization (Reliable-loc) exploiting the verifiable cues in the sequential LiDAR data. First, we propose a Monte Carlo Localization (MCL) based on spatially verifiable cues, utilizing the rich information embedded in local features to adjust the particles' weights hence avoiding the particles converging to erroneous regions. Second, we propose a localization status monitoring mechanism guided by the sequential pose uncertainties and adaptively switching the localization mode using the temporal verifiable cues to avoid the crash of the localization system. To validate the proposed Reliable-loc, comprehensive experiments have been conducted on a large-scale heterogeneous point cloud dataset consisting of high-precision vehicle-mounted mobile laser scanning (MLS) point clouds and helmet-mounted WLS point clouds, which cover various street scenes with a length of over 20km. The experimental results indicate that Reliable-loc exhibits high robustness, accuracy, and efficiency in large-scale, complex street scenes, with a position accuracy of 1.66m, yaw accuracy of 3.09 degrees, and achieves real-time performance. For the code and detailed experimental results, please refer to https://github.com/zouxianghong/Reliable-loc.
... In [105] authors investigated the effect of geometric instability on alignment and proposed a novel learning-based approach to predict misalignment. A comparative evaluation of several misaligned point cloud detection methods for point cloud registration problem, where multiple point clouds need to be aligned or merged together is presented in [106]. Similarly, in [107], researchers proposed a novel system to detect alignment errors in point cloud registration. ...
Automated driving systems (ADSs) aim to improve the safety, efficiency and comfort of future vehicles. To achieve this, ADSs use sensors to collect raw data from their environment. This data is then processed by a perception subsystem to create semantic knowledge of the world around the vehicle. State-of-the-art ADSs’ perception systems often use deep neural networks for object detection and classification, thanks to their superior performance compared to classical computer vision techniques. However, deep neural network-based perception systems are susceptible to errors, e.g., failing to correctly detect other road users such as pedestrians. For a safety-critical system such as ADS, these errors can result in accidents leading to injury or even death to occupants and road users. Introspection of perception systems in ADS refers to detecting such perception errors to avoid system failures and accidents. Such safety mechanisms are crucial for ensuring the trustworthiness of ADSs. Motivated by the growing importance of the subject in the field of autonomous and automated vehicles, this paper provides a comprehensive review of the techniques that have been proposed in the literature as potential solutions for the introspection of perception errors in ADSs. We classify such techniques based on their main focus, e.g., on object detection, classification and localisation problems. Furthermore, this paper discusses the pros and cons of existing methods while identifying the research gaps and potential future research directions.
... We instead use odometry uncertainty as a prior, which we combine with additional loop evidence computed after registration. Finally, a range of methods aims at verifying pose estimates by detecting point cloud misalignment [30], [32], [37]. We fuse misalignment detection [32] together with place similarity and odometry uncertainty to achieve robust unified loop detection and verification. ...
Robust SLAM in large-scale environments requires fault resilience and awareness at multiple stages, from sensing and odometry estimation to loop closure. In this work, we present TBV (Trust But Verify) Radar SLAM, a method for radar SLAM that introspectively verifies loop closure candidates. TBV Radar SLAM achieves a high correct-loop-retrieval rate by combining multiple place-recognition techniques: tightly coupled place similarity and odometry uncertainty search, creating loop descriptors from origin-shifted scans, and delaying loop selection until after verification. Robustness to false constraints is achieved by carefully verifying and selecting the most likely ones from multiple loop constraints. Importantly, the verification and selection are carried out after registration when additional sources of loop evidence can easily be computed. We integrate our loop retrieval and verification method with a fault-resilient odometry pipeline within a pose graph framework. By evaluating on public benchmarks we found that TBV Radar SLAM achieves 65% lower error than the previous state of the art. We also show that it's generalizing across environments without needing to change any parameters.
... However, there have been some works on detecting wrong loop closures [12], [13], [14], which is a closely related problem, since an invalid map merge is fairly similar to a wrong global loop closure. Furthermore there is some research on map quality measures [15], [16], [17], which could be used to detect invalid merges by computing a map quality measure and assuming that a invalid merge must have happened if it drops below a certain threshold. We adapt the method in [12] (histogram) and [16] (entropy) to compare our methods against. ...
... Another idea to detect invalid merges is to use some kind of map quality metric in order to assess the quality of the map and to use a low quality as an indicator that a invalid merge has happened. Different map quality metrics have been proposed in literature, e.g., [15], [17]. Some of these metrics measure aspects of the map's quality that are not suitable for detecting invalid merges since they are designed to respond to noisy maps or slightly misaligned scans. ...
For Lifelong SLAM, one has to deal with temporary localization failures, e.g., induced by kidnapping. We achieve this by starting a new map and merging it with the previous map as soon as relocalization succeeds. Since relocalization methods are fallible, it can happen that such a merge is invalid, e.g., due to perceptual aliasing. To address this issue, we propose methods to detect and undo invalid merges. These methods compare incoming scans with scans that were previously merged into the current map and consider how well they agree with each other. Evaluation of our methods takes place using a dataset that consists of multiple flat and office environments, as well as the public MIT Stata Center dataset. We show that methods based on a change detection algorithm and on comparison of gridmaps perform well in both environments and can be run in real-time with a reasonable computational cost.
... Similarly, [15] relies on the condition number to determine the health of the optimization process and includes partial constraints along non-degenerate direction for sensor fusion. Other methods, such as [16,17] rely on the final alignment of scans to capture the adequacy of geometric constraints provided by the environment for correct solution convergence. However, all these methods either rely on the point cloud registration process or its result to determine the performance of the pose estimation process, and do not exploit the information provided by the point cloud data directly to facilitate the estimation process itself. ...
LiDAR-based localization and mapping is one of the core components in many modern robotic systems due to the direct integration of range and geometry, allowing for precise motion estimation and generation of high quality maps in real-time. Yet, as a consequence of insufficient environmental constraints present in the scene, this dependence on geometry can result in localization failure, happening in self-symmetric surroundings such as tunnels. This work addresses precisely this issue by proposing a neural network-based estimation approach for detecting (non-)localizability during robot operation. Special attention is given to the localizability of scan-to-scan registration, as it is a crucial component in many LiDAR odometry estimation pipelines. In contrast to previous, mostly traditional detection approaches, the proposed method enables early detection of failure by estimating the localizability on raw sensor measurements without evaluating the underlying registration optimization. Moreover, previous approaches remain limited in their ability to generalize across environments and sensor types, as heuristic-tuning of degeneracy detection thresholds is required. The proposed approach avoids this problem by learning from a collection of different environments, allowing the network to function over various scenarios. Furthermore, the network is trained exclusively on simulated data, avoiding arduous data collection in challenging and degenerate, often hard-to-access, environments. The presented method is tested during field experiments conducted across challenging environments and on two different sensor types without any modifications. The observed detection performance is on par with state-of-the-art methods after environment-specific threshold tuning 1.
... However, there have been some works on detecting wrong loop closures [12], [13], [14], which is a closely related problem, since an invalid map merge is fairly similar to a wrong global loop closure. Furthermore there is some research on map quality measures [15], [16], [17], which could be used to detect invalid merges by computing a map quality measure and assuming that a invalid merge must have happened if it drops below a certain threshold. We adapt the method in [12] (histogram) and [16] (entropy) to compare our methods against. ...
... Another idea to detect invalid merges is to use some kind of map quality metric in order to assess the quality of the map and to use a low quality as an indicator that a invalid merge has happened. Different map quality metrics have been proposed in literature, e.g., [15], [17]. Some of these metrics measure aspects of the map's quality that are not suitable for detecting invalid merges since they are designed to respond to noisy maps or slightly misaligned scans. ...
... Localization failure detection methods have been proposed by some researchers [3], [4]. Owing to recent advances in machine learning (ML) algorithms, some researchers have proposed ML-based failure detection methods [5]- [9]. However, these methods do not consider the entire relation of sensor measurement. ...
... The MLbased methods enable us to implicitly model complex relationships. Almqvist et al. compared several misaligned point cloud detection methods, including the ML-based methods [9], and demonstrated that AdaBoost [37] executed a better performance. Through experiments conducted in this work, we demonstrate that the proposed method outperforms the AdaBoost-based failure detector. ...
Most of the recent automated driving systems assume the accurate functioning of localization. Unanticipated errors cause localization failures and result in failures in automated driving. An exact localization failure detection is necessary to ensure safety in automated driving; however, detection of the localization failures is challenging because sensor measurement is assumed to be independent of each other in the localization process. Owing to the assumption, the entire relation of the sensor measurement is ignored. Consequently, it is difficult to recognize the misalignment between the sensor measurement and the map when partial sensor measurement overlaps with the map. This paper proposes a method for the detection of localization failures using Markov random fields with fully connected latent variables. The full connection enables to take the entire relation into account and contributes to the exact misalignment recognition. Additionally, this paper presents localization failure probability calculation and efficient distance field representation methods. We evaluate the proposed method using two types of datasets. The first dataset is the SemanticKITTI dataset, whereby four methods are compared with the proposed method. The comparison results reveal that the proposed method achieves the most accurate failure detection. The second dataset is created based on log data acquired from the demonstrations that we conducted in Japanese public roads. The dataset includes several localization failure scenes. We apply the failure detection methods to the dataset and confirm that the proposed method achieves exact and immediate failure detection.
... Some examples of methods that have been used in practice to assess the alignment quality include point-to-point or point-to-plane distances [9], [10], point-to-distribution [11], [12] or distribution-to-distribution [13], [14] likelihood estimates, mean map entropy [15] or dense radar-image comparison [16]. However, except for some notable exceptions [12], [17], few studies in the literature have specifically and methodically targeted the measurement of alignment correctness. ...
... Some examples of methods that have been used in practice to assess the alignment quality include point-to-point or point-to-plane distances [9], [10], point-to-distribution [11], [12] or distribution-to-distribution [13], [14] likelihood estimates, mean map entropy [15] or dense radar-image comparison [16]. However, except for some notable exceptions [12], [17], few studies in the literature have specifically and methodically targeted the measurement of alignment correctness. ...
... (d) Misaligned Pa ∪ P b colored by quality measure when a small misalignment is added. However, this measure has been shown to be highly sensitive to the environment and the choice of the outlier threshold [12], [19] when trying to detect small errors. Consequently, this is a poor measure for alignment correctness classification. ...
Robust perception is an essential component to enable long-term operation of mobile robots. It depends on failure resilience through reliable sensor data and preprocessing, as well as failure awareness through introspection, for example the ability to self-assess localization performance. This paper presents CorAl: a principled, intuitive, and generalizable method to measure the quality of alignment between pairs of point clouds, which learns to detect alignment errors in a self-supervised manner. CorAl compares the differential entropy in the point clouds separately with the entropy in their union to account for entropy inherent to the scene. By making use of dual entropy measurements, we obtain a quality metric that is highly sensitive to small alignment errors and still generalizes well to unseen environments. In this work, we extend our previous work on lidar-only CorAl to radar data by proposing a two-stage filtering technique that produces high-quality point clouds from noisy radar scans. Thus we target robust perception in two ways: by introducing a method that introspectively assesses alignment quality, and applying it to an inherently robust sensor modality. We show that our filtering technique combined with CorAl can be applied to the problem of alignment classification, and that it detects small alignment errors in urban settings with up to 98% accuracy, and with up to 96% if trained only in a different environment. Our lidar and radar experiments demonstrate that CorAl outperforms previous methods both on the ETH lidar benchmark, which includes several indoor and outdoor environments, and the large-scale Oxford and MulRan radar data sets for urban traffic scenarios The results also demonstrate that CorAl generalizes very well across substantially different environments without the need of retraining.
... Hence, machine learning approaches have been applied recently. Almqvist et al. [Almqvist et al., 2018] applied several threshold-and machine-learning-based methods to classify misaligned point clouds. Alsayed et al. [Alsayed et al., 2017, Alsayed et al., 2018 presented a machine-learning-based failure detection method for 2D LiDAR SLAM. ...
... The localization state classifier classifies whether localization has failed. The classifier can be implemented using any methods such as threshold-and machine-learning-based methods [Almqvist et al., 2018]. Output form of the classifier is changed according to its implementation. ...
Reliability is a key factor for realizing safety guarantee of full autonomous robot systems. In this paper, we focus on reliability in mobile robot localization. Monte Carlo localization (MCL) is widely used for mobile robot localization. However, it is still difficult to guarantee its safety because there are no methods determining reliability for MCL estimate. This paper presents a novel localization framework that enables robust localization, reliability estimation, and quick re-localization, simultaneously. The presented method can be implemented using similar estimation manner to that of MCL. The method can increase localization robustness to environment changes by estimating known and unknown obstacles while performing localization; however, localization failure of course occurs by unanticipated errors. The method also includes a reliability estimation function that enables us to know whether localization has failed. Additionally, the method can seamlessly integrate a global localization method via importance sampling. Consequently, quick re-localization from failures can be realized while mitigating noisy influence of global localization. Through three types of experiments, we show that reliable MCL that performs robust localization, self-failure detection, and quick failure recovery can be realized.
... Similarly, [15] relies on the condition number to determine the health of the optimization process and includes partial constraints along non-degenerate direction for sensor fusion. Other methods, such as [16,17] rely on the final alignment of scans to capture the adequacy of geometric constraints provided by the environment for correct solution convergence. However, all these methods either rely on the point cloud registration process or its result to determine the performance of the pose estimation process, and do not exploit the information provided by the point cloud data directly to facilitate the estimation process itself. ...
LiDAR-based localization and mapping is one of the core components in many modern robotic systems due to the direct integration of range and geometry, allowing for precise motion estimation and generation of high quality maps in real-time. Yet, as a consequence of insufficient environmental constraints present in the scene, this dependence on geometry can result in localization failure, happening in self-symmetric surroundings such as tunnels. This work addresses precisely this issue by proposing a neural network-based estimation approach for detecting (non-)localizability during robot operation. Special attention is given to the localizability of scan-to-scan registration, as it is a crucial component in many LiDAR odometry estimation pipelines. In contrast to previous, mostly traditional detection approaches, the proposed method enables early detection of failure by estimating the localizability on raw sensor measurements without evaluating the underlying registration optimization. Moreover, previous approaches remain limited in their ability to generalize across environments and sensor types, as heuristic-tuning of degeneracy detection thresholds is required. The proposed approach avoids this problem by learning from a corpus of different environments, allowing the network to function over various scenarios. Furthermore, the network is trained exclusively on simulated data, avoiding arduous data collection in challenging and degenerate, often hard-to-access, environments. The presented method is tested during field experiments conducted across challenging environments and on two different sensor types without any modifications. The observed detection performance is on par with state-of-the-art methods after environment-specific threshold tuning.
