No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Distinctive 2D and 3D Features for Automated Large-Scale Scene Analysis in Urban Areas
... Feature extraction, as a vital step of classification, aims to find discriminative features describing 3D distribution from massive points [8], which directly affects the accuracy of subsequent point cloud processing. Commonly used point cloud features include RGB colors [9], intensity features [10][11][12][13], geometric features [9,[14][15][16][17], etc. The intensity and RGB information are not always available, which limits the generality of methods using those features. ...
... Ibrahim et al. extracted ground points by counting the point number in cylinder neighborhoods and set the density threshold according to the average point density and the number of drive lines [29]. In addition to directly using point density as a feature description, projecting 3D points onto the XOY plane is another common strategy [14,19,30,31]. Li et al. projected the points onto a rectangular grid in the XOY plane, counting the number of points in each sub-grid, and taking it as the projection density. ...
... Then, the angular resolution is estimated by NARP for non-ground points. In the step of feature extraction, some commonly used multi-dimensional geometric features [14,19,20,30,52] are extracted for each non-ground point, with traditional projection density replaced by relative projection density. Finally, a Random Forest (RF) classifier is used to label each point and evaluate feature importance. ...
Point cloud classification is a key step for three-dimensional (3D) scene analysis in terrestrial laser scanning but is commonly affected by density variation. Many density-adaptive methods are used to weaken the impact of density variation and angular resolution, which denotes the angle between two horizontally or vertically adjacent laser beams and are commonly used as known parameters in those methods. However, it is difficult to avoid the case of unknown angular resolution, which limits the generality of such methods. Focusing on these problems, we propose a density-adaptive feature extraction method, considering the case when the angular resolution is unknown. Firstly, we present a method for angular resolution estimation called neighborhood analysis of randomly picked points (NARP). In NARP, n points are randomly picked from the original data and the k nearest points of each point are searched to form the neighborhood. The angles between the beams of each picked point and its corresponding neighboring points are used to construct a histogram, and the angular resolution is calculated by finding the adjacent beams of each picked point under this histogram. Then, a grid feature called relative projection density is proposed to weaken the effect of density variation based on the estimated angular resolution. Finally, a 12-dimensional feature vector is constructed by combining relative projection density and other commonly used geometric features, and the semantic label is generated utilizing a Random Forest classifier. Five datasets with a known angular resolution are used to validate the NARP method and an urban scene with a scanning distance of up to 1 km is used to compare the relative projection density with traditional projection density. The results demonstrate that our method achieves an estimation error of less than 0.001° in most cases and is stable with respect to different types of targets and parameter settings. Compared with traditional projection density, the proposed relative projection density can improve the performance of classification, particularly for small-size objects, such as cars, poles, and scanning artifacts.
... The neighborhood size was chosen such that it minimized the eigentropy E of the vector (λ1/Λ, λ2/Λ, λ3/Λ), where E represents the point cloud adjacency relationship. According to the best neighbor principle proposed by Weinmann et al. [44], ...
... According to [44], these features were defined by the local domain of each point in the point cloud. The eigenvalues for each point λ 1 ≥ λ 2 ≥ λ 3 were calculated of the covariance matrix of the positions of the neighbors. ...
... The neighborhood size was chosen such that it minimized the eigentropy E of the vector (λ 1 /Λ, λ 2 /Λ, λ 3 /Λ), where E represents the point cloud adjacency relationship. According to the best neighbor principle proposed by Weinmann et al. [44], Λ = ∑ 3 i=1 λ i , which is in accordance with the optimal adjacency: ...
The use of 3D point cloud-based technology for quantifying standing wood and stand parameters can play a key role in forestry ecological benefit assessment and standing tree cultivation and utilization. With the advance of 3D information acquisition techniques, such as light detection and ranging (LiDAR) scanning, the stand information of trees in large areas and complex terrain can be obtained more efficiently. However, due to the diversity of the forest floor, the morphological diversity of the trees, and the fact that forestry is often planted as large-scale plantations, efficiently segmenting the point cloud of artificially planted forests and extracting standing wood feature parameters remains a considerable challenge. An effective method based on energy segmentation and PointCNN is proposed in this work to address this issue. The network is enhanced for learning point cloud features by geometric feature balance model (GFBM), enabling the efficient segmentation of tree point clouds from forestry point cloud data collected by terrestrial laser scanning (TLS) in outdoor environments. The 3D Forest software is then used to obtain single wood point cloud after semantic segmentation, and the extracted single wood point cloud is finally employed to extract standing wood feature parameters using TreeQSM. The point cloud semantic segmentation method is the most important part of our research. According to our findings, this method can segment datasets of two different artificially planted woodland point clouds with an overall accuracy of 0.95 and a tree segmentation accuracy of 0.93. When compared with the manual measurements, the root-mean-square error (RMSE) for tree height in the two datasets are 0.30272 and 0.21015 m, and the RMSEs for the diameter at breast height are 0.01436 and 0.01222 m, respectively. Our method is a robust framework based on deep learning that is applicable to forestry for extracting the feature parameters of artificially planted trees. It solves the problem of segmenting tree point clouds in artificially planted trees and provides a reliable data processing method for tree information extraction, trunk shape analysis, etc.
... The determination of how much k neighbors has been studied in (Hackel et al., 2016;Weinmann et al., 2013) where a fixed number of points for all points was applied. Others applied a different and changing k number for each individual point according to a specific condition (Demantké et al., 2011;Weinmann et al., 2014;Weinmann, Urban, et al., 2015). ...
... For a (2D) planar structure, (λ 1 λ 2 ) are much larger than λ 3 , while a (3D) volumetric structure has similar Eigenvalues (Dittrich et al., 2017). In (Weinmann et al., 2014;Weinmann, Urban, et al., 2015), they replaced the Eigenvalues with their normalized values (e 1 , e 2 , e 3 ) where (e = λ / ∑ λ =1 ). Another common set of features is the moment features which implemented previously is (Hackel et al., 2016), those features were derived from the dot product of the coordinates' array and the Eigenvectors of the covariance matrix. ...
... The first subset of features, covariance features, are derived from the normalized Eigenvalues (e 1 , e 2 , e 3 ) where e = λ i /(λ 1 + λ 2 + λ 3 ). The covariance features are similar to what have been previously used in the research of (Weinmann, Urban, et al., 2015), except for the "Sum" feature that is derived from the summation of the three Eigenvalues, not the normalized ones. Another feature is added to the covariance set is the verticality which has been driven before in the research of (Demantké et al., 2011). ...
Mobile LiDAR, Road Features, Machine Learning, Neighborhood Selection. Mobile LiDAR systems are distinguished with large and highly accurate point clouds data acquisition for road environments. Road features extraction is becoming one of the most important applications of LiDAR point cloud, and is used largely in road maintenance and autonomous driving vehicles. The main step in Mobile LiDAR processing is point classification This classification relies on the geometric definition of the points and their surroundings, as well as the classification methods used. The neighbors of each point is helpful to find more meaningful information than the raw coordinates. On the other hand, machine learning algorithms have proved their efficiency in LiDAR point cloud classification. This research compares results of using three machine learning classifiers, namely Random Forest, Gaussian Naïve Bayes, and Quadratic Discriminate Analysis along with using three neighborhood search methods, namely k nearest neighbors, spherical and cylindrical. A part of the pre-labelled benchmark dataset (Paris Lille 3D) with about 98 million points was tested. Results showed that the using Random Forest classifer with the cylindirical neighborhood search method acheived the highest overall accuracy of 92.39%.
... Histogram features, such as the fast point feature histogram [10], accumulate information about the spatial interconnection between a point and its neighbors into a histogram representation [11,12]. Covariance features, including line, plane, and volume attributes, are calculated from the covariance matrix of all points in the point's neighborhood [13,14]. Although this manual-constructed feature is useful for land cover classification, it cannot produce three-dimensional land cover classification with sufficient quality owing to the complexity and diversity of actual geo-objects. ...
... Histogram features, such as the fast point feature histogram [10], accumulate information about the spatial interconnection between a point and its neighbors into a histogram representation [11,12]. Covariance features, including line, plane, and volume attributes, are calculated from the covariance matrix of all points in the point's neighborhood [13,14]. Although this manual-constructed feature is useful for land cover classification, it cannot produce threedimensional land cover classification with sufficient quality owing to the complexity and diversity of actual geo-objects. ...
... Remote Sens. 2021,13, 4928 ...
The heterogeneity of urban landscape in the vertical direction should not be neglected in urban ecology research, which requires urban land cover product transformation from two-dimensions to three-dimensions using light detection and ranging system (LiDAR) point clouds. Previous studies have demonstrated that the performance of two-dimensional land cover classification can be improved by fusing optical imagery and LiDAR data using several strategies. However, few studies have focused on the fusion of LiDAR point clouds and optical imagery for three-dimensional land cover classification, especially using a deep learning framework. In this study, we proposed a novel prior-level fusion strategy and compared it with the no-fusion strategy (baseline) and three other commonly used fusion strategies (point-level, feature-level, and decision-level). The proposed prior-level fusion strategy uses two-dimensional land cover derived from optical imagery as the prior knowledge for three-dimensional classification. Then, a LiDAR point cloud is linked to the prior information using the nearest neighbor method and classified by a deep neural network. Our proposed prior-fusion strategy has higher overall accuracy (82.47%) on data from the International Society for Photogrammetry and Remote Sensing, compared with the baseline (74.62%), point-level (79.86%), feature-level (76.22%), and decision-level (81.12%). The improved accuracy reflects two features: (1) fusing optical imagery to LiDAR point clouds improves the performance of three-dimensional urban land cover classification, and (2) the proposed prior-level strategy directly uses semantic information provided by the two-dimensional land cover classification rather than the original spectral information of optical imagery. Furthermore, the proposed prior-level fusion strategy provides a series that fills the gap between two- and three-dimensional land cover classification.
... To describe the shape of the neighborhood, dimensionality features can be constructed by the eigenvalues of the local structure tensor [5]. The eigenvalues are extended in both 3D and 2D space by Weinmann, and he combined with projection density, height difference and height standard deviation for classification [6] [7] [8]. The grid projection density is greatly affected by the point cloud density, and it is difficult to distinguish the geometric features of ground objects themselves. ...
... Because the calculation of geometric features only relies on the coordinate information of the points, we only involve geometric features in our investigations. As a part of geometric features, grid features have also been studied in many studies [7][21], among which rectangular grid is the most commonly used grid. Grid feature is influenced by the local structure and point density of the target object. ...
... Besides the grid features, the 3D and 2D features in [9] are the same in our method. As the 3D and 2D neighborhoods of the two methods are both generated adaptively by the method in [7], parameters requiring manual setting are related with the grid features. The main parameters of grid feature proposed in this paper are grid size, which are set as 1m*1m in this test. ...
... In our study, we propose to view the problem of infrastructure detection in rail corridors as a classification problem, where each point of a LiDAR acquisition belongs either to an object of interest or to a dedicated class containing all objects not related to railroad infrastructure. The classification task from LiDAR point clouds has been thoroughly investigated (Weinmann et al., 2015;Vicari et al., 2019), with promising algorithms, especially in urban areas. However, there exists only a few studies on the classification of all key elements in rail corridors (Arastounia, 2012). ...
... λi, in accordance with the optimal neighborhood principle advocated in Weinmann et al. (2015): ...
... As presented in Demantké et al. (2011) and Weinmann et al. (2015), these eigenvalues allow us to qualify the shape of the local neighborhood by deriving the following values: ...
Railroad environments are peculiar, as they combine dense urban areas, along with rural parts. They also display a very specific spatial organization. In order to monitor a railway network a at country scale, LiDAR sensors can be equipped on a running train, performing a full acquisition of the network. Then most processing steps are manually done. In this paper, we propose to improve performances and production flow by creating a classification of the acquired data. However, there exists no public benchmark, and little work on LiDAR data classification in railroad environments. Thus, we propose a weakly supervised method for the pointwise classification of such data. We show that our method can be improved by using the l0-cut pursuit algorithm and regularize the noisy pointwise classification on the produced segmentation. As production is envisaged in our context, we designed our implementation such that it is computationally efficient. We evaluate our results against a manual classification, and show that our method can reach a FScore of 0.96 with just a few samples of each class.
... Gathering meaningful information from 3D point clouds is a very important topic in remote sensing, robotics, and computer vision fields [1]. Point cloud classification is a key task for many applications such as object detection [2,3], urban mapping [4,5], vegetation mapping [6], defining building structures [7], city modeling [8] and interpretation of complex 3D scenes [9]. ...
... In the scope of this work, the semantic ground-truth segments are trained and tested using both the Support Vector Machine (SVM) [17] and Random Forest (RF) [18] classifiers separately to evaluate the success of the classifiers and features. The SVM and RF classifiers are widespread for the classification of 3D point clouds in the literature [1,3,19,20]. The evaluation results are given in this paper as the rate of the true detected semantic segments to the number of all semantic segments in the test part for each classifier. ...
... Each decision tree is trained with differently selected subsets of the data. In the test, the trained RF classifies the consequent new data according to the majority vote of all decision trees [1,25]. ...
This paper presents a segmentation-based classification technique for 3D point clouds. This technique is supervised and needs a ground-truth data for the training process. In this work, the Stanford Large-Scale 3D Indoor Spaces (S3DIS) dataset has been used for the classification of points with the segmentation pre-processing. The dataset consists of a huge amount of points and has semantic ground-truth segments (structures and objects). The main problem in this study is to classify raw points according to the predefined objects and structures. For this purpose, each semantic segment in the training part is segmented separately by a novel successful segmentation algorithm at first. The extracted features of each sub-segments resulted from the segmentation of the semantic segments in the training part are trained using the classifier, and a trained model is obtained. Finally, the raw data reserved for testing are segmented using the same segmentation parameters as used for training, and the result segments are classified using the trained model. The method is tested using two classifiers which are Support Vector Machine (SVM) and Random Forest (RF) with different segmentation parameters. The quantitative results show that RF gives a very useful classification output for such complicated data.
... The most popular features in the remote sensing community are based on the eigenvalues of the point neighbourhood. Early work by Pauly et al. (2003) and Vandapel et al. (2004) introduced the concept, which was extended by Gross (2009) andWeinmann et al. (2015b). The other common features are proposed by Rusu (2010) and implemented by him in his Point Cloud Library (PCL). ...
... The Linear and Quadratic Discriminant Analyses, along with the Naive Bayes proved unable to effectively classify the bolt points and were not considered further. When comparing the remaining three classifiers, the Random Forest produced higher accuracies on the minority bolt class than the Support Vector Machines; these results agree with those found by Bassier et al. (2019), Kogut and Weistock (2019) and Weinmann et al. (2015aWeinmann et al. ( , 2015b. However, the MLP outperformed both the SVM and the RF, this is in contrast to the results observed by Bassier et al. (2019) and Weinmann et al. (2015aWeinmann et al. ( , 2015b. ...
... When comparing the remaining three classifiers, the Random Forest produced higher accuracies on the minority bolt class than the Support Vector Machines; these results agree with those found by Bassier et al. (2019), Kogut and Weistock (2019) and Weinmann et al. (2015aWeinmann et al. ( , 2015b. However, the MLP outperformed both the SVM and the RF, this is in contrast to the results observed by Bassier et al. (2019) and Weinmann et al. (2015aWeinmann et al. ( , 2015b. It is hypothesised that this difference may be due to the larger number of hyperparameters required to produce a stable result from the MLP classifier, as discussed by Nygren and Jasinski (2016). ...
Rock bolts are a crucial part of underground infrastructure support; however, current methods to locate and record their positions are manual, time consuming and generally incomplete. This paper describes an effective method to automatically locate supporting rock bolts from a 3D laser scanned point cloud. The proposed method utilises a machine learning classifier combined with point descriptors based on neighbourhood properties to classify all data points as either ‘bolt’ or ‘not-bolt’ before using the Density Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm to divide the results into candidate bolt objects. The centroids of these objects are then computed and output as simple georeferenced 3D coordinates to be used by surveyors, mine managers and automated machines. Two classifiers were tested, a random forest and a shallow neural network, with the neural network providing the more accurate results. Alongside the different classifiers, different input feature types were also examined, including the eigenvalue based geometric features popular in the remote sensing community and the point histogram based features more common in the mobile robotics community. It was found that a combination of both feature sets provided the strongest results. The obtained precision and recall scores were 0.59 and 0.70 for the individual laser points and 0.93 and 0.86 for the bolt objects. This demonstrates that the model is robust to noise and misclassifications, as the bolt is still detected even if edge points are misclassified, provided that there are enough correct points to form a cluster. In some cases, the model can detect bolts which are not visible to the human interpreter.
... Moreover, 3D descriptors can be computed for each point (Demantke et al., 2011;Weinmann et al., 2015a). These descriptors add strong insights on the local geometry of the data. ...
... Automatic interpretation of large 3D point clouds acquired from terrestrial and mobile LiDAR scanning systems has become an important topic in the remote sensing community (Munoz et al., 2009;Weinmann et al., 2015a;Xu et al., 2019), yet it presents numerous challenges. Indeed, the high volume and the irregular structure of LiDAR point clouds make assigning a semantic label to each point a difficult endeavor. ...
... Consequently, the sought semantic labeling can be expected to display high spatial regularity. In order to take into account the expected spatial regularity, Weinmann et al. (2015a) propose to classify a point cloud using descriptors computed on a local neighborhood for each point. However, the resulting classification is not regular in general, as observed in Figure 6.1b. ...
Thanks to their ever improving resolution and accessibility, \gls{acr::lidar} sensors are increasingly used for mapping cities. Indeed, these sensors are able to efficiently capture high-density scans, which can then be used to produce geometrically detailed reconstructions of complex scenes. However, such reconstruction requires organizing the scan with a fitting data structure, such as point clouds or meshes. Point clouds provide such a representation in a compact way, but their discrete nature prevents some applications such as visualization or simulation. Meshes allow for a continuous representation of surfaces, but are not well suited for representing complex objects, whose level of detail can exceed the resolution. To address these limitations, we propose to reconstruct a continuous geometry of the acquisition where sufficient geometric information is available only. This leads us to create a reconstruction mixing triangles, edges and points. We call such collection of objects a simplicial complex.In this thesis, we study the creation of geometrically detailed \gls{acr::3d} models of urban scenes, based on simplicial complexes. We show that simplicial complexes are a suitable alternative to such meshes. Indeed, they are fast to compute, and can be simplified while maintaining high geometric geometric fidelity with respect to the input scan. We argue that simplicial complexes convey valuable geometric information which can in turn be used for the semantization of \gls{acr::3d} point clouds. We also think that they can serve as input for multi-scale reconstructions of urban scenes.We first present an efficient algorithm for computing simplicial complexes from \gls{acr::lidar} scans of urban scenes. Since the reconstructed simplicial complexes can be very large, they can be difficult to process on a standard computer. To handle this challenge, we investigate different approaches for their spatial generalization by approximating large and geometrically simple areas with elementary primitives. To this end, we propose a new algorithm to compute piecewise-planar approximations of \gls{acr::3d} point clouds, based on a global optimization approach. Next, we propose two different applications of simplicial complexes. The first one is a polygonalization method improving the creation of light yet geometrically accurate \gls{acr::3d} models. The second one is a weakly-supervised classification method using \gls{acr::3d} local and global descriptors.
... Processing of point cloud data, such as scans acquired by LiDAR systems, is a topic of interest in the fields of machine vision and robotics [1]. For a machine to understand the contents of a scanned scene, it is often necessary to semantically segment the scene by labelling each point. ...
... However, only a few works *Correspondence: yasuhiro.yao.tc@hco.ntt.co.jp † Yasuhiro Yao and Katie Xu contributed equally to this work. 1 NTT Media Intelligence Laboratories, Yokosuka 239-0847, Japan Full list of author information is available at the end of the article have addressed this issue with specific regard to semantic segmentation of point clouds. We propose to integrate pseudo-labelling with PointNet [2] to form a technique which can semantically label a point cloud scene given only a few labelled points. ...
... Feature-based pointwise classification such as [1,7,8] has traditionally been the method of choice semantic segmentation tasks [9]. Descriptive pointwise features are computed based on a local neighbourhood and used to train a classifier such as a random forest or a support vector machine. ...
Abstract Manually labelling point cloud scenes for use as training data in machine learning applications is a time- and labour-intensive task. In this paper, we aim to reduce the effort associated with learning semantic segmentation tasks by introducing a semi-supervised method that operates on scenes with only a small number of labelled points. For this task, we advocate the use of pseudo-labelling in combination with PointNet, a neural network architecture for point cloud classification and segmentation. We also introduce a method for incorporating information derived from spatial relationships to aid in the pseudo-labelling process. This approach has practical advantages over current methods by working directly on point clouds and not being reliant on predefined features. Moreover, we demonstrate competitive performance on scenes from three publicly available datasets and provide studies on parameter sensitivity.
... Advances in machine learning and the rapidly growing availability of 3D data have led to several supervised learning approaches for concept classification. Respective approaches include the classification of structures according to semantic categories such as facades, roofs, different forms of vegetation or pole/trunk structures using pointwise hand-crafted geometric descriptors on a single optimal scale [1,2,3,4] or multiple scales [5,6], additionally leveraging contextual information [7,8,9,10], as well as deep-learning strategies [11,12,13,14,15,16,17,18,19,20,21,22]. Furthermore, a few works also focused on the individual classification of points according to being or not being on edges based on multi-scale features and a randomforest-based classification [23], multi-scale features and a dedicated neural network based edge detection classifier [24], neural-network-based pointwise distance estimation to the next sharp geometric feature [25], binarypattern-based filtering on local topology graphs [26], neural-network-based edge-aware point set consolidation leveraging an edge-aware loss [27], training two networks based on PointNet++ [14] to classify points into corners and edges and subsequently applying nonmaximal suppression and inferring feature curves [28], the learning of multi-scale local shape properties (e.g., normal and curvature) [29], and the computation of a scalar sharpness field defined on the underlying Moving Least-Squares surface of the point cloud whose local maxima correspond to sharp edges [30,31]. ...
... The impact of the number of scales is shown in Table 5. For the experiments, we chose to use 2 i neighbors per scale where i is distributed evenly-spaced over the interval (3,7]. While the performance of our algorithm using 4, 8, or 16 scales is very similar, using 2 scales performs much worse. ...
Extracting high-level structural information from 3D point clouds is challenging but essential for tasks like urban planning or autonomous driving requiring an advanced understanding of the scene at hand. Existing approaches are still not able to produce high-quality results consistently while being fast enough to be deployed in scenarios requiring interactivity. We propose to utilize a novel set of features describing the local neighborhood on a per-point basis via first and second order statistics as input for a simple and compact classification network to distinguish between non-edge, sharp-edge, and boundary points in the given data. Leveraging this feature embedding enables our algorithm to outperform the state-of-the-art techniques in terms of quality and processing time.
... The segmentation methodology is based on Grilli et al. (2019), where supervised machine learning techniques were applied to infer per-point labels on the point cloud of the temple of Neptune in Paestum. Following Weinmann et al. (2015), the supervised semantic segmentation of point clouds consists of: (1) class determination, (2) data annotation/labelling, (3) point neighbourhood selection, (4) feature extraction, (5) algorithm training and (6) label inference and evaluation. Figure 2. The two ancient Greek temples, both of the Doric order. ...
... The comparison result is visualised with the confusion matrix, which analyses the amount of correct and false predictions. Furthermore, based on the confusion matrix, the overall accuracy, precision, recall and F1-score metrics are calculated (Hackel et al. 2016;Weinmann, 2015). The overall accuracy measures the overall ability of the model to correctly assign a label to all points; the precision represents the performance of the model in relation to false positives, while the recall is in relation to the false negatives; the F1-score measures the performance of the model by taking into account both precision and recall values. ...
3D point clouds are robust representations of real-world objects and usually contain information about the shape, size, position and radiometry of the scene. However, unstructured point clouds do not directly exploit the full potential of such information and thus, further analysis is commonly required. Especially when dealing with cultural heritage objects which are, typically, described by complex 3D geometries, semantic segmentation is a fundamental step for the automatic identification of shapes, erosions, etc. This paper focuses on the efficient extraction of semantic classes that would support the generation of geometric primitives such as planes, spheres, cylinders, etc. Our semantic segmentation approach relies on supervised learning using a Random Forest algorithm, while the geometric shapes are identified and extracted with the RANSAC model fitting algorithm. In this way the parametric modelling procedure in a HBIM environment is easily enabled. Our experiments show the efficient label transferability of our 3D semantic segmentation approach across different Doric Greek temples, with qualitatively and quantitatively evaluations.
... The 3D covariance matrix from the neighboring points' coordinates have been used for describing the local 3D structure [26], as well as covariance of angular measures and point distances [27]. Weinmann et al. [28] presented 2D and 3D point cloud features for automated large-scale scene analysis, including basic geometric properties (e.g. absolute height, radius, local point density, local normal vector), 3D structure and shape features (general distribution, normalized eigenvalues, linearity, planarity, scattering, omnivariance, anisotropy, eigenentropy, local surface variation, etc.). ...
... absolute height, radius, local point density, local normal vector), 3D structure and shape features (general distribution, normalized eigenvalues, linearity, planarity, scattering, omnivariance, anisotropy, eigenentropy, local surface variation, etc.). One question in [28] is that they compute features at multiple scales, so this method is time-consuming. Hackel et al. [15] proposed a fast semantic segmentation method for 3D point clouds based on carefully handling of points' neighborhood relations. ...
Three-dimensional (3D) semantic segmentation of point clouds is important in many scenarios, such as automatic driving, robotic navigation, while edge computing is indispensable in the devices. Deep learning methods based on point sampling prove to be computation and memory efficient to tackle large-scale point clouds (e.g. millions of points). However, some local features may be abandoned while sampling. In this paper, We present one end-to-end 3D semantic segmentation framework based on dilated nearest neighbor encoding. Instead of down-sampling point cloud directly, we propose a dilated nearest neighbor encoding module to broaden the network’s receptive field to learn more 3D geometric information. Without increase of network parameters, our method is computation and memory efficient for large-scale point clouds. We have evaluated the dilated nearest neighbor encoding in two different networks. The first is the random sampling with local feature aggregation. The second is the Point Transformer. We have evaluated the quality of the semantic segmentation on the benchmark 3D dataset S3DIS, and demonstrate that the proposed dilated nearest neighbor encoding exhibited stable advantages over baseline and competing methods.
... To efficiently discriminate the leaf from timber points, we removed the noise points found in the points classified as "timber". To reach this, first, we recalculated the eigenentropy feature following the same previous procedure (Geometry-based calculation) [36]. Then, extreme eigenentropy values (0.03 ≤ x ≤ 0.75 ≡ 3th ≤ x ≤ 75th percentile; eigenentropy values ranging between 0 and 1) were removed of "timber" points [31]. ...
... We observed that the fixed "Ln" value slightly influenced the geometry-based description of points, as highlighted by the low uncertainties in the timber-leaf discrimination (the overall accuracy is 0.98). Even if it could be conditioned by the quality of TLS data (i.e., point density and spacing), we recommend performing the neighbourhood points using variables "Ln" values [36]. Nevertheless, the combined use of the RF algorithm and a filtering approach allowed us to discriminate the timber from leaf points two times and to generate good input data for the stem detection and reconstruction. ...
Timber assortments are some of the most important goods provided by forests worldwide. To quantify the amount and type of timber assortment is strongly important for socio-economic purposes, but also for accurate assessment of the carbon stored in the forest ecosystems, regardless of their main function. Terrestrial laser scanning (TLS) became a promising tool for timber assortment assessment compared to the traditional surveys, allowing reconstructing the tree architecture directly and rapidly. This study aims to introduce an approach for timber assortment assessment using TLS data in a mixed and multi-layered Mediterranean forest. It consists of five steps: (1) pre-processing, (2) timber-leaf discrimination, (3) stem detection, (4) stem reconstruction, and (5) timber assortment assessment. We assume that stem form drives the stem reconstruction, and therefore, it influences the timber assortment assessment. Results reveal that the timber-leaf discrimination accuracy is 0.98 through the Random Forests algorithm. The overall detection rate for all trees is 84.4%, and all trees with a diameter at breast height larger than 0.30 m are correctly identified. Results highlight that the main factors hindering stem reconstruction are the presence of defects outside the trunk, trees poorly covered by points, and the stem form. We expect that the proposed approach is a starting point for valorising the timber resources from unmanaged/managed forests, e.g., abandoned forests. Further studies to calibrate its performance under different forest stand conditions are furtherly required.
... The point clouds classification is essential step for any MLS databased application. Thus, the classification step has a great interest in several researches (Munoz et al., 2009;Weinmann et al., 2015;Xiong et al., 2011). ...
... The third features subset is derived from height of points including the maximum difference in height within the neighbourhood as well as the standard deviation of the height values of whole points. (Breiman, 2001), which combines multiple of weak learners to the sake of a stronger one (Weinmann et al., 2015). The simple and powerful GNB classifiers are predictive modelling and used for continuous data where the mean and standard deviation of the training data are used to show the distribution of the data. ...
3D road mapping is essential for intelligent transportation system in smart cities. Road features can be utilized for road maintenance, autonomous driving vehicles, and providing regulations to drivers. Currently, 3D road environment receives its data from Mobile Laser Scanning (MLS) systems. MLS systems are capable of rapidly acquiring dense and accurate 3D point clouds, which allow for effective surveying of long road corridors. They produce huge amount of point clouds, which requires automatic features classification algorithms with acceptable processing time. Road features have variant geometric regular or irregular shapes. Therefore, most researches focus on classification of one road feature such as road surface, curbs, building facades, etc. Machine learning (ML) algorithms are widely used for predicting the future or classifying information to help policymakers in making necessary decisions. This prediction comes from a pre-trained model on a given data consisting of inputs and their corresponding outputs of the same characteristics. This research uses ML algorithms for mobile LiDAR data classification. First, cylindrical neighbourhood selection method was used to define point’s surroundings. Second, geometric point features including geometric, moment and height features were derived. Finally, three ML algorithms, Random Forest (RF), Gaussian Naïve Bayes (GNB), and Quadratic Discriminant Analysis (QDA) were applied. The ML algorithms were used to classify a part of Paris-Lille-3D benchmark of about 1.5 km long road in Lille with more than 98 million points into nine classes. The results demonstrated an overall accuracy of 92.39%, 78.5%, and 78.1% for RF, GNB, and QDA, respectively.
... More particularly, we use geometric features based on the combination of the eigenvalues ! ≥ " ≥ # ≥ 0 of the covariance tensor computed within a local neighbourhood of a point, as used by (Weinmann et al., 2015;Hackel at al., 2016) as implemented in CloudCompare (2021). More particularly, the metrics that were experimentally found to be of significance for highlighting the fine details, were surface variation and normal change rate. ...
... and the linear segment detector LSD (vonGioi et al., 2010). Edge detection directly in the 3D space has also been revised in the literature.Weinmann et al. (2015) classified edges along with corners and planes using features based on eigenvalues in 3D point clouds.Hackel et al. (2016) used eigenvalues and Markov Random Fields (MRF) to create wireframe models. Jain et al. (2010) extracted 3D lines under an optimization formulation, minimizing the reprojection error of the segment end-points and en ...
Mesh models generated by multi view stereo (MVS) algorithms often fail to represent in an adequate manner the sharp, natural edge details of the scene. The harsh depth discontinuities of edge regions are eventually a challenging task for dense reconstruction, while vertex displacement during mesh refinement frequently leads to smoothed edges that do not coincide with the fine details of the scene. Meanwhile, 3D edges have been used for scene representation, particularly man-made built environments, which are dominated by regular planar and linear structures. Indeed, 3D edge detection and matching are commonly exploited either to constrain camera pose estimation, or to generate an abstract representation of the most salient parts of the scene, and even to support mesh reconstruction. In this work, we attempt to jointly use 3D edge extraction and MVS mesh generation to promote edge detail preservation in the final result. Salient 3D edges of the scene are reconstructed with state-of-the-art algorithms and integrated in the dense point cloud to be further used in order to support the mesh triangulation step. Experimental results on benchmark dataset sequences using metric and appearance-based measures are performed in order to evaluate our hypothesis.
... e required features as input of ML algorithms. K nearest neighbor (KNN), spherical neighborhood and cylindrical neighborhood selection methods are the most common types of neighborhood selection methods in previous studies, and hence the optimum neighborhood selection method is still under investigation (Hackel et al., 2016;M. Weinmann et al., 2014M. Weinmann et al., , 2015aM. Weinmann et al., , 2015bMallet et al., 2011 andM. Weinmann et al., 2017). ...
... N selection method assigns a fixed number of k nearest points to the query point  of LiDAR data using the Euclidean distance (Hackel et al., 2016;Linsen & Prautzsch, 2001;M. Weinmann et al., 2013). Other studies applied a changing number of k neighbors for each point according to a specific condition (Demantké et al., 2011;M. Weinmann et al., 2014M. Weinmann et al., , 2015aM. Weinmann et al., , 2015b. Niemeyer et al. (2014) used the k nearest neighbors for points in the 2D projection of the MLS point cloud. ...
Road features extraction is essential for autonomous driving vehicles and road maintenance. Mobile Laser Scanning (MLS) systems have proven their capability for dense and accurate LiDAR point cloud data acquisition of road features. Usually, MLS data are received in the format of XYZ coordinates and sometimes with intensity values. Thus, the first step in MLS data processing is point classification, which mainly relays on the geometric distribution of surrounding points. However, processing such huge data is costly and time- consuming. Therefore, in this research, different neighborhood selection methods, including k nearest neighbors, spherical and cylindrical methods are evaluated to reveal the suitable method for MLS data classification. In addition, a data sub-sampling method based on minimum point spacing is applied in order to reduce the processing time. A set of point features, including covariance, moment and height was first extracted based on the three neighborhood selection methods. Random forest classifier was then used to classify a part of the benchmark dataset of Paris–Lille-3D, which belongs to NPM3D Benchmark suite research project. The dataset is divided into three main parts; Lille 1, Lille 2 and Paris. Lille 1 and Lille 2 were used in this research with about 1.5 km longitudinal road and about 98.1 million total number of points. Six scenarios were evaluated; three for the full dataset and three for the sub-sampled dataset using the aforementioned neighborhood selection methods. The results showed that the cylindrical neighborhood selection method achieved the highest classification accuracy of 92.39% and 90.26% for the full and sub-sampled datasets, respectively. The data sub-sampling has showed a good performance, whereas the dataset was reduced by about half and processing time was reduced by almost half with close classification accuracy using the cylindrical neighborhood selection method.
... Thus, in order to express the orderliness of a local seafloor surface, a function E expressing the measure of entropy of the eigenvalues is shown in (4) (Weinmann et al., 2015b;2016). ...
... This may be caused by a "binning" effect resulting from the selected range of analysis scales, where some seafloor features in the study area are broader or finer than the range of scales selected, and would be grouped into the finest or broadest scales available. This binning effect has been observed to occur when optimizing neighborhoods sizes for point cloud segmentation (Weinmann et al., 2015a(Weinmann et al., , 2015b. We suspect the selection of a suitable scale range to be essential to the success of implementing this method. ...
Seafloor substrate mapping has become increasingly important to guide the management of marine ecosystems. Full coverage substrate maps, however, cannot easily be created from point samples (e.g. grabs, videos) as a result of the time required for collection and their discrete spatial extent. Instead, relationships between substrate types and surrogate variables as obtained from bathymetric or backscatter data can be modelled to build predictive substrate maps. As calculation of these surrogate variables is scale-dependent, the scale(s) of analysis need(s) to be selected first, with multiple scales likely required to adequately capture substrate characteristics. This paper proposes an objective and automatic self-adaptive analysis scale determination approach at each bathymetric point to extract terrain features (e.g. slope, aspect, etc). Object-based image analysis (OBIA) is also used to calculate additional texture features for segmented backscatter image objects. Random Forest classification is then used to model the relationship between these extracted features and substrate types interpreted from ground-truth video data, and full-coverage seafloor substrate maps are produced. The proposed method was applied on two datasets from Newfoundland, Canada, and demonstrated good performance in terms of both overall (>80%) and per-class accuracies. The proposed method is easily transferable to other study areas and provides an objective, repeatable means for classifying seafloor substrates for environmental protection and management of marine habitats.
... Ning et al. [30] applied the local features calculated by the covariance matrix to the machine learning classification algorithm for tree extraction and achieved good classification results. Based on this, we selected 6 features that have a strong description ability for outdoor scene PCD, namely linearity , flatness , divergence , anisotropy , characteristic entropy , and curvature variation [31], these features can be calculated by Equation (2): ...
Urban trees are vital elements of outdoor scenes via mobile laser scanning (MLS), accurate individual trees detection from disordered, discrete, and high-density MLS is an important basis for the subsequent analysis of city management and planning. However, trees cannot be easily extracted because of the occlusion with other objects in urban scenes. In this work, we propose a coarse-to-fine individual trees detection method from MLS point cloud data (PCD) based on treetop points extraction and radius expansion. Firstly, an improved semantic segmentation deep network based on PointNet is applied to segment tree points from the scanned urban scene, which combining spatial features and dimensional features. Next, through calculating the local maximum, the candidate treetop points are located. In addition, the optimized treetop points are extracted after the tree point projection plane was filtered to locate the candidate treetop points, and a distance rule is used to eliminate the pseudo treetop points then the optimized treetop points are obtained. Finally, after the initial clustering of treetop points and vertical layering of tree points, a top-down layer-by-layer segmentation based on radius expansion to realize the complete individual extraction of trees. The effectiveness of the proposed method is tested and evaluated on five street scenes in point clouds from Oakland outdoor MLS dataset. Furthermore, the proposed method is compared with two existing individual trees segmentation methods. Overall, the precision, recall, and F-score of instance segmentation are 98.33%, 98.33%, and 98.33%, respectively. The results indicate that our method can extract individual trees effectively and robustly in different complex environments.
... Here, we used Cloud Compare (version 2.11, Cloud Compare, GPL software) to calculate the six geometric features ( Table 2) to classify lianas and trees. These features are often used to separate leaf and wood points from point clouds [42,43]. Table 2. Six geometric features extracted from the point cloud. ...
Lianas are self-supporting systems that are increasing their dominance in tropical forests due to climate change. As lianas increase tree mortality and reduce tree growth, one key challenge in ecological remote sensing is the separation of a liana and its host tree using remote sensing techniques. This separation can provide essential insights into how tropical forests respond, from the point of view of ecosystem structure to climate and environmental change. Here, we propose a new machine learning method, derived from Random Forest (RF) and eXtreme Gradient Boosting (XGBoosting) algorithms, to separate lianas and trees using Terrestrial Laser Scanning (TLS) point clouds. We test our method on five tropical dry forest trees with different levels of liana infestation. First, we use a multiple radius search method to define the optimal radius of six geometric features. Second, we compare the performance of RF and XGBoosting algorithms on the classification of lianas and trees. Finally, we evaluate our model against independent data collected by other projects. Our results show that the XGBoosting algorithm achieves an overall accuracy of 0.88 (recall of 0.66), and the RF algorithm has an accuracy of 0.85 (recall of 0.56). Our results also show that the optimal radius method is as accurate as the multiple radius method, with F1 scores of 0.49 and 0.48, respectively. The RF algorithm shows the highest recall of 0.88 on the independent data. Our method provides a new flexible approach to extracting lianas from 3D point clouds, facilitating TLS to support new studies aimed to evaluate the impact of lianas on tree and forest structures using point clouds.
... uclidean distance (Linsen & Prautzsch 2001). The number of neighbours is determined according to a fixed number k for all points in the MLS point cloud (Weinmann et al. 2013;Hackel et al. 2016). In addition, the k number of points may be changing for each individual point according to a specific condition (Demantké et al. 2011;Weinmann et al. 2014;M. Weinmann et al. 2015;Martin Weinmann et al. 2015). While most researchers used KNN in the 3D environment, (Niemeyer et al. 2014) used the KNN in the 2D projection of MLS point cloud. ...
3D road mapping is essential for intelligent transportation system in smart cities. Road environment receives its data from mobile laser scanning (MLS) systems in the format of LiDAR point clouds, which are distinguished with their accuracy and high density. In this paper, a mobile LiDAR data classification method based on machine learning (ML) is presented. First, data subsampling and slicing are applied, followed by cylindrical neighbourhood selection method to determine the neighbourhood of each point. Second, a new LiDAR-based point feature namely Zmod is introduced, and used along with existing fifteen geometric features as input for a ML algorithm. Finally, Random Forest classifier is applied to a part of (Paris-Lille-3D) MLS point clouds belonging to NPM3D Benchmark. The dataset is about 1.5 km long road in Lille, France with more than 98 million points. The use of Zmod improved the accuracy from 90.26% to 95.23% and achieved sufficient results for classes with low points' portion in the dataset. In addition, the Zmod is the third important feature in the classification process among the sixteen features with about 14.63%. Moreover, using the first six important features achieved almost the maximum overall accuracy with about 60% reduction in the processing time.
... In point-based classification, the individual point does not have sufficient characteristics to support the classification. There is much research that focuses on the point clouds neighborhood selection (Filin and Pfeifer, 2006), feature extraction, and feature selection (Weinmann et al., 2015;Gupta et al., 2020). However, the point-based methods still suffer from the point clouds density anisotropy, unreasonable neighborhood, and noise. ...
Point clouds classification is the basis for 3D spatial information extraction and applications. The point-clusters-based methods are proved to be more efficient and accurate than the point-based methods, however, the precision of the classification is significantly affected by the segmentation errors. The traditional single-scale point clouds segmentation methods cannot segment complex objects well in urban scenes which will result in inaccurate classification. In this paper, a new multi-scale point clouds segmentation method for urban scene point clouds classification is proposed. The proposed method consists of two stages. In the first stage, to ease the segmentation errors caused by density anisotropy and unreasonable neighborhood, a multi-resolution supervoxels segmentation algorithm is proposed to segment the objects into small-scale clusters. Firstly, the point cloud is segmented into initial supervoxels based on geometric and quantitative constraints. Secondly, robust neighboring relationships between supervoxels are obtained based on kd-tree and octree. Furthermore, the resolution of supervoxels in the planar and low-density region is optimized. In the second stage, planar supervoxels are clustered into the large-scale planar point clusters based on the region growing algorithm. Finally, a mix of small-scale and large-scale point clusters is obtained for classification. The performance of the segmentation method in classification is compared with other segmentation methods. Experimental results revealed that the proposed segmentation method can significantly improve the efficiency and accuracy of point clouds classification than other segmentation methods.
... All points within their respective supervoxel have similar features, and the centroid points are ordered in a mesh-like shape to simplify the complex computation of plane shape features. Specifically, an adjacency map containing the adjacent connections relations among supervoxels is simultaneously generated, which presents coterminous connection information that can greatly reduce the cost of neighborhood searching and improve the robustness and accuracy during neighbor calculation [40,41]. ...
As an essential part of point cloud processing, autonomous classification is conventionally used in various multifaceted scenes and non-regular point distributions. State-of-the-art point cloud classification methods mostly process raw point clouds, using a single point as the basic unit and calculating point cloud features by searching local neighbors via the k-neighborhood method. Such methods tend to be computationally inefficient and have difficulty obtaining accurate feature descriptions due to inappropriate neighborhood selection. In this paper, we propose a robust and effective point cloud classification approach that integrates point cloud supervoxels and their locally convex connected patches into a random forest classifier, which effectively improves the point cloud feature calculation accuracy and reduces the computational cost. Considering the different types of point cloud feature descriptions, we divide features into three categories (point-based, eigen-based, and grid-based) and accordingly design three distinct feature calculation strategies to improve feature reliability. Two International Society of Photogrammetry and Remote Sensing benchmark tests show that the proposed method achieves state-of-the-art performance, with average F1-scores of 89.16 and 83.58, respectively. The successful classification of point clouds with great variation in elevation also demonstrates the reliability of the proposed method in challenging scenes.
... HREE-Dimensional (3D) visualization of urban scenes plays an extremely vital role in the process of digital city construction. Point clouds are an effective and visualized manner for recording and describing 3D urban information, which is widely used in urban monitoring [1][2][3], 3D model reconstruction [4][5][6], and artificial facility interpretation [7][8][9][10]. Photogrammetric stereo matching [11][12][13][14] and laser scanning technology [15,16] are the main methods to acquire the largerscale urban point clouds. ...
Array interferometric synthetic aperture radar (Array InSAR) has a 3-D resolution capability and solves the layover problem in interferometric SAR (InSAR) by arranging multiple antennas in the cross-orbit direction. Airborne Array InSAR point clouds are obtained from two scans for complete building information in urban areas, resulting in very low overlapping point cloud. The existing methods are difficult to extract the identical features for the registration of Array InSAR point clouds. To this end, a robust registration approach Array InSAR point clouds in urban areas is proposed in this study. The main contribution of this article is raising the theoretically optimal transformation for achieving point cloud registration, considering the constraint from parallel facades of a certain building. Point density estimation is adopted to retain building facade points for initial registration. The facade pairs of a specific building are then matched and divided into two categories by judging whether one contains the concave–convex features or not, for performing rotation rectification and fine shift fixation, respectively. Experimental results of both simulated and real data validate the feasibility and reliability of our approach. For the simulated data, the results reach an average rotation error of about 0.01° and an average translation error of less than 0.8 m. For the real data, two evaluation criteria are designed for the lack of reference data. The results reach an average of 0.4° of the defined angle difference and less 0.8-m distance difference from the source facades center to the normal extension of the target facades.
... Mekânsal alanlarda elde edilen verilerin otomatik analizi, fotogrametri, uzaktan algılama, bilgisayarlı görü ve robotik uygulamalarda yoğun olarak çalışılan bir konu haline gelmiştir. [4][5][6][7][8][9]. ...
... The definition of these features is shown in Table 2.1, while Figure 2.5 illustrates how these features highlight different parts of the point cloud according to the local geometry. Weinmann et al. (2015), did the same analysis using the k-NN of each point. Moreover, they also proposed other features to describe the geometrical structure of the local neighbourhoods, as for example Eigentropy. ...
Graphs are powerful mathematical structures representing a set of objects and the underlying links between pairs of objects somehow related. They are becoming increasingly popular in data science in general and in particular in image or 3D point cloud analysis. Among the wide spectra of applications, they are involved in most of the hierarchical approaches.Hierarchies are particularly important because they allow us to efficiently organize the information required and to analyze the problems at different levels of detail. In this thesis, we address the following topics. Many morphological hierarchical approaches rely on the Minimum Spanning Tree (MST). We propose an algorithm for MST computation in streaming based on a graph decomposition strategy. Thanks to this decomposition, larger images can be processed or can benefit from partial reliable information while the whole image is not completely available.Recent LiDAR developments are able to acquire large-scale and precise 3D point clouds. Many applications, such as infrastructure monitoring, urban planning, autonomous driving, precision forestry, environmental assessment, archaeological discoveries, to cite a few, are under development nowadays. We introduce a ground detection algorithm and compare it with the state of the art. The impact of reducing the point cloud density with low-cost scanners is studied, in the context of an autonomous driving application. Finally, in many hierarchical methods similarities between points are given as input. However, the metric used to compute similarities influences the quality of the final results. We exploit metric learning as a complementary tool that helps to improve the quality of hierarchies. We demonstrate the capabilities of these methods in two contexts. The first one,a texture classification of 3D surfaces. Our approach ranked second in a task organized by SHREC’20 international challenge. The second one learning the similarity function together with the optimal hierarchical clustering, in a continuous feature-based hierarchical clustering formulation.
... Consequently, the scalability of approaches towards data-intensive processing is becoming more and more important. This has, for instance, been addressed with recent approaches focusing on the enrichment of acquired data with semantic information (Hackel et al., 2016;Smeeckaert et al., 2013;Weinmann et al., 2015) or the creation of 3D models from the acquired data (Lafarge and Mallet, 2012;Vosselman et al., 2015). Furthermore, it has to be taken into account that, simultaneously to the consideration of larger and larger areas of interest, the level of detail of the acquired data is steadily increasing due to the increasing performance of respective acquisition systems in recent years. ...
... Early research on ALS point cloud classification focused on extracting handcrafted features, such as eigenvalues [11,12], shape and geometry features [13][14][15] and using traditional supervised classifiers, such as support vector machine (SVM) [16][17][18], random forests [19,20], AdaBoost [14,21], Markov random field (MRF) [22][23][24], conditional random field (CRF) [25,26] and so on. However, these traditional machine learning-based methods rely heavily on professional experience and have limited generalizability when applied to complex, large-scale scenes [27]. ...
In various applications of airborne laser scanning (ALS), the classification of the point cloud is a basic and key step. It requires assigning category labels to each point, such as ground, building or vegetation. Convolutional neural networks have achieved great success in image classification and semantic segmentation, but they cannot be directly applied to point cloud classification because of the disordered and unstructured characteristics of point clouds. In this paper, we design a novel convolution operator to extract local features directly from unstructured points. Based on this convolution operator, we define the convolution layer, construct a convolution neural network to learn multi-level features from the point cloud, and obtain the category label of each point in an end-to-end manner. The proposed method is evaluated on two ALS datasets: the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen 3D Labeling benchmark and the 2019 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest (DFC) 3D dataset. The results show that our method achieves state-of-the-art performance for ALS point cloud classification, especially for the larger dataset DFC: we get an overall accuracy of 97.74% and a mean intersection over union (mIoU) of 0.9202, ranking in first place on the contest website.
... Change of curvature = 3 1 + 2 + 3 Weinmann et al., 2015b). Even if all the eigenfeatures have been extracted, it must consider they may contain redundant of irrelevant information with respect to the classification task. ...
Recent years showed a gradual transition from terrestrial to aerial survey thanks to the development of UAV and sensors for it. Many sectors benefited by this change among which geological one; drones are flexible, cost-efficient and can support outcrops surveying in many difficult situations such as inaccessible steep and high rock faces. The experiences acquired in terrestrial survey, with total stations, GNSS or terrestrial laser scanner (TLS), are not yet completely transferred to UAV acquisition. Hence, quality comparisons are still needed. The present paper is framed in this perspective aiming to evaluate the quality of the point clouds generated by an UAV in a geological context; data analysis was conducted comparing the UAV product with the homologous acquired with a TLS system. Exploiting modern semantic classification, based on eigenfeatures and support vector machine (SVM), the two point clouds were compared in terms of density and mutual distance. The UAV survey proves its usefulness in this situation with a uniform density distribution in the whole area and producing a point cloud with a quality comparable with the more traditional TLS systems.
... The handcrafted features normally refer to the geometric features that are generated from a 2D or 3D neighborhood, such as eigenvalue-based features [20], height features [21] and echo features [22]. A comprehensive description of handcrafted features derived from 2D and 3D neighborhoods can be found in the work of Weinmann et al. [23]. As for the classifiers, various classical machine learning models are available for ALS point clouds, such as Decision Tree (DT) [1], Bayesian discriminant classifiers [24], Support Vector Machines (SVM) [2,3], Adaboost [25,26] and Random Forest (RF) [4,5]. ...
The success achieved by deep learning techniques in image labelling has triggered a growing interest in applying deep learning for 3D point cloud classification. To provide better insights into different deep learning architectures and their applications to ALS point cloud classification, this paper presents a comprehensive comparison among three state-of-the-art deep learning networks: PointNet++, SparseCNN and KPConv, on two different ALS datasets. The performances of these three deep learning networks are compared w.r.t. classification accuracy, computation time, generalization ability as well as the sensitivity to the choices of hyper-parameters. Overall, we observed that PointNet++, SparseCNN and KPConv all outperform Random Forest on the classification results. Moreover, SparseCNN leads to a slightly better classification result compared to PointNet++ and KPConv, while requiring less computation time and memory. At the same time, it shows a better ability to generalize and is less impacted by the different choices of hyper-parameters.
... Good feature is important for high-precision point cloud classification. Commonly used point cloud features can be generally divided into point-based (Blomley et al., 2014;Weinmann et al., 2015;Zhang et al., 2016), segment-based (Aijazi et al., 2013;Xiang et al., 2018;Zhou et al., 2012) and learning-based (Graham et al., 2018;Tchapmi et al., 2017) levels. ...
Point cloud classification is quite a challenging task due to the existence of noises, occlusion and various object types and sizes. Currently, the commonly used statistics-based features cannot accurately characterize the geometric information of a point cloud. This limitation often leads to feature confusion and classification mistakes (e.g., points of building corners and vegetation always share similar statistical features in a local neighbourhood, such as curvature, sphericity, etc). This study aims at solving this problem by leveraging the advantage of both the supervoxel segmentation and multi-scale features. For each point, its multi-scale features within different radii are extracted. Simultaneously, the point cloud is partitioned into simple supervoxel segments. After that, the class probability of each point is predicted by the proposed SegMSF approach that combines multi-scale features with the supervoxel segmentation results. At the end, the effect of data noises is supressed by using a global optimization that encourages spatial consistency of class labels. The proposed method is tested on both airborne laser scanning (ALS) and mobile laser scanning (MLS) point clouds. The experimental results demonstrate that the proposed method performs well in terms of classifying objects of different scales and is robust to noise.
... In previous work, to create discriminative features, studies have exploited both point geometry and inherent attributes. Many contextual features extracted from spatial distributions and directions of points have shown their effectiveness in the classification task , such as eigenvalue based features from covariance matrix of point coordinates (Chehata et al., 2009;Weinmann et al., 2015a;Weinmann et al., 2015c), waveform-based features from transformation (Jutzi and Gross, 2010;Zhang et al., 2011), 2D projected patterns , elevation values and height differences (Maas, 1999;Gorgens et al., 2017;Sun et al., 2018), and orientations of points from normal vectors (Rabbani et al., 2006). However, designing good handcrafted features is a critical and difficult task, which requires a good understanding of the scanned objects and highly depends on empirical tests . ...
Semantic labeling is an essential but challenging task when interpreting point clouds of 3D scenes.
As a core step for scene interpretation, semantic labeling is the task of annotating every point in the point cloud with a label of semantic meaning, which plays a significant role in plenty of point cloud related applications.
For airborne laser scanning (ALS) point clouds, precise annotations can considerably broaden its use in various applications.
However, accurate and efficient semantic labeling is still a challenging task, due to the sensor noise, complex object structures, incomplete data, and uneven point densities.
In this work, we propose a novel neural network focusing on semantic labeling of ALS point clouds, which investigates the importance of long-range spatial and channel-wise relations and is termed as global relation-aware attentional network (GraNet).
GraNet first learns local geometric description and local dependencies using a local spatial discrepancy attention convolution module (LoSDA).
In LoSDA, the orientation information, spatial distribution, and elevation information are fully considered by stacking several local spatial geometric learning modules and the local dependencies are learned by using an attention pooling module.
Then, a global relation-aware attention module (GRA), consisting of a spatial relation-aware attention module (SRA) and a channel relation-aware attention module (CRA), is presented to further learn attentions from the structural information of a global scope from the relations and enhance high-level features with the long-range dependencies.
The aforementioned two important modules are aggregated in the multi-scale network architecture to further consider scale changes in large urban areas.
We conducted comprehensive experiments on two ALS point cloud datasets to evaluate the performance of our proposed framework.
The results show that our method can achieve higher classification accuracy compared with other commonly used advanced classification methods.
For the ISPRS benchmark dataset, our method improves the overall accuracy (OA) to 84.5% and the average F_1 measure (AvgF_1) to 73.6%, which outperforms other baselines.
Besides, experiments were conducted using a new ALS point cloud dataset covering highly dense urban areas and a newly published large-scale dataset.
... In previous work, to create discriminative features, studies have exploited both point geometry and inherent attributes. Many contextual features extracted from spatial distributions and directions of points have shown their effectiveness in the classification task , such as eigenvalue based features from covariance matrix of point coordinates (Chehata et al., 2009;Weinmann et al., 2015a;Weinmann et al., 2015c), waveform-based features from transformation (Jutzi and Gross, 2010;Zhang et al., 2011), 2D projected patterns , elevation values and height differences (Maas, 1999;Gorgens et al., 2017;Sun et al., 2018), and orientations of points from normal vectors (Rabbani et al., 2006). However, designing good handcrafted features is a critical and difficult task, which requires a good understanding of the scanned objects and highly depends on empirical tests . ...
In this work, we propose a novel neural network focusing on semantic labeling of ALS point clouds, which investigates the importance of long-range spatial and channel-wise relations and is termed as global relation-aware attentional network (GraNet). GraNet first learns local geometric description and local dependencies using a local spatial discrepancy attention convolution module (LoSDA). In LoSDA, the orientation information, spatial distribution, and elevation differences are fully considered by stacking several local spatial geometric learning modules and the local dependencies are embedded by using an attention pooling module. Then, a global relation-aware attention module (GRA), consisting of a spatial relation-aware attention module (SRA) and a channel relation aware attention module (CRA), are investigated to further learn the global spatial and channel-wise relationship between any spatial positions and feature vectors. The aforementioned two important modules are embedded in the multi-scale network architecture to further consider scale changes in large urban areas. We conducted comprehensive experiments on two ALS point cloud datasets to evaluate the performance of our proposed framework. The results show that our method can achieve higher classification accuracy compared with other commonly used advanced classification methods. The overall accuracy (OA) of our method on the ISPRS benchmark dataset can be improved to 84.5% to classify nine semantic classes, with an average F1 measure (AvgF1) of 73.5%. In detail, we have following F1 values for each object class: powerlines: 66.3%, low vegetation: 82.8%, impervious surface: 91.8%, car: 80.7%, fence: 51.2%, roof: 94.6%, facades: 62.1%, shrub: 49.9%, trees: 82.1%. Besides, experiments were conducted using a new ALS point cloud dataset covering highly dense urban areas.
... Vilariño et al. (2017) employ a two-phase region growing algorithm for the segmentation of point clouds and then classify the segments by a rule-based classification method. In contrast to classification after segmentation, classification of individual points without segmentation (Weinmann et al., 2015a, * Corresponding author Blomley andWeinmann, 2017, Bauchet andLafarge, 2019) and segmentation as a refinement step after point-wise classification have also been explored. Polewski et al. (2014) similarly employ point classification as the initial step of segmentation for the detection of fallen trees. ...
Classification and segmentation of buildings from airborne lidar point clouds commonly involve point features calculated within a local neighborhood. The relative change of the features in the immediate surrounding of each point as well as the spatial relationships between neighboring points also need to be examined to account for spatial coherence. In this study we formulate the point labeling problem under a global graph-cut optimization solution. We construct the energy function through a graph representing a Markov Random Field (MRF). The solution to the labeling problem is obtained by finding the minimum-cut on this graph. We have employed this framework for three different labeling tasks on airborne lidar point clouds. Ground filtering, building classification, and roof-plane segmentation. As a follow-up study on our previous ground filtering work, this paper examines our building extraction approach on two airborne lidar datasets with different point densities containing approximately 930K points in one dataset and 750K points in the other. Test results for building vs. non-building point labeling show a 97.9% overall accuracy with a kappa value of 0.91 for the dataset with 1.18 pts/m2 average point density and a 96.8% accuracy with a kappa value of 0.90 for the dataset with 8.83 pts/m2 average point density. We can achieve 91.2% overall average accuracy in roof plane segmentation with respect to the reference segmentation of 20 building roofs involving 74 individual roof planes. In summary, the presented framework can successfully label points in airborne lidar point clouds with different characteristics for all three labeling problems we have introduced. It is robust to noise in the calculated features due to the use of global optimization. Furthermore, the framework achieves these results with a small training sample size.
... We build on several important works to compare the effects of geometric features. First, there is the study by Weinmann et al. [12,85] and Dittrich et al. [86] in which the issue is addressed of how to increase the distinctiveness of geometric features and select the most relevant ones for point cloud classification. It is stated that point cloud features and class characteristics have significant variance which require robust classification models such as Random Forests (RF) [87] or Bagged Trees. ...
Interpreting 3D point cloud data of the interior and exterior of buildings is essential for automated navigation, interaction and 3D reconstruction. However, the direct exploitation of the geometry is challenging due to inherent obstacles such as noise, occlusions, sparsity or variance in the density. Alternatively, 3D mesh geometries derived from point clouds benefit from preprocessing routines that can surmount these obstacles and potentially result in more refined geometry and topology descriptions. In this article, we provide a rigorous comparison of both geometries for scene interpretation. We present an empirical study on the suitability of both geometries for the feature extraction and classification. More specifically, we study the impact for the retrieval of structural building components in a realistic environment which is a major endeavor in Building Information Modeling (BIM) reconstruction. The study runs on segment-based structuration of both geometries and shows that both achieve recognition rates over 75% F1 score when suitable features are used.
... A significant number of 3D approaches have been developed in response to previous limitations (Lopes et al., 2019), some concerned with laser scanning of the city and massive analysis of point clouds (Bonczak & Kontokosta, 2019;Czyńska & Rubinowicz, 2019;Weinmann, Urban, Hinz, Jutzi, & Mallet, 2015), which are considered as time-consuming and costly procedures (Yabuki et al., 2011). On the other hand, image simulation of the existing condition and proposed alternative is another approach, which sets the stage for a before-after comparison and the visual evaluation of a development project from Key Observation Points (KOPs) (An & Powe, 2015;Palmer, 2019b). ...
Rapid urban development has posed destructive effects on the visual quality of many cities over the past few decades. Consequently, Visibility Analysis (VA) methods have been adopted to evaluate and prevent the visual influences of the development over valuable scenes. Shiraz' strategic view of the Quran Gate, in especial, is known for its remarkable landmarks, while the visual effects of the recent rapid urban growth on the view have been a matter of concern. The future of the view is even more at risk due to the lack of synergy between the urban development plan and historic preservation regulation. Consequently, this study aims to undertake a Visibility Analysis of the height regulation of the Shiraz Development Plan on the view of the Quran Gate. In this essence, two 3D models of the study area are produced based on the existing condition and the Shiraz Development Plan scenario. By choosing seventeen key viewpoints, visibility analysis of landmarks and viewpoints was conducted. The results demonstrate the destructive visual effects of the development plan on the strategic view. It is suggested that the proposed cost-efficient methodology can minimize subjective evaluation and contribute to the synergy between development and conservation plans.
... To this end, solutions for optimizing the neighborhood are commonly proposed, e.g., optimal neighborhood adaptation (Belton and Lichti, 2006;Demantke et al., 2011;Weinmann et al., 2015a), multi-scale neighborhood aggregation (Kang and Yang, 2018;Xu et al., 2014;Zhang et al., 2016;Blomley and Weinmann, 2017), eliminating the strong assumption in fixed neighborhood setting. Regarding feature extraction, to exploit the importance of the features for classification, various types of features such as eigenvalue-based (Chehata et al., 2009;Weinmann et al., 2015a;Weinmann et al., 2015c), waveformbased (Jutzi and Gross, 2010;Zhang et al., 2011), 2D image (Zhao et al., 2018), and height features (Maas, 1999;Gorgens et al., 2017;Sun et al., 2018) have been studied. Nevertheless, the performance of these aforementioned traditional methods for both neighborhood selection and feature extraction are highly associated with the knowledge of point clouds. ...
Semantic interpretation of the 3D scene is one of the most challenging problems in point cloud processing, which also deems as an essential task in a wide variety of point cloud applications. The core task of semantic interpretation is semantic labeling, namely, obtaining a unique semantic label for each point in the point cloud. Despite several reported approaches, semantic labeling continues to be a challenge owing to the complexity of scenes, objects of various scales, and the non-homogeneity of unevenly distributed points. In this paper, we propose a novel method for obtaining semantic labels of airborne laser scanning (ALS) point clouds involving the embedding of local context information for each point with multi-scale deep learning, nonlinear manifold learning for feature dimension reduction, and global graph-based optimization for refining the classification results. Specifically, we address the tasks of learning discriminative features and global labeling smoothing. The key contribution of our study is threefold. First, a hierarchical data augmentation strategy is applied to enhance the learning of deep features based on the PointNet++ network and simultaneously eliminate the artifacts caused by division and sampling while dealing with large-scale datasets. Subsequently, the learned hierarchical deep features are globally optimized and embedded into a low-dimensional space with a nonlinear manifold-based joint learning method with the removal of redundant and disturbing information. Finally, a graph-structured optimization based on the Markov random fields algorithm is performed to achieve global optimization of the initial classification results that are obtained using the embedded deep features by constructing a weighted indirect graph and solving the optimization problem with graph-cuts. We conducted thorough experiments on ALS point cloud datasets to assess the performance of our framework. Results indicate that compared to other commonly used advanced classification methods, our method can achieve high classification accuracy. The overall accuracy (OA) of our approach on the ISPRS benchmark dataset can scale up to 83.2% for classifying nine semantic classes, thereby outperforming other compared point-based strategies. Additionally, we evaluated our framework on a selected portion of the AHN3 dataset, which provided OA up to 91.2%.
... The classifier prioritizes the most dominant classes in order to obtain a better result with the minimum of operations. [23] obtain an F1 score index of more than 90% for the road and façade classes, while for less dominant classes (motorcycles, traffic sign and pedestrians), the F1 index does not exceed 25%. The computational cost increases according to the number of points, as features are extracted point by point and not grouped in objects. ...
The application of Deep Learning techniques to point clouds for urban object classification is limited by the large number of samples needed. Acquiring and tagging point clouds is more expensive and tedious labour than its image equivalent process. Point cloud online datasets contain few samples for Deep Learning or not always the desired classes This work focuses on minimizing the use of point cloud samples for neural network training in urban object classification. The method proposed is based on the conversion of point clouds to images (pc-images) because it enables: the use of Convolutional Neural Networks, the generation of several samples (images) per object (point clouds) by means of multi-view, and the combination of pc-images with images from online datasets (ImageNet and Google Images). The study is conducted with ten classes of objects extracted from two street point clouds from two different cities. The network selected for the job is the InceptionV3. The training set consists of 5000 online images with a variable percentage (0% to 10%) of pc-images. The validation and testing sets are composed exclusively of pc-images. Although the network trained only with online images reached 47% accuracy, the inclusion of a small percentage of pc-images in the training set improves the classification to 99.5% accuracy with 6% pc-images. The network is also applied at IQmulus & TerraMobilita Contest dataset and it allows the correct classification of elements with few samples.
... First, in order to highlight the rod-shaped features and the planar features, the point cloud is contracted by Laplacian smoothing, and then the point cloud is formed into a rod-shaped, planar, and mixed ground by clustering, finally, they identified the rod-like features through some combination rules. Multi-level neighborhoods can be implemented by setting the multi-level neighborhood size [11] or adaptively determining the neighborhood size based on the scale parameter [12]. In addition to multi-level neighborhoods, multi-scale features based on multi-level segmentation have also been studied by many scholars. ...
Airborne Light Detection And Ranging (LiDAR) has become an important means for efficient and high-precision acquisition of 3D spatial data of large scenes. It has important application value in digital cities and location-based services. The classification and identification of point cloud is the basis of its application, and it is also a hot and difficult problem in the field of geographic information science.The difficulty of LiDAR point cloud classification in large-scale urban scenes is: On the one hand, the urban scene LiDAR point cloud contains rich and complex features, many types of features, different shapes, complex structures, and mutual occlusion, resulting in large data loss; On the other hand, the LiDAR scanner is far away from the urban features, and is like a car, a pedestrian, etc., which is in motion during the scanning process, which causes a certain degree of data noise of the point cloud and uneven density of the point cloud.Aiming at the characteristics of LiDAR point cloud in urban scene.The main work of this paper implements a method based on the saliency dictionary and Latent Dirichlet Allocation (LDA) model for LiDAR point cloud classification. The method uses the tag information of the training data and the tag source of each dictionary item to construct a significant dictionary learning model in sparse coding to expresses the feature of the point set more accurately.And it also uses the multi-path AdaBoost classifier to perform the features of the multi-level point set. The classification of point clouds is realized based on the supervised method. The experimental results show that the feature set extracted by the method combined with the multi-path classifier can significantly improve the cloud classification accuracy of complex city market attractions.
The fast semantic segmentation algorithm of 3D laser point clouds for large scenes is of great significance for mobile information measurement systems, but the point cloud data is complex and generates problems such as disorder, rotational invariance, sparsity, severe occlusion, and unstructured data. We address the above problems by proposing the random sampling feature aggregation module ATSE module, which solves the problem of effective aggregation of features at different scales, and a new semantic segmentation framework PointLAE, which effectively presegments point clouds and obtains good semantic segmentation results by neural network training based on the features aggregated by the above module. We validate the accuracy of the algorithm by training on Semantic3D, a public dataset of large outdoor scenes, with an accuracy of 90.3, while verifying the robustness of the algorithm on Mvf CNN datasets with different sparsity levels, with an accuracy of 86.2, and on Bjfumap data aggregated by our own mobile environmental information collection platform, with an accuracy of 77.4, demonstrating that the algorithm is good for mobile information complex scale data in mobile information collection with great recognition effect.
Detecting narrow butt weld seam with high precision has become an urgent problem with the wide application of laser welding technology. Many previous methods use line laser to locate the welds. However, these methods can only get a single position of the weld seam in each shooting and the detection scope is limited to the laser projection area, leading to low detection efficiency. To extract the narrow butt welds more efficiently, this paper combines the passive methods with the active methods, and proposes a 3D narrow butt weld seam detection system based on the binocular consistency analysis. Specifically, the active light method of fringe projection profilometry is adopted to capture the 3D information of the weldment. The weld seam extraction network based on binocular spatial information mining (BSMNet) is designed to analyze the corresponding passive light data and locate the weld seam position. Besides, a data annotation method based on binocular consistency correction is proposed to achieve more accurate data annotation for the BSMNet training. The experimental results show the max error of the detection is about 0.081mm, and the mean error is about 0.021mm.
The enrichment of the point clouds with colour images improves the visualisation of the data as well as the segmentation and recognition processes. Coloured point clouds are becoming increasingly common, however, the colour they display is not always as expected. Errors in the colouring of point clouds acquired with Mobile Laser Scanning are due to perspective in the camera image, different resolution or poor calibration between the LiDAR sensor and the image sensor. The consequences of these errors are noticeable in elements captured in images, but not in point clouds, such as the sky. This paper focuses on the correction of the sky-coloured points, without resorting to the images that were initially used to colour the whole point cloud. The proposed method consists of three stages. First the region of interest where the erroneously coloured points are accumulated, is selected. Second, the sky-coloured points are detected by calculating the colour distance in the Lab colour space to a sample of the sky-colour. And third, the colour of the sky-coloured detected points is restored from the colour of the nearby points. The method is tested in ten real case studies with their corresponding point clouds from urban and rural areas. In two case studies, sky-coloured points were assigned manually and the remaining eight case studies, the sky-coloured points are derived from the acquisition errors. The algorithm for sky-coloured points detection obtained an average F1-score of 94.7%. The results show a correct reassignment of colour, texture, and patterns, while improving the point cloud visualisation.
Due to the large scale and complex terrain of some outcrops, it is difficult to carry out comprehensive and detailed stratum identification using traditional geological methods. The emergence of oblique photogrammetry technology provides a new way to construct high-precision 3D models of outcrops, but the existing object extraction methods of 3D models cannot be applied to the differentiation of strata in an outcrop. Therefore, this paper proposes a novel identification method for outcropping strata from oblique photogrammetric data using a spatial case-based reasoning (SCBR) model. The method includes (a) constructing point cloud of the outcrop based on the photogrammetric data; (b) segmenting the point cloud into voxels; (c) creating an integrated expression of the spatial and attribute features of a stratum case; (d) creating an integrated similarity reasoning model of the spatial and attribute features; and (e) dividing the strata after reasoning. Furthermore, this study carried out stratum identification experiments on two cases of study in Fugu, Shaanxi, China. After Performing the SCBR model, the overall accuracies were found to reach 94%. After the stratum division after reasoning was carried out, the verification accuracies were found to exceed 99%. Based on the results of the manual stratum differentiation, the differentiation results of the proposed SCBR model were found to be much more consistent with the actual situation. The experiments show that the proposed method is very effective for differentiating outcropping strata and provides a new approach for the extraction of 3D objects generated with oblique photogrammetric data.
Urban object segmentation and classification tasks are critical data processing steps in scene understanding, intelligent vehicles and 3D high-precision maps. Semantic segmentation of 3D point clouds is the foundational step in object recognition. To identify the intersecting objects and improve the accuracy of classification, this paper proposes a segment-based classification method for 3D point clouds. This method firstly divides points into multi-scale supervoxels and groups them by proposed inverse node graph (IN-Graph) construction, which does not need to define prior information about the node, it divides supervoxels by judging the connection state of edges between them. This method reaches minimum global energy by graph cutting, obtains the structural segments as completely as possible, and retains boundaries at the same time. Then, the random forest classifier is utilized for supervised classification. To deal with the mislabeling of scattered fragments, higher-order CRF with small-label cluster optimization is proposed to refine the classification results. Experiments were carried out on mobile laser scan (MLS) point dataset and terrestrial laser scan (TLS) points dataset, and the results show that overall accuracies of 97.57% and 96.39% were obtained in the two datasets. The boundaries of objects were retained well, and the method achieved a good result in the classification of cars and motorcycles. More experimental analyses have verified the advantages of the proposed method and proved the practicability and versatility of the method.
The ongoing transformation of our real-world into digital representations increasingly asks for tools allowing the understanding of unstructured data. Contributions to this problem are coming from the field of computer vision, where different strategies try to detect and classify objects in data sets. On the one hand, such strategies are model-driven, which are typically focused on either a specific type of data, a context (indoor, outdoor), or a set of objects. Other approaches are based on machine learning concepts, providing higher flexibility but needing a costly step of annotating data which is the prerequisite to training a learning process. That leads to problems when the data are not available. To overcome such constraints, we propose substituting the implicitly contained knowledge in the data, which is accessed through a training process, by explicit formulated knowledge in a fully semantic-guided approach. The semantics’ role is to express all relevant knowledge of the objects’ representation inside the data sets, and the al- gorithms that address this representation. The approach also contains a learning phase since it adapts the processing according to the objects’ diversity and data features. The semantic is expressed via an ontological model and uses standard Web technologies like SPARQL queries, providing great flexibility. The ontological model describes the object, the data, and the algorithms. It allows the selection and execution of algorithms adapted to the data and objects. Similarly, processing results are classified and enrich the ontological model using SPARQL construct queries. The semantic formulated through SPARQL also acts as a connection between the knowledge within the ontological model and the processing branch. It pro- vides the capability to adapt the algorithms’ sequence to an individual state of the processing chain and makes the solution adaptable. Its ability to adjust to different objects’ recognition in different contexts is shown in two case studies. The first case study aims at recognizing small objects in an indoor point cloud acquired by low-cost technology. In contrast, the second case study aims at identifying larger objects in an outdoor point cloud acquired by laser scanning. For both of these case studies, our semantic-guided approach obtained good results. The recognition quality was significantly improved through a learning phase auto adapting to the context, allowing them to recognize cars, motorcycles, and traffic signs better than other literature approaches.
İç ve dış mekânlarda bulunan yapı ve nesneler Lidar (ışık algılayan ve mesafe ölçen) sistemler ile taranarak nokta bulutu halinde, üç boyutlu (3D) ve renkli olarak dijital ortamlara aktarılabilmektedir. Lidar taramasıyla elde edilen, yapı ve nesneler hakkında detaylı bilgi sağlayan bu 3D nokta bulutu verisinin elemanları olan noktalar, organize edilmiş bir veri yapısı içerisinde konumlandırılmamış olarak düzensiz bir şekilde gelmektedir. Günümüzde Lidar teknolojisindeki gelişmeler, nokta bulutu verilerinin kalitesini artırmasının (daha az konum hatası ve daha yüksek çözünürlüklü olarak) yanında, çok yüksek miktarlarda düzensiz veri yığınını da getirmiştir. Çok yüksek boyuttaki bir veriden benzer özellikteki ve konumsal yakınlığı olan verileri gruplayarak, işlenecek veri sayısını düşürmekle birlikte veriden daha anlamlı bilgiler çıkarılmasını da sağlayan işleme segmentasyon denilmektedir. Segmentasyon, 3D nokta bulutu işlemeyi de kapsayan bilgisayarlı görme alanında büyük miktarda veri ile uğraşmayı gerektiren uygulamalar için oldukça yüksek bir öneme sahiptir. Segmentasyon işleminin de karmaşık veriler üzerinde istenilen özelliklerde ve sürede sonuç vermesi, bilgisayarlı görme alanında ayrı bir uğraş konusu olmuştur. Tez çalışmasında, 3D nokta bulutu işlemede, uygulamanın başarısını önemli oranda etkileyen segmentasyon işleminin daha başarılı ve hızlı bir şekilde yapılabilmesi için yeni bir voksel tabanlı segmentasyon yöntemi geliştirilmiştir. Geliştirilen yöntem, yüzeylerdeki lokal nokta gruplarının oluşturdukları düzlemsel yapıların eğim açıları ve ağırlık merkezleri gibi basit geometrik özelliklerini kullanarak segmentasyon işlemini gerçekleştirebilmiştir. Tez kapsamında, literatürde kullanılan veri setlerinin özellikleri dikkate alınarak, benzer şekilde bir adet iç ve iki adet dış mekânsal ortam, bir karasal Lidar sistemi ile taranarak üç farklı 3D nokta bulutu verisi temin edilmiştir. Elde edilen ham nokta verileri, oluşturulan veri setinin kullanım amacına göre indirgeme, kırpma ve gürültü giderme gibi ön işlemlerden geçirildikten sonra, segmentasyon referans segmentleri de hazırlanarak üç adet veri seti oluşturulmuştur. Tez kapsamında hazırlanan veri setlerine ek olarak, literatürden de iki adet daha segmentasyon veri seti temin edilmiş ve böylece toplam beş adet veri seti segmentasyon karşılaştırmasında kullanılmıştır. Veri setlerinin temin edilmesinin ardından, yöntemlerin nicel değerler üzerinden karşılaştırması aşamasına kadar olan geliştirme ve iyileştirme aşamaları iki ayrı koldan eş zamanlı olarak ilerlemiştir. Bunlardan birisi sekiz dallı ağaç (octree) organizasyonu ile veriyi vokselleme tekniğinin, düzlem özelliği göstermeyen vokseller için yeniden düzlem uydurma ön işleminin ve geliştirilen segmentasyon yönteminin kodlanması aşamalarıdır. Diğeri ise karşılaştırma için literatürde başarı göstermiş segmentasyon yöntemlerinin belirlenmesi, bunların temin edilmesi veya yeniden kodlanması ve nicel karşılaştırma için doğruluk ve F1 skor değerleri hesaplama yöntemlerinin kodlanması aşamalarıdır. Bütün bu geliştirme, iyileştirme ve kodlama aşamaları tamamlandıktan sonra uygulanan yöntemlerin tez kapsamında kullanılan veri setleri üzerindeki segmentasyon çıktılarının doğruluk ve F1 skor sonuçları alınarak, başarı ve çalışma süresi açısından karşılaştırma analizleri yapılmıştır. Geliştirilen yöntem, 0.81 ortalama doğruluk değeri ve 0.69 ortalama F1 skor değeri ile literatürde bulunan ve benzer şekilde noktaların geometrik özelliklerini kullanarak segmentasyon yapan diğer yöntemlere göre segmentasyon başarısı ve hız açısından üstünlük elde etmiştir. Tez kapsamında ayrıca, nokta bulutundaki noktaların renk değerlerinin farklılıkları da belirli etki oranlarında segmentasyona dâhil edilmiş ve renk kalitesi yüksek iç mekân verisinde başarı arttırılmıştır. Tez kapsamında daha sonra, geliştirilen yöntemin farklı parametre değerleri ile literatürden alınan yüksek miktarda noktadan oluşan bir iç mekân anlamsal segmentasyon (semantic segmentation) veri seti (S3DIS) üzerindeki ham nokta bulutu sınıflandırmasında ara işlem olarak kullanımı da incelenmiştir. Sınıflandırma işlemi için, öncelikle geliştirilen yöntemle segmentasyon yapılarak veri segmentlere ayrılmış ve her segmentten bir özellik vektörü çıkarılmıştır. Daha sonra da, bu özellik vektörleri kullanılarak sınıflandırma yapılmıştır. Segmentasyon tabanlı sınıflandırma işlemi, Destek Vektör Makinesi (DVM) ve Rastgele Orman (RO) olarak iki farklı sınıflandırıcı ile ayrı ayrı uygulanmıştır. Sınıflandırma işlemlerinin sonuçları da noktaların sınıf etiketlerinin doğruluk ve F1 skor değerleri üzerinden karşılaştırılmıştır. Karşılaştırma sonuçlarına göre, ham nokta bulutundaki noktaların sınıflandırma başarıları DVM için 0.76 doğruluk ve 0.48 F1 skor iken RO için 0.83 doğruluk ve 0.70 F1 skor olmuştur. Sonuçlara bakıldığında kullanılan veri ve özellik setlerine göre RO sınıflandırıcısı DVM sınıflandırıcısından daha iyi sonuç vermiştir.
The perception system has become a topic of great importance for autonomous vehicles, as high accuracy and real-time performance can ensure safety in complex urban scenarios. Clustering is a fundamental step for parsing point cloud due to the extensive input data (over 100,000 points) of a wide variety of complex objects. It is still challenging to achieve high precision real-time performance with limited vehicle-mounted computing resources, which need to balance the accuracy and processing time. We propose a method based on a Two-Layer-Graph (TLG) structure, which can be applied in a real autonomous vehicle under urban scenarios. TLG can describe the point clouds hierarchically, we use a range graph to represent point clouds and a set graph for point cloud sets, which reduce both processing time and memory consumption. In the range graph, Euclidean distance and the angle of the sensor position with two adjacent vectors (calculated from continuing points to different direction) are used as the segmentation standard, which use the local concave features to distinguish different objects close to each other. In the set graph, we use the start and end position to express the whole set of continuous points concisely, and an improved Breadth-First-Search (BFS) algorithm is designed to update categories of point cloud sets between different channels. This method is evaluated on real vehicles and major datasets. The results show that TLG succeeds in providing a real-time performance (less than 20 ms per frame), and a high segmentation accuracy rate (93.64%) for traffic objects in the road of urban scenarios.
The automatic classification of 3-D point clouds is publicly known as a challenging task in a complex road environment. Specifically, each point is automatically classified into a unique category label, and then, the labels are used as clues for semantic analysis and scene recognition. Instead of heuristically extracting handcrafted features in traditional methods to classify all points, we put forward an end-to-end octree-based fully convolutional network (FCN) to classify 3-D point clouds in an urban road environment. There are four contributions in this paper. The first is that the integration and comprehensive uses of OctNet and FCN greatly decrease the computing time and memory demands compared with a dense 3-D convolutional neural network (CNN). The second is that the octree-based network is strengthened by means of modifying the cross-entropy loss function to solve the problems of an unbalanced category distribution. The third is that an Inception-ResNet block is united with our network, which enables our 3-D CNN to effectively learn how to classify scenes containing objects at multiple scales and improve classification accuracy. The last is that an open source data set (HuangshiRoad data set) with ten different classes is introduced for 3-D point cloud classification. Three representative data sets [Semantic3D, WHU_MLS (blocks I and II), and HuangshiRoad] with different covered areas and numbers of points and classes are selected to evaluate our proposed method. The experimental results show that the overall classification accuracy is appreciable, with 89.4% for Semantic3D, 82.9% for WHU_MLS block I, 91.4% for WHU_MLS block II, and 94% for HuangshiRoad. Our deep learning approach can efficiently classify 3-D dense point clouds in an urban road environment measured by a mobile laser scanning (MLS) system or static LiDAR.
In this paper, we introduce a non-associative higher-order graphical model to tackle the problem of semantic labeling of 3D point clouds. For this task, existing higher-order models overlook the relationships between the different classes and simply encourage the nodes in the cliques to have consistent labelings. We address this issue by devising a set of non-associative context patterns that describe higher-order geometric relationships between different class labels within the cliques. To this end, we propose a method to extract informative cliques in 3D point clouds that provide more knowledge about the context of the scene. We evaluate our approach on three challenging outdoor point cloud datasets. Our experiments evidence the benefits of our non-associative higher-order Markov networks over state-of-the-art point cloud labeling techniques.
In this paper, we investigate the potential of a Conditional Random Field (CRF) approach for the classification of an airborne LiDAR
(Light Detection And Ranging) point cloud. This method enables the incorporation of contextual information and learning of specific
relations of object classes within a training step. Thus, it is a powerful approach for obtaining reliable results even in complex urban
scenes. Geometrical features as well as an intensity value are used to distinguish the five object classes building, low vegetation, tree,
natural ground, and asphalt ground. The performance of our method is evaluated on the dataset of Vaihingen, Germany, in the context
of the 'ISPRS Test Project on Urban Classification and 3D Building Reconstruction'. Therefore, the results of the 3D classification
were submitted as a 2D binary label image for a subset of two classes, namely building and tree.
The increasing use of electrical energy has yielded more necessities of electric utilities including transmission lines and electric pylons which require a real-time risk monitoring to prevent massive economical damages. Recently, Airborne Laser Scanning (ALS) has become one of primary data acquisition tool for corridor mapping due to its ability of direct 3D measurements. In particular, for power-line risk management, a rapid and accurate classification of power-line objects is an extremely important task. We propose a 3D classification method combining results obtained from multiple classifier trained with different features. As a base classifier, we employ Random Forests (RF) which is a composite descriptors consisting of a number of decision trees populated through learning with bootstrapping samples. Two different sets of features are investigated that are extracted in a point domain and a feature (i.e., line & polygon) domain. RANSAC and Minimum Description Length (MDL) are applied to create lines and a polygon in each volumetric pixel (voxel) for the line & polygon features. Two RFs are trained from the two groups of features uncorrelated by Principle Component Analysis (PCA), which results are combined for final classification. The experiment with two real datasets demonstrates that the proposed classification method shows 10% improvements in classification accuracy compared to a single classifier.
3D reconstruction of trees is of great interest in large-scale 3D city modelling. Laser scanners provide geometrically accurate 3D point
clouds that are very useful for object recognition in complex urban scenes. Trees often cause important occlusions on building façades.
Their recognition can lead to occlusion maps that are useful for many façade oriented applications such as visual based localisation
and automatic image tagging. This paper proposes a pipeline to detect trees in point clouds acquired in dense urban areas with only
laser informations (x,y, z coordinates and intensity). It is based on local geometric descriptors computed on each laser point using a
determined neighbourhood. These descriptors describe the local shape of objects around every 3D laser point. A projection of these
values on a 2D horizontal accumulation space followed by a combination of morphological filters provides individual tree clusters. The
pipeline is evaluated and the results are presented on a set of one million laser points using a man made ground truth.
This paper describes a publicly available 3D database from the rueMadame, a street in the 6th Parisian district. Data have been acquired by the Mobile Laser Scanning (MLS) system L3D2 and correspond to a 160 m long street section. Annotation has been carried out in a manually assisted way. An initial annotation is obtained using an automatic segmentation algorithm. Then, a manual refinement is done and a label is assigned to each segmented object. Finally, a class is also manually assigned to each object. Available classes include facades, ground, cars, motorcycles, pedestrians, traffic signs, among others. The result is a list of (X, Y, Z, reflectance, label, class) points. Our aim is to offer, to the scientific community, a 3D manually labeled dataset for detection, segmentation and classification benchmarking. With respect to other databases available in the state of the art, this dataset has been exhaustively annotated in order to include all available objects and to allow point-wise comparison.
We present a method for reconstructing surfaces from point sets. The main novelty lies in a structure-preserving approach where the input point set is first consolidated by structuring and resampling the planar components, before reconstructing the surface from both the consolidated components and the unstructured points. The final surface is obtained through solving a graph-cut problem formulated on the 3D Delaunay triangulation of the structured point set where the tetrahedra are labeled as inside or outside cells. Structuring facilitates the surface reconstruction as the point set is substantially reduced and the points are enriched with structural meaning related to adjacency between primitives. Our approach departs from the common dichotomy between smooth/piecewise-smooth and primitive-based representations by gracefully combining canonical parts from detected primitives and free-form parts of the inferred shape. Our experiments on a variety of inputs illustrate the potential of our approach in terms of robustness, flexibility and efficiency.
3D scene analysis by automatically assigning 3D points a semantic label has become an issue of major interest in recent years. Whereas
the tasks of feature extraction and classification have been in the focus of research, the idea of using only relevant and more distinctive
features extracted from optimal 3D neighborhoods has only rarely been addressed in 3D lidar data processing. In this paper, we focus on
the interleaved issue of extracting relevant, but not redundant features and increasing their distinctiveness by considering the respective
optimal 3D neighborhood of each individual 3D point. We present a new, fully automatic and versatile framework consisting of four
successive steps: (i) optimal neighborhood size selection, (ii) feature extraction, (iii) feature selection, and (iv) classification. In a
detailed evaluation which involves 5 different neighborhood definitions, 21 features, 6 approaches for feature subset selection and 2
different classifiers, we demonstrate that optimal neighborhoods for individual 3D points significantly improve the results of scene
interpretation and that the selection of adequate feature subsets may even further increase the quality of the derived results.
Due to ever more efficient and accurate laser scanning technologies, the analysis of 3D point clouds has become an important task
in modern photogrammetry and remote sensing. To exploit the full potential of such data for structural analysis and object detection,
reliable geometric features are of crucial importance. Since multiscale approaches have proved very successful for image-based applications,
efforts are currently made to apply similar approaches on 3D point clouds. In this paper we analyse common geometric
covariance features, pinpointing some severe limitations regarding their performance on varying scales. Instead, we propose a different
feature type based on shape distributions known from object recognition. These novel features show a very reliable performance on a
wide scale range and their results in classification outnumber covariance features in all tested cases.
The mapping of road environments is an important task, providing important input data for a broad range of scientific disciplines.
Pole-like objects, their visibility and their influence onto local light and traffic noise conditions are of particular interest for traffic
safety, public health and ecological issues. Detailed knowledge can support the improvement of traffic management, noise reducing
infrastructure or the planning of photovoltaic panels. Mobile Mapping Systems coupled with computer aided mapping work-flows
allow an effective data acquisition and provision. We present a classification work flow focussing on pole-like objects. It uses
rotation and scale invariant point and object features for classification, avoiding planar segmentation and height slicing steps. Single
objects are separated by connected component and Dijkstra-path analysis. Trees and artificial objects are separated using a graph
based approach considering the branching levels of the given geometries. For the focussed semantic groups, classification accuracies
higher than 0.9 are achieved. This includes both the quality of object aggregation and separation, where the combination of Dijkstrapath
aggregation and graph-based classification shows good results. For planar objects the classification accuracies are lowered,
recommending the usage of planar segmentation for classification and subdivision issues as presented by other authors. The
presented work-flow provides sufficient input data for further 3D reconstructions and tree modelling.
We propose an automatic and robust approach to detect, segment and classify urban objects from 3D point clouds. Processing is carried out using elevation images and the result is reprojected onto the 3D point cloud. First, the ground is segmented and objects are detected as discontinuities on the ground. Then, connected objects are segmented using a watershed approach. Finally, objects are classified using SVM with geometrical and contextual features. Our methodology is evaluated on databases from Ohio (USA) and Paris (France). In the former, our method detects 98% of the objects, 78% of them are correctly segmented and 82% of the well-segmented objects are correctly classified. In the latter, our method leads to an improvement of about 15% on the classification step with respect to previous works. Quantitative results prove that our method not only provides a good performance but is also faster than other works reported in the literature.
The ability to reliably detect vegetation is an important requirement for outdoor navigation with mobile robots as it enables the robot to navigate more efficiently and safely. In this paper, we present an approach to detect flat vegetation, such as grass, which cannot be identified using range measurements. This type of vegetation is typically found in structured outdoor environments such as parks or campus sites. Our approach classifies the terrain in the vicinity of the robot based on laser scans and makes use of the fact that plants exhibit specific reflection properties. It uses a support vector machine to learn a classifier for distinguishing vegetation from streets based on laser reflectivity, measured distance, and the incidence angle. In addition, it employs a vibration-based classifier to acquire training data in a self-supervised way and thus reduces manual work. Our approach has been evaluated extensively in real world experiments using several mobile robots. We furthermore evaluated it with different types of sensors and in the context of mapping, autonomous navigation, and exploration experiments. In addition, we compared it to an approach based on linear discriminant analysis. In our real world experiments, our approach yields a classification accuracy close to 100%.
This paper provides a comprehensive overview of urban reconstruction. While there exists a considerable body of literature, this topic is still under active research. The work reviewed in this survey stems from the following three research communities: computer graphics, computer vision and photogrammetry and remote sensing. Our goal is to provide a survey that will help researchers to better position their own work in the context of existing solutions, and to help newcomers and practitioners in computer graphics to quickly gain an overview of this vast field. Further, we would like to bring the mentioned research communities to even more interdisciplinary work, since the reconstruction problem itself is by far not solved.
In this paper we present an approach for automatic analysis of urban accessibility using 3D point clouds. Our approach is based on range images and it consists in two main steps: urban objects segmentation and curbs detection. Both of them are required for accessibility diagnosis and itinerary planning.
Our method automatically segments facades and urban objects using two hypotheses: facades are the highest vertical structures in the scene and objects are bumps on the ground on the range image. The segmentation result is used to build an urban obstacle map. After that, the gradient is computed on the ground range image. Curb candidates are selected using height and geodesic features. Then, nearby curbs are reconnected using Bézier curves. Finally, accessibility is defined based on geometrical features and accessibility standards.
Our methodology is tested on two MLS databases from Paris (France) and Enschede (The Netherlands). Our experiments show that our method has good detection rates, is fast and presents few false alarms. Our method outperforms other works reported in the literature on the same databases.
Identifying damaged buildings after natural disasters such as earthquake is important for the planning of recovery actions. We present a segment-based approach to classifying damaged building roofs in aerial laser scanning data. A challenge in the supervised classification of point segments is the generation of training samples, which is difficult because of the complexity of interpreting point clouds. We evaluate the performance of three different classifiers trained with a small set of training samples and show that feature selection improves the training and the accuracy of the resulting classification. When trained with 50 training samples, a linear discriminant classifier using a subset of six features reaches a classification accuracy of 85%.
The automatic analysis of large 3D point clouds represents a crucial task in photogrammetry, remote sensing and computer vision.
In this paper, we propose a new methodology for the semantic interpretation of such point clouds which involves feature relevance
assessment in order to reduce both processing time and memory consumption. Given a standard benchmark dataset with 1.3 million 3D
points, we first extract a set of 21 geometric 3D and 2D features. Subsequently, we apply a classifier-independent ranking procedure
which involves a general relevance metric in order to derive compact and robust subsets of versatile features which are generally
applicable for a large variety of subsequent tasks. This metric is based on 7 different feature selection strategies and thus addresses
different intrinsic properties of the given data. For the example of semantically interpreting 3D point cloud data, we demonstrate the
great potential of smaller subsets consisting of only the most relevant features with 4 different state-of-the-art classifiers. The results
reveal that, instead of including as many features as possible in order to compensate for lack of knowledge, a crucial task such as scene
interpretation can be carried out with only few versatile features and even improved accuracy.
We present a novel and robust method for modeling cities from 3D-point data. Our algorithm provides a more complete description than existing approaches by reconstructing simultaneously buildings, trees and topologically complex grounds. A major contribution of our work is the original way of modeling buildings which guarantees a high generalization level while having semantized and compact representations. Geometric 3D-primitives such as planes, cylinders, spheres or cones describe regular roof sections, and are combined with mesh-patches that represent irregular roof components. The various urban components interact through a non-convex energy minimization problem in which they are propagated under arrangement constraints over a planimetric map. Our approach is experimentally validated on complex buildings and large urban scenes of millions of points, and is compared to state-of-the-art methods.
We present a method for automatic object localization and recognition in 3D point clouds representing outdoor urban scenes. The
method is based on the implicit shape models (ISM) framework, which recognizes objects by voting for their center locations. It
requires only few training examples per class, which is an important property for practical use. We also introduce and evaluate an
improved version of the spin image descriptor, more robust to point density variation and uncertainty in normal direction estimation.
Our experiments reveal a significant impact of these modifications on the recognition performance. We compare our results against the
state-of-the-art method and get significant improvement in both precision and recall on the Ohio dataset, consisting of combined aerial
and terrestrial LiDAR scans of 150,000 m2 of urban area in total.
The selection of suitable features and their parameters for the classification of three-dimensional laser range data is a crucial issue for high-quality results. In this paper we compare the performance of different histogram descriptors and their parameters on three urban datasets recorded with various sensors—sweeping SICK lasers, tilting SICK lasers and a Velodyne 3D laser range scanner. These descriptors are 1D, 2D, and 3D histograms capturing the distribution of normals or points around a query point. We also propose a novel histogram descriptor, which relies on the spectral values in different scales. We argue that choosing a larger support radius and a z-axis based global reference frame/axis can boost the performance of all kinds of investigated classification models significantly. The 3D histograms relying on the point distribution, normal orientations, or spectral values, turned out to be the best choice for the classification in urban environments.
Full-waveform lidar data are increasingly being available. Morphological features can be retrieved from the echoes composing the waveforms, and are now extensively used for a large variety of land-cover mapping issues. However, the genuine contribution of these features with respect to those computed from standard discrete return lidar systems has been barely theoretically investigated. This paper therefore aims to study the potential of full-waveform data through the automatic classification of urban areas in building, ground, and vegetation points. Two waveform processing methods, namely a non-linear least squares method and a marked point process approach, are used to fit the echoes both with symmetric and asymmetric modeling functions. The performance of the extracted full-waveform features for the classification problem are then compared to a large variety of multiple-pulse features thanks to three feature selection methods. A support vector machines classifier is finally used to label the point cloud according to various scenarios based on the rank of the features. This allows to find the best classification strategy as well as the minimal feature subsets allowing to achieve the highest classification accuracy possible for each of the three feature selection methods.The results show that the echo amplitude as well as two features computed from the radiometric calibration of full-waveform data, namely the cross-section and the backscatter coefficient, significantly contribute to the high classification accuracies reported in this paper (around 95%). Conversely, features extracted from the non Gaussian modelling of the echoes are not relevant for the discrimination of vegetation, ground, and buildings in urban areas.
Various multi-echo and Full-waveform (FW) lidar features can be processed. In this paper, multiple classifers are applied to lidar feature selection for urban scene classification. Random forests are used since they provide an accurate classification and run efficiently on large datasets. Moreover, they return measures of variable importance for each class. The feature selection is obtained by backward elimination of features depending on their importance. This is crucial to analyze the relevance of each lidar feature for the classification of urban scenes. The Random Forests classification using selected variables provide an overall accuracy of 94.35%.
