ArticlePDF Available

Rapid Localization and Extraction of Street Light Poles in Mobile LiDAR Point Clouds: A Supervoxel-Based Approach

Authors:

Abstract

This paper presents a supervoxel-based approach for automated localization and extraction of street light poles in point clouds acquired by a mobile LiDAR system. The method consists of five steps: pre-processing, localization, segmentation, feature extraction, and classification. First, the raw point clouds are divided into segments along the trajectory, the ground points are removed and the remaining points are segmented into supervoxels. Then, a robust localization method is proposed to accurately identify the pole-like objects. Next, a localization-guided segmentation method is proposed to obtain pole-like objects. Subsequently, the pole features are classified using the support vector machine and random forests. The proposed approach was evaluated on three datasets with 1,055 street light poles and 701 million points. Experimental results show that our localization method achieved an average recall value of 98.8%. Comparative study proved that our method is more robust and efficient than other existing methods for localization and extraction of street light poles.
Dear Readers,
Thank you for your interest in our work.
Our full papers can be downloaded from our website at
https://uwaterloo.ca/geospatial-sensing/publication/journal-papers-and-book-
chapters.
Thank you.
Jonathan Li, PhD, PEng
Professor, University of Waterloo
... An issue with these methods is that the accuracy of feature classes ranged from 66.7 to 94.3%, and the methods were only tested on 900m of roadway, which is not enough for it to be considered a viably tested method. Some researches such as El-Halawany and Lichti (El-Halawany et al. (2011), Zheng et al. (2017, and Wu et al. (2017) have solely focused on automated streetpole extraction from mobile LiDAR. Their different methods of segmentation and clustering have led in some cases to a success rate of over 90% as well as a low false classification rate. ...
... Regarding the features themselves in their unobstructed environments, some urban mobile LiDAR studies addressed occluded objects. Studies by Wu et al. (2017) and Zheng et al. (2017) constructed automated methods for extracting light poles from mobile data, addressed the occlusion issue, and found that complete objects were captured well over 90% of the time. Yang et al. (2015) utilized methods to extract multiple features and also concluded that complete features were almost always captured despite potential occlusions. ...
... Mobile LiDAR's high energy and dense, emission of over 2000 points per meter limits occlusion and penetrates tree canopies. In fact, the existing mobile LiDAR studies relating to street features and tree collection show very limited (if any) occlusion from high-density mobile LiDAR (Yang et al. 2015;Lin et al. 2014;Wu et al. 2017;Zheng et al. 2017). Therefore, mobile LiDAR's results clearly show how volumetric calculations are more justifiable than simple crude counts of features. ...
Article
Full-text available
This study investigates the feasibility of utilizing high-density mobile Light Detection and Ranging (LiDAR) to measure urban streetscape features. The results suggest that mobile LiDAR’s density allows for much smaller voxels than in the previous research and the ability to measure small urban streetscape features in 3D. This includes street trees, light/lampposts, street furniture, traffic and commercial signage, building window proportions, awnings, and enclosed courtyard restaurants. Moreover, mobile LiDAR facilitated measuring and categorizing these streetscape features in walkable, downtown-like streetscape environments. The ability to compartmentalize such streetscapes into smaller cubic foot voxels to be quantitatively measured and categorized could supplement or replace conventional audit-based streetscape measurement. This study introduces new methods—based on voxel data analysis—to compile accurate descriptive statistics of streetscape features and how they can be represented in 3D.
... As dimensional objects expanding upon the concept of the "superpixel" [17], "supervoxels" [18] are generated by partitioning 3D space as point clusters. Supervoxels have been increasingly applied to describe adjoining points related to the same objects [16,19]. Transferring the original point cloud to the "supervoxel cloud" propagates simple point-based classification to an object-based level. ...
Article
Full-text available
As an essential part of point cloud processing, autonomous classification is conventionally used in various multifaceted scenes and non-regular point distributions. State-of-the-art point cloud classification methods mostly process raw point clouds, using a single point as the basic unit and calculating point cloud features by searching local neighbors via the k-neighborhood method. Such methods tend to be computationally inefficient and have difficulty obtaining accurate feature descriptions due to inappropriate neighborhood selection. In this paper, we propose a robust and effective point cloud classification approach that integrates point cloud supervoxels and their locally convex connected patches into a random forest classifier, which effectively improves the point cloud feature calculation accuracy and reduces the computational cost. Considering the different types of point cloud feature descriptions, we divide features into three categories (point-based, eigen-based, and grid-based) and accordingly design three distinct feature calculation strategies to improve feature reliability. Two International Society of Photogrammetry and Remote Sensing benchmark tests show that the proposed method achieves state-of-the-art performance, with average F1-scores of 89.16 and 83.58, respectively. The successful classification of point clouds with great variation in elevation also demonstrates the reliability of the proposed method in challenging scenes.
... As dimensional objects expanding upon the concept of the "superpixel" (Achanta et al., 2012), "supervoxel" (Papon et al., 2013), are generated by partitioning 3D space as point clusters. Supervoxels have been increasingly applied to describe adjoining points related to the same objects (Wu et al., 2016, Zhu et al., 2017. Transferring the original point cloud to the "supervoxel cloud" propagates simple point-based classification to an object-based level. ...
Article
Full-text available
State-of-the-art point cloud classification methods mostly process raw point clouds, using a single point as the basic unit and calculating point cloud features by searching local neighbors via the k-neighborhood method. Such methods tend to be computationally inefficient and have difficulty obtaining accurate feature descriptions due to inappropriate neighborhood selection. In this paper, we propose a robust and effective point cloud classification approach that integrates point cloud supervoxels and their locally convex connected patches into a random forest classifier. We apply a centroid cloud extracted from supervoxels into the proposed classifier, which effectively improves the point cloud feature calculation accuracy and reduces the computational cost. Considering the different types of point cloud feature descriptions, we divide features into three categories (point-based, eigen-based, and grid-based) and accordingly design three distinct feature calculation strategies to improve feature reliability. The proposed method achieves state-of-the-art performance, with average F1-scores of 89.16%, respectively. The successful classification of point clouds with great variation in elevation also demonstrates the reliability of the proposed method in challenging scenes to some extents.
Article
Mobile Laser Scanning (MLS) systems provide highly dense 3D point clouds that enable the acquisition of accurate traffic facilities information for intelligent transportation system. Road guardrails with safety features that can separate traffic and define moving spaces for pedestrians and vehicles face challenges such as diverse guardrail types and continuous slopes in point clouds data. This paper proposes a novel approach for rapidly extracting urban road guardrails from MLS point clouds, combining a proposed multi-level filtering with a modified Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering, and adapting for most types of guardrails and rough slope roads. We develop a multi-level filter to detect the road surface and remove the undesirable points. Through a proposed modified DBSCAN clustering, the guardrails are extracted after a four-step screening, which includes the limits based on the number of points, the fitting error, the bounding box size and the average reflection intensity for each cluster. The proposed method achieves high precisions of 97.2% and 96.4% respectively for the lane-separating guardrails and the anti-fall guardrails on the dataset. Extensive experiments with test dataset captured by a RIEGL VMX-450 MLS, show that our method outperforms the state-of-the-art method to extract 3D guardrails from point clouds.
Article
The widespread use of light detection and ranging (LiDAR) data provides a promising source for the automatic detection and inventory of roadside assets. One of the essential elements in roadside furniture is light poles. There is limited research on the mapping of light poles using point-cloud data on rural highways. In this environment, the placement of light poles within roadside clear zones often poses a safety concern, as they are related to an increased risk of collisions. Only a limited number of studies have explored the relationship between light poles and safety because of the time-consuming and labor-intensive practices of collecting light pole assets data using traditional manual methods. This paper proposes an automated approach to mapping the locations of light poles. First, the scanning vehicle trajectory is extracted, smoothed, and then used to segment the point-cloud data into smaller overlapped batches of data. Several filters are applied to extract pole-like objects from the data. The segments are combined back together, and a density-based clustering algorithm is used to group the remaining points into clusters. A geometric filter is finally applied to extract light poles. The model is tested on 28 km of data on three rural highways in Alberta, Canada. The proposed algorithm is found to be accurate relative to previous studies, with average precision, recall, and F1 scores exceeding 98% for the test segments. The proposed work can assist in the automation of light pole inventory and road safety audits by transportation agencies.
Article
According to the spatial structure characteristics of road curbs and road surfaces, a robust method for automatic extraction of road boundaries, road curbs and road surfaces was proposed using mobile laser scanning (MLS) point cloud data. Firstly, ground filtering was performed to separate ground points and non-ground points according to the angle between the normal vector of the point cloud and the direction vector of the z-axis. Secondly, based on the vertical and linear features of the road curb, the MLS trajectory points were used to extract road curb and road boundary points. Then, Euclidean clustering and fitting were performed on the road boundary point segments. Adjacent clusters were merged, and sparse points were densified. In addition, based on the principle that road surfaces are within road boundaries, road surface points were obtained in scanning line order. Two MLS point clouds with different resolutions and road roughness were tested. Compared with the manually calibrated reference road curb, the extraction completenesses of the road curb from the two datasets were 95.66% and 96.45%, respectively, and the extraction correctnesses of the road curb were 96.34% and 99.10%, respectively, with both qualities over 92%. The algorithm can effectively extract straight or curved road boundaries and road curbs from the point cloud data containing vehicles, pedestrians and obstacle occlusions in an urban environment, and is applicable to MLS point cloud data with different resolutions and roughness.
Article
Accurate and reliable localization in urban environments is critical for high-level autonomous driving systems. In comparison to other localization strategies, map-based methods are more reliable and have higher localization accuracy. However, due to higher storage and update costs, dense map-based localization methods are inefficient for widespread use. In urban road environments, structural features, such as road curbs that limit the road passable area, and pole-like features, such as streetlights, tree trunks, and traffic lights, exhibit exceptional long-term stability and extraction consistency. The maps based on road curbs and pole-like features are unaffected by dynamic obstacles, such as vehicles and pedestrians, making them ideal for vehicle localization in urban environments. In this article, we design a lightweight feature map for LiDAR localization in urban environments based on road curbs and pole-like features, and propose a novel, fast, and accurate extraction strategy for these features, implementing the entire pipeline of feature extraction, map construction, and localization. We combine the orderliness of 2-D range images with the spatial properties of 3-D point clouds to extract road curbs and poles quickly and accurately. The proposed feature extraction methods and localization system are evaluated on datasets collected in multiple environments and times. The experimental results demonstrate that the method proposed in this article is more efficient and robust at features’ extraction and vehicle localization compared with other state-of-the-art approaches.
Article
Full-text available
Convolutional neural network (CNN), one of the branches of deep neural networks, has been widely used in image recognition, natural language processing, and other related fields with great success recently. This paper proposes a novel framework with CNN to classify objects in a point cloud captured by LiDAR on urban streets. The proposed BA-CNN algorithm is composed of five steps: (i) removing ground points, (ii) clustering objects, (iii) transforming to bearing angle images, (iv) ROI selection, and (V) identifying objects by CNN. In the first step, ground points are removed by the multi-threshold-based ground detection to reduce the processing time. Then, a flood-fill-based clustering method is used for object segmentation. Those individual point cloud objects are converted to bearing angle (BA) images. Then, a well-trained CNN is used to classify objects with BA images. The main contribution of this paper is proposing an efficient recognition method that uses the information from point clouds only. In contrast, because most 3D object classifiers use the fusion of point clouds and color images, their models are very complicated and take a colossal amount of memory to store the parameters. Since the ground point detection and object clustering process all points along with the scanline-major order and layer-major order, the proposed algorithm performs better in terms of time consumption and memory consumption. In the experiment, three scenes from KITTI dataset are used for training and testing the proposed BA-CNN classifier, and the proposed BA-CNN achieves high classification accuracy.
Article
Full-text available
Detecting small objects (e.g., manhole covers, license plates, and roadside milestones) in urban images is a long-standing challenge mainly due to the scale of small object and background clutter. Although convolution neural network (CNN)-based methods have made significant progress and achieved impressive results in generic object detection, the problem of small object detection remains unsolved. To address this challenge, in this study we developed an end-to-end network architecture that has three significant characteristics compared to previous works. First, we designed a backbone network module, namely Reduced Downsampling Network (RD-Net), to extract informative feature representations with high spatial resolutions and preserve local information for small objects. Second, we introduced an Adjustable Sample Selection (ADSS) module which frees the Intersection-over-Union (IoU) threshold hyperparameters and defines positive and negative training samples based on statistical characteristics between generated anchors and ground reference bounding boxes. Third, we incorporated the generalized Intersection-over-Union (GIoU) loss for bounding box regression, which efficiently bridges the gap between distance-based optimization loss and area-based evaluation metrics. We demonstrated the effectiveness of our method by performing extensive experiments on the public Urban Element Detection (UED) dataset acquired by Mobile Mapping Systems (MMS). The Average Precision (AP) of the proposed method was 81.71%, representing an improvement of 1.2% compared with the popular detection framework Faster R-CNN.
Article
Full-text available
Object detection in three-dimensional (3-D) laser scanning point clouds of complex urban environment is a challenging problem. Existing methods are limited by their robustness to complex situations such as occlusion, overlap, and rotation or by their computational efficiency. This paper proposes a high computationally efficient method integrating supervoxel with Hough forest framework for detecting objects from 3-D laser scanning point clouds. First, a point cloud is over-segmented into spatially consistent supervoxels. Each supervoxel together with its first-order neighborhood is grouped into one local patch. All the local patches are described by both structure and reflectance features, and then used in the training stage for learning a random forest classifier as well as the detection stage to vote for the possible location of the object center. Second, local reference frame and circular voting strategies are introduced to achieve the invariance to the azimuth rotation of objects. Finally, objects are detected at the peak points in 3-D Hough voting space. The performance of our proposed method is evaluated on real-world point cloud data collected by the up-to-date mobile laser scanning system. Experimental results demonstrate that our proposed method outperforms state-of-the-art 3-D object detection methods with high computational efficiency.
Article
Full-text available
Updating road markings is one of the routine tasks of transportation agencies. Compared with traditional road inventory mapping techniques, vehicle-borne mobile light detection and ranging (LiDAR) systems can undertake the job safely and efficiently. However, current hurdles include software and computing challenges when handling huge volumes of highly dense and irregularly distributed 3-D mobile LiDAR point clouds. This paper presents the development and implementation aspects of an automated object extraction strategy for rapid and accurate road marking inventory. The proposed road marking extraction method is based on 2-D georeferenced feature (GRF) images, which are interpolated from 3-D road surface points through a modified inverse distance weighted (IDW) interpolation. Weighted neighboring difference histogram (WNDH)-based dynamic thresholding and multiscale tensor voting (MSTV) are proposed to segment and extract road markings from the noisy corrupted GRF images. The results obtained using 3-D point clouds acquired by a RIEGL VMX-450 mobile LiDAR system in a subtropical urban environment are encouraging.
Article
Full-text available
This paper proposes a novel, automated algorithm for rapidly extracting urban road facilities, including street light poles, traffic signposts, and bus stations, for transportation-related applications. A detailed description and implementation of the proposed algorithm is provided using mobile laser scanning data collected by a state-of-the-art RIEGL VMX-450 system. First, to reduce the quantity of data to be handled, a fast voxel-based upward growing method is developed to remove ground points. Then, off-ground points are clustered and segmented into individual objects via Euclidean distance clustering and voxel-based normalized cut segmentation, respectively. Finally, a 3-D object matching framework, benefiting from a locally affine-invariant geometric constraint, is developed to achieve the extraction of 3-D objects. Quantitative evaluations show that the proposed algorithm attains an average completeness, correctness, quality, and F1-measure of 0.949, 0.971, 0.922, and 0.960, respectively, in extracting 3-D light poles, traffic signposts, and bus stations. Comparative studies demonstrate the efficiency and feasibility of the proposed algorithm for automated and rapid extraction of urban road facilities.
Article
Full-text available
This paper presents a spatial-related traffic sign inspection process for sign type, position, and placement using mobile laser scanning (MLS) data acquired by a RIEGL VMX-450 system and presents its potential for traffic sign inventory applications. First, the paper describes an algorithm for traffic sign detection in complicated road scenes based on the retroreflectivity properties of traffic signs in MLS point clouds. Then, a point cloud-to-image registration process is proposed to project the traffic sign point clouds onto a 2-D image plane. Third, based on the extracted traffic sign points, we propose a traffic sign position and placement inspection process by creating geospatial relations between the traffic signs and road environment. For further inventory applications, we acquire several spatial-related inventory measurements. Finally, a traffic sign recognition process is conducted to assign sign type. With the acquired sign type, position, and placement data, a spatial-associated sign network is built. Experimental results indicate satisfactory performance of the proposed detection, recognition, position, and placement inspection algorithms. The experimental results also prove the potential of MLS data for automatic traffic sign inventory applications.
Article
Full-text available
This paper proposes a novel framework for automated detection of urban road manhole covers using mobile laser scanning (MLS) data. First, to narrow searching regions and reduce the computational complexity, road surface points are segmented from a raw point cloud via a curb-based road surface segmentation approach and rasterized into a georeferenced intensity image through inverse distance weighted interpolation. Then, a supervised deep learning model is developed to construct a multilayer feature generation model for depicting high-order features of local image patches. Next, a random forest model is trained to learn mappings from high-order patch features to the probabilities of the existence of urban road manhole covers centered at specific locations. Finally, urban road manhole covers are detected from georeferenced intensity images based on the multilayer feature generation model and random forest model. Quantitative evaluations show that the proposed algorithm achieves an average completeness, correctness, quality, and F1-measure of 0.955, 0.959, 0.917, and 0.957, respectively, in detecting urban road manhole covers from georeferenced intensity images. Comparative studies demonstrate the advantageous performance of the proposed algorithm over other existing methods for rapid and automated detection of urban road manhole covers using MLS data.
Article
Full-text available
This paper presents a survey of literature about road feature extraction, giving a detailed description of a Mobile Laser Scanning (MLS) system (RIEGL VMX-450) for transportation-related applications. This paper describes the development of automated algorithms for extracting road features (road surfaces, road markings, and pavement cracks) from MLS point cloud data. The proposed road surface extraction algorithm detects road curbs from a set of profiles that are sliced along vehicle trajectory data. Based on segmented road surface points, we create Geo-Referenced Feature (GRF) images and develop two algorithms, respectively, for extracting the following: 1) road markings with high retroreflectivity and 2) cracks containing low contrast with their surroundings, low signal-to-noise ratio, and poor continuity. A comprehensive comparison illustrates satisfactory performance of the proposed algorithms and concludes that MLS is a reliable and cost-effective alternative for rapid road inspection.
Article
Full-text available
This paper proposes a novel algorithm for extracting street light poles from vehicleborne mobile light detection and ranging (LiDAR) point-clouds. First, the algorithm rapidly detects curb-lines and segments a point-cloud into road and nonroad surface points based on trajectory data recorded by the integrated position and orientation system onboard the vehicle. Second, the algorithm accurately extracts street light poles from the segmented nonroad surface points using a novel pairwise 3-D shape context. The proposed algorithm is tested on a set of point-clouds acquired by a RIEGL VMX-450 mobile LiDAR system. The results show that road surfaces are correctly segmented, and street light poles are robustly extracted with a completeness exceeding 99%, a correctness exceeding 97%, and a quality exceeding 96%, thereby demonstrating the efficiency and feasibility of the proposed algorithm to segment road surfaces and extract street light poles from huge volumes of mobile LiDAR point-clouds.