ArticlePDF Available

Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods

Authors:

Abstract and Figures

Light detection and ranging (LiDAR) sensors can create high-quality scans of an environment. However, LiDAR point clouds are affected by harsh weather conditions since airborne particles are easily detected. In literature, conventional filtering and artificial intelligence (AI) filtering methods have been used to detect, and remove, airborne particles. In this paper, a convolutional neural network (CNN) model was used to classify airborne dust particles through a voxel-based approach. The CNN model was compared to several conventional filtering methods, where the results show that the CNN filter can achieve up to 5.39 % F1 score improvement when compared to the best conventional filter. All the filtering methods were tested in dynamic environments where the sensor was attached to a mobile platform, the environment had several moving obstacles, and there were multiple dust cloud sources.
Content may be subject to copyright.
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2023.0322000
Dust De-Filtering in LiDAR Applications with
Conventional and CNN Filtering Methods
TYLER PARSONS1, JAHO SEO1, BYEONGJIN KIM2, HANMIN LEE2, JI-CHUL KIM2, and
MOOHYUN CHA2
1Department of Automotive and Mechatronics Engineering, Ontario Tech University, Oshawa, Ontario, L1G 0C5, Canada (e-mail:
tyler.parsons1@ontariotechu.net, Jaho.Seo@ontariotechu.ca)
2Department of Smart Industrial Machine Technologies at the Korea Institute of Machinery and Materials (KIMM), Daejeon, South Korea (e-mail:
bjkim@kimm.re.kr; hmlee@kimm.re.kr; jckim@kimm.re.kr; mhcha@kimm.re.kr)
Corresponding author: Jaho Seo (e-mail: Jaho.Seo@ontariotechu.ca).
This work was supported in part by the Korea Institute of Machinery and Materials (KIMM).
ABSTRACT Light detection and ranging (LiDAR) sensors can create high-quality scans of an environment.
However, LiDAR point clouds are affected by harsh weather conditions since airborne particles are easily
detected. In literature, conventional filtering and artificial intelligence (AI) filtering methods have been used
to detect, and remove, airborne particles. In this paper, a convolutional neural network (CNN) model was
used to classify airborne dust particles through a voxel-based approach. The CNN model was compared to
several conventional filtering methods, where the results show that the CNN filter can achieve up to 5.39 %
F1 score improvement when compared to the best conventional filter. All the filtering methods were tested
in dynamic environments where the sensor was attached to a mobile platform, the environment had several
moving obstacles, and there were multiple dust cloud sources.
INDEX TERMS Artificial intelligence (AI), autonomous navigation, convolutional neural network (CNN),
dust de-filtering, light detection and ranging (LiDAR).
I. INTRODUCTION
LIGHT Detection and Ranging (LiDAR) is commonly
used in autonomous navigation applications to create
high-resolution maps of the surrounding area [1]. In au-
tonomous vehicles, LiDAR plays a key role in obstacle de-
tection and avoidance since collisions can be extremely dan-
gerous and expensive. Although LiDAR can produce high-
resolution point clouds, harsh environmental factors can de-
grade the quality of the scanned environment. Factors such
as dust, snow, rain, and other small airborne particles can
be detected because of the short wavelength (900 nm) of the
signal [2], thus causing a lot of noise in a point cloud. Because
of this, it becomes difficult to distinguish between obstacles
and airborne particles.
In literature, there exists several methods to distinguish
between airborne particles and solid objects. One approach
is to fuse multiple sensors together [3], [4], such as a depth
camera and LiDAR [5]. By combining multiple sensors, dis-
crepancies between the sensors can be easily detected. How-
ever, hardware and physical limitations (not enough room for
more sensors) may not make this approach ideal. Modern
technological advances have allowed for smaller sensors and
all-in-one packages which can overcome this issue, but this
requires additional costs which may not be ideal. So, another
approach is to use conventional particle filtering methods
on the LiDAR point cloud. In literature, there are several
conventional algorithms that can be used, such as the Radius
Outlier Removal (ROR) filter [6], [7], Dynamic Radius Out-
lier Removal (DROR) filter [8], Statistical Outlier Removal
(SOR) filter [9], and Low-Intensity Outlier Removal (LIOR)
filter [10]. These methods make use of the point cloud’s
geometry and light intensity to filter outlier points that have
properties of airborne particles. However, the conventional
methods sometimes remove points that are not airborne parti-
cles, which results in the removal of important environmental
features such as obstacles.
Another method of airborne particle removal can be with
Artificial-Intelligence (AI) techniques. Some well studied AI
techniques that have been applied to detect airborne particles
are the Random Forest (RF) classifier [11], Support Vector
Machine (SVM) classifier [11], [12], K-Nearest Neighbors
(KNN) classifier [12], Neural Network (NN) classifier [11],
and Deep Neural Network (DNN) classifier [13]. These ap-
proaches require a large amount of data to train a model such
that it can accurately make predictions in real-time. Thus, the
data and features used to train the model has a large influence
VOLUME 11, 2023 1
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
on the quality of the predictions. Existing literature states
that a combination of geometry features and light-intensity
features through a voxel-based approach is ideal for training
AI methods for airborne particle detection [1], [11], [13].
Compared to the conventional methods, some AI approaches
can remove airborne particles with greater accuracy while still
maintaining important environmental information [1], [13].
In the mentioned literature, both conventional and AI filter-
ing methods have been discussed. As explained in the men-
tioned works, conventional filtering methods perform well
under ideal conditions when dust and non-dust particles are
easily distinguishable. However, it is speculated that a CNN
based approach can outperform the conventional filtering
methods, especially in unideal conditions. This assumption
is supported by the fact that AI based approaches have been
used for airborne particle classification in the mentioned
works, and have a good performance. Several research papers
have used CNNs for airborne particle filtering with slightly
different approaches. For example, Heinzler et al. proposed a
CNN model named WeatherNet which is based on LiLaNet
to filter fog and rain in point clouds [14]. They also propose a
method of automatically labelling experimental data by cross-
referencing a sample environment scan under ideal conditions
(no rain or fog). They transform the 3D LiDAR data into
two 2D images, one for depth and one for intensity. This is
done by "unralvelling" the 360
°
scan into a 2D matrix with
pixel intensities corresponding to the depth and intensity of
the captured points. In the 2D matrix, each row corresponds
to one of the 32 vertically stacked send/recieve modules,
and each column corresponds to one of the 1800 segments
over the whole scanning range. Another study expands on
the work of Heinzler et al. by adding a local radius outlier
removal (LROR) filter after using a CNN to filter rain and
fog [15]. Their CNN model is named SunnyNet and is based
upon the WeatherNet model with some minor modifications.
Similarily, they also use the 2D depth and one for intensity
images for their network. Sebastian et al. conduct an in depth
study of snowy, foggy, and rainy conditions in 3D LiDAR
applications [16]. What they found was that the eignevectors
and eigen values can effectively capture different weather
conditions. This concept was extended to not only detecting
the atmospheric conditions, but the road conditions as well.
Specifically, their model can detect snow, light fog, dense fog,
and rain in the atmophere, and full snow coverage, slushy,
and wet road conditions. Similarily, they also used the 2D
depth image representation of the 3D LiDAR data for their
CNN structure named RangeWeatherNet, which is a redesign
of the DarkNet archetechture. In [17], the authors propose
a different method to represent the LiDAR data: the bird’s-
eye view. The bird’s-eye view is a depth image projected
on a different axis compared to other literature. They used
this for their CNN model named MobileWeatherNet, which
is a modified structure based on the work of Simonyan and
Zisserman [18], to detect rain and fog. Their work shows that
the bird’s-eye view is better for classifying different weather
conditions.
The mentioned works highlight the novelty of each respec-
tive filter, however, a comparitive study between a selection
of conventional filtering methods and CNN based approach
have not been explored. Additionally, many of the existing
CNN based approaches for LiDAR filtering and weather
applicaions do not consider dust as one of the conditions.
In this paper, a novel CNN based approach is proposed to
classify and filter airborne dust particles in LiDAR point
clouds for autonomous excavation applications. In existing
studies, CNN based approaches have been used for airborne
particle filtering. However, in these studies the voxel-feature
tabular data is converted to an image-like structure before
being fed into the model. In this study, the tabular data is used
as the input to the CNN model, and reshaping functions are
used to convert the tabular data into an image-like structure
directly within the model. Therefore, the training process
can optimize the conversion between the tabular data and
image structure. Additionally, this work is applied to dust de-
filtering applications only, however, the same methodology
can be applied in other airborne particle filtering applications
under various adverse weaather conditions such as snow, fog,
and rain. The proposed model will be compared to conven-
tional filtering methods with ground-truth data to evaluate
the effectiveness of the model. The main contributions of our
work can be summarized as follows.
A large-scale voxel-based dataset was collected and la-
belled for the CNN training.
A novel CNN structure was proposed that converts the
voxelized features into a image-like structure for feature
extraction.
This study considers several conventional filtering meth-
ods, such as the ROR, DROR, SOR, and LIOR, provides
a comprehensive analysis on each filter’s de-dusting
performance compared to the developed CNN model.
The voxelized classification results from the CNN were
converted to a point-based classification as to properly
compare to the results generated from the conventional
methods.
The remainder of this paper is divided into the following
sections. Section II describes the theoretical background and
working principles of the conventional filtering methods and
the CNN architecture. Section III describes the collection
and preparation of the training data, as well as the metrics
used to quantify the improvements of the CNN approach.
Section IV covers a complete analysis of the classification
results of the conventional methods and CNN method, the
improvements made are highlighted in this section. Finally,
Section V discusses the concluding remarks, limitations, and
future work for the proposed area of research.
II. CONVENTIONAL AND PROPOSED FILTERING
METHODOLOGY
This section will present the operating principle and theoret-
ical background regarding the selected conventional filtering
algorithms and the proposed CNN-based filtering method.
2VOLUME 11, 2023
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
A. RADIUS OUTLIER REMOVAL FILTER (ROR)
The ROR filter eliminates outlier points in 3D space through
a geometrical approach [7]. By iterating though each point
in the point cloud, a sphere can be created with a prede-
fined search radius (SR), and the neighboring points can be
counted. If the number of neighbors is less than a defined
threshold, the point will be classified as an outlier (dust) and
removed from the point cloud. The input parameters for this
algorithm are the SR and minimum acceptable number of
neighboring points. An example of the ROR filter being ap-
plied to 2D point data can be seen in Fig. 1. In 3D, the circles
created by the SR would simply be modelled as spheres.
The ROR has some drawbacks with LiDAR applications.
Specifically, due to the sensor resolution, the density of the
point cloud changes with respect to the radial distance. As a
result, non-dust points may be classified as dust, resulting in
a filtered point cloud that removed important environmental
information.
B. DYNAMIC RADIUS OUTLIER REMOVAL FILTER (DROR)
The DROR filter was developed to resolve the problem seen
in the ROR regarding the changing point cloud density for
further objects [7]. It operates like the ROR, except that the
SR is a function of the point’s coordinates and the sensor
resolution. The SR can be calculated in (1), where xand yare
the coordinates of the point, βis a multiplier constant greater
than 1, and αis the angular resolution of the LiDAR.
SR =βpx2+y2α(1)
There should also be minimum SR (SRmin) defined for
points that are near the sensor, otherwise no neighboring
points will be detected for a SR defined in (1). So, the SR is
calculated for all points, but when it is below SRmin, the SR is
simply set to SRmin. The input parameters for the DROR filter
are the multiplier constant (β), angular resolution (α), min-
imum SR (SRmin), and the minimum acceptable neighboring
points. An example of the DROR filter being applied to 2D
FIGURE 1. ROR example for 2D data. In this example, the non-dust
threshold was set to 2.
point data can be seen in Fig. 2. In 3D, the circles created by
the SR would simply be modelled as spheres.
C. STATISTICAL OUTLIER REMOVAL FILTER (SOR)
The SOR filter detects outliers in the point cloud using a
statistical approach [19]. First, the point cloud statistics need
to be calculated. By iterating through all points in the point
cloud, the average distance (dk) of knearest neighbors can
be calculated. Based on the average distances calculated, the
mean average distance (µ) and deviation (σ) can be calcu-
lated. With these parameters, the statistical outliers can be
identified using (2), where βis a specified multiplier.
µβαdkµ+βα(2)
Based on (2), each points average distance for knearest
neighbors can be calculated, and if it is outside of the statis-
tical range, it is classified as dust. The drawback with this
approach is that it is computationally expensive. Since the
mean average distance and deviations need to be calculated
to classify them, the point cloud needs to be iterated over
twice. The input parameters for this approach are the number
of neighboring points and the constant multiplier.
D. LOW-INTENSITY OUTLIER REMOVAL FILTER (LIOR)
The ROR, DROR, and SOR all rely strictly on the geometry
of the points within the point cloud. However, LiDAR can
measure the intensity of the signal obtained reflecting off an
object. The LIOR approach uses this valuable information in
addition to the geometry of the points [10]. As discussed in
[1], the LIOR filter consists of two stages.
In the first stage, the points within the point cloud are fil-
tered with respect to a predefined intensity value. The second
FIGURE 2. DROR example for 2D data. In this example, the non-dust
threshold was set to 2.
VOLUME 11, 2023 3
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
stage uses the geometry information of the points removed in
the first stage. The second stage can use either the ROR or the
DROR filter previously discussed, thus creating the LIOR-
ROR or LIOR-DROR filters. If the point is classified as an
outlier in the first and second stage, then it is classified as
dust. If the point is calculated as an outlier in the first stage
but not the second stage, then it is not classified as dust and
is kept in the original point cloud.
Since LIOR is paired with either ROR or DROR, the input
parameters are the same except for the intensity threshold
used in the first stage. To properly select the intensity thresh-
old, a study should be conducted with ground-truth labelled
data. The LIOR filter adds an additional measure to the ROR
and DROR filters by identifying possible dust points, then the
second stage can preserve non-dust points that may have a
relatively smaller intensity value.
E. PROPOSED FILTERING METHODOLOGY:
CONVOLUTIONAL NEURAL NETWORK FILTER (CNN)
In literature, the CNN [20] is commonly used in object detec-
tion and recognition in images. The CNN can be applied in
many applications ranging from medical image classification
for disease diagnosis [21] to facial recognition [22]. As seen
in a majority of applications, the CNN model consists of an
input layer, several hidden layers, and an output layer used
for classification [23]. The operation types within the hidden
layers consist of the convolutional layer (feature extraction),
pooling layer, and fully connected layer [23]. The structure
of the CNN, the order in which the layers occur, and the
parameters selected have a large influence on the performance
of the model. Thus, these metrics need to be carefully selected
to yield a high-quality model.
Sometimes, a CNN may be applied to tabular data rather
than images, as seen in [24] and [25]. In these cases, the
tabular data is converted into an image-like structure that can
be used by the CNN. As explained by Zhu et al. [25], the
tabular data is converted to an image where each entry in the
tabular data is converted to a pixel. They state that the location
of the pixels should be optimized such that similar features are
close together in the image.
As seen in literature, LiDAR data can be voxelized [26],
and features can be calculated using a principle component
analysis (PCA), which will be discussed in detail in the next
section. This results in a collection of tabular data where each
voxel consists of several metrics calculated using the PCA.
So, the input of the CNN model is tabular data representing
a voxel containing several LiDAR points. As described in
literature, the tabular data should be converted into an image-
like structure for the CNN to function as intended.
The proposed CNN model takes tabular data as the input
and is immediately passed through a dense layer. Rather than
converting the tabular data to an image before being trained,
the proposed CNN model converts the tabular data to an
image after the first dense layer. This process increases the
dimensionality of the input vector, then the data can simply be
rearranged into a 2D image. In doing so, the model can train
the dense layer weights such that the optimal tabular data-to-
image conversion is obtained.
After the first dense layer, the resulting vectors can be
flattened and reshaped into an image. Then, the first 2D con-
volution layer is applied. For this process, a ReLU activation
function was selected, which can be formulated in (3). The
ReLU function outputs the input (x) if it is positive, and 0
otherwise [27]. The ReLU function was selected because of
its simple implementation, yet effective performance. The
ReLU function was used for each convolutional layer.
f(x) = max(0,x)(3)
After the first convolutional layer, a 2D max pooling func-
tion is used on the tensors produced. 2D max pooling works
by creating a pool (typically a 2D matrix of a specified size)
and sliding it over a tensor in the X and Y directions. The
maximum value found within a pool will be added to output
tensor [28]. This process down samples the input tensor and
extracts important features. A visual representation of the 2D
max pooling operation can be seen in Fig. 3. The 2D max
pooling operation is done after every convolutional layer.
In total, there are 3 2D convolutional layers paired with 2D
max pooling in series. After the final 2D max pooling layer,
the tensor is flattened and used as the input to the final dense
layer with a sigmoid activation function, which is ideal for
binary classification. The sigmoid activation function can be
formulated in (4), where it takes any real number as an input,
and the output is in the range of 0 to 1 [29]. This models the
probability of the voxel being either dust or non-dust.
f(x) = 1
1 + ex(4)
A graphical representation of the proposed CNN structure
can be seen in Fig. 4, where the dimension of the tensors is
highlighted at each layer. The first convolutional layer has
32 3 x 3 filters, the second convolutional layer has 128 3
x 3 filters, and the third convolutional layer has 256 3 x 3
filters. Each 2D max pooling layer has a pool size of 2 x 2.
FIGURE 3. Example of a 2D max pooling operation.
4VOLUME 11, 2023
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
FIGURE 4. The proposed structure for dust de-filtering.
The proposed CNN model was assembled using the Python
TensorFlow library [30].
III. EXPERIMENT
To train any AI model, a large labelled dataset is a necessity.
So, the team conducted several experiments and test runs to
gather enough data to train a robust model.
The training data used was gathered from 2 experiments.
In both experiments, a VLP-16 LiDAR sensor was used [31].
The first experiment consisted of a stationary sensor with a
non-dust target and dust cloud that varied in distance from
the sensor. The distances of the dust cloud and non-dust
target for the first experiment can be seen in Table 1. For
the first experiment, only 1 leaf blower was active for all test
runs. The second experiment was a more realistic scenario
with dynamic environmental factors. Specifically, the VLP-
16 was mounted on a mobile platform and there were dynamic
objects in the environment (people walking), as well as a dust
cloud generated from up to two leaf blowers. This results
in a dust cloud that varied in density depending on how
many leaf blowers (i.e., 1 or 2 blowers) were active. The
initial distances of the dust cloud and non-dust target for the
second experiment can also be seen in Table 1. For the second
experiment, the number of active leaf blowers was changed
for each test run. Meaning that each test run had 1 leaf blower
active, then an additional leaf blower was activated to create
the dense dust cloud. Also, the distance of the dynamic non-
dust obstacle was varied as it moved through the environment.
An example of the data collection environment can be seen in
Fig. 5. To label the LiDAR data collected, the LiDAR labeller
app in MATLAB was used [32].
As seen in Fig. 5, only the data directly in front of the
TABLE 1. First and second experiment dust and non-dust target distances.
Experiment Test Dust Cloud Non-Dust
Run Distance Target Distance
First
1 4 m 5 m
2 5 m 10 m
3 8 m 10 m
4 10 m 15 m
Second
1 4 m 5 m
2 6 m 5 m
3 7 m 10 m
4 11 m 10 m
sensor was considered. There are two reasons for this. The
first reason is that the dust clouds were generated only in front
of the sensor. If the full field of view of the LiDAR sensor
was considered, a significant amount of non-dust particles
would be captured from behind the sensor. If this dataset was
used for training, it would result in a biased model due to the
imbalanced classes. The second reason is that the proposed
dust de-filtering technology was developed for vehicles that
only travel in the forwards direction. Meaning that only the
environment directly in front of the vehicle needs to be con-
sidered. Additionally, having less points to process can reduce
the processing time, which is ideal for real-time applications.
To extract features in the point cloud to use for training, the
point clouds were voxelized. Specifically, an octree structure
[33] was used for the voxelization. The octree decomposi-
tion begins with a region of interest (ROI), and the ROI is
continuously segmented into children voxels given that some
condition is met. Some conditions may include the maximum
number of children partitions, the minimum voxel dimension,
or the voxel capacity. In some cases, the voxels may be
different sizes.
For this application, each voxel must contain at least 4
points to do the PCA. The PCA of a point distribution in
a voxel is derived from the least squares estimation [26].
Single value decomposition can be used to derive the nor-
mal vector of the best-fit plane by solving an eigensystem
[34]. The eigenvalues found using the PCA can be used to
compute several important geometrical traits of the voxelized
data. The calculated eigenvalues can be ordered as follows,
λ0λ1λ2[35]. The geometric features calculated using
the eigenvalues are based on the roughness [11], planarity
[36], and curvature [36]. The selected geometric features can
be formulated in (5) to (7).
f1=λ2
λ0
(5)
f2=λ1
λ0
(6)
f3=λ0
λ0+λ1+λ2
(7)
In addition to the geometric features in (5) to (7), intensity
features were also used. The mean and deviation intensities
of the points in each voxel were calculated as the final two
features used for training. The mean and deviation of intensity
VOLUME 11, 2023 5
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
FIGURE 5. The environment in which the training and testing LiDAR data was collected. The MATLAB LiDAR labeler app was used to label the dust points
seen in red.
calculations can be seen in (8) and (9), where intiis the
intensity of point iin the voxel, nis the number of points in a
voxel, and int is the mean intensity seen within the voxel.
f4=1
n
n
X
i=1
inti(8)
f5=v
u
u
t
1
n1
n
X
i=1
(intiint)2(9)
A summary of the training dataset can be seen in Table 2.
When training the CNN, the dataset is stratified and split 70%
for training, and 30% validation.
A. METRICS FOR IMPROVEMENT
To quantify the performance of the proposed CNN model, it
will be compared to the conventional filtering methods. Since
the conventional filtering methods are applied to the points,
and the CNN filtering method is applied to the voxelized
point cloud, some conversions are needed as to properly
compare the results side-by-side. To do so, the resulting voxel
classifications are applied to all points within the voxel. For
example, if a voxel is classified as dust by the CNN, then all
points within the voxel are dust.
To quantify performance of the discussed filters, the F1
score will be computed. The F1 score is defined as the har-
monic mean of precision and recall [37]. The F1 score was
selected since it considers true positives, false positives, and
false negatives in its calculation. The mathematical formula-
tion of precision, recall, and the F1 score can be seen in (10) to
(12). In (10) and (11), TP represents the true positives, which
are the ground-truth dust particles that have been successfully
classified as dust, FP represents the false positives, which
are the non-dust particles that are classified as dust, and FN
represents the false negatives, which are the dust particles that
TABLE 2. Voxelized training data for the CNN model.
Non-Dust Voxels Dust Voxels
253, 362 10, 742
are classified as non-dust. The F1 score formulated in (12)
considers both the precision (p) and recall (r).
p=TP
TP +FP (10)
r=TP
TP +FN (11)
F1 = 2
r1+p1(12)
IV. RESULTS AND DISCUSSION
In this section, a complete analysis of the conventional meth-
ods and CNN model will be conducted for airborne dust par-
ticle filtering. To compare the performance of each filtering
method, 4 frames worth of LiDAR data will be tested. The
previous research only tested the conventional methods with
the static environment [1], so the 4 frames will be selected
from the dynamic experiment since it is closer to the real-
world application of the proposed research.
A. CONVENTIONAL METHODS
As previously discussed, the ROR, DROR, SOR, and LIOR
filtering methods will be selected to compare against the
proposed CNN model. The parameters selected for the con-
ventional filtering methods can be summarized in Table 3.
In Table 3, all values were selected based on trial and error
to achieve the best performance for the respective filtering
algorithm, except for the intensity threshold in the LIOR
filters. For this, a study was conducted using the ground-truth
labelled data. The intensity distribution was examined for the
4 test frames, and the threshold was selected based on these
results. The intensity distribution plots can be seen in Fig.
6. From the plots seen in Fig. 6, the recommended threshold
intensity for identifying dust particles is about 3. Some non-
dust particles do go less than 3. However, since the LIOR
filters are paired with ROR and DROR, this portion of the
algorithm aims to preserve these outliers.
The performance of the conventional filters can be sum-
marized in Table 4. Examining the results presented in Table
4, the best conventional airborne dust filtering algorithm is
6VOLUME 11, 2023
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
TABLE 3. Parameters for the ROR, DROR, SOR, and LIOR conventional
filters.
Filter Parameter Value
ROR SR 0.04 m
Minimum Acceptable Points 3
DROR
SRmin 0.04 m
Minimum Acceptable Points 3
Multiplier Constant (β) 0.05
Angular Resolution (α) 0.25
SOR kNearest Neighbors 3
Multiplier Constant 0.2
LIOR-ROR
SR 0.043 m
Intensity Threshold 3
Minimum Acceptable Points 6
LIOR-DROR
SRmin 0.05 m
Intensity Threshold 3
Minimum Acceptable Points 4
Multiplier Constant (β) 0.03
Angular Resolution (α) 0.25
the LIOR-DROR, whereas the best conventional filter that
does not use the point intensity is the DROR filter. The worst
conventional filter is the ROR filter, with an F1 score ranging
from 13.99 % to 26.65 %. The results show that when a
conventional filter is paired with a two stage LIOR approach,
the F1 score can improve. This is seen when comparing the
ROR filter to the LIOR-ROR, and the DROR filter to the
LIOR-DROR.
Since the LIOR-DROR filter was the best amongst the
conventional filters, the LIOR-DROR F1 scores will be used
to compare to the CNN filter.
B. CONVOLUTIONAL NEURAL NETWORK
The CNN model was trained using the voxelized dataset
discussed in Section III in about 4 hrs. Examining Table 2, the
dataset consists of approximately 4 % dust voxels. Meaning
that the training dataset is extremely biased towards non-dust
voxels. Unfortunately, there are no publically available dust
LiDAR datasets, so the training data was limited to the data
collected in the discussed experiments. Additionally, manu-
TABLE 4. F1 score for the conventional and CNN filters on 4 different
LiDAR frames from the dynamic environment.
Testing Frame
Filter 1 2 3 4
ROR 26.65 % 17.10 % 13.99 % 25.91 %
DROR 72.39 % 75.69 % 80.10 % 45.28 %
SOR 27.62 % 48.97 % 64.64 % 28.69 %
LIOR-ROR 64.04 % 41.98 % 30.55 % 57.29 %
LIOR-DROR 82.76 % 85.93 % 91.07 % 81.85 %
CNN 82.16 % 88.87 % 95.57 % 87.24 %
ally labelling all the point clouds in the MATLAB LiDAR
labeller app was a time-consuming process.
So, a few different data processing techniques were applied
to observe which method yielded the best F1 score in the vali-
dation stage of training. First, the minority class (dust voxels)
was oversampled using the synthetic minority oversampling
technique (SMOTE) [38]. The SMOTE works by operating in
the feature space of the minority class to generate synthetic
data. The synthetic data is created by selecting features along
line segments that join any/all of the minority class’s nearest
neighbors [38].
The other approach was to stratify the dataset. By allowing
the same fraction of the minority class to be in the training
and validation partitions, the imbalance can be minimized.
Through experimentation, it was found that the SMOTE tech-
nique could not accurately produce synthetic dust voxel data,
resulting in a CNN model that had a low F1 score when tested
with the mentioned 4 frames. So, the CNN model was trained
using the stratified dataset. The testing results for the CNN
model can be seen in Table 4, along with the conventional
filtering results.
Examining the F1 scores in Table 4, the CNN model outper-
formed the best conventional filter (LIOR-DROR) for 3 out
of 4 of the testing point clouds. The performance of the CNN
was comparable to that of the LIOR-DROR filter for the first
testing dataset with a difference of 0.6 %. However, the other
3 frames saw an improvement of up to 5.39 %, thus validating
the CNN filtering method as an effective means of classifying
FIGURE 6. Intensity distribution plots for the non-dust (a) and dust (b) particles.
VOLUME 11, 2023 7
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
FIGURE 7. Testing frame 1 before (a) and after (b) CNN airborne dust filtering. The red points are ground truth labelled dust points.
FIGURE 8. Testing frame 2 before (a) and after (b) CNN airborne dust filtering. The red points are ground truth labelled dust points.
airborne dust particles. A side-by-side comparison of each
testing frame before and after applying the CNN filter can
be seen in Fig. 7 to Fig. 10. Examining Fig. 7 to Fig. 10, most
of the dust (red LiDAR points) were effectively eliminated
while still preserving important environmental information,
such as the ground and solid obstacles. This performance is
also reflected in the F1 scores found in Table 4.
V. CONCLUSION
In this paper, a CNN model was proposed to classify airborne
dust particles in autonomous excavation applications. Since
CNNs are commonly applied to image-based classification,
the tabular data (voxel features) was converted to an image-
like structure by using the CNNs dense layer and reshaping
function. In doing so, the proposed model can optimize the
weights of the dense layer such that the best image structure
is obtained. This differs from existing research where the
tabular data is converted to an image-like structure before
being fed to the CNN model. In existing studies, extensive
research was conducted to find the optimal tabular data-
to-image conversion method. In this study, this process is
encorporated into the model itslef, so the training process
can find the appropriate conversion. Additionally, this study
conducted a comprehensive analysis of the proposed CNN
model and several conventional de-dust filtering methods.
This comparison provides insight regarding the performance
of each filtering method in de-dusting applications.
The proposed CNN model was compared to several con-
ventional filtering methods, namely the ROR, DROR, SOR,
LIOR-ROR, and LIOR-DROR for 4 frames selected from the
dynamic environment experiment. The results show that the
CNN model can outperform the best conventional filtering
method in 3 of the test frames, whereas the performance was
comparable in the one without improvement.
Some drawbacks of this research include the limited train-
ing data for dust classification. Since there are no publicly
available dust LiDAR datasets, the training data needs to be
collected and labelled manually through the discussed exper-
iments. This can be done in the MATLAB LiDAR labeller
app; however, it is time consuming to gather enough data
for training a CNN model. Additionally, the amount of dust
samples in the training data was very limited (approximately 4
% dust samples). If more dust samples were collected, the F1
score of the CNN model may increase in the proposed testing
cases.
Future work for this research includes applying the CNN
model in real-time. This paper mainly focuses on the val-
idation of the proposed CNN model, which was conducted
offline after the data was collected. Additionally, the perfor-
mance may be improved by fusing multiple sensors together.
For example, LiDAR data can be fused with depth camera
data, which is what will be achieved in the future work as
8VOLUME 11, 2023
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
FIGURE 9. Testing frame 3 before (a) and after (b) CNN airborne dust filtering. The red points are ground truth labelled dust points.
FIGURE 10. Testing frame 4 before (a) and after (b) CNN airborne dust filtering. The red points are ground truth labelled dust points.
well. Finally, the background environment can be added after
the removal of the dust particles. In the proposed research,
the airborne dust particles are removed from the point cloud.
However, since non-dust objects may exist behind a dust
cloud, the dust cloud can be blocking the non-dust objects.
When this happens, the dust cloud can be removed through
filtering, but the non-dust objects behind the dust cloud are
not detected. This presents a safety concern since informa-
tion is missing regarding the obstacles that exist behind dust
clouds. So, future work aims to fuse a static environment scan
with the filtered point cloud such that the environment behind
the dust cloud is maintained. This can be done by filtering the
point cloud captured in real-time, and adding non-dust objects
from the static environment scan that were blocked by the dust
cloud. However, this poses an additional concern regarding
the dynamic non-dust objects. So, additional methods may be
used to track dynamic non-dust objects in real time and add
them to the filtered environment (in their predicted location)
if they are blocked by a dust cloud.
ACKNOWLEDGMENT
The authors of this paper would like to thank Ali Afza-
laghaeinaeini for his previous research in this area. Afza-
laghaeinaeini helped to capture and label the experimental
data used in this study. He also has conducted detailed studies
of the conventional de-dust filtering methods, thus paving the
way for our research. This research was funded by the Korean
Institute of Machinery and Materials.
REFERENCES
[1] A. Afzalaghaeinaeini, J. Seo, D. Lee, and H. Lee, ‘‘Design of dust-
filtering algorithms for LiDAR sensors using intensity and range infor-
mation in off-road vehicles,’’ Sensors, vol. 22, no. 11, May, 2022, DOI:
10.3390/s22114051.
[2] T. G. Phillips, N. Guenther, and P. R. McAree, ‘‘When the dust settles:
The four behaviors of LiDAR in the presence of fine airborne particu-
lates,’ J. Field Robotics, vol. 34, no. 5, pp. 985–1009, Feb. 2017, DOI:
10.1002/rob.21701.
[3] T. Peynot, and A. Kassir, ‘‘Laser-camera data discrepancies and reliable
perception in outdoor robotics,’ in 2010 IEEE/RSJ International Conference
on Intelligent Robots and Systems, Taipei, Tiwan, 2010, pp. 2625–2632.
[4] M. P. Gerado-Castro, T. Peynot, and F. Ramos , ‘Laser-radar data
fusion with gaussian process implicit surfaces,’ in Field and Service
Robotics,105th ed. New York, New York, USA: Cham: Springer Inter-
national Publishing, 2015, pp. 289–302. [Online]. Available: https://doi-
org.uproxy.library.dc-uoit.ca/10.1007/978-3-319-07488-7_20.
[5] G. Xie, J. Zhang, J. Tang, H. Zhao, N. Sun, and M. Hu, ‘Obstacle detection
based on depth fusion of lidar and radar in challenging conditions,’ Indus-
trial Robot, vol. 48, no. 6, pp. 792–802, Feb. 2017, DOI: 10.1002/rob.21701.
[6] A. Aldoma et al., ‘‘Tutorial: Point cloud library: Three-dimensional
object recognition and 6 DOF pose estimation,’ IEEE Robotics &
Automation Magazine, vol. 19, no. 3, pp. 80–91, Sept. 2012, DOI:
10.1109/MRA.2012.2206675.
[7] N. Charron, S. Phillips, and S. L. Waslander, ‘De-noising of LiDAR point
clouds corrupted by snowfall,’’ in 2018 15th Conference on Computer and
Robot Vision (CRV), Toronto, On., Canada, 2018, pp. 254–261.
VOLUME 11, 2023 9
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
[8] Y. Cao, H. Huang, and D. Yu, ‘‘Filter methods for removing falling
snow from light detection and ranging point clouds in snowy weather,’’
Sens. Mater., vol. 34, no. 12, pp. 4507–4519, Nov. 2022, DOI:
10.18494/SAM4047.
[9] R. B. Rusu, and S. Cousins, ‘‘3D is here: Point Cloud Library (PCL),’’ in
2011 IEEE International Conference on Robotics and Automation, Shang-
hai, China, 2011, pp. 1–4.
[10] J. I. Park et al., ‘‘Fast and accurate desnowing algorithm for LiDAR
point clouds,’ IEEE Access, vol. 8, pp. 160202–160212, Aug. 2020, DOI:
10.1109/ACCESS.2020.3020266.
[11] L. Stanislas, N. Suenderhauf, and T. Peynot, ‘Lidar-based detection of
airborne particles for robust robot perception,’ in Proceedings of the Aus-
tralasian Conference on Robotics and Automation (ACRA) 2018, Austrailia,
2018, pp. 1–8.
[12] A. U. Shamsudin et al., ‘‘Fog removal using laser beam penetration, laser
intensity, and geometrical features for 3D measurements in fog-filled room,’’
Advanced Robotics, vol. 30, no. 11 12 pp. 729–743, Feb. 2016, DOI:
10.1080/01691864.2016.1164620.
[13] A. Afzalaghaeinaeini, ‘‘Design of dust-filtering algorithms for LiDAR
sensors in off-road vehicles using the AI and non-AI methods,’’ M.S. thesis,
Dept. Auto. Mech. Eng., Ont. Tech., Oshawa, Ont., 2022.
[14] R. Heinzler, F. Piewak, P. Schindler and W. Stork, ‘‘CNN-based
LiDAR point cloud de-noising in adverse weather,’’ IEEE Robotics
and Automation Letters, vol. 5, no. 2pp. 2514–2521, Apr. 2020,
DOI:10.1109/LRA.2020.2972865.
[15] Z. Luo, J. Ma, G. Xiong, X. Hu, Z. Zhou and J. Xu, ‘‘Semantic segmen-
tation based rain and fog filtering only by LiDAR point clouds,’’ in 2022
IEEE International Conference on Unmanned Systems (ICUS), Guangzhou,
China, 2022, pp. 90–95.
[16] G. Sebastian, T. Vattem, L. Lukic, C. Bürgy and T. Schumann,
‘‘RangeWeatherNet for LiDAR-only weather and road condition classifi-
cation,’ in 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan,
2021, pp. 777–784.
[17] M. P. Da Silva, D. Carneiro, J. Fernandes and L. F. Texeira, ‘‘MobileWeath-
erNet for LiDAR-only weather estimation,’’ in 2023 International Joint
Conference on Neural Networks (IJCNN), Gold Coast, Australia, 2023, pp.
1–8.
[18] K. Simonyan and A. Zisserman, ‘‘Very deep convolutional networks for
large-scale image recognition,’ in International Conference on Learning
Representations, 2015.
[19] R. B. Rusu et. al, ‘‘Towards 3D Point cloud based object maps for house-
hold environments,’’ Robotics and Autonomous Systems, vol. 56, no 11 pp.
927–941, Aug. 2008, DOI: 10.1016/j.robot.2008.08.005.2020.2972865.
[20] Y. LeCun, Y. Bengio, and G. Hinton, ‘‘Deep learning,’ Nature, vol. 521,
no 7553 pp. 436–444, May. 2015, DOI: 10.1038/nature14539.
[21] S. S. Yadav, and S. M. Jadhav, ‘‘Deep convolutional neural network based
medical image classification for disease diagnosis,’ Journal of Big Data,
vol. 6, no 1pp. 1–18, Dec. 2019, DOI: 10.1186/s40537-019-0276-2.
[22] Q. Sun, and A. Redei, ‘‘Knock knock, who’s there: Facial recognition
using CNN-based classifiers,’ International Journal of Advanced Com-
puter Science and Applications, vol. 13, no 1pp. 9–16, 2022, DOI:
10.14569/IJACSA.2022.0130102.
[23] X. He et al, ‘Simple-Encoded evolving convolutional neural net-
work and its application to skin disease image classification,’ Swarm
and Evolutionary Computation, vol. 67, pp. 100955, Aug. 2021, DOI:
10.1016/j.swevo.2021.100955.
[24] M. Artzi et al, ‘‘Classification of pediatric posterior fossa tumors using
convolutional neural network and tabular data,’’ IEEE Access, vol. 9, pp.
91966–91973, June 2021, DOI: 10.1109/ACCESS.2021.3085771.
[25] Y. Zhu et al, ‘‘Converting tabular data into images for deep learning with
convolutional neural networks,’’ Scientific Reports, vol. 11, no. 1, pp. 11325,
May 2021, DOI: 10.1038/s41598-021-93376-5.
[26] M. Wang, and Y. H. Tseng, ‘‘Incremental segmentation of lidar point
clouds with an octree-structured voxel space,’’ The Photogrammetric
Record, vol. 26, no. 133, pp. 32–57, Mar. 2011, DOI: 10.1111/j.1477-
9730.2011.00624.x.
[27] C. Banerjee, T. Mukherjee, and E. Pasiliao Jr., ‘‘An empirical study on
generalizations of the relu activation function,’’ in Proceedings of the 2019
ACM Southeast Conference, Kennesaw, GA, USA, 2011, pp. 164–167.
[28] B. Zhao, Y. S. Chong, and A. Tuan Do, Area and energy efficient 2D
max-pooling for convolutional neural network hardware accelerator,’ in
IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics
Society, Singapore, 2020, pp. 423–427.
[29] G. Mourgias-Alexandris et al., ‘‘An all-optical neuron with sigmoid acti-
vation function,’ Opt. Express, vol. 27, no. 7, pp. 9620–9630, Apr. 2019,
DOI: 10.1364/OE.27.009620.
[30] TensorFlow. [Online]. Available: https://www.tensorflow.org/.
[31] Velodyne LiDAR puck. [Online]. Available:
https://www.amtechs.co.jp/product/VLP-16-Puck.pdf.
[32] Getting started with the LiDAR labeler. [Online]. Available:
https://www.mathworks.com/help/lidar/ug/lidar-labeler-get-started.html.
[33] Su. Y-T, J. Bethel, and S. Hu, ‘‘Octree-based segmentation for terrestrial
LiDAR point cloud data in industrial applications,’’ ISPRS Journal of Pho-
togrammetry and Remote Sensing, vol. 133, pp. 59–74, Mar. 2016, DOI:
10.1016/j.isprsjprs.2016.01.001.
[34] G. Golub, and C. F. Van Loan, ‘‘Matrix computations, 3rd edition,’
The Mathematical Gazette, vol. 83, no. 498, pp. 556–557, 1999, DOI:
10.2307/3621013.
[35] J-F. Lalonde et al., ‘Natural terrain classification using three-dimensional
ladar data for ground robot mobility,’’ Journal of Field Robotics, vol. 23, no.
10, pp. 839–861, Jun. 2006, DOI: 10.1002/rob.20134.
[36] Y. Li et al., ‘‘Deep learning for LiDAR point clouds in autonomous
driving: A review,’’ IEEE Transactions on Neural Networks and
Learning Systems, vol. 32, no. 8, pp. 3412–3432, Aug. 2021, DOI:
10.1109/TNNLS.2020.3015992.
[37] K. Takahashi et al., ‘Confidence interval for micro-averaged F1 and
macro-averaged F1 scores,’ Applied Intelligence, vol. 52, no. 2, pp. 4961–
4972, Jul. 2021, DOI: 10.1007/s10489-021-02635-5.
[38] N. V. Chawla et al., ‘‘SMOTE: Synthetic minority over-sampling tech-
nique,’ The Journal of Artificial Intelligence Research, vol. 16, pp. 321–
357, Jun. 2011, DOI: 10.1613/jair.953.
10 VOLUME 11, 2023
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
T. Parsons et al.: Dust De-Filtering in LiDAR Applications with Conventional and CNN Filtering Methods
TYLER PARSONS received a Bachelor’s in Engi-
neering (Honours) degree in 2021, and a Master of
Applied Science degree in 2023 from Ontario Tech
University, Ontario, Canada. His main area of re-
search for graduate studies was in route optimiza-
tion and global path planning. Since graduating, he
has been working as an Associate Researcher at the
Autonomous Vehicle and Electro-Hydraulic Con-
trol (AVEC) lab at Ontario Tech University. His
current area of research includes advance sensing
technology and path planning optimization.
JAHO SEO received his BS degree in agricultural
machinery and process engineering from Seoul
National University, Seoul, Korea in 1999, his ME
degree in mechanical engineering from the Univer-
sity of Quebec (Ecole de Technologie Superieure),
Montreal, Canada in 2006, and his PhD in me-
chanical engineering from University of Waterloo,
Waterloo, Canada in 2011. He was with the De-
partment of Mechanical and Mechatronics Engi-
neering of University of Waterloo as a postdoctoral
fellow in 2011, the Department of System Reliability of Korea Institute of
Machinery & Materials (KIMM) as a senior researcher during 2012–2016,
and the Department of Biosystems Machinery Engineering of Chungnam
National University, Korea as an assistant professor during 2016–2017. Since
2017, he has been an assistant professor at the Department of Automotive
and Mechatronics Engineering, Ontario Tech University where he has been
involved in research on the development of autonomous control systems for
intelligent mobile machines.
BYEONGJIN KIM received his B.S. degree in
Electrical Engineering from Pohang University of
Science and Technology (POSTECH) in 2015, and
his Ph.D. degree in Convergence IT Engineering
from Pohang University of Science and Technol-
ogy (POSTECH) in 2021. Currently, he is working
as a Senior Researcher at the Department of Smart
Industrial Machine Technologies of the Korea In-
stitute of Machinery and Materials (KIMM). His
research interests include sensing systems under
extreme environment, autonomous driving of unmanned systems and field
robotics.
HANMIN LEE is a principal researcher and the
Head of the Department of Smart Industrial Ma-
chine Technologies at the Korea Institute of Ma-
chinery and Materials (KIMM). He received his
B.S., M.S., and Ph.D. in mechanical engineering
from Korea Advanced Institute of Science and
Technology (KAIST) in 1998, 2000, and 2005,
respectively. His current research interest is au-
tonomous driving and manipulation in off-road en-
vironments.
JI-CHUL KIM received his Bachelor’s degree in
Mechanical Engineering from Yonsei University
in 2007, his Master’s and Ph.D. degrees in Me-
chanical Engineering from KAIST with a focus on
Robotics in 2009 and 2014, respectively. Currently,
he works as a Senior Researcher at the Depart-
ment of Smart Industrial Machine Technologies
of the Korea Institute of Machinery and Materials
(KIMM). His main research interests lie in un-
manned systems technology and automatic con-
trol.
MOOHYUN CHA received his B.S. degree from
POSTECH (Pohang University of Science and
Technology) and an M.S. degree from KAIST.
Currently, he is working as a Principal Researcher
at the Department of Smart Industrial Machine
Technologies of the Korea Institute of Machinery
and Materials (KIMM). His research interests in-
clude various VR (virtual reality) applications for
engineering and training and computer graphics &
data processing for very large engineering datasets.
VOLUME 11, 2023 11
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3362804
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
... The first focuses on point cloud segmentation and object detection and recognition using LiDAR. The second area aims to establish the robustness of LiDAR point clouds against weather and external environmental factors [4,5]. ...
... These techniques utilize the geometry and light intensity of the point cloud to effectively eliminate outlier points that resemble airborne particles. However, a drawback of the method is that it removes object points, which can lead to slower algorithm performance when attempting to restore the point clouds using conventional methods [5]. ...
Article
Full-text available
The paper presents a novel method for real-time detection of off-road drivable regions in LiDAR point clouds without relying on deep learning. Our method directly distinguishes drivable regions while accounting for noise caused by snow and fog. By focusing on the scanning characteristics of rotating LiDAR, our PIIC approach achieves robust performance in challenging off-road environments. The proposed technique divides the space into sections and segments drivable areas by counting points in each section. It allows noise influence to be countable using a simple probabilistic model. The developed solution offers significant advancements in autonomous driving in off-road environments since the method offers drivability in real-time with noise immunity. Experimental results demonstrate the efficiency of the proposed method in hazardous situations such as heavy snow, fog, and crosstalk from other LiDAR signals. Importantly, the low computational load enables real-time operation and provides promising opportunities for fusion with other recognition methods.
Article
Although Light Detection and Ranging (LiDAR) is a sensor type for autonomous vehicles, it is recognized as an essential tool in various fields, such as drones, Unmanned Surface Vehicles (USVs), Unmanned Ground Vehicles (UGVs), and surveillance facilities. This is because LiDAR has advantages including a high resolution, 360-degree sensing capabilities, and consistent performance during the day and at night. However, LiDAR has the fatal disadvantage of performing poorly in snow, rain, and fog. Therefore, studies are attempting to remove noise points caused by snow, rain, and fog to ensure that LiDAR performs well even under extreme weather conditions. These studies began in approximately 2020, coinciding with accelerated research on autonomous driving, and they are still being actively pursued today. In particular, with the development of artificial intelligence technology in addition to the existing conventional methods, various approaches are being developed, but papers that thoroughly analyze and organize this research have not yet been published. Accordingly, in this study, we aim to comprehensively review the studies that have attempted to remove LiDAR sensor noise under extreme weather conditions through various methods. This review discusses the latest LiDAR denoising algorithms, with a particular focus on techniques implemented under snowy conditions. These algorithms can be categorized into five types: distance-based, intensity-based, fusion-based, learning-based, and other methods. In addition, this paper covers extreme weather datasets and related perception studies.
Article
Full-text available
Although the LiDAR sensor provides high-resolution point cloud data, its performance degrades when exposed to dust environments, which may cause a failure in perception for robotics applications. To address this issue, our study designed an intensity-based filter that can remove dust particles from LiDAR data in two steps. In the first step, it identifies potential points that are likely to be dust by using intensity information. The second step involves analyzing the point density around selected points and removing them if they do not meet the threshold criterion. To test the proposed filter, we collected experimental data sets under the existence of dust and manually labeled them. Using these data, the de-dusting performance of the designed filter was evaluated and compared to several types of conventional filters. The proposed filter outperforms the conventional ones in achieving the best performance with the highest F1 score and removing dust without sacrificing the original surrounding data.
Article
Full-text available
Artificial intelligence (AI) has captured the public's imagination. Performance gains in computing hardware, and the ubiquity of data have enabled new innovations in the field. In 2014, Facebook's DeepFace AI took the facial recognition industry by storm with its splendid performance on image recognition. While newer models exist, DeepFace was the first to achieve near-human level performance. To better understand how this breakthrough performance was achieved, we developed our own facial image detection models. In this paper, we developed and evaluated six Convolutional Neural Net (CNN) models inspired by the DeepFace architecture to explore facial feature identification. This research made use of the You Tube Faces (YTF) dataset which included 621,126 images consisting of 1,595 identities. Three models leveraged pretrained layers from VGG16 and InceptionResNetV2, whereas the other three did not. Our best model achieved a 84.6% accuracy on the test dataset.
Article
Full-text available
A binary classification problem is common in medical field, and we often use sensitivity, specificity, accuracy, negative and positive predictive values as measures of performance of a binary predictor. In computer science, a classifier is usually evaluated with precision (positive predictive value) and recall (sensitivity). As a single summary measure of a classifier’s performance, F 1 score, defined as the harmonic mean of precision and recall, is widely used in the context of information retrieval and information extraction evaluation since it possesses favorable characteristics, especially when the prevalence is low. Some statistical methods for inference have been developed for the F 1 score in binary classification problems; however, they have not been extended to the problem of multi-class classification. There are three types of F 1 scores, and statistical properties of these F 1 scores have hardly ever been discussed. We propose methods based on the large sample multivariate central limit theorem for estimating F 1 scores with confidence intervals.
Article
Full-text available
Purpose To the industrial application of intelligent and connected vehicles (ICVs), the robustness and accuracy of environmental perception are critical in challenging conditions. However, the accuracy of perception is closely related to the performance of sensors configured on the vehicle. To enhance sensors’ performance further to improve the accuracy of environmental perception, this paper aims to introduce an obstacle detection method based on the depth fusion of lidar and radar in challenging conditions, which could reduce the false rate resulting from sensors’ misdetection. Design/methodology/approach Firstly, a multi-layer self-calibration method is proposed based on the spatial and temporal relationships. Next, a depth fusion model is proposed to improve the performance of obstacle detection in challenging conditions. Finally, the study tests are carried out in challenging conditions, including straight unstructured road, unstructured road with rough surface and unstructured road with heavy dust or mist. Findings The experimental tests in challenging conditions demonstrate that the depth fusion model, comparing with the use of a single sensor, can filter out the false alarm of radar and point clouds of dust or mist received by lidar. So, the accuracy of objects detection is also improved under challenging conditions. Originality/value A multi-layer self-calibration method is conducive to improve the accuracy of the calibration and reduce the workload of manual calibration. Next, a depth fusion model based on lidar and radar can effectively get high precision by way of filtering out the false alarm of radar and point clouds of dust or mist received by lidar, which could improve ICVs’ performance in challenging conditions.
Article
Automatically searched architectures by neural architecture search (NAS) methods have shown promising performance in various visual recognition tasks. Among NAS methods, evolutionary neural architecture methods are popular because of their potential to find the global optimal convolutional neural networks (CNNs). These methods usually use an individual to represent a CNN, while often facing two challenges: 1) since a CNN has numerous encoded parameters and weights, the length of each individual is long which causes a large search space, and 2) due to the unknown optimal depth of a CNN, it is necessary to deal with a variable-length optimization problem, which leads to the search in a messy way since search spaces with different dimensions may have different optimal solutions. In this paper, we propose a genetic algorithm with a simple encoding scheme (SEECNN) for evolving CNNs to address image classification problems. In our encoding scheme, the parameters and weights of each layer are encoded into an individual and the whole population represents an entire CNN. Over the course of evolution, three offspring subpopulations are separately produced by genetic operators on three subpopulations. In each subpopulation, the lengths of individuals are short and equal. Afterward, we design a stable search strategy to update the population based on the performance improvement, where we only insert, replace, and remove one individual to generate candidate populations. SEECNN is compared with 24 well-known algorithms on nine benchmark datasets and five state-of-the-art methods on a real-world skin disease image classification case. The results demonstrate its effectiveness.