Conference PaperPDF Available

Mine Machinery Detection (MMD) from Remote sensing images: A comparative study

Authors:

Figures

Content may be subject to copyright.
Mine Machinery Detection (MMD) from Remote
sensing images: A comparative study
Ashish Soni
Computer Science and Engineering
MITS, Gwalior
Phagwara, Punjab
aashishsoni3874@gmail.com
Shreya Rakshit
Dept. of Mining Engineering
IIT (ISM) Dhanbad
Dhanbad, Jharkhand
shreyarakshit2017@gmail.com
Radhakanta Koner*
Dept. of Mining Engineering
IIT (ISM) Dhanbad
Dhanbad, Jharkhand
rkoner@iitism.ac.in
Abstract—In the age of this industry 4.0, application of
different automation techniques and different state of
technology for seamless management of different unit
operations in mining is very important. If we undergo a drone
survey in a mine, even if in a very big mine, it does not require
more than a few hours. So, within those few hours, we are
going to get a sufficient number of images with that we can
automatically fetch out cars/machines and thereby we can
introduce an audit system in the mine frequently and this can
be a very accurate system to rely on. Here in this particular
investigation, we have tried to start a work that basically aims
to detect “manmade object”, particularly like car/machine
automatically. To detect the object with high precision, deep
learning approach has been used which includes Yolo, Faster
R-CNN, Mask R-CNN, Cascade R-CNN. The comparative
study has shown that quality and quantity of data has greatly
influences the performance of the model.
Keywords—Object Detection, Mining Machinery, Drone
Survey, Remote Sensing
I. INTRODUCTION
The open-pit mining is contributing the major production in
mining and mineral industry. The open-pit mining required
the deployment of large earth moving machinery, vehicle
and other large machineries for rapid access and conveyance
of the coal and mineral and effectively achieving higher
productivity. The effective management of these assets and
keeping track of these utilities is very much required for the
mine management. A few methodologies are available to
keeping records of these items. The available methodologies
are mostly manual and laborious. Another disadvantage is
these are not readily available to the management desk of
planners and engineers who are seating in the top hierarchy.
The principle of Industry 4.0 laid down the basic changes in
the conventional practice of mining engineering that
requires fast and easy access of all the resources data to be
made available on real-time basis seamlessly to all stake-
holders. Remote sensing technology is very useful technique
to do that effectively in some part of the open-pit mining
operation. In a large mechanized open-pit mines there are a
good number of machines and vehicle that are operating on
continuous basis and needs effective supervision of the
management. The auditing and continuous monitoring of
these machines are very much required.
The object detection algorithms have worked on different
application like worked on the different application such as
urban planning[1], road planning[2] and so on. These
applications mostly focused on the development part of the
object detection algorithms. In the mining industry
conventional mechanism is very effective, although, several
add-ons in terms of technology could benefit the miner
safety and optimize the production. For instance,
Nascimento [3] has shown the expensive and tedious
problem faced by yhr mining industry. Angelo et al.[4] has
demonstrated the used of deep learning based object
detection algorithms in the inspection of mining industry. In
this experimentation, the authors used Yolo technique to
detect the fault in the conveyor belt. In addition to detection,
a comparative study suggested that GPU-based system is
providing the almost real-time detection.
One of the major application of Deep learning and remote
sensing technology is surveillance. However, such
application has been helpful for the defense sector.
Nowadays the specialized sensor such as synthetic aperture
radar (SAR)
has been helpful for detecting mines. In this study [5],
author has introduced the UAV platform equipped with
SAR sensor the capture the several location to locate the
buried mines. This approach used the advance
photogrammetric technique to locate and orient the mine but
it lacks the used of advance machine learning technique to
detect it efficiently.
Herein a comparative study has been performed using
digital image processing technique to detect the mine
machinery in
the open-pit mines from UAV (Unmanned Aerial Vehicle)
images. The detail methodology is explained in the flow
chart in [1].
The main objective of this project is to deploy an object
detection approach to a highly realistic scenario such as
mining region, where detecting a mining machinery could
have drastic significance. For this project, we have basically
used publicly available dataset of several mine regions along
with Drone image collected to nearby mine region. For
preparing training dataset, collected images had to undergo
annotation process as per the requirement of the associated
algorithms. To detect the object, several object detection has
been utilized considering the current scenario. For this
purpose, YOLO algorithm, Faster R-CNN, Mask R-CNN
and Cascade Mask R-CNN, with polygon annotation was
This work is sponsored by Coal India Limited (CIL), R&D Board, Project
No: CIL/R&D/01/77/2022 dated 19 January, 2022
.
* Corresponding Author
2022 OPJU International Technology Conference on Emerging Technologies for Sustainable Development (OTCON) | 978-1-6654-9294-2/23/$31.00 ©2023 IEEE | DOI: 10.1109/OTCON56053.2023.10114015
978-1-6654-9294-2/23/$31.00 ©2023 IEEE
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
used. The annotation was performed with utmost accuracy
for getting a better result.
Fig. 1. Flow of the study.
II. OBJECT
DETECTION
ALGORITHMS
A. You only look once (YoLo)
In 2015, Redmon et al.[6] have presented a sophisticated
object detection framework known as YOLO. The
framework has presented the advantageous over the
classification-based algorithms. This approach worked as a
regression, and separates the spatially box and and class
probabilities. It uses the single pipe line of neural network to
associate the class probabilities with the respective class. In
detail, CNN-based pipeline uses the input image and
partition into NxN grid and anticipates the class
probabilities of N bounding boxes. Yolo is Superfast
approach and have potential to work on the real-world
challenges, however, the architecture can only detect the
multiple objects from an image[7].
B. Faster R-CNN
The R-CNN and Fast R-CNN are the popular object
detection framework, however, the architecture used the
selective search approach to locate the object position in the
input image. The approach work effectively, however, due
to the searching approach is costlier approach without
learning. Ren et al. proposed the learning based approach
known as region proposals network (RPN) to locate the
position. Faster R-CNN constitutes of two separate CNN
network: regional proposal network to generate the area of
interest (or region of interest) and integrating network to
detect the objects. The author has proposed the idea of
anchor boxes to identify the area of suggestion and predict it
using CNN based architecture. The Fig 2 shows the
graphical representation of the anchor configured around the
image. In the standard configuration, the 3 size of AOI is
defined with 3 aspect ratio, as a result 9 AOI. The AOI’s is
further processed by RPN to classifies the object from
background. Therefore, Faster RCNN is considered as the
accurate object detection approach.
Fig. 2. AOI’s creation of Image [8]
C. Mask R-CNN
Mask R-CNN is an extended version of the Faster R-CNN
model. Similar to the Faster R-CNN, the framework uses the
CNN to generate. While object detection is all about
creating a bounding box surrounding the object, instance
segmentation indicates the actual boundary of the object. So,
it helps us not only to localize and classify the objects, but
also to know its orientation. Therefore, the Mask R-CNN
identifies the object using segmentation framework, along
with threshold limit IoU, that will result a segmentation
mask. Overall, the Mask R-CNN is similar to the Faster R-
CNN with added advantage of differentiating the foreground
to the background.
D. Cascade R-CNN
Cascade Mask R-CNN performs both object detection and
instance segmentation[9], [10]. It is a multi-stage extension
of the Mask R-CNN, where the cascade of Mask R-CNN
stages is trained sequentially, using the output of one stage
to train the next. This architecture was developed to address
problems with degrading performance with increased IoU
thresholds. It also possesses the benefits of instance
segmentation and henceforth polygon annotation is
mandatory.
III. D
ATASET
For this project, two different sources of image information
have been introduced 1. Drone image [11] and google earth
image. Firstly, Drone image is collected in the collaboration
with geotechnical scientist. In the process, the drone DJI
Phantom 4 Pro UAV has been. The image acquisition is
constituting of 75% horizontal and 65% vertical
overlapping. The differential Global positioning system
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
(DGPS), which helps to orient the image in proper N-S
alignment.
In the Deep learning approach, Quantity and quality of
image always place a vital role in the training and testing.
Therefore, this study used the Google Earth Image to
enhance the quantity of images. During collection, the study
focused on several mining region such as Cadia mine,
Boddington gold mine, Agnew gold mine, Cannington
mine, Coolgardie gold mine, Argyle diamond mine, Daunia
mine from Australia; Gloria mine, Grootegeluk coal mine
from South Africa; some mines from Chile; Jharia coalfield
from India. For the annotation purposes, the annotation tool
has been used to prepare the bounding box for the target
region using visual interpretation.
IV. EXPERIMENTAL SETUP
A. Evaluation metrics
Object detection is a complex approach, which requires
sophisticated metric to evaluate the framework accuracy.
Several literatures have recommended the use of Mean
average precision (mAP) to evaluate the object detection
framework [12]. Generally, an object detection framework
work on the two simultaneous process such segmentation
and classification, and mAP has been the best fit for this
scenario. In this context, algorithms used to categorize the
background from the foreground and mean precision
calculates area under precision-recall (mentioned [1] and
[2]). In general term, precision-recall provides the degree of
confidence of predicting the correct object. In addition, the
researcher also integrated the degree of confidence based on
inter section over union (IoU) to check model performance
in the different threshold. For instance, mAP@0.5 represent
the average precision for the threshold on or above 50%.
Precision =



(1)
Recall =



(2)
Where, tp=true-positive, fp =false-positive, fn=false
negative
B. Parameter configuration
This section elaborated the deep learning parameter for the
purpose of object detection in mining region. For
comparison, the model constitutes with basic parameter has
been set to same value such as the learning rate is 0.01,
momentum of 0.7, Relu activation function for non-
linearity, with maximum epoch of 1500 and allow the
patient level to reduce the unnecessary iteration. To adjust
the weight during the training the back-propagation
algorithms such as Stochastic Gradient Descent (SGD) has
been used[13]. While performing the experiment and
validity of the model, the dataset has been tested on two
different testing ratios. The details of the first and second
experiments are given in table I.
TABLE I. DETAILED CONFIGURATION OF MODEL PARAMETERS (WHERE
‘M’ IS MASK R-CNN, ‘C’ IS CSACADE R-CNN, ‘FR’ IS FASTER-RCNN,
TR:V:TE IS TRAINING:VALIDATION:TESTING, BS IS BATCH SIZE)
Model First Training Second Training
Tr:V:Te BS Iterat
ion
Tr:V:Te BS Iterati
on
Yolo 70:20:10 16 1500 60:20:20 16 1500
FR 70:20:10 64 1500 60:20:20 64 1500
M 70:20:10 128 1500 60:20:20 128 1500
C 70:20:10 128 1500 60:20:20 128 1500
V. RESULT AND DISCUSSION
Table II illustrates the performance of object detection
algorithms using different threshold values based on IOU.
For comparison, two separated experiments have been
performed to using two test ratios (shown in table I). It has
been observed that accuracy of sufficient enough to detect
the mining machinery, however, it greatly depends upon the
threshold limit set for the IoU. For instance, In the first
experiment, the Yolo has achieved the highest accuracy of
0.993 with threshold of 0.5, And the accuracy range
decreased 0.812 as the IoU limit increased to 0.95. Similar
changes have been observed in the other model as well.
Therefore, based on the threshold of 0.5 the accuracy is
arranged in the following order: Yolo > Masked R-CNN >
Cascade R-CNN > Faster R-CNN.
Deep learning framework are greatly influence by the
datasets in term of Quality and Quantity. The Quality of the
dataset is quite similar, the quantity of the data for training
has been modified to test adaptability of the model on lower
volume of the data. In this context, object detection has been
tested on the same setting such First experiment. The Yolo
frameworks has shown the weaker side such 0.362 and
0.241
TABLE II. ASSESMENT OF OBJECT DETECTION METHOD (WHERE ‘M’ IS
MASK R-CNN, ‘C’ IS CSACADE R-CNN, ‘B’ IS BOUNDING BOX AND ‘S’ IS
SEGMENTATION )
Model
First Experiment Second Experiment
mAP@
0.5
mAP@
0.75
mAP
@0.5
:0.95
mAP@
0.5
mAP@
0.75
mAP
@0.5:
0.95
Yolo 0.993 NA 0.812 0.362 NA 0.241
Faster
RCNN 0.986 0.974 0.879 0.611 0.135 0.265
M B 0.991 0.925 0.774 0.735 0.213 0.331
S 0.991 0.808 0.665 0.735 0.082 0.287
C B 0.989 0.949 0.807 0.632 0.134 0.284
S 0.989 0.822 0.666 0.669 0.126 0.243
for threshold 0.5 and 0.95, simultaneously. The performance
of Yolo suggested that it need rich amount of the data get
the better performance. On the other hand, the other model
has performed relatively better than Yolo. Masked R-CNN
has shown the highest accuracy of the 0.735 within the
threshold limit of 0.5. And the accuracies are in the order (in
0.5 limit): Mask R-CNN > Cascade R-CNN > Faster R-
CNN >Yolo. The similar degradation in the performance
has been observed like first experiment.
Another reason of the degrading performance of the second
model with respect to the first model can be calculating
precision using the training dataset itself in the first set. To
analyze its relevance, we exercised the YOLO training
algorithm using both the dataset again with a larger number
of training images. The same training datasets were
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
augmented (vertical flip, horizontal flip and 90 deg.
rotation) to increase the number of training images.
TABLE III. A
SSESMENT OF OBJECT
D
ETECTION
M
ETHOD USING
AUGMENTED DATASET
Mo
del
First Experiment Second Experiment
mAP
@0.5
mAP@
0.75
mAP@0.
5:0.95
mAP
@0.5
mAP@
0.75
mAP@0.
5:0.95
Yol
o 0.995 NA 0.839 0.815 NA 0.37
In table III, we have shown the performances of YOLO
models using augmented training dataset. Assessment of the
falsely detected objects are shown in the YOLO model
using augmented training dataset column of table IV. In this
case, the precision of the second model has highly
improved. We can draw a conclusion that increasing the
number of training images will deliver a great result.
The object detection algorithms mostly focus on the
annotation or bounding box. Therefore, the accuracy of the
bounding box is also concerned in the performance
estimation. Due to this reason, mAP of bounding box is also
evaluated in the Mask and Cascade R-CNN, where
relatively higher accuracy has been achieved in the 0.5
threshold.
Cascade R-CNN is mainly used for addressing problems of
the degrading percentage with increased IoU threshold. It is
clearly noticeable that the difference between mAP@0.5
and mAP@0.75 has decreased in Cascade Mask R-CNN in
comparison with the Mask R-CNN.
The graphical comparison has also shown in the Figure 3. In
the figure, three tile is considered to make the fair
comparison. In all the cases of Yolo prediction, a red
rectangular box is visible and faster R-CNN is missing most
of mining machinery. The missing detection has been
observed through the example of Faster R-CNN, which
shows the weaker side of the Faster R-CNN.
The Masked R-CNN and cascade R-CNN has almost similar
result. Both algorithms performed relatively better in the
first experiment with simultaneous segmentation and
detection (shown in table II). In the Experiment I, some
misclassification has been observed in the tile 1. Tile 2 and
3 are showing similar result like Yolo, with relatively higher
precision in term of bounding box. In the experiment II,
Masked R-CNN and Cascade R-CNN has precisely detected
the object, which is not recovered by Yolo and Faster-
RCNN in the tile 1. On the contrary, the misclassification
has been observed by Masked and Cascade R-CNN in the
tile 3.
Fig. 3. Visual Comparison between different framework results.
The table IV has shown the detailed analysis of object
uncovered while performing the algorithms. In total 13
objected is targeted to evaluated the performance and
characterized into missed and false detection along with
overlap error (or localization error) for bounding box. In the
experiment I, the false detection rate for all the cases are
similar i.e., 23.08 expect Cascade R-CNN produces the
drastic decrement of +15%. The percentage of missing
object in all the algorithms are identical with 15.38%.
TABLE IV. A
SSESMENT OF FALSE LY DETECTED OBJECT
,
WHERE
S
ST
’,
R
,
ATD
,
M
’,
HS
ARE THE SHADOW
,
STONE
,
RODS
,
A
UGMENTED
T
RAINING
D
ATASET
,
MORE THAN ONE BOUNDING BOXES FOR ONE OBJECT AND
H
EIGHT
S
HORTENED
Model
Yolo Yolo Using ATD Faster RCNN Mask RCNN Cascade Mask RCNN
First
Experimen
t
Second
Experimen
t
First
Experimen
t
Second
Experimen
t
First
Experimen
t
Second
Experimen
t
First
Experimen
t
Second
Experimen
t
First
Experimen
t
Second
Experimen
t
Object
present 13 13 13 13 13 13 13 13 13 13
Confidenc
e 0.7 0.6 0.7 0.6 0.7 0.6 0.7 0.6 0.7 0.6
false
detection
1 (st) + 2
(r) 2 (r) 2 (r) 2 (r) + 1
(st)
2 (r) +1
(st) 1 (r)
1 (st) + 2
(r)
2 (st) + 2
(r) + 1 (s)
2 (st) + 2
(r) + 1 (s)
1 (st) + 1
(r)
false
detection
3/13*100
= 23.08
2/13*100
= 15.38
2/13*100
= 15.38
3/13*100
= 23.08
3/13*100
= 23.08
1/13*100
= 7.69
3/13*100
= 23.08
5/13*100
= 38.46
5/13*100
= 38.46
2/13*100
= 15.38
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
(%)
Missed
object 2 4 2 3 2 4 2 2 2 4
Percentag
e of missed
object
2/13*100
= 15.38
4/13*100
= 30.76
2/13*100
= 15.38
3/13*100
= 23.08
2/13*100
= 15.38
4/13*100
= 30.76
2/13*100
= 15.38
2/13*100
= 15.38
2/13*100
= 15.38
4/13*100
= 30.76
localizatio
n error 0 0 0 0 0 0 0 1 (m) 0 1 (hs) + 1
(m)
localizatio
n error
(%)
0/13*100
= 0
0/13*100
= 0
0/13*100
= 0
0/13*100
= 0
0/13*100
= 0
0/13*100
= 0
0/13*100
= 0
1/13*100
= 7.69
0/13*100
= 0
2/13*100
= 15.38
In the experiment II, false detection rate is relatively low
compared to experiment for all the cases. However, the
missing object is almost double in the case of Yolo, Faster
R-CNN, Cascade-RCNN. The Masked R-CNN has shown
the similar performance of 15.3%. Overall, the object
detection model performed significantly better for the
detecting mining machinery.
Table V shows the required training time of the different
models used in this study. So, the sequence of the training
time is Faster R-CNN > YOLO > Cascade Mask R-CNN >
Mask R-CNN.
TABLE V. REQUIRED TRAINING TIME
Model First Experiment Second Experiment
Yolo
45 min. 23 sec.
28 min. 28 sec.
Faster
RCNN 1 hr. 19 min. 57 sec 1 hr. 21 min. 05 sec
Mask
RCNN 17 min. 26 sec. 13 min. 20 sec.
Cascade
Mask
RCNN
18 min. 36 sec. 13 min. 53 sec.
VI. LIMITATIONS
In this experiment, some false detections and missed objects
have been encountered. Also decrease in the size of training
dataset results in less mAP value. Greater number of
iterations in the training procedure may lead to greater mAP
and less error; but would take much time and computation
power. So, we need to optimize the training time and the
accuracy intended. Some adjustment could also be made to
the confidence level as per the requirement. It should be
decreased appropriately to get less percentage of missed
objects, if less number of images are being used. But the
false detection may increase in this case just like the Mask
RCNN method with respect to the first and second
experiment.
VII. CONCLUSION
Mining machinery plays an important role for smooth
conduction of mining operation. Performing the object
detection from high resolution images is an active research
topic. However, the significant variation in the machinery
size and shape makes it very challenging in remote sensing
imagery. In this paper, we have performed comparative
analysis to evaluated the latest object detection technique on
mining site. This study has included the Yolo, Faster R-
CNN, Masked R-CNN, Cascade R-CNN. It has been
observed that algorithms are greatly influenced by the
quality and quantity of the datasets. Therefore, algorithms
have performed relatively better in the experiment I. This
study has also tried to decrease the quantity of training size,
and noted that Yolo architecture is prone to training size.
For the impeccable annotation prediction, the threshold limit
has been varied i.e., 50%, 70%, 95%, and 50% threshold has
produced the highest accuracy for all the algorithms.
The application of remote sensing image has set the baseline
for the extraction for MMD. In the future, we would like to
test the proposed method for more datasets to resolve the
noise, to achieve better performance. Furthermore, it would
be more interesting to test refinement of target feature at
edges, while adding the latest object detection technique.
VIII. ACKNOWLEDGEMENT
We thank our mentor Dr. Radhakanta Koner from Indian
Institute of Technology who provided insight and expertise
that greatly assisted the research, although they may not
agree with all of the interpretations/conclusions of this
paper.
We thank Dr. Ashish Soni for assistance with
Conceptualization and methodology, and Shreya Rakshit for
data creation, implementation and overall analysis.
IX. REFERENCES
[1] M. Knura, F. Kluger, M. Zahtila, J. Schiewe, B.
Rosenhahn, and D. Burghardt, “Using object
detection on social media images for urban bicycle
infrastructure planning: a case study of Dresden,”
ISPRS Int. J. Geo-Information, vol. 10, no. 11, p.
733, 2021.
[2] D. Zhu, G. Xu, J. Zhou, E. Di, and M. Li, “Object
detection in complex road scenarios: improved
YOLOv4-tiny algorithm,” in 2021 2nd Information
Communication Technologies Conference (ICTC),
2021, pp. 75–80.
[3] R. Nascimento, R. Carvalho, S. Delabrida, A. G.
Bianchi, R. A. R. Oliveira, and L. G. U. Garcia, “An
integrated inspection system for belt conveyor
rollers,” in Proc. 19th Int. Conf. Enterprise Inf.
Sys.(ICEIS), 2017, vol. 2, pp. 190–200.
[4] T. D. Angelo, M. Mendes, B. Keller, R. Ferreira,
and S. Delabrida, “Deep Learning-Based Object
Detection for Digital Inspection in the Mining
Industry,” pp. 633–640, 2019, doi:
10.1109/ICMLA.2019.00116.
[5] E. Schreiber, A. Heinzel, M. Peichl, M. Engel, and
W. Wiesbeck, “Advanced buried object detection by
multichannel, UAV/drone carried synthetic aperture
radar,” in 2019 13th European Conference on
Antennas and Propagation (EuCAP), 2019, pp. 1–5.
[6] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi,
“You only look once: Unified, real-time object
detection,” in Proceedings of the IEEE conference
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
on computer vision and pattern recognition, 2016,
pp. 779–788.
[7] A. Dhillon and G. K. Verma, “Convolutional neural
network: a review of models, methodologies and
applications to object detection,” Prog. Artif. Intell.,
vol. 9, no. 2, pp. 85–112, 2020.
[8] C. Arya et al., “Object detection using deep
learning: a review,” in Journal of Physics:
Conference Series, 2021, vol. 1854, no. 1, p. 12012.
[9] Z. Cai and N. Vasconcelos, “Cascade r-cnn: Delving
into high quality object detection,” in Proceedings
of the IEEE conference on computer vision and
pattern recognition, 2018, pp. 6154–6162.
[10] Z. Cai and N. Vasconcelos, “Cascade R-CNN: high
quality object detection and instance segmentation,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no.
5, pp. 1483–1498, 2019.
[11] S. Layek, V. G. K. Villuri, R. Koner, and K. Chand,
“Rainfall \& Seismological Dump Slope Stability
Analysis on Active Mine Waste Dump Slope with
UAV,” Adv. Civ. Eng., vol. 2022, 2022.
[12] C. Chen, M.-Y. Liu, O. Tuzel, and J. Xiao, “R-CNN
for small object detection,” in Asian conference on
computer vision, 2016, pp. 214–230.
[13] R. G. J. Wijnhoven and P. H. N. de With, “Fast
training of object detection using stochastic gradient
descent,” in 2010 20th International conference on
pattern recognition, 2010, pp. 424–427.
Authorized licensed use limited to: Indian Institute of Technology (ISM) Dhanbad. Downloaded on May 09,2023 at 05:36:05 UTC from IEEE Xplore. Restrictions apply.
... For example, in the unstable dump slope area, UAV deployment captures high-resolution aerial image data at different altitudes without risking human life [17]. Many studies show the capability of UAV application and its uses, such as landslides, mine dumps, hill slopes, and vehicle detection [12,13,[18][19][20][21][22]. This technology is rapidly growing in geotechnical investigations and survey [23][24][25][26], due to its low operating cost, easy handling, and high mobility [27,28]. ...
Article
In large open-pit coal mine projects, internal dump slope failure is a serious issue for industry. Slope failure affects haul roads, mine machinery, and miners' lives and hampers coal production operations. Dump slope failure events will not occur suddenly, and progressive displacement will occur in the slope before failure. Therefore, dump slope monitoring is essential for the safety of humans and machinery. Generally, the mine slope displacement is measured using a conventional survey approach. However, conventional surveys cannot measure minor changes in a large dump area. Therefore, massive dump displacement monitoring is a tedious task for surveyors. This study used the unmanned aerial vehicle (UAV) close-range photogrammetry survey to introduce an intelligent and reliable approach for dump change detection and displacement monitoring. A UAV close-range photogrammetry survey was conducted to collect RGB images, and dump surface point cloud data (PCD) were generated using the structure-from-motion (SfM) computing algorithm. The ground control points (GCPs) were also measured using a coordinate measurement survey for geo-reference points. For dump surface PCD and 3D reconstruction, the Pix-4D Mapper image processing tool was used. PCDs were generated for two months and used for dump slope surface change and displacement monitoring using the cloud-to-cloud (C2C) computing algorithm. The PCD was compared with the reference PCD, and the results of this investigation indicated minor and major changes on the dump slope and top surfaces. This approach is inexpensive and reliable for large dump slope change detection and displacement monitoring.
Article
Full-text available
Mine waste dump material has no economic value to the industry. Therefore, the mine waste is dumped, forming slopes. Mine waste dump slopes obtain 30% to 50% of the mining area. To reduce the land occupancy of these slopes, they are created with high altitudes. Hence, they are susceptible to failure. Slope stability analysis is a major aspect of geotechnical engineering. Slope stability analyses are mostly done with assumptions on the geometry. This is avoided in this paper with the usage of UAVs. The 3D model is created from UAV imagery of a coal mine in Raniganj coalfield, India. The model is fine-tuned with the DGPS survey. Geotechnical data were collected and tested in the laboratory for various numerical analyses. An active mine waste dump slope is analyzed for slope failure. Earthquakes and rainfall cannot be controlled, and their effects on the stability of the mine waste dump slope were examined. The study extracted various factor of safety (FOS) analyses on static, seismic, and rainfall conditions. The seismic condition simulates a condition of the slope to be failed with low (0.948) FOS. However, rainfall condition predicts the slope to be more stable. The deformation pattern and magnitude of the slope failure are also discussed.
Article
Full-text available
Accomplished and accurate object detection has been an important topic in the progress of computer vision systems. With the arrival of deep learning techniques, the purity for object detection has increased drastically. The paper aims to inclusive state of the art technique for the object detection with the goal of obtain high accuracy with a real time performance. A major challenge in many of the object detection system is the docility on other computer vision techniques for helping the deep learning-based perspective, which leads to slow and minimal performance. In this paper, I use a completely deep learning-based approach to solve the problems of object detection in an end to end fashion using wireless sensor network. Even if, many techniques have been developed, but I have discussed some famous and basic idea of object detection using deep learning. In the end i have also given their general applications and results.
Article
Full-text available
With cities reinforcing greener ways of urban mobility, encouraging urban cycling helps to reduce the number of motorized vehicles on the streets. However, that also leads to a significant increase in the number of bicycles in urban areas, making the question of planning the cycling infrastructure an important topic. In this paper, we introduce a new method for analyzing the demand for bicycle parking facilities in urban areas based on object detection of social media images. We use a subset of the YFCC100m dataset, a collection of posts from the social media platform Flickr, and utilize a state-of-the-art object detection algorithm to detect and classify moving and parked bicycles in the city of Dresden, Germany. We were able to retrieve the vast majority of bicycles while generating few false positives and classify them as either moving or stationary. We then conducted a case study in which we compare areas with a high density of parked bicycles with the number of currently available parking spots in the same areas and identify potential locations where new bicycle parking facilities can be introduced. With the results of the case study, we show that our approach is a useful additional data source for urban bicycle infrastructure planning because it provides information that is otherwise hard to obtain.
Article
Deep learning has developed as an effective machine learning method that takes in numerous layers of features or representation of the data and provides state-of-the-art results. The application of deep learning has shown impressive performance in various application areas, particularly in image classification, segmentation and object detection. Recent advances of deep learning techniques bring encouraging performance to fine-grained image classification which aims to distinguish subordinate-level categories. This task is extremely challenging due to high intra-class and low inter-class variance. In this paper, we provide a detailed review of various deep architectures and model highlighting characteristics of particular model. Firstly, we described the functioning of CNN architectures and its components followed by detailed description of various CNN models starting with classical LeNet model to AlexNet, ZFNet, GoogleNet, VGGNet, ResNet, ResNeXt, SENet, DenseNet, Xception, PNAS/ENAS. We mainly focus on the application of deep learning architectures to three major applications, namely (i) wild animal detection, (ii) small arm detection and (iii) human being detection. A detailed review summary including the systems, database, application and accuracy claimed is also provided for each model to serve as guidelines for future work in the above application areas.
Article
In object detection, the intersection over union (IoU) threshold is frequently used to define positives/negatives. The threshold used to train a detector defines its quality . While the commonly used threshold of 0.5 leads to noisy (low-quality) detections, detection performance frequently degrades for larger thresholds. This paradox of high-quality detection has two causes: 1) overfitting, due to vanishing positive samples for large thresholds, and 2) inference-time quality mismatch between detector and test hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, composed of a sequence of detectors trained with increasing IoU thresholds, is proposed to address these problems. The detectors are trained sequentially, using the output of a detector as training set for the next. This resampling progressively improves hypotheses quality, guaranteeing a positive training set of equivalent size for all detectors and minimizing overfitting. The same cascade is applied at inference, to eliminate quality mismatches between hypotheses and detectors. An implementation of the Cascade R-CNN without bells or whistles achieves state-of-the-art performance on the COCO dataset, and significantly improves high-quality detection on generic and specific object datasets, including VOC, KITTI, CityPerson, and WiderFace. Finally, the Cascade R-CNN is generalized to instance segmentation, with nontrivial improvements over the Mask R-CNN.
Conference Paper
In the mining industry, belt conveyors are essential for transporting large quantities of materials efficiently and inexpensively. The rollers are one of the main components of a belt conveyor. Their maintenance and operation are critical for material transportation during the mining activities. Despite that, its inspection has been done in the same way for almost 20 years, with little technological innovation. The inspection of the belt conveyor rollers automatically via object detection and thermal imaging can improve this process. Therefore, we propose a new system capable of running the detection of defective rollers in real time and with better precision and recall metrics than those of previous works. We based the object detection models on the YOLOv2 deep learning architecture. Comparing our results with previous work, we observe reductions on false discovery and false negative rates of 52.46% and 61.03%, respectively. We also evaluated three computing infrastructures, and the GPU-based one was the only that enabled real-time performance.
Conference Paper
The great innovations of Synthetic Aperture Radar (SAR) technology during the past years stimulated new applications in several areas. While in the past SAR was primarily operated for airborne and spaceborne applications, novel operations for quite low altitude like surveillance of cities, local agricultural applications, or even buried object detection, are of new interest. For such operations the well-known and established SAR system concepts should apply similarly, while the technology has to be transferred to the state of the art and new platforms like unmanned aerial vehicles (UAVs) or drones. The paper addresses the special concept for a SAR to detect buried mines. It combines a novel DLR approach based on multistatic observation with the capability to create nearly arbitrary azimuth sampling trajectories. By providing very high resolution it is possible to even identify man-made objects like landmines in the SAR image by their spatial radar-cross section (RCS) distribution. These capabilities allow advanced detection capabilities and satisfy the ultimate demand for buried object identification, both being great improvement in landmine detection and related activities. Beside this concept the paper addresses an idea on robust high-precision positioning, which is indispensable for a successful SAR system. It is based on a photogrammetric method providing the trajectory, the orientation of the UAV and the three-dimensional (3D) ground surface as a side effect as well. Measurement results are shown confirming the feasibility of the proposed approach.
Article
In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference-time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R-CNN is shown to surpass all single-model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R-CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code will be made available at https://github.com/zhaoweicai/cascade-rcnn.