No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Global Road Damage Detection: State-of-the-art Solutions
Abstract
This paper summarizes the Global Road Damage Detection Challenge (GRDDC), a Big Data Cup organized as a part of the IEEE International Conference on Big Data’2020. The Big Data Cup challenges involve a released dataset and a well-defined problem with clear evaluation metrics. The challenges run on a data competition platform that maintains a leaderboard for the participants. In the presented case, the data constitute 26336 road images collected from India, Japan, and the Czech Republic to propose methods for automatically detecting road damages in these countries. In total, 121 teams from several countries registered for this competition. The submitted solutions were evaluated using two datasets test1 and test2, comprising 2,631 and 2,664 images. This paper encapsulates the top 12 solutions proposed by these teams. The best performing model utilizes YOLO-based ensemble learning to yield an F1 score of 0.67 on test1 and 0.66 on test2. The paper concludes with a review of the facets that worked well for the presented challenge and those that could be improved in future challenges.
... Using this dataset, the first road damage detection challenge was undertaken in the same year with the participation of the big data conference. Experimental investigations showed that the models developed based on this dataset do not have a comprehensive performance under realistic conditions [13]. ...
... To solve these problems, the RDD2020 dataset [14] was subsequently presented. In the big data conference of the year 2020, RDD2020 dataset was used as reference for the global road damage detection challenge (GRDDC) [13]. ...
... In 2020, 121 teams participated in the GRDDC 2020 [13]. The goal of this challenge was to present different road damage detection methods based on deep learning. ...
In this paper, based on the data augmentation techniques of bounding box augmentation and the road damage generative adversarial network based augmentation, a robust road damage detection method has been presented. To this end, first, Iran road damage dataset has been collected by means of a dashboard‐installed mobile phone. After processing these images by the blind referenceless image spatial quality evaluator technique, the substandard and inferior data have been automatically eliminated. In the second step, based on the YOLOv5 with several different baseline models, an algorithm has been developed for detecting the road surface damages. In the third step, by using the traditional as well as the bounding box augmentation and road damage generative adversarial network based augmentation techniques, the precision and the robustness of road damage detector under different environmental and field conditions have been improved. Finally, through the ensemble of the best models, the final detector accuracy has been enhanced. The findings of this article indicate that by using the proposed approach, the values of mAP and F1‐score are improved by 13.79% and 7.58%, respectively. The dataset and parts of the code are available at: https://github.com/IranRoadDamageDataset/IRRDD.
... Water can also collect in potholes and cause erosion, contaminating soil and water. There are numerous classifications for damaged roads [2][3][4][5][6] ...
... In Table 3-1, the details of each class are mentioned. were grouped together in the dataset as a single category [3][4][5][6] . The Road Damage Dataset, RDD2022, has 47,420 images of roads in six different countries (Japan, India, the Czech Republic, Norway, the United States, and China) as stated by Arya et al. [6] in their statement. ...
... In point of fact, rutting, bumping, potholes, and separation are all different classifications of road deterioration; nevertheless, distinguishing between these four categories has proven difficult when relying just on visual portrayals of the condition of the road. Consequently, they were categorised collectively and assigned the label D40 [3][4][5][6] . It is noteworthy to mention that D00 and D10 are classified as linear cracks, despite their distinct characteristics of being a longitudinal and lateral crack, respectively. ...
The present research examines the development of a road damage detection model and its potential implications on the economy and society. The timely and accurate identification of deteriorated road conditions has the potential to significantly reduce vehicular accidents, improve traffic flow, and ultimately save lives. The adoption of this technology can also yield substantial financial benefits for governmental entities and road management associations by enhancing the prioritization and scheduling of maintenance and repair tasks. Notably, deep learning techniques, particularly the YOLOv7 model, have contributed to notable improvements in detection accuracy, reduced computational complexity, and overall effectiveness. This study involved conducting several experiments comparing the performance of YOLOv5 and YOLOv7 models. The results highlight the superior performance of the YOLOv7 architecture in accurately identifying damaged areas and predicting object categories.
The utilisation of the YOLOv7 model in deep learning and computer vision has led to significant advancements in the precision and efficiency of road damage detection models. The findings from a series of experiments comparing YOLOv5 and YOLOv7 models demonstrate the enhanced effectiveness of the YOLOv7 architecture in accurately detecting damaged regions and predicting object categories. Various methodologies, including hyperparameter optimization and image augmentation techniques, were employed to improve precision. The initial experiment incorporated rotation, hue saturation value configuration, image scaling, and flipping techniques resulting in an average accuracy of 79.75% across all test image categories using YOLOv7. However, Experiment 2 revealed limitations in the implementation of the Gaussian Blur method on YOLOv7, leading to a bias toward blurred images and a consequent compromise in overall precision, resulting in an accuracy of 19.75%. In Experiment 3, the YOLOv5 algorithm was used with a refined dataset, yielding an average accuracy of 55.75% across all categories, which was lower than the accuracy achieved in the first experiment using YOLOv7.
This study suggests employing the YOLOv7-trained model as a means of detecting road damage, in conjunction with various image augmentation techniques. According to the findings, the model attained an F1 score of 75%. The adoption of this technology in practical applications is justified by its exceptional performance, which provides significant insights into the state of road conditions and enables timely maintenance and repair interventions. It is also recommended to utilise the designated software in this study to facilitate the convenient implementation of the proposed model by end-users. Further research and improvement of the model may reveal its complete capabilities in augmenting road infrastructure administration and guaranteeing more secure and effective transportation systems.
... This RDD2020 data was made publicly available and formed the basis for organizing the second challenge of the road damage detection challenge series, named the Global Road Damage Detection Challenge, GRDDC'2020 [20]. It invited the participants to propose models capable of efficiently detecting road damage in India, Japan, and the Czech Republic. ...
... For one-stage, the team adopted YOLOv5 [46] and YOLOv7 [47]. YOLOv5 [48] is choosen because it showed high performance in solutions proposed through GRDDC'2020 [20] targeting the same domain of road damage detection. The YOLOv7 [47] is selected because it has several trainable bag-of-freebies, e.g., model re-parametrization, and auxiliary head, and is a new state-of-the-art real-time object detector. ...
... Likewise, the team adopted Cascade RCNN [39] models for two-stage detectors after analyzing the top-ranking solutions [21]- [23] for GRDDC'2020 [20]. From the anlaysis, the team inferred that YOLO series models perfer to have a higher recall rate, while Cascade RCNN series models prefer to have a higher precision rate and works better for small damage. ...
This paper summarizes the Crowdsensing-based Road Damage Detection Challenge (CRDDC), a Big Data Cup organized as a part of the IEEE International Conference on Big Data’2022. The Big Data Cup challenges involve a released dataset and a well-defined problem with clear evaluation metrics. The challenges run on a data competition platform that maintains a real-time online evaluation system for the participants. In the presented case, the data constitute 47,420 road images collected from India, Japan, the Czech Republic, Norway, the United States, and China to propose methods for automatically detecting road damages in these countries. More than 70 teams from 19 countries registered for this competition. The submitted solutions were evaluated using five leaderboards based on performance for unseen test images from the aforementioned six countries. This paper encapsulates the top 11 solutions proposed by these teams. The best-performing model utilizes ensemble learning based on YOLO and Faster-RCNN series models to yield an F1 score of 76% for test data combined from all 6 countries. The paper concludes with a comparison of current and past challenges and provides direction for the future.
... This RDD2020 data was made publicly available and formed the basis for organizing the second challenge of the road damage detection challenge series, named the Global Road Damage Detection Challenge, GRDDC'2020 [19]. It invited the participants to propose models capable of efficiently detecting road damage in India, Japan, and the Czech Republic. ...
... For one-stage, the team adopted YOLOv5 [45] and YOLOv7 [46]. YOLOv5 [47] is choosen because it showed high performance in solutions proposed through GRDDC'2020 [19] targeting the same domain of road damage detection. The YOLOv7 [46] is selected because it has several trainable bag-of-freebies, e.g., model re-parametrization, and auxiliary head, and is a new state-of-the-art real-time object detector. ...
... Likewise, the team adopted Cascade RCNN [38] models for two-stage detectors after analyzing the top-ranking solutions [20]- [22] for GRDDC'2020 [19]. From the anlaysis, the team inferred that YOLO series models perfer to have a higher recall rate, while Cascade RCNN series models prefer to have a higher precision rate and works better for small damage. ...
This paper summarizes the Crowdsensing-based Road Damage Detection Challenge (CRDDC), a Big Data Cup organized as a part of the IEEE International Conference on Big Data'2022. The Big Data Cup challenges involve a released dataset and a well-defined problem with clear evaluation metrics. The challenges run on a data competition platform that maintains a real-time online evaluation system for the participants. In the presented case, the data constitute 47,420 road images collected from India, Japan, the Czech Republic, Norway, the United States, and China to propose methods for automatically detecting road damages in these countries. More than 60 teams from 19 countries registered for this competition. The submitted solutions were evaluated using five leaderboards based on performance for unseen test images from the aforementioned six countries. This paper encapsulates the top 11 solutions proposed by these teams. The best-performing model utilizes ensemble learning based on YOLO and Faster-RCNN series models to yield an F1 score of 76% for test data combined from all 6 countries. The paper concludes with a comparison of current and past challenges and provides direction for the future.
... Mandal et al [68] trained three deep learning models, YOLO, CenterNet, and EfficientDet models, using 21,041 pavement images included in a big dataset released by IEEE global road detection challenges [69]. This dataset has a collection of road images taken from Japan, the Czech Republic, and India. ...
... Recently, some popular datasets on pavement cracks have been published. For example, CRACK500 [70], EdmCrack600 [3], GAPs [22] and also IEEE on global road detection challenge [69]. A summary of these asphalt pavement crack datasets is included in Table 4. ...
This study was conducted to answer two research questions. How do the chosen CNN models perform on publicly available asphalt pavement crack datasets for classification and segmentation? And how can these CNN models be fine-tuned to improve their performance? For classification, a ResNet50 model with transfer learning was explored, using eight different epochs and two optimizers, a total of sixteen combinations, and the optimal combination of these two hyperparameters was identified. On the other hand, the segmentation task was accomplished using a modified-UNet model on two different datasets, and the results were presented and discussed.
The dataset for classification was derived from a publicly available dataset, EdmCrack600. It was given the name AugCrack132. It has three principal types of asphalt pavement cracks: alligator, longitudinal, and transverse. The training dataset consists of 132 ground truth images, and the testing dataset has 56 raw crack images. The optimal classification accuracy occurred at epoch 500 with the 'adaptive moment estimation' or “Adam” optimizer algorithm, while the least accuracy occurred at epoch 40 with 'stochastic gradient descent' or “SGD” optimizer algorithm. The classification accuracy on the overall dataset varied from 8% to 58%, the F1 score was from 1.463% to 59%, the precision ranged from less than 1% to 68%, and the recall varied from 8.9% to 59%.
The modified-UNet model was trained and tested on two published pavement crack datasets to segment asphalt pavement cracks. The first dataset included 470 images and corresponding masks obtained from the EdmCrack600 dataset through a preprocessing, and it was named EdmCrack470. Furthermore, 206 images from the CRACKTREE260 dataset were used to evaluate the modified-UNet model, as a result, the dataset has been named CRACTREE206 here. At epoch 30, the model achieved an IoU of 67%, precision 96%, recall 65%, and F1 78% score for the EdmCrack470 dataset. Moreover, the values of the evaluation metrics for CRACKTREE206 are: IoU 60%, precision 95%, recall 58%, and F1 72%. In addition, the predicted masks were assessed based on three criteria.
The research highlights that transferred-ResNet50 has successfully classified the pavement crack types in some hyperparameter combinations. Hence, this model should be applied to a more organized classification dataset where ground truths of the cracks need to be more specific with severity levels. Also, this study recommended considering more hyperparameter combinations to evaluate the model performance for classification tasks.
Furthermore, the modified-UNet model could contribute to pavement crack segmentation. The addition of multi-scale layers in both encoder-decoder networks can be used to improve the performance. This inclusion will assist the model in differentiating the cracks from noises and redundant background information. Also, some additional data preprocessing and cleaning are the potential tasks to improve the accuracy of the predicted shapes.
From this study, it is clear that state-of-the-art CNN models can be used for evaluating both new and existing pavement crack datasets. Creating a new model for specific datasets is challenging because the biased parameters of the model would be less effective for another dataset. Also, it can raise complexities when the model framework is simple or complicated compared to the data size and structure. Therefore, fine-tuning the existing models can be more productive in pavement crack classification and segmentation tasks.
Keywords: Classification, Segmentation, Pavement Crack, Convolutional Neural Networks.
... This is to promote the automation of pavement inspection by introducing new technologies such as AI and robots, and it is desirable to improve the technology for automated pavement inspection. As an example of the automation of pavement inspection performed to date, a road surface damage detection system using the method proposed in the Road Damage Detection Challenge held by the IEEE in 2018 was used by several municipalities in Japan [1]. Feedback obtained using this system suggests that the accuracy of the system needs to be improved. ...
... Feedback obtained using this system suggests that the accuracy of the system needs to be improved. There are many papers on models for detecting road damage like this example, and several were presented at the RDD Challenge [1][2][3][4]. Although many teams proposed a variety of models for the RDD Challenge, most of them used general methods to improve their models, and any papers improved their models by focusing on the unique features of road images are not found. ...
The social infrastructure, including roads and bridges built during period of rapid economic growth in Japan, is now aging, and there is a need to strategically maintain and renew the social infrastructure that is aging. On the other hand, road maintenance in rural areas is facing serious problems such as reduced budgets for maintenance and a shortage of engineers due to the declining birthrate and aging population. Therefore, it is difficult to visually inspect all roads in rural areas by maintenance engineers, and a system to automatically detect road damage is required. This paper reports practical improvements to the road damage model using YOLOv5, an object detection model capable of real-time operation, focusing on road image features
... One crucial component of this task is monitoring road surface defects, which is labour-intensive and requires domain expertise. Recently, with the evolution of computer vision and machine/deep learning (ML/DL) techniques, researchers have successfully applied them to detect road defects automatically [1]- [4]. ...
... In association with IEEE Big Data 2020, a global road damage detection challenge has been released with 121 teams' participation. Many solutions [1] have been proposed to address the road defects problem. For instance, Hegde et al. [38] used Yolov5 [39] with Data and Test Time Augmentation, and three ensemble approaches. ...
This paper proposes an effective deep learning-based model for crack detection in images acquired by different acquisition systems (e.g., cameras mounted on vehicles and drones and smartphone cameras) in six countries. By utilizing successful training procedures and including multi-scale feature extraction models, the ensemble model is built using effective variations of the cutting-edge object detection technique, Yolov7. The top crack detection models are fused using the non-maximum suppression method to create the proposed ensemble model. The proposed crack detection model is trained and validated using the crowdsensing-based road damage detection challenge (CRDDC2022). With the test set, the proposed model produced an average F1 score of 0.65 with all leaderboards of CRDDC2022 (all countries, India, Japan, Norway, and the United States leaderboards). Our approach is ranked in the 5th position in the CRDDC2022 challenge 1 . The source code is available at https://github.com/AmmarOkran/CRDD2022.
... Various solutions have already been developed to solve the road damage detection task in Global Road Damage Detection Challenge 2020 (GRDDC'2020) [10], and data augmentation techniques and algorithms that positively improve the load damage detection performance have been studied. Furthermore, with the development of deep learning technology, many teams used deep learning-based object detection solutions [10]. ...
... Various solutions have already been developed to solve the road damage detection task in Global Road Damage Detection Challenge 2020 (GRDDC'2020) [10], and data augmentation techniques and algorithms that positively improve the load damage detection performance have been studied. Furthermore, with the development of deep learning technology, many teams used deep learning-based object detection solutions [10]. Among them, YOLO-based solutions obtained a high F1 score. ...
The importance of road damage detection work is continuously increasing, and various methods are being developed to reduce the cost of time and economy. Therefore, the road damage dataset was collected from six countries, such as China, the Czech Republic, India, Japan, Norway, and United States, by various sources such as drones, cars, and motorbikes for making a robust and powerful automatic road state monitoring system. We solved the road damage detection task using YOLO, a deep learning based technology. We adopted the image tiling technique to properly use the high resolution road damage images captured in Norway with other similar size resolution images and trained twelve YOLOv5x models to use the ensemble method for detecting the four road damage types. Finally, our solution obtained an average F1 score of 0.6744 and an inference speed of 1 FPS.
... Various solutions have already been developed to solve the road damage detection task in Global Road Damage Detection Challenge 2020 (GRDDC'2020) [10], and data augmentation techniques and algorithms that positively improve the load damage detection performance have been studied. Furthermore, with the development of deep learning technology, many teams used deep learning-based object detection solutions [10]. ...
... Various solutions have already been developed to solve the road damage detection task in Global Road Damage Detection Challenge 2020 (GRDDC'2020) [10], and data augmentation techniques and algorithms that positively improve the load damage detection performance have been studied. Furthermore, with the development of deep learning technology, many teams used deep learning-based object detection solutions [10]. Among them, YOLO-based solutions obtained a high F1 score. ...
The importance of road damage detection work is increasing continuously, and various methods are being developed to perform the work in terms of time and economy efficiently. We tried to solve the road damage detection task using YOLO, and trained the model using the road damage dataset collected from various sources such as drones, cars, and motorbikes in six countries with multi resolution. To efficiently use high resolution images captured in Norway, we adopted the image tiling technique and trained 12 YOLOv5x models to use it ensemble in inference. Finally, we obtained an average F1 score of 0.6744 and 2nd place in the Crowded Sensing-based Road Damage Detection Challenge.
... From an academic perspective, camera-based detection of road damages reached its preliminary peak in the Global Road Damage Detection Challenge (GRDDC) 2020 [33,34]. The RDDS 2020 was shared among researchers with solely focusing on detection performance, not accounting for speed/applicability. ...
... The base models are evaluated on the GRDDC evaluation server results (cf. Table 4, Arya et al. [33]). The YOLOv4-CSP model is the best non-ensemble solution with #4 on the leaderboard, utilizing the tools applied in Table 3 with a runtime of 26 ms. ...
While autonomous driving technology made significant progress in the last decade, road damage detection as a relevant challenge for ensuring safety and comfort is still under development. This paper addresses the lack of algorithms for detecting road damages that meet autonomous driving systems’ requirements. We investigate the environmental perception systems’ architecture and current algorithm designs for road damage detection. Based on the autonomous driving architecture, we develop an end-to-end concept that leverages data from low-cost pre-installed sensors for real-time road damage and damage severity detection as well as cloud- and crowd-based HD Feature Maps to share information across vehicles. In a design science research approach, we develop three artifacts in three iterations of expert workshops and design cycles: the end-to-end concept featuring road damages in the system architecture and two lightweight deep neural networks, one for detecting road damages and another for detecting their severity as the central components of the system. The research design draws on new self-labeled automotive-grade images from front-facing cameras in the vehicle and interdisciplinary literature regarding autonomous driving architecture and the design of deep neural networks. The road damage detection algorithm delivers cutting-edge performance while being lightweight compared to the winners of the IEEE Global Road Damage Detection Challenge 2020, which makes it applicable in autonomous vehicles. The road damage severity algorithm is a promising approach, delivering superior results compared to a baseline model. The end-to-end concept is developed and evaluated with experts of the autonomous driving application domain.
... In this study, we utilized the open-source dataset RDD2022 [13], consisting of road images from various countries. For experimental validation, a total of 4398 road images from China were selected. ...
Road defect detection is a crucial task for promptly repairing road damage and ensuring road safety. Traditional manual detection methods are inefficient and costly. To overcome this issue, we propose an enhanced road defect detection algorithm called BL-YOLOv8, which is based on YOLOv8s. In this study, we optimized the YOLOv8s model by reconstructing its neck structure through the integration of the BiFPN concept. This optimization reduces the model’s parameters, computational load, and overall size. Furthermore, to enhance the model’s operation, we optimized the feature pyramid layer by introducing the SimSPPF module, which improves its speed. Moreover, we introduced LSK-attention, a dynamic large convolutional kernel attention mechanism, to expand the model’s receptive field and enhance the accuracy of object detection. Finally, we compared the enhanced YOLOv8 model with other existing models to validate the effectiveness of our proposed improvements. The experimental results confirmed the effective recognition of road defects by the improved YOLOv8 algorithm. In comparison to the original model, an improvement of 3.3% in average precision mAP@0.5 was observed. Moreover, a reduction of 29.92% in parameter volume and a decrease of 11.45% in computational load were achieved. This proposed approach can serve as a valuable reference for the development of automatic road defect detection methods.
... In this work, the dataset we use is the GRDDC2020 dataset [54], which is used for the Global Road Damage Detection Challenge. The dataset includes a total of 21,041 samples of road defects in three countries, namely Czechia, India and Japan, and the annotation file for the given test set is only officially available and we cannot obtain it. ...
In recent years, the number of traffic accidents caused by road defects has increased dramatically all over the world, and the repair and prevention of road defects is an urgent task. Researchers in different countries have proposed many models to deal with this task, but most of them are either highly accurate and slow in detection, or the accuracy is low and the detection speed is high. The accuracy and speed have achieved good results, but the generalization of the model to other datasets is poor. Given this, this paper takes YOLOv5s as a benchmark model and proposes an optimization model to solve the problem of road defect detection. First, we significantly reduce the parameters of the model by pruning the model and removing unimportant modules, propose an improved Spatial Pyramid Pooling-Fast (SPPF) module to improve the feature signature fusion ability, and finally add an attention module to focus on the key information. The activation function, sampling method, and other strategies were also replaced in this study. The test results on the Global Road Damage Detection Challenge (GRDDC) dataset show that the FPS of our proposed model is not only faster than the baseline model but also improves the MAP by 2.08%, and the size of this model is also reduced by 6.07 M.
... On the one hand, detection-based road crack recognition works [9][10][11] follow the pipeline of prevalent two-dimensional object detectors [12][13][14], where an RGB feature map extracted by convolutional backbone is fed into a region proposal network and detection head for boundingbox regression and classification. The recently held road damage detection challenge motivates to develop various road damage detectors [15][16][17][18]. Maeda et al. [10] capture large-scale road damage images using a smartphone mounted on a car, and use the off-the-shelf single-shot detector [12] for eight damage classifications. ...
Prior convolution-based road crack detectors typically learn more abstract visual representation with increasing receptive field via an encoder–decoder architecture. Despite the promising accuracy, progressive spatial resolution reduction causes semantic feature blurring, leading to coarse and incontiguous distress detection. To these ends, an alternative sequence-to-sequence perspective with a transformer network termed TransCrack is introduced for road crack detection. Specifically, an image is decomposed into a grid of fixed-size crack patches, which is flattened with position embedding into a sequence. We further propose a pure transformer-based encoder with multi-head reduced self-attention modules and feed-forward networks for explicitly modelling long-range dependencies from the sequential input in a global receptive field. More importantly, a simple decoder with cross-layer aggregation architecture is developed to incorporate global with local attentions across different regions for detailed feature recovery and pixel-wise crack mask prediction. Empirical studies are conducted on three publicly available damage detection benchmarks. The proposed TransCrack achieves a state-of-the-art performance over all counterparts by a substantialmargin, and qualitative results further demonstrate its superiority in contiguous crack recognition and fine-grained profile extraction.
This article is part of the theme issue ‘Artificial intelligence in failure analysis of transportation infrastructure and materials’.
... The efficiency of facility monitoring can be considerably increased in terms of speed and accuracy by applying this strategy. Deeksha Arya et al. (2020) [28] The Global Road Damage Detection Challenge (GRDDC) was a Big Data Cup held in 2020 in conjunction with the IEEE International Conference on Big Data. It used a dataset of 26,336 road photographs from India, Japan, and the Czech Republic to create algorithms for automatically detecting road damage in these nations. ...
Road damage detection is a crucial task for maintaining infrastructure and maximising road safety. Recent developments in artificial intelligence (AI) have opened up new opportunities for automating the road damage detection process. In this paper, we present a thorough review of recent research on artificial intelligence-based techniques and methods for road damage detection. We discuss the different types of road damages, AI models used for detection, datasets, evaluation metrics, and challenges associated with this field. We also provide an overview of potential future research directions.
... In another work, Arya et al. [36] reported a set of state-ofthe-art solutions in global road damage detection and classification tasks. For example, Pham et al. [37] experimented with a study with Detectron2, implementing Faster R-CNN. ...
This paper presents a novel automated road damage detection approach using Unmanned Aerial Vehicle (UAV) images and deep learning techniques. Maintaining road infrastructure is critical for ensuring a safe and sustainable transportation system. However, the manual collection of road damage data can be labor-intensive and unsafe for humans. Therefore, we propose using UAVs and Artificial Intelligence (AI) technologies to improve road damage detection’s efficiency and accuracy significantly. Our proposed approach utilizes three algorithms, YOLOv4, YOLOv5, and YOLOv7, for object detection and localization in UAV images. We trained and tested these algorithms using a combination of the RDD2022 dataset from China and a Spanish road dataset. The experimental results demonstrate that our approach is efficient and achieves 59.9% mean average precision mAP@.5 for the YOLOv5 version, 73.20% mAP@.5 for the YOLOv7 version, and 65.70% mAP@.5 for a YOLOv5 model with a Transformer Prediction Head. These results demonstrate the potential of using UAVs and deep learning for automated road damage detection and pave the way for future research in this field.
... The YOLOv5s-M model achieved good F1-scores of 0.6709 and 0.6601 on the two test datasets, respectively. The results and rankings in RDD2020 (Arya et al., 2020) as well as our results, are shown in Table 8. The term "Ensemble" indicates that the method uses ensemble learning, and "Single" represents that the method is a single network without ensemble learning. ...
Road pavement damage affects driving comfort markedly, threatens driving safety, and may even cause traffic accidents. The traffic management department conventionally captures pavement damage information mainly using manual and vehicle-mounted equipment, which is not conducive to the detection of large-scale road pavement distress. Street-view images can provide full-view images of urban roads where the data is updated regularly by navigation map service companies, making it possible to rapidly detect pavement damage in urban areas. This paper presents a new pavement damage detection approach that is built upon an improved YOLOv5 network and street-view images. The proposed model can deal with a large-scale detection layer to improve the detection precision of large distress targets, achieving thus both cross-layer and cross-scale feature fusion by using the Generalized Feature Pyramid Network (Generalized-FPN) structure. The improved network also involves a diagonal Intersection over Union loss for regression calculation of the boundary box and builds the decoupled Head structure to achieve the decoupling detection of prediction and regression. As a result, the fusion of the weak feature information in feature layers is enhanced at different spatial scales, a more suitable method is achieved for pavement damage detection in the complex context of multi-scale street-view images, and the accuracy of the modified network is much improved in the detection of pavement distress from street-view imagery. Furthermore, We created a large image sample set for model training and testing, and a total of 156,304 street-view images, obtained from Fengtai District, Beijing, China was used for demonstrating the usefulness of the proposed network. The findings indicated that the proposed approach could effectively achieve pavement damage detection of urban roads from street-view images, with a precision average of 79.8% on the test samples. Moreover, the developed model was applied for pavement damage detection for all the roads in Fengtai District, Beijing, indicating that our method can offer viable damage data for road maintenance planning.
... Data augmentation is an ordinarily engaged method to maximize the diversity and size of the labelled training sets by leveraging input transformation to conserve the corresponding output labels [6] [32]. In ML, data augmentation is commonly used to resolve class imbalance issues, address over-fitting problems in DL, and enhance convergence, which leads to better results. ...
Road surface condition detection is a significant application for many intelligent transportation systems (ITSs) to uphold favourable driving conditions and avoid accidents. However, the accurate detection and classification of road damages have become more challenging. Thus, this paper proposed an enhanced sensor based technology that determines road damage by employing a deep learning (DL) algorithm. Initially, the road damage image is acquired from mobilephone devices and pre-processed using the adaptive Gaussian bilateral filter (AGBF). Then, after data augmentation, the proposed method has applied the super and semi-supervised remora adversarial resolution learning generative (S3-RARLG) model for road damage detection and classification. Initially, a Super-Generative Adversarial Network (SGAN) is used in S3-RARLG to maximize road image clarity and improve road damage detection performance. Then semi-supervised learning is adapted to address the insufficiency of label images by maximizing the expandability of training data. Finally, adversarial learning is enhanced by integrating with SGAN to improve detection performance. Moreover, in S3-RARLG, the weight updation is executed using bionic remora meta-heuristic optimization (BRMO). The proposed ESSR-GAN is simulated on the Python platform, and the performance metrics are determined to demonstrate the effectiveness of the proposed model using the road damage dataset 2019. As a result, the proposed ESSR-GAN accomplishes a higher accuracy of 99.12%, and the acquired results outperform the existing architectures.
... Following the work of Maeda et al. [84], progressive growing-generative adversarial networks (PG-GANs) [90] with Poisson blending have been used to generate new training data (RDD-2019) in order to improve the accuracy of road damage detection. More recently, transfer learning (TL)-based road damage detection model [91] has been proposed introducing large-scale open-source dataset RDD2020 [92,93] considering multiple countries. In [94], EfficientDet-D7 has been employed for the detection of asphalt pavement distress. ...
... YOLOv5 has been widely welcomed by academic and engineering communities since its release. The USC-InfoLab team reaped the GRDDC'2020 championship using YOLOv5 [26], which also proved the effectiveness of YOLOv5 for crack detection. Therefore, we designed an improved network based on YOLOv5 to detect cracks. ...
Efficient detection of pavement cracks can effectively prevent traffic accidents and reduce road maintenance costs. In this paper, an improved YOLOv5 network combined with a Bottleneck Transformer is proposed for crack detection, called YOLOv5-CBoT. By combining the CNN and Transformer, YOLOv5-CBoT can better capture long-range dependencies to obtain more global information, so as to adapt to the long-span detection task of cracks. Moreover, the C2f module, which is proposed in the state-of-the-art object detection network YOLOv8, is introduced to further optimize the network by paralleling more gradient flow branches to obtain richer gradient information. The experimental results show that the improved YOLOv5 network has achieved competitive results on RDD2020 dataset, with fewer parameters and lower computational complexity but with higher accuracy and faster inference speed.
... Their detection techniques include Yolov5, Ensemble model, Ensemble Prediction, and data augmentation such as TTA are also adopted as efficient methods to lift performance. Duplicate or overlapped predictions generated in the process are filtered using the non-maximum suppression (NMS) algorithm [5] It is compared with HED, RCF, SegNet, SRN, U-net, SE, CrackForest, DeepCrack, and Crack-Tree (Traditional low-level feature-based method). The performance efficiency is 6 FPS, slower than HED, RFC, and SRN. ...
As the pace grows in the development of image processing techniques and the current applications rise in machine learning and deep learning techniques for visual inspections and physical assessment, this article reviews the existing literature. It provides a detailed synthesis of the overview of surface pavement conditions, computer-vision-based technologies for road damage detection, various datasets and data collection methods. We analyse and compare different machine-learning methods and models proposed in the literature and identify challenges that need to be addressed in the future in road surface defect detection.
... F1 score is calculated as the harmonic mean of precision and recall values. Details of the parameters and some equations [53][54][55] are given below: ...
Autonomous vehicles have gained popularity in recent years, but they are still not compatible with other vulnerable components of the traffic system, including pedestrians, bicyclists, motorcyclists, and occupants of smaller vehicles such as passenger cars. This incompatibility leads to reduced system performance and undermines traffic safety and comfort. To address this issue, the authors considered pedestrian crosswalks where vehicles, pedestrians, and micro-mobility vehicles collide at right angles in an urban road network. These road sections are areas where vulnerable people encounter vehicles perpendicularly. In order to prevent accidents in these areas, it is planned to introduce a warning system for vehicles and pedestrians. This procedure consists of multi-stage activities by sending warnings to drivers, disabled individuals, and pedestrians with phone addiction simultaneously. This collective autonomy is expected to reduce the number of accidents drastically. The aim of this paper is the automatic detection of a pedestrian crosswalk in an urban road network, designed from both pedestrian and vehicle perspectives. Faster R-CNN (R101-FPN and X101-FPN) and YOLOv7 network models were used in the analytical process of a dataset collected by the authors. Based on the detection performance comparison between both models, YOLOv7 accuracy was 98.6%, while the accuracy for Faster R-CNN was 98.29%. For the detection of different types of pedestrian crossings, YOLOv7 gave better prediction results than Faster R-CNN, although quite similar results were obtained.
... Following the work of Maeda et al. (2018), progressive growing-generative adversarial networks (PG-GANs) (Maeda et al., 2021) with Poisson blending have been used to generate new training data (RDD-2019) in order to improve the accuracy of road damage detection. More recently, transfer learning (TL)-based road damage detection model (Arya et al., 2021a) has been proposed introducing large-scale open-source dataset RDD2020 (Arya et al., 2020(Arya et al., , 2021b considering multiple countries. In Naddaf-Sh et al. (2020), EfficientDet-D7 has been employed for the detection of asphalt pavement distress. ...
Objective:Computer vision-based up-to-date accurate damage classification and localization are of decisive importance for infrastructure monitoring, safety, and the serviceability of civil infrastructure. Current state-of-the-art deep learning (DL)-based damage detection models, however, often lack superior feature extraction capability in complex and noisy environments, limiting the development of accurate and reliable object distinction. Method: To this end, we present DenseSPH-YOLOv5, a real-time DL-based high-performance damage detection model where DenseNet blocks have been integrated with the backbone to improve in preserving and reusing critical feature information. Additionally, convolutional block attention modules (CBAM) have been implemented to improve attention performance mechanisms for strong and discriminating deep spatial feature extraction that results in superior detection under various challenging environments. Moreover, additional feature fusion layers and a Swin-Transformer Prediction Head (SPH) have been added leveraging advanced self-attention mechanism for more efficient detection of multiscale object sizes and simultaneously reducing the computational complexity. Results: Evaluating the model performance in large-scale Road Damage Dataset (RDD-2018), at a detection rate of 62.4 FPS, DenseSPH-YOLOv5 obtains a mean average precision (mAP) value of 85.25 %, F1-score of 81.18 %, and precision (P) value of 89.51 % outperforming current state-of-the-art models. Significance: The present research provides an effective and efficient damage localization model addressing the shortcoming of existing DL-based damage detection models by providing highly accurate localized bounding box prediction. Current work constitutes a step towards an accurate and robust automated damage detection system in real-time in-field applications.
... Perkembangan ekonomi ini tidak terlepas dari peran penting infrastruktur jalan sebagai penghubung antar wilayah, sehingga menyebabkan perubahan kondisi angkutan barang dan jasa sesuai dengan volume dan berat beban yang memuat jalan tersebut [1]. Kondisi jalan yang baik memudahkan warga beraktivitas untuk menunjang kegiatan sosial ekonominya [2]. Akibat perubahan beban dan kepadatan, jalan seringkali menunjukkan kerusakan yang dapat berbahaya bagi pengguna jalan [3]. ...
Jalan Soekarno Hatta merupakan jalan kabupaten yang sekaligus menjadi penghubung antara Kota Labuan Bajo dengan jalan Trans Flores yang merupakan jalur keluar menuju kabupaten lain di Pulau Flores. Jalan ini memegang peranan penting dalam meningkatkan pertumbuhan ekonomi karena menjadi pusat wisata kuliner, pertokoan, persewaan peralatan renang, dan berbagai aktivitas khas daerah pariwisata lainnya. Tujuan penelitian ini untuk mengetahui jenis dan tingkat kerusakan jalan dan memberikan alternatif perbaikan kerusakan jalan. Penelitian ini dilakukan pada ruas jalan Soekarno Hatta dengan panjang jalan 3500 m ,lebar 7 m dan hanya mempunyai 1 arah. Penelitian ini menggunakan metode Bina Marga No.18/T/BNKT/1990. Kerusakan yang terjadi ada 4 macam, yaitu retak, alur, lubang dan tambalan, dan kekasaran permukaan. Kerusakan yang paling dominan adalah retak-retak dan kekasaran permukaan, yang disebabkan oleh beban yang melampaui batas yang dapat dipikul oleh lapisan permukaan dan juga akibat tidak segera ditanganinya kerusakan-kerusakan lain, pengaruh cuaca (terutama hujan) dan genangan air yang mempercepat terbentuknya lubang dan berbagai kerusakan lainnya. Perbaikan kerusakan dapat dilakukan dengan penambalan lubang (patching), penambahan lapisan perkerasan (overlay), dan dengan melakukan palapisan taburan aspal pada daerah yang mengalami retak. Nilai prioritas kondisi ruas jalan ini adalah 7 sehingga dimasukkan dalam program pemeliharaan rutin. Kata kunci : Kerusakan jalan, metode Bina Marga, Jalan Soekarno Hatta, Labuan Bajo
... Specific to road damage detection, Arya et al. [12], [13] reported a set of state-of-the-art solutions in global roadway Abstract-Maintaining the roadway infrastructure is one of the essential factors in enabling a safe, economic, and sustainable transportation system. Manual roadway damage data collection is laborious and unsafe for humans to perform. ...
Maintaining the roadway infrastructure is one of the essential factors in enabling a safe, economic, and sustainable transportation system. Manual roadway damage data collection is laborious and unsafe for humans to perform. This area is poised to benefit from the rapid advance and diffusion of artificial intelligence technologies. Specifically, deep learning advancements enable the detection of road damages automatically from the collected road images. This work proposes to collect and label road damage data using Google Street View and use YOLOv7 (You Only Look Once version 7) together with coordinate attention and related accuracy fine-tuning techniques such as label smoothing and ensemble method to train deep learning models for automatic road damage detection and classification. The proposed approaches are applied to the Crowdsensing-based Road Damage Detection Challenge (CRDDC2022), IEEE BigData 2022. The results show that the data collection from Google Street View is efficient, and the proposed deep learning approach results in F1 scores of 81.7% on the road damage data collected from the United States using Google Street View and 74.1% on all test images of this dataset. With these results, we received rank 2 (silver prize) as a data contributor and rank 3 (bronze prize) as the predictive model in this competition among 54 leaders (private companies and academic institutions) in this area.
... Wang et al. [22] introduce data augmentation to balance the training sample, and adopt Faster RCNN [12] to classify different cracks. Besides, Team IMSC [44] trains a YOLOv5 detector [19] with parameter adjustment and architecture finetuned to detect various types of pavement distress. The tricks of ensemble learning and test time augmentation (TTA) also facilitate the model performance. ...
Recent advances on road damage detection relies on a large amount of labeled data, whilst collecting pavement image is labor-intensive and time-consuming. Unsupervised Domain Adaptation (UDA) provides a promising solution to adapt a source domain to the target domain, however, cross-domain crack detection is still an open problem. In this paper, we propose domain adaptive road damage detection termed as DA-RDD, by incorporating image-level with instance-level feature alignment for domain-invariant representation learning in an adversarial manner. Specifically, importance weighting is introduced to evaluate the intermediate samples for image-level alignment between domains, and we aggregate RoI-wise feature with multi-scale contextual information to recover the crack details for progressive domain alignment at instance level. Additionally, a large-scale road damage dataset (based on Road Damage Dataset 2020 (RDD2020)) named as RDD2021 is constructed with
$100k$
synthetic labeled distress images. Extensive experimental results on damage detection across different countries demonstrate the universality and superiority of DA-RDD, and empirical studies on RDD2021 further claim its effectiveness and advancement. To our best knowledge, it is the first time to investigate domain adaptative pavement crack detection, and we expect the contributions in this work would facilitate the development of generalized road damage detection in the future.
... Specific to road damage detection, Arya et al. [10] reported a set of state-of-the-art solutions in global roadway damage detection and classification tasks. For instance, Pham et al. [11] experimented with Detectron2's implementation of Faster R-CNN implementation in this study. ...
Maintaining the roadway infrastructure is one of the essential factors in enabling a safe, economic, and sustainable transportation system. Manual roadway damage data collection is laborious and unsafe for humans to perform. This area is poised to benefit from the rapid advance and diffusion of artificial intelligence technologies. Specifically, deep learning advancements enable the detection of road damages automatically from the collected road images. This work proposes to collect and label road damage data using Google Street View and use YOLOv7 (You Only Look Once version 7) together with coordinate attention and related accuracy fine-tuning techniques such as label smoothing and ensemble method to train deep learning models for automatic road damage detection and classification. The proposed approaches are applied to the Crowdsensing-based Road Damage Detection Challenge (CRDDC2022), IEEE BigData 2022. The results show that the data collection from Google Street View is efficient, and the proposed deep learning approach results in F1 scores of 81.7% on the road damage data collected from the United States using Google Street View and 74.1% on all test images of this dataset.
... For each damage in the image from the training set, the damage class and its corresponding bounding box coordinates are labeled. RDD-2020 dataset has also been used as the benchmark dataset by Global Road Damage Detection Challenge (GRDDC) [105], performance of state-of-the-art solutions can be found in [91]. ...
Deep learning breakthrough stimulates new research trends in civil infrastructure inspection, whereas the lack of quality-guaranteed, human-annotated, free-of-charge, and publicly available defect datasets with sufficient amounts of data hinders the progress of deep learning in defect inspection. To boost research in deep learning-based visual defect inspection, this paper first reviews and summarizes 40 publicly available defect datasets, covering common defects in various types of buildings and infrastructures. The taxonomy of the datasets is proposed based on specific deep learning objectives (classification, segmentation, and detection). Clarifications are also made for each dataset regarding its corresponding data volume, data resolution, data source, defect categories covered, infrastructure types focused, material types targeted, algorithms adopted for validation, annotation levels, context levels, and publication license for future utilization. Consequently, the summarized defect datasets offer around 13.38M labeled images, cover more than 5 defect types, 5 infrastructure types, 5 material types, and 3 levels of image context. Given that the crack is a common interest in civil engineering, this paper further combines existing datasets with self-labeled crack images to establish a benchmark dataset providing more than 15,000 and 11,000 labeled images for crack classification and segmentation, respectively. Based on the established crack dataset, experiments are conducted for classification, segmentation, and the subsequent non-maximum suppression-based detection tasks. The proposed multi-branch self-attention module and multi-stage-fused attentional pyramid network have been successfully adapted into the state-of-the-art (SOTA) classification network-Swin Transformer and segmentation networks including DeepLab V3+, DenseNet, and Full Resolution ResNet. The resulting classification network achieves 88.0% accuracy, and the adapted segmentation models reach 77.8%,77.6%,76.9% mIoU (mean Intersection over Union), respectively. Moreover, a comprehensive comparison between 11 SOTA classification algorithms and 12 SOTA segmentation algorithms has been conducted. The algorithms proposed in this work are shown to achieve satisfactory performance with an acceptable efficiency on modern graphic processing units. Detailed suggestions are provided for constructing high-quality datasets and inspection algorithms. Finally, this paper remarks on the quantity, diversity, difficulty, and scalability of the reviewed defect datasets, feasibility on robotic platforms, superiority of proposed algorithms, and criticality of algorithm comparison results, formulating a solid baseline for future defect inspection research.
... Recent researches have shown the potential of using image processing techniques and deep learning in the detection of road damages ( [4], [5], [6], [7]). Most of the past researches ( [7], [8], [9]) and Global Road Damage Detection Challenge (GRDDC) organized as a part of the 2020 IEEE International Conference on Big Data [10] in this direction focus on the detection of cracks, potholes, and their variants. However, detection of road rutting using deep learning still remains an open research problem. ...
Road rutting is a severe road distress that can cause premature failure of road incurring early and costly maintenance costs. Research on road damage detection using image processing techniques and deep learning are being actively conducted in the past few years. However, these researches are mostly focused on detection of cracks, potholes, and their variants. Very few research has been done on the detection of road rutting. This paper proposes a novel road rutting dataset comprising of 949 images and provides both object level and pixel level annotations. Object detection models and semantic segmentation models were deployed to detect road rutting on the proposed dataset, and quantitative and qualitative analysis of model predictions were done to evaluate model performance and identify challenges faced in the detection of road rutting using the proposed method. Object detection model YOLOX-s achieves mAP@IoU=0.5 of 61.6% and semantic segmentation model PSPNet (Resnet-50) achieves IoU of 54.69 and accuracy of 72.67, thus providing a benchmark accuracy for similar work in future. The proposed road rutting dataset and the results of our research study will help accelerate the research on detection of road rutting using deep learning.
... The proposed method categorized the damages into four main types: longitudinal/parallel cracks, transverse/perpendicular cracks, alligator/complex cracks, and potholes. The proposed dataset in [4] was made publicly available [5] and formed the basis for the organization of the Global Road Damage Detection Challenge (GRDDC), which led to many innovative solutions that are summarized in [3]. In this work, an image-based system for road infrastructure monitoring, based on YOLOv5 detection model is presented, which classifies the defects into three main damage types. ...
Road infrastructure is positively associated with a country’s socio-economic growth and therefore road maintenance is of great importance for every country. One of the critical maintenance steps is road damage detection, which typically requires large amounts of time and high costs. In this work, the YOLOv5 two-stage detector is leveraged, in order to create an image-based solution for road defect detection and classification. The damages are classified into three main categories: cracks, potholes, and blurred markings. The YOLOv5 can achieve a relatively high detection accuracy with a score of Intersection over Union (IoU) up to 88.89% and classification accuracy with an F1 score up to 80.72%. The precision and recall scores are 84.26% and 78.38%, respectively.
Efficient identification of road defects is a critical concern for road safety and infrastructure upkeep. This research employs drone-captured imagery and advanced object detection algorithms to expedite defect recognition, with a specific focus on determining the optimal algorithm for prompt and precise detection. The importance of timely road defect detection, crucial for mitigating potential hazards, remains central. A comprehensive comparative analysis of contemporary object detection algorithms, encompassing YOLOv5s, YOLOv5m, YOLOv5l, YOLOv5x, and YOLOv7. The results of this study highlight YOLOv7 as the most efficient, with a notable mAP of 68.3%, closely followed by YOLOv5l (66.8%), YOLOv5m (66.3%), YOLOv5x (66%), and YOLOv5s (63%). The integration of drone-derived imagery, capturing distinct gradients, significantly enhances defect detection accuracy. Beyond road safety, this study offers valuable insights to computer vision and machine learning practitioners. By bridging technological innovation with practical implementation, it holds potential to advance road safety and transportation infrastructure quality and the use of revolutionary drone technology.
4 Deep Learning (DL) has proven its efficacy in extracting useful distress information from image-5 based data of infrastructure assets such as pavements. Despite the overwhelming research on this 6 topic, state-of-the-art DL approaches fail to perform satisfactorily on independent datasets as noted 7 from an object detection-based competition. Besides, a lack of clarification in computing DL 8 performance measures and inadequate discussion on DL implementation framework still exist. To 9 this end, this paper contributes to the body of knowledge by synthesizing the performance of DL 10 models from the existing relevant literature using the-approach. 11 Meta-analysis requires an estimate of the uncertainty in the reported performance measure (i.e., 12 F1-score) to assign weights to individual studies and compute an overall performance measure for 13 a group of studies. Hence, this paper introduces a statistical approach to calculate the uncertainty 14 in the reported F1-score to compute the within-study variance. The methods, statistics, and results 15 presented in this paper will help understand the requisites for future studies on DL in pavement 16 distress evaluation, ultimately improving pavement asset management. 17 18
This paper presents a comprehensive review of recent advancements in image processing and deep learning techniques for pavement distress detection and classification, a critical aspect in modern pavement management systems. The conventional manual inspection process conducted by human experts is gradually being superseded by automated solutions, leveraging machine learning and deep learning algorithms to enhance efficiency and accuracy. The ability of these algorithms to discern patterns and make predictions based on extensive datasets has revolutionized the domain of pavement distress identification. The paper investigates the integration of unmanned aerial vehicles (UAVs) for data collection, offering unique advantages such as aerial perspectives and efficient coverage of large areas. By capturing high-resolution images, UAVs provide valuable data that can be processed using deep learning algorithms to detect and classify various pavement distresses effectively. While the primary focus is on 2D image processing, the paper also acknowledges the challenges associated with 3D images, such as sensor limitations and computational requirements. Understanding these challenges is crucial for further advancements in the field. The findings of this review significantly contribute to the evolution of pavement distress detection, fostering the development of efficient pavement management systems. As automated approaches continue to mature, the implementation of deep learning techniques holds great promise in ensuring safer and more durable road infrastructure for the benefit of society.
The current road damage detection (RDD) algorithms fail to achieve automatic and accurate evaluation and application in the traffic scenarios. In this paper, we propose a RDD algorithm YOLOv7-RDD based on the YOLOv7 model. The data augmentation method CutPaste is introduced for the first time, which can learn the irregularity of damage characteristics, construct pseudo damage samples with high similarity, and create a priori conditions for features extracted. We introduce the CBAM module into the ELAN module to resist the influence of interfering information. And it makes the model focus more on the feature of small objects and reduce the difficulty of the hard objects. In addition, we propose a new dataset RDDBJ, which contains five categories of road damage in 5390 images. And they are high-resolution from a top view, which are more suitable for detection and localization than others. Experiments on the RDDBJ dataset shows that the mAP reaches by 61.9% and is improved by 3.3% compared to the baseline, which is competitive and inspiring.KeywordsRoad Damage DetectionYOLOv7 ModelCutPasteCBAM
Object detection plays a vital role in computer vision. As a typical application of object detection, road damage detection has strong practicability. Road damage includes longitudinal crack, transverse crack, etc. Compared with universal objects, the length-width ratio of road damage is usually extreme, and the road damage appears like a line. Thus, in the paper, we utilize this characteristic of road damage to propose linear refinement attention. Our linear refinement attention consists of two parts: linear pooling and linear convolution. They are connected in series and refine the intermediate feature maps’ global and local features, respectively. They both first capture high-level semantic features, then transform the captured features into weight, and finally utilize the weight to refine the input. The most common dataset for road damage detection is Road Damage Dataset 2020 (RDD2020). According to our observation, many images in RDD2020 lack labels, leading to unreasonable evaluation. Thus, we optimized the dataset to ensure a professional and accurate evaluation. We conducted many experiments, and the results show that our method can capture the features of road damage more accurately and improve the mAP steadily compared with the baseline.KeywordsObject DetectionRoad Damage DetectionRDD2020Linear Refinement
Automated detection of pavement distress can prevent deterioration of premature surface disintegration in pavements. Potholes that are a common sight in harsh and cold terrains are a severe threat to road safety and a major contributing factor to pavement distress. To facilitate timely detection and repair of potholes, a computationally light and feasible, intelligent pavement pothole detection system is proposed by developing a novel workflow for image-based detection and severity assessment. A single-stage CNN architecture, RetinaNet is modified and optimised to best detect potholes and used in combination with a novel pothole depth estimation algorithm. A comparative evaluation of the model’s performance against the existing state-of-the-art model on the benchmark dataset establishes the proposed model’s high performance and applicability in real-time scenarios. The depth estimation algorithm is based on a 3D road surface model generated by employing the photogrammetric process of structure from motion (SfM). The point cloud data obtained thereafter, is used for accurate measurement of pothole depth. The comparison of the derived depth with the onsite depth measurement of the pothole reveals a mean error below 5%. This method leads to a practical and intelligent solution to be implemented as part of a potential pavement health assessment system for future practice.
The pothole is a common road defect that seriously affects traffic efficiency and personal safety. Road evaluation and maintenance and automatic driving take pothole detection as their main research part. In the above scenarios, accuracy and real-time pothole detection are the most important. However, the current pothole detection methods can not meet the accuracy and real-time requirements of pothole detection due to their multiple parameters and volume. To solve these problems, we first propose a lightweight one-stage object detection network, the AAL-Net. In the network, we design an LF (lightweight feature extraction) module and use the NAM (Normalization-based Attention Module) attention module to ensure the accuracy and real time of the pothole detection process. Secondly, we make our own pothole dataset for pothole detection. Finally, in order to simulate the real road scene, we design a data augmentation method to further improve the detection accuracy and robustness of the AAL-Net. The metrics F1 and GFLOPs show that our method is better than other deep learning models in the self-made dataset and the pothole600 dataset and can well meet the accuracy and real-time requirements of pothole detection.
Pavement condition evaluation is essential to time the preventative or rehabilitative actions and control distress propagation. Failing to conduct timely evaluations can lead to severe structural and financial loss of the infrastructure and complete reconstructions. Automated computer-aided surveying measures can provide a database of road damage patterns and their locations. This database can be utilized for timely road repairs to gain the minimum cost of maintenance and the asphalt's maximum durability. This paper introduces a deep learning-based surveying scheme to analyze the image-based distress data in real-time. A database consisting of a diverse population of crack distress types such as longitudinal, transverse, and alligator cracks, photographed using mobile-device is used. Then, a family of efficient and scalable models that are tuned for pavement crack detection is trained. Proposed models, resulted in F1-scores, ranging from 52% to 56%, and average inference time from 178-10 images per second. Finally, the performance of the object detectors are examined, and error analysis is reported against various images. The source code is available at https://github.com/mahdi65/roadDamageDetection2020.
Machine learning can produce promising results when sufficient training data are available; however, infrastructure inspections typically do not provide sufficient training data for road damage. Given the differences in the environment, the type of road damage and the degree of its progress can vary from structure to structure. The use of generative models, such as a generative adversarial network (GAN) or a variational autoencoder, makes it possible to generate a pseudoimage that cannot be distinguished from a real one. Combining a progressive growing GAN along with Poisson blending artificially generates road damage images that can be used as new training data to improve the accuracy of road damage detection. The addition of a synthesized road damage image to the training data improves the F ‐measure by 5% and 2% when the number of original images is small and relatively large, respectively. All of the results and the new Road Damage Dataset 2019 are publicly available (https://github.com/sekilab/RoadDamageDetector).
Cyber attacks cause great damage to our national security, ranging from individual internet user to biggest governmental/industrial organizations, such as Equifax (Data Breach 145.5 Million Accounts, reported in July 2017) or Uber (Data Breach 57 Million Records, reported in November 2017). The cyber assault has significantly increased in breadth and depth. This paper introduces CVExplorer, a novel interactive system for visualizing cybersecurity threats reported in the National Vulnerability Database. The proposed system aims to work as a reporting and alerting tool that can help enhance the security against cyber attacks can potentially reduce network vulnerabilities. The CVExplorer system containing multiple linked views allows users to visualize the relationships of various dimensions in the large number of vulnerability reports, such as types and levels of vulnerability, vendors, and products. The CVExplorer provides an intuitive interface and supports a range of interactive features, such as filtering and ordering by vulnerability severity ratings, allowing users to narrow down topics of interest quickly. To demonstrate the effectiveness of the proposed system, we demonstrate the CVExplorer on two case studies of Common Vulnerabilities and Exposures retrieved from the National Vulnerability Database.
Research on damage detection of road surfaces using image processing techniques has been actively conducted. This study makes three contributions to address road damage detection issues. First, to the best of our knowledge, for the first time, a large‐scale road damage data set is prepared, comprising 9,053 road damage images captured using a smartphone installed on a car, with 15,435 instances of road surface damage included in these road images. Next, we used state‐of‐the‐art object detection methods using convolutional neural networks to train the damage detection model with our data set, and compared the accuracy and runtime speed on both, using a GPU server and a smartphone. Finally, we demonstrate that the type of damage can be classified into eight types with high accuracy by applying the proposed object detection method. The road damage data set, our experimental results, and the developed smartphone application used in this study are publicly available (https://github.com/sekilab/RoadDamageDetector/).
The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision
and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised
annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection.
This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification
and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object
or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history
of the challenge, and proposes directions for future improvement and extension.
Many municipalities and road authorities seek to implement automated evaluation of road damage. However, they often lack technology, know-how, and funds to afford state-of-the-art equipment for data collection and analysis of road damages. Although some countries, like Japan, have developed less expensive and readily available Smartphone-based methods for automatic road condition monitoring, other countries still struggle to find efficient solutions. This work makes the following contributions in this context. Firstly, it assesses usability of Japanese model for other countries. Secondly, it proposes a large-scale heterogeneous road damage dataset comprising 26,620 images collected from multiple countries (India, Japan, and the Czech Republic) using smartphones. Thirdly, it proposes models capable of detecting and classifying road damages in more than one country. Lastly, the study provides recommendations for readers, local agencies, and municipalities of other countries when one other country publishes its data and model for automatic road damage detection and classification. A part of the proposed dataset was utilized for Global Road Damage Detection Challenge’2020 and can be accessed at (https://github.com/sekilab/RoadDamageDetector/).
The Pascal Visual Object Classes (VOC) challenge consists of two components: (i) a publicly available dataset of images together with ground truth annotation and standardised evaluation software; and (ii) an annual competition and workshop. There are five challenges: classification, detection, segmentation, action classification, and person layout. In this paper we provide a review of the challenge from 2008–2012. The paper is intended for two audiences: algorithm designers, researchers who want to see what the state of the art is, as measured by performance on the VOC datasets, along with the limitations and weak points of the current generation of algorithms; and, challenge designers, who want to see what we as organisers have learnt from the process and our recommendations for the organisation of future challenges. To analyse the performance of submitted algorithms on the VOC datasets we introduce a number of novel evaluation methods: a bootstrapping method for determining whether differences in the performance of two algorithms are significant or not; a normalised average precision so that performance can be compared across classes with different proportions of positive instances; a clustering method for visualising the performance across multiple algorithms so that the hard and easy images can be identified; and the use of a joint classifier over the submitted algorithms in order to measure their complementarity and combined performance. We also analyse the community’s progress through time using the methods of Hoiem et al. (Proceedings of European Conference on Computer Vision, 2012) to identify the types of occurring errors. We conclude the paper with an appraisal of the aspects of the challenge that worked well, and those that could be improved in future challenges.
Transfer learning-based road damage detection for multiple countries
- D Arya
- H Maeda
- S K Ghosh
- D Toshniwal
- A Mraz
- T Kashiyama
- Y Sekimoto
D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, A. Mraz, T. Kashiyama,
and Y. Sekimoto, "Transfer learning-based road damage detection for
multiple countries," arXiv preprint arXiv:2008.13101, 2020.