Article

Road-Sign Detection and Recognition Based on Support Vector Machines

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Traditional methods [6][7][8][9] use dimension reduction methods, like principal component analysis and the Karhunen-Loève transform, to extract effective and lower-dimension features and select a corresponding classification method, like Fisher, support vector machines (SVM), and multi-layer perceptron (MLP). Traditional methods require a smaller dataset and less computation; however, the accuracy relies on the data distribution and is relatively lower. ...
... Zaklouta et al. [7] compared the K-d trees with random forest using four types of histogram of oriented gradients (HOG) and distance transform and achieved the best accuracy with tree classifiers. Maldonado-Bascón et al. [8] trained a shape-based support vector machine (SVM) on the distance to border vectors, and Fleyeh et al. [9] proposed a two stage SVM method, with the first stage to classify the shape and the second stage to determine the pictogram. ...
... The traffic sign detection task requires the detection of the location of the traffic sign in the picture, while we simultaneously detect and classify the location. The dataset we selected is the ITSDB dataset for traffic sign detection under the ice and snow environment, and we chose the Libra-RCNN [8] network proposed by CVPR in 2019, which obtained the best results in COCO and the best performance in ImageNet, and the HRNetv2p-w18 detection network [7], which is also one of the advanced networks to test on our dataset to verify the robustness and our dataset's challenge. ...
Article
Full-text available
Traffic sign recognition in poor environments has always been a challenge in self-driving. Although a few works have achieved good results in the field of traffic sign recognition, there is currently a lack of traffic sign benchmarks containing many complex factors and a robust network. In this paper, we propose an ice environment traffic sign recognition benchmark (ITSRB) and detection benchmark (ITSDB), marked in the COCO2017 format. The benchmarks include 5806 images with 43,290 traffic sign instances with different climate, light, time, and occlusion conditions. Second, we tested the robustness of the Libra-RCNN and HRNetv2p on the ITSDB compared with Faster-RCNN. The Libra-RCNN performed well and proved that our ITSDB dataset did increase the challenge in this task. Third, we propose an attention network based on high-resolution traffic sign classification (PFANet), and conduct ablation research on the design parallel fusion attention module. Experiments show that our representation reached 93.57% accuracy in ITSRB, and performed as well as the newest and most effective networks in the German traffic sign recognition dataset (GTSRB).
... RS satellite data can address the challenges of assessing forest conditions and evaluating the existing historical trends of forest degradation and deforestation [7]. The information of LU/LC changes is essential for various management and decision-making activities [59]. The changes in LU/LC due to human activities and natural phenomena have caused many problems in climate change, natural hazards, land shifting, and global warming at regional and global levels. ...
... PCA method has been used for many years in change detection and has become the most popular method due to its easiness and ability to enhance change information [14,59]. There are two methods for applying PCA for change detection: ...
Article
Full-text available
The change detection (CD) methods explore the potential of remote sensing (RS) spatial datasets in various land use/land cover (LU/LC) applications. These methods are used to analyze the LU/LC dynamics using various high and medium-resolution multi-spectral remote sensing satellite datasets (Landsat-TM/ETM+/OLI, IRS LISS-3 & 4, Sentinel-2, SPOT, and ASTER). The study’s objective is to summarize multiple changes in the last two decades in land use applications at the regional and international levels using traditional and advance change detection methods. Mapping of LU/LC dynamics at regional and global scales is essential for various land use applications (vegetation monitoring, crop cultivation monitoring, urban planning, landslide, and socio-economic dynamics). The review study showed that machine learning and deep learning techniques play an essential role in classification and change detection applications. The deep learning methods more effectively identify the changes in LU/LC (due to human activities and natural phenomena) than other traditional methods. The present study analyzes the conventional and advanced methods of change detection methods and various challenges and problems facing during the change detection.
... Tsai et al. (2008) transferred the road images from RGB colour space to an eigenvalue space and classified them using a radial basis function (RBF) classifier. Maldonado-Bascón et al. (2007) also used the colour space of HSI and then segmented the images using the support vector machine (SVM) classifier. As another example, Lillo-Castellano et al. (2015) used Lab and HSI colour spaces for coloured and white signs, respectively. ...
... This idea is thoroughly tested in the Evaluations section. There, we compare the results of our technique to those obtained by some of the most popular colour-based segmentation techniques including SVM (Maldonado-Bascón et al. 2007), KNN (Lillo-Castellano et al. 2015), and K-means (Li et al. 2015). Also, as is evident from the discussions above, the images are usually transformed into a colour space other than RGB before segmentation to overcome unwanted illumination effects. ...
Article
Full-text available
Automatic detection and recognition of traffic signs from images is an important topic in many applications. At first, we segmented the images using a classification algorithm to delineate the areas where the signs are more likely to be found. In this regard, shadows, objects having similar colours, and extreme illumination changes can significantly affect the segmentation results. We propose a new shape-based algorithm to improve the accuracy of the segmentation. The algorithm works by incorporating the sign geometry to filter out the wrong pixels from the classification results. We performed several tests to compare the performance of our algorithm against those obtained by popular techniques such as Support Vector Machine (SVM), K-Means, and K-Nearest Neighbours. In these tests, to overcome the unwanted illumination effects, the images are transformed into colour spaces Hue, Saturation, and Intensity, YUV, normalized red green blue, and Gaussian. Among the traditional techniques used in this study, the best results were obtained with SVM applied to the images transformed into the Gaussian colour space. The comparison results also suggested that by adding the geometric constraints proposed in this study, the quality of sign image segmentation is improved by 10%–25%. We also comparted the SVM classifier enhanced by incorporating the geometry of signs with a U-Shaped deep learning algorithm. Results suggested the performance of both techniques is very close. Perhaps the deep learning results could be improved if a more comprehensive data set is provided.
... Initial work done on traffic sign and traffic light detection [4], [6], [11], [17], [21], [24] mainly focus on traditional image processing based techniques and machine learning Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka © 2022 IEEE. Personal use of this material is permitted. ...
... Similarly, for traffic lights, distinct features of traffic lights such as colour and shape have been used in [11], while [4] uses HOG features. Machine learning based techniques such as support vector machines (SVMs) and random forests have been used for traffic sign classification in [21] and [6]. Hidden Markov models and SVMs have been used as machine learning based techniques for traffic light detection and classification in [11] and [17]. ...
Preprint
Recent work done on traffic sign and traffic light detection focus on improving detection accuracy in complex scenarios, yet many fail to deliver real-time performance, specifically with limited computational resources. In this work, we propose a simple deep learning based end-to-end detection framework, which effectively tackles challenges inherent to traffic sign and traffic light detection such as small size, large number of classes and complex road scenarios. We optimize the detection models using TensorRT and integrate with Robot Operating System to deploy on an Nvidia Jetson AGX Xavier as our embedded device. The overall system achieves a high inference speed of 63 frames per second, demonstrating the capability of our system to perform in real-time. Furthermore, we introduce CeyRo, which is the first ever large-scale traffic sign and traffic light detection dataset for the Sri Lankan context. Our dataset consists of 7984 total images with 10176 traffic sign and traffic light instances covering 70 traffic sign and 5 traffic light classes. The images have a high resolution of 1920 x 1080 and capture a wide range of challenging road scenarios with different weather and lighting conditions. Our work is publicly available at https://github.com/oshadajay/CeyRo.
... To solve the problems in the traditional method, related research [4,5] adds machine learning to the traditional method, which divides the detection into two steps, first locating the region of traffic signs in the image using the traditional method, and subsequently classifying the traffic signs in the region using a support vector machine (SVM) classifier. The SVM classifier is effective in mitigating the effects of external factors, but this method still suffers from the need to design hand-made features for different traffic signs. ...
Article
Full-text available
Existing algorithms have difficulty in solving the two tasks of localization and classification simultaneously when performing traffic sign detection on realistic images of complex traffic scenes. In order to solve the above problems, a new road traffic sign dataset is created, and based on the YOLOv4 algorithm, for the complexity of realistic traffic scene images and the large variation in the size of traffic signs in the images, the multi-scale feature extraction module, cascade feature fusion module and attention mechanism module are designed to improve the algorithm’s ability to locate and classify traffic signs simultaneously. Experimental results on the newly created dataset show that the improved algorithm achieves a mean average precision of 84.44%, which is higher than several major CNN-based object detection algorithms for the same type of task.
... Li et al. [25] developed a new traffic sign detection method by integrating the image segmentation based on color invariants and the shape matching based on pyramid histogram of oriented gradients (PHOG) features. Based on support vector machines (SVM), Maldonado-Bascón et al. [26] proposed an automatic road-sign detection and recognition system. Salti et al. [5] addressed the problem of traffic sign detection in mobile mapping data by combining solid image analysis and pattern recognition techniques. ...
Article
Full-text available
Traffic sign detection is extremely important in autonomous driving and transportation safety systems. However, the accurate detection of traffic signs remains challenging, especially under extreme conditions. This paper proposes a novel model called Traffic Sign Yolo (TS-Yolo) based on the convolutional neural network to improve the detection and recognition accuracy of traffic signs, especially under low visibility and extremely restricted vision conditions. A copy-and-paste data augmentation method was used to build a large number of new samples based on existing traffic-sign datasets. Based on You Only Look Once (YoloV5), the mixed depth-wise convolution (MixConv) was employed to mix different kernel sizes in a single convolution operation, so that different patterns with various resolutions can be captured. Furthermore, the attentional feature fusion (AFF) module was integrated to fuse the features based on attention from same-layer to cross-layer scenarios, including short and long skip connections, and even performing the initial fusion with itself. The experimental results demonstrated that, using the YoloV5 dataset with augmentation, the precision was 71.92, which was increased by 34.56 compared with the data without augmentation, and the mean average precision mAP_0.5 was 80.05, which was increased by 33.11 compared with the data without augmentation. When MixConv and AFF were applied to the TS-Yolo model, the precision was 74.53 and 2.61 higher than that with data augmentation only, and the value of mAP_0.5 was 83.73 and 3.68 higher than that based on the YoloV5 dataset with augmentation only. Overall, the performance of the proposed method was competitive with the latest traffic sign detection approaches.
... Most available traffic sign datasets are composed of still images. For example, UAH Dataset [35], CVL Dataset [30], German Traffic Sign Detection Benchmark(GTSDB) [25], Russian Traffic SignDataset(RTSD) [49], and Tsinghua-Tencent 100K(TT100K) [64]. Even though the MASTIF dataset [47] provides some annotated video sequences, only a few frames in a long sequence are labeled. ...
Article
Full-text available
Continuously detecting traffic signs in a video sequence is necessary for autonomous or assisted driving scenarios, since a vehicle needs the information from the signs to facilitate navigation. Single-image based traffic sign detector may fail in many cases, when the car moves fast on the road, resulting in motion blur, partial occlusion, and abrupt environmental change. In this paper, we propose an effective methodology, called detection-by-tracking, for robust traffic sign detection in videos, so as to improve the detection performance beyond a basic object detector. We explore the temporal cues among frames to help with the proposal reasoning for further regression. The correlations of spatial location and appearance similarity for the same sign in adjacent frames are considered in our approach. Experimental results show that the proposed detection-by-tracking mechanism is helpful, with improved detection performance to a large extent. Moreover, the idea of the detection-by-tracking can also be generalized to other scenarios for object detection tasks in videos.
... In the detection stage, the input images are preprocessed, enhanced and then, segmented according to their color or geometry. Color-based methods usually use normalized RGB space [7][8][9][10] or HSV space [11][12][13][14][15][16] or YUV space [17] to distinguish between traffic and non traffic signs. However, these methods are generally affected by the weather conditions and the illumination variations. ...
Chapter
Full-text available
This chapter deals with a design of a new speed control method using artificial intelligence techniques applied to an autonomous electric vehicle. In this research, we develop an Advanced Driver Assistance System (ADAS) which aims to enhance the driving manner and the safety, especially when traveling too fast. The proposed model is a complete end-to-end vehicle speed system controller that proceeds from a detected speed limit sign to the regulation of the motor’s speed. It recognizes the speed limit signs before extracting from them, a speed information that will be sent, as reference, to a NARMA-L2 based controller. The study is developped specially for electric vehicle using Brushless Direct Current (BLDC) motor. The simulation results, implemented using Matlab-Simulink, show that the speed of the electric vehicle is controlled successfully with different speed references coming from the image processing unit.
... In addition, rapid and accurate recognition of traffic signs is important for improving traffic safety, the primary goal of Intelligent Transportation Systems and Vision Zero initiatives [5][6][7]. While significant effort to develop a robust sign recognition system has been made by both academics and industry practitioners [8][9][10][11][12][13][14][15], the sensitivity of the developed systems to recognize signs in varying real-world conditions is still hypothetical. Our descriptive research, then, concentrates on the question: how does the diversity of weather and reflectivity conditions of the physical world influence the recognition performance of a TSR system, given that the system is developed by incremental training resources? ...
Article
Full-text available
Automatic recognition of traffic signs in complex, real-world environments has become a pressing research concern with rapid improvements of smart technologies. Hence, this study leveraged an industry-grade object detection and classification algorithm (You-Only-Look-Once, YOLO) to develop an automatic traffic sign recognition system that can identify widely used regulatory and warning signs in diverse driving conditions. Sign recognition performance was assessed in terms of weather and reflectivity to identify the limitations of the developed system in real-world conditions. Furthermore, we produced several editions of our sign recognition system by gradually increasing the number of training images in order to account for the significance of training resources in recognition performance. Analysis considering variable weather conditions, including fair (clear and sunny) and inclement (cloudy and snowy), demonstrated a lower susceptibility of sign recognition in the highly trained system. Analysis considering variable reflectivity conditions, including sheeting type, lighting conditions, and sign age, showed that older engineering-grade sheeting signs were more likely to go unnoticed by the developed system at night. In summary, this study incorporated automatic object detection technology to develop a novel sign recognition system to determine its real-world applicability, opportunities, and limitations for future integration with advanced driver assistance technologies.
... -Traffic Sign Detection focuses on the detection and interpretation of traffic signs. This can be achieved with semi-supervised 1 SVMs [155]. The icon for traffic sign detection is a stop sign. ...
Thesis
Machine learning, also known as artificial intelligence, has become a much-researched topic in recent years. Many everyday life applications in a wide variety of fields make use of these powerful self-learning systems. Among such applications are safety-critical software systems, such as autonomous driving systems. However, like any computer system, machine learning systems are not safe from attacks by organizations with malicious intentions. To analyze how dangerous attacks are to safety-critical systems, we estimate the threat that attacks pose to the systems that contain machine learning and humans, such as road users, if the systems are not secured against attacks. We evaluate attacks on machine learning systems and subsystems in autonomous vehicles and combine both evaluations to assess the actual danger that attacks pose to autonomous vehicles. We find that many attacks are already mitigated by the distributed nature of embedded systems and security measures in place as of today. The greatest threat is posed by attacks that require access to only the inputs and outputs of the machine learning system. These include adversarial example attacks that manipulate inputs to provoke false outputs. We also conduct interviews with industry experts to analyze how machine learning systems are currently developed in practice and identify areas for potential and need for improvement. As a result of this analysis, we set up a list of requirements that can help create more secure machine learning systems. Machine learning systems are sensitive to small changes in the input data. For example, when images are slightly manipulated in a specific way they are misclassified even though they were classified correctly before the manipulations were applied. These altered images are called adversarial examples and pose a serious threat. This work deals with this form of attack in more detail and analyzes how the computation of manipulated images can be sped up with the help of masks. We propose an algorithm that selects random pixels in the mask, manipulates them and merges the changes that have the biggest influence on the output of the machine learning system regarding the attackers' goal to create the adversarial example. We run several experiments using different types and sizes of masks and find that masks can indeed have a positive impact on the effectiveness and efficiency of the attack. In addition, it may be possible to add masks to existing adversarial example attack algorithms, which also improves them. We show this by running experiments using other attack algorithms. We also discuss prerequisites under which an improvement of attack algorithms by using masks is possible. We combine the various small perturbations that turn images into adversarial examples into a universal adversarial perturbation. This is a special modification that does not cause misclassification for only one image, as is the case with adversarial examples but causes misclassification of multiple images. Our experiments show that the universal adversarial perturbations we compute cause misclassification for a large number of images, but the changes in the images need to be very strong, making them easy for a human to detect. Therefore, universal adversarial perturbations need to be obscured differently. For that we use masks, for example, to perturb only the border of the image. These manipulations could be seen as a decorative element. We also see that it is difficult to compute universal adversarial perturbations that cause misclassification for 100% of the images in a dataset.
... The system here detects different shapes like circle, triangle, rectangle and octagons. This was used in Spanish traffic sign detection technique (Maldonado-Bascón et al., 2007;Farah et al., 2018). Colour indexing another method used for identifying road signs, but the computation time was not feasible in complex situations when the traffic was dense. ...
Article
In this paper we have designed and constructed an IoT-based platform which can automatically send information about the road signs. Here, we will demonstrate the basic idea of how to set-up a communication between the upcoming vehicle and the sign boards. This system will play an important role for the recognition and detection of specific locations like markets, schools, speed breakers, universities, hospitals, offices etc. Detecting and recognising traffic signs is a challenging problem. Traffic sign recognition (TSR) is an issue of concern for drivers because of the speed in which they tend to travel at, especially on the highways. We present a device that will detect the road sign with the help of IoT using a very simple logic.
... First, they generate the edge map of the region of interest (ROI) and then aim to find the HAZMAT attributes using shape-line attributes methods such as line or circle Hough transform (Pao et al., 1992), (Loy and Barnes, 2004). These approaches may also find the candidate shape based on the SVM classifiers (Maldonado-Bascon et al., 2007) after an initial content recognition, or based on Gaussian-kernel SVMs. Shape-based methods are not occlusion-invariant, and also very sensitive to perspective distortions. ...
Article
Full-text available
One of the most challenging and non-trivial tasks in robot-based rescue operations is the Hazardous Materials (HAZMAT) sign detection in dangerous operation fields, in order to prevent further unexpected disasters. Each HAZMAT sign has a specific meaning that the rescue robot should detect and interpret it to take a safe action, accordingly. Accurate HAZMAT detection and real-time processing are the two most important factors in such robotics applications. Furthermore, the rescue robot should cope with some secondary challenges such as image distortion and restricted CPU and computational resources, embedded in the robot. In this research, we propose a CNN-Based pipeline called DeepHAZMAT for HAZMAT sign detection and segmentation in four steps: (1) Input data volume optimisation before feeding into the CNN network, (2) Application of a YOLO-based structure to collect the required visual information from the hazardous areas, (3) HAZMAT sign segmentation and separation from the background using adaptive GrabCut technique, and (4) Post-processing optimisation using morphological operators and convex hull algorithms. In spite of the utilisation of a very limited CPU and memory resources, the experimental results show the proposed method has successfully maintained a better performance in terms of detection-speed and detection-accuracy, compared to classical and modern state-of-the-art methods.
... In particular, techniques based on color and shape features have become popular, and these are referred to as traditional methods. Examples of some traditional methods are the HOG feature descriptor, color segmentation and the Hough transform (De La Escalera et al., 2004), (Dalal & Triggs, 2005), (Garcia-Garrido et al., 2006), (Maldonado-Bascón et al., 2007). Many detection methods have been developed with traditional methods in the past decades. ...
Article
Full-text available
With the ever increasing number of vehicles on the roads, traffic signs are becoming more and more important every passing day. Despite the fact that traffic signs are simple and easy to understand, in congested traffic drivers may miss them. Considering that even milliseconds can make a huge difference in preventing accidents, it would make a big help if a system could assist the driver with traffic signs. In order to achieve this, a traffic sign recognition system needs to be implemented. Accordingly, this study aims to develop a Turkish traffic sign detection and recognition system using the Faster R-CNN algorithm. The proposed solution utilizes TensorFlow framework and specifically makes use of the Faster R-CNN Inception-v2-COCO to train the object detection model. For training purposes, indigenous dataset is created containing 54 classes and 10842 Turkish traffic sign images. The training process of the model is carried out twice with step numbers 51,217 and 200,000, respectively. Then, these two models are used to detect 10 Turkish traffic sign images taken both daytime and nighttime. The results indicate that the proposed system’s average precision is 67.2% and average recall is 78.3% when trained with 51,217 steps; on the other hand, the average precision increases to 76% and average recall to 82.8% when trained with 200,000 steps.
... It is very difficult for the visually impaired people, which the traffic sign may be missed prominently. The traffic signs play a second part that is to normalize the traffic and the next is to indicate the road state stated from Maldonado et al. [1]. This sign can be categorized as the shape and color. ...
Article
Full-text available
Independent mobility involves various challenges to Visual Impairment or Blindness (VIB) people. Most of the mobile devices are accessible to people with VIB that makes the use of available specific applications in online stores. Some applications support the independent mobility for VIB users in safely crossing road. The traffic sign detection and recognition (TSDR) is an essential challenge to VIB people. Existing research offers various techniques to detect the traffic sign in an open road environment. However, this system did not correctly recognizes the traffic sign. This research addressed the problem of traffic sign recognition to support the VIB people for safely crossing the road. Traffic sign detection and recognition are achieved by using novel Random Gradient Succession with Momentum (RGSM) with novel shape specific feature extraction method. Finally, the CNN classifier will be utilized to categorize the trained output labels, which then converts the traffic sign into the audio signal in both the training phase and the testing phase. The results are estimated for the performance measures like accuracy, specificity, precision, F-score, Jaccard coefficient, kappa, and Dice coefficient. Estimation of the results shows a better improvement for this parameter on comparing the proposed system with that of the existing methods. The proposed traffic sign detection system involves the robust audio signal processing that increased the feature extraction and classification performances. The suggested solution solves the obstacles faced by visually impaired peoples for independent mobility.
... In the last decade, numerous computing-based technologies have been investigated for automatic traffic sign detection, including traditional machine learning-related methods [1][2][3][4][5], and more recently deep learning methods [6][7][8][9][10][11]. The performance of the conventional machine learning-based traffic sign detection is highly dependent on the types of extracted traffic sign features such as shape and color for classification [2,3]. ...
Article
Full-text available
Traffic sign detection (TSD) using convolutional neural networks (CNN) is promising and intriguing for autonomous driving. Especially, with sophisticated large-scale CNN models, TSD can be performed with high accuracy. However, the conventional CNN models suffer the drawbacks of being time-consuming and resource-hungry, which limit their application and deployments in various platforms of limited resources. In this paper, we propose a novel real-time traffic sign detection system with a lightweight backbone network named Depth Separable DetNet (DS-DetNet) and a lite fusion feature pyramid network (LFFPN) for efficient feature fusion. The new model can achieve a performance trade-off between speed and accuracy using a depthwise separable bottleneck block, a lite fusion module, and an improved SSD detection front-end. The testing results on the MS COCO and the GTSDB datasets reveal that 23.1% mAP with 6.39 M parameters and only 1.08B FLOPs on MSCOCO, 81.35% mAP with 5.78 M parameters on GTSDB. With our model, the run speed is 61 frames per second (fps) on GTX 1080ti, 12 fps on Nvidia Jetson Nano and 16 fps on Nvidia Jetson Xavier NX.
... However, it is hard to find adaptive thresholding like [9] algorithm which helps to introduce no distortion in any sign sample due to lack of information of the original level of intensity. In order to remove feature vectors, the pixels that cover the area of the hand configuration are calculated with the mask, as described in [8] . The problem which is under consideration is that a multi-class one with samples that do not belong to any sign. ...
Article
Full-text available
Social correspondence is one of the most significant columns that the public dependent on. Notably, language is the best way to communicate and associate with one another both verbally and nonverbally. There is a persistent communication gap among deaf and non-deaf communities because non-deaf people have less understanding of sign languages. Every region/country has its sign language. In Pakistan, the sign language of Urdu is a visual gesture language that is being used for communication among deaf peoples. However, the dataset of Pakistan Sign Language (PSL) is not available publicly. The dataset of PSL has been generated by acquiring images of different hand configurations through a webcam. In this work, 40 images of each hand configuration with multiple orientations have been captured. In addition, we developed, an interactive android mobile application based on machine learning that minimized the communication barrier between the deaf and non-deaf communities by using the PSL dataset. The android application recognizes the Urdu alphabet from input hand configuration.
... Semantic segmentation is a classification task in which patches of images are grouped together which belong to the same object class [259]. Various applications of semantic segmentation are explored in [260][261][262][263][264][265][266]. ...
Article
Since the dawn of Humanity, to communicate both abstract and concrete ideas, visualization through visual imagery has been an effective way. With the advancement of scientific technologies, vision has been imparted to machines like humans do. Computer vision give ability to machines, to receive and analyze visual data on its own, and then make decisions about it, hence computer vision is more than machine learning applied. So, visualization of computer models to learn without being explicitly programmed using machine learning algorithms is called Visual learning. This work aims to review the state-of-the-art in computer vision by highlighting the contributions, challenges and applications. We first provide an overview of important visual learning approaches and their recent developments, and then describes their applications in diverse vision tasks, such as image classification, object detection, object recognition, visual saliency detection, semantic and instance segmentation, human pose estimation and image retrieval. Hardware constraints are also highlighted for better understanding of model selection. Finally, some important challenges, trends and outlooks are also discussed for better design and training of learning modules, along with several directions that may be further explored in the future.
... To minimize the effect of environment on the test images different color and shape-based approaches are used [12]. The most popular color-based detection methods are HSV Transformation [9]and YCb-Cr color space transform [6]. Due to illumination and weather change the color information can be unreliable. ...
Conference Paper
Autonomous cars must take real-time decisions about surroundings to reduce death rates during traffic accidents. Traffic related information's are available in the Traffic sign. It assists to drive better and safer. In the traditional method, the traffic sign is detected by manually or computer vision methods. Those are time consuming processes.No-one will be aware of all traffic signs, so it will let everyone know and learn the signs easily. Traffic sign recognition is just one of the problems that computer vision and deep learning can solve Convolution Neural Network (CNN) architecture.Machinesareable-to identify traffic signs from the German Traffic Signal Recognition Benchmark (GTSRB) dataset that contains forty-three classes. The proposed system has three working stages: image pre-processing, detection, and recognition. Initially, the traffic sign image is pre-processed, and the detailed information present in the traffic sign image is detected by using the histogram equalization method, which improves the contrast of the traffic sign image. After preprocessing, the features of the images are extracted by using CNN architecture with three non-linear activation functions such as Re-Lu, Leaky-Re-Lu and sigmoid. The experimental results compare the results of the above three non-linear activation functions.The activation function Re-Lu and Leaky Re-Lu achieved accuracy above 95%.After feature extraction, the output layer is used to predict the traffic sign images.
... The shape of regulatory signs is limited to either a circular, triangular, or equilateral polygon and their color consists of mainly primary colors such as red or yellow. Therefore, in the beginning there were detection methods based on color segmentation [41][42][43][44], and shape detection [45,46]. Recently, most methods are based on machine learning algorithms [47][48][49][50][51][52][53]. ...
Article
Full-text available
This paper proposes a map-matching based precise positioning system for highways. The proposed system fuses multiple sensors such as a camera, conventional GPS, and vehicle motion sensors widely adapted in mass-produced vehicles. While conventional map-matching based systems consume a large amount of computation because of the landmark detection, the proposed system achieves both precise positioning and small computations by introducing a detection-verification-cascade strategy: the proposed system generates vehicle position candidates per lane by utilizing lane endpoints detectable with small computation, and then it selects the candidate on an ego-lane by verifying the existence of road signs within the region of interest generated by a map and each candidate. To reduce the computation further, the proposed system adapts an extended Kalman filter (EKF) as a positioning filter instead of a particle filter. As the noise distribution of the EKF is limited to a unimodal, the proposed system maintains multiple EKFs to track the candidates for each lane until the ego lane is identified. After ego-lane identification, the system leaves only one EKF to track the candidate on an ego-lane. The proposed system achieves 0.17m average positioning error in highway situations and it takes about 53ms to process the whole procedure from landmark detection to position estimation in low-end hardware equipped with a cortex-A9 CPU whose clock is 1.0 GHz.
... Furthermore, there is a task that identifies and separates joints called sample segmentation [2,3]. Medical image semantic segmentation has a variety of applications, such as road sign detection [4], colon crypt segmentation [5], landuse classification, and land surface classification [6]. It is also widely used in medicine, such as brain and tumor detection [7] and discovering and tracking medical devices in surgery [8]. ...
Article
Full-text available
Image medical semantic segmentation has been employed in various areas, including medical imaging, computer vision, and intelligent transportation. In this study, the method of semantic segmenting images is split into two sections: the method of the deep neural network and previous traditional method. The traditional method and the published dataset for segmentation are reviewed in the first step. The presented aspects, including all-convolution network, sampling methods, FCN connector with CRF methods, extended convolutional neural network methods, improvements in network structure, pyramid methods, multistage and multifeature methods, supervised methods, semiregulatory methods, and nonregulatory methods, are then thoroughly explored in current methods based on the deep neural network. Finally, a general conclusion on the use of developed advances based on deep neural network concepts in semantic segmentation is presented.
... As a typical pattern recognition task, the accuracy of the road signs perception mainly depends on the feature extractor and the classifier [137]. In the beginning, ML approaches, like SVM [76,77] and RF [78], were used as classifiers with hand-crafted features. These ML approaches are still insufficient to deal with the not typical (or regular or conforming) images. ...
Article
Full-text available
Intelligent Transportation Systems, or ITS for short, includes a variety of services and applications such as road traffic management, traveler information systems, public transit system management, and autonomous vehicles, to name a few. ITS are expected to be an integral part of urban planning and future smart cities, contributing to improved road and traffic safety, transportation and transit efficiency, as well as to increased energy efficiency and reduced environmental pollution. On the other hand, ITS pose a variety of challenges due to its scalability and diverse quality-of-service needs, as well as the massive amounts of data it will generate. In this survey, we explore the use of Machine Learning (ML), which has recently gained significant traction, to enable ITS. We provide a thorough survey of the current state-of-the-art of how ML technology has been applied to a broad range of ITS applications and services, such as cooperative driving and road hazard warning, and identify future directions for how ITS can further use and benefit from ML technology.
... There are several complex methods that had been suggested for identifying ROI. Recognizing the object or region of interest (ROI) in a realistic view is difficult since the content of raw images comprises several non-uniform sub-regions and the severity of inhomogeneities [20]. Figure 6b shows the ROI detection from the OpenCV library ...
Article
Full-text available
Everyday humans use cars to move faster, and the world is a chaotic place, and a little distraction or a mistake could be the reason for an accident and bring people great pain. An assistance system that can distinguish and detect signs on the roads and brings the driver's attention to road signs and make them aware of their meaning could be beneficial. The most important part of the Traffic Sign Recognition System is the algorithm. In this paper, a new way toward Traffic Sign Recognition algorithm taking the advantages of Color Segmentation, support vector machines, and histograms of oriented gradients on the GTSRB dataset is proposed. The unsupervised shuffled frog-leaping algorithm is employed for segmenting the images. The results show remarkable improvements by using meta-heuristic algorithms.
... Among them, GLCM, LBP, and gradient cooccurrence matrix methods are classical texture feature extraction algorithms, but their performance is not ideal due to the imaging of ultrasound images, and they do not detect changes in texture features well. e methods in [17,18] are methods that have been applied in recent years to extract texture features in medical images. From the results, it can be seen that its performance is better compared to the traditional methods. ...
Article
Full-text available
The cultural tourism industry combines the similarities of the cultural industry and tourism industry, which can be the most effective way to meet people's spiritual and cultural needs as well as their leisure needs simultaneously, and has a vast development potential. However, there are numerous and dispersed areas where the market value of cultural tourism resources is clustered, and frequently, each city has the clustering area with the highest market value concentration of cultural tourism resources. This feature of the spatial distribution of the market value of cultural tourism resources is significant for promoting the development of cultural tourism as a whole and constructing the industry's overall structure. It has broad application potential for extracting and differentiating cultural tourism industry characteristics. Texture feature extraction is typically performed using dual-tree complex wavelet transform (DT-CWT) and Gabor wavelet. In this paper, we propose a multiscale DT-CWT and Gabor-based method for identifying the cultural tourism industry. The method first decomposes the images of cultural tourism into multiscale space using a Gaussian pyramid, then extracts the multiscale features of the images using DT-CWT and Gabor, and lastly achieves feature fusion. Using a support vector machine (SVM) classifier to achieve classification, the effectiveness of the feature extraction method is determined. The experimental findings demonstrate that the method proposed in this paper can achieve a high rate of recognition.
... For instance, autonomous driving applications require vehicles to quickly and accurately recognize traffic signs on the roadside. The authors in [20] proposed a road sign intelligence system based on Support Vector Machines (SVMs) for detecting and recognizing road signs. The authors in [21] adopted a Region-Convolutional Neural Network (R-CNN) approach for large-scale detection and recognition of traffic signs through automatic end-to-end learning. ...
Preprint
With the gradual popularization of self-driving, it is becoming increasingly important for vehicles to smartly make the right driving decisions and autonomously obey traffic rules by correctly recognizing traffic signs. However, for machine learning-based traffic sign recognition on the Internet of Vehicles (IoV), a large amount of traffic sign data from distributed vehicles is needed to be gathered in a centralized server for model training, which brings serious privacy leakage risk because of traffic sign data containing lots of location privacy information. To address this issue, we first exploit privacy-preserving federated learning to perform collaborative training for accurate recognition models without sharing raw traffic sign data. Nevertheless, due to the limited computing and energy resources of most devices, it is hard for vehicles to continuously undertake complex artificial intelligence tasks. Therefore, we introduce powerful Spike Neural Networks (SNNs) into traffic sign recognition for energy-efficient and fast model training, which is the next generation of neural networks and is practical and well-fitted to IoV scenarios. Furthermore, we design a novel encoding scheme for SNNs based on neuron receptive fields to extract information from the pixel and spatial dimensions of traffic signs to achieve high-accuracy training. Numerical results indicate that the proposed federated SNN outperforms traditional federated convolutional neural networks in terms of accuracy, noise immunity, and energy efficiency as well.
... Typically, traffic signs, lights, and road markings are designed to be easily distinguished from their surroundings [10]. ...
Article
Full-text available
High-definition (HD) maps determine the location of the vehicle under limited visibility based on the location information of safety signs detected by sensors. If a safety sign disappears or changes, incorrect information may be obtained. Thus, map data must be updated daily to prevent accidents. This study proposes a map update system (MUS) framework that maps objects detected by a road map detection system and the object present in the HD map. Based on traffic safety signs notified by the Korean National Police Agency, 151 types of objects, including traffic signs, traffic lights, and road markings, were annotated manually and semi-automatically. Approximately 3,000,000 annotations were trained based on the you only look once (YOLO) model, suitable for real-time detection by grouping safety signs with similar properties. The object coordinates were then extracted from the mobile mapping system point cloud, and the detection location accuracy was verified by comparing and evaluating the center point of the object detected in the MUS. The performance of the groups with and without specified properties was compared and their effectiveness was verified based on the dataset configuration. A model trained with a Korean road traffic dataset on our testbed achieved a group model of 95% mAP and no group model of 70.9% mAP.
... The characteristics of the classical works is to find efficient handcrafted image features, such as color, shape, and edge features [12,28,36,37]. Berkaya et al. [2] utilized multi-features with an SVM classifier to detect traffic signs. ...
Article
Full-text available
As a crucial sub-task for autonomous driving and intelligent transportation, traffic sign detection attracts a lot of researchers’ attention. Although prior works have achieved promising results, it still suffers from two problems, the need of massive labeled data and the slow inference speed on high-resolution images, which are also the common problem of generic object detection. Motivated by these problems, we propose a lightweight traffic sign detection model and employ an automatic data labeling pipeline. The detection model utilizes a cascaded structure that consists of a localization module and a recognition module, achieving quite fast speed on GPU and edge devices. We discard all complex structures and operations to boost the speed of the model, making it easy for deployment. Then, we propose a two-stage automatic data labeling pipeline to reduce the cost of data labeling work. With only traffic sign template images, a synthetic dataset is constructed for generating initial pseudo labels in the first stage. In the second stage, we propose to use an image pretext model to refine the initial labels. The accuracy of the final pseudo labels is nearly 100%. We test the proposed method on TT-100K, GTSDB, and GTSRB datasets, and the results show that the model trained with the pseudo labels only has a negligible accuracy loss compared with the model trained by real labels. The proposed model’s calculation latency is around 1 ms on GPU, and the accuracy is still on par with the state-of-the-art models.
... Given the importance of this task, the detection of traffic sign is an active field of research in the artificial intelligence community. In the early days, a diverse set of solutions have been proposed, ranging from the ones using model-based systems with handcrafted features [3] to the ones exploiting learning-based methods with simple features [26]. Nowadays, the state-of-the-art has been using deep learning-based methods [1,48], which is also the focus of this study. ...
Article
Full-text available
Deep learning has become a standard approach to machine vision in recent years. Despite several advances, it requires large amounts of annotated data. Nonetheless, in many applications, large-scale data acquisition and annotation is expensive and data imbalance is an intrinsic problem. To address these challenges, we propose a novel synthetic database generation method that only requires (i) arbitrary natural images, i.e., does not demand real images from the target domain, and (ii) templates of the traffic signs. Our method does not aim at overcoming the training with real data but to be a compatible option when there is a lack of real data. Results with data of multiple countries show that the synthetic database generated without human effort is effective for training a deep traffic sign detector. On large datasets, training with a fully synthetic dataset almost matches the performance of training with a real one. When compared to training with a smaller dataset of real images, training with synthetic images increased the accuracy by 12.25%. The proposed method also improves the performance of the detector when target-domain data are available.
Article
Traffic sign detection has made great progress with the rise of deep learning in recent years. As a result of the complex and changeable traffic environment, detecting small traffic signs in a real-world scene is still a challenging problem. In this paper, a novel group multi-scale attention pyramid network is proposed to address the problem. Specifically, to aggregate the feature at different scales and suppress the messy information in the background, an effective multi-scale attention module is proposed. Furthermore, a feature fusion module, named adaptive pyramid convolution, is further designed, which can drive the network to learn the optimal feature fusion pattern and construct an informative feature pyramid for detecting traffic signs in different sizes. Extensive experimental results on the public traffic sign detection datasets demonstrate the effectiveness and superiority of the proposed method when compared with several state-of-the-art methods.
Article
Wayfinding signs must have prominent visibility in the environment to provide people with the information they need to navigate to an intended destination. Therefore, designers seek to design signs that are clearly visible while esthetically harmonious with the ambient environment. To find highly conspicuous colors, 1632 images showing combinations of various colors and environments were created. And then evaluation factors such as sign conspicuity, color contrast, color harmony, sign color preference, and expected sign usability were tested with 33 designers. Through statistical analysis of experimental results, the colors with high conspicuity and harmony were organized in different types of environments, and all correlations between the evaluation factors were analyzed. The results showed that all evaluation factors had a significantly high correlation except color harmony. Based on these results, we conclude that when designing a sign, it should focus on increasing the sign conspicuity by using color difference rather than color harmony between the sign and the ambient environment.
Article
Full-text available
Recently, intelligent transportation systems (ITS) attracts more and more attention for its wide applications. Traffic sign detection and recognition (TSDR) system is an essential task of ITS. It enhances the safety by informing the drivers about the current state of traffic signs and offering valuable information about precautions. This paper reviews the popular traffic sign detection methods (TSD) prevalent in recent literature. The methods are divided into color-based, shape-based, and machine learning based ones. Color space, segmentation method, features, and shape detection method are the terms considered in the review of the detection module. The paper presents a comparison between these methods. Furthermore, a list of publicly available data sets and a discussion on possible future works are provided.
Chapter
The Traffic Sign Recognition (TSR) system is an essential part of traffic management, being support system for driver and intelligent vehicle. Traffic Sign Detection (TSD) is the prerequisite step of the automatic TSR system. This paper proposes a TSD framework by exploring fuzzy image processing and invariant geometric moments. In this framework, fuzzy inference system is applied to convert the HSV image into gray tone. Then, statistical threshold is used for the segmentation. After shape verification of every connected component using Hu moments and Quadratic Discriminant Analysis (QDA) model, the candidate signs are detected by referencing bounding box parameters. The framework is simulated in different complex scenarios of day and night mode. Experimental result shows that its performance is satisfactory and recommendable with the state of the art research. The proposed framework yields 94.86% F-measure in case of Bangladesh road signs and 93.01% in German road signs.
Chapter
The objective of this chapter is to detect and recognize the traffic signs using two methods. Traffic signs are silent speakers to the road users. It gives advance information about road conditions ahead. In this chapter, an effective method is proposed to identify the traffic signs in an automated manner. The traffic sign recognition for advanced driver-assistance system of Method I comprises of three stages: (1) detection using Haar-like features with AdaBoost method, (2) feature extraction using speeded up robust features, and (3) classification using support vector machine. Method II comprises of two stages: (1) detection of traffic sign using convolution neural network and (2) classification using convolution neural network. The performance of the system is analyzed. From the experimental results, it is observed that the Method II using convolution neural network gives a better performance compared with Method I
Article
This paper presents a polytopic linear parameter varying (LPV) controller design methodology for nonlinear anti-vibration suspension based on stochastic road identification. As the main nonlinear component in semi-active suspension, the continuous damping control (CDC) damper value is mapped by damping velocity and control current through a bench experiment. linear parameter varying gain-scheduled method is applied for dealing with the nonlinear deterministic model. As the uncertainties caused by road stochastic excitation to suspension cannot be ignored, it is essential to identify the excitation signal as a measurable variable parameter. Empirical-mode-decomposition (EMD) and support vector machine (SVM) are utilized to identify and classify the road stochastic signal input into standard level, so polytopic LPV controller can be designed with CDC damper velocity, control current, and road level as variable parameters. Meanwhile the exponential increase of high-order plant elements in multi-parameter LPV model makes it too complicated for real-time computing. Hence the tensor product model transformation is utilized to identify the orthogonal basis of system plant so that low-rank approximation of it can be implemented by truncated high-order singular value decomposition (T−HOSVD) method. Real-time simulation shows the performance and feasibility of controller, which can efficaciously identify the road stochastic excitation as measurable variable parameters for nonlinear suspension polytopic LPV control.
Chapter
Dental healthcare providers need to examine a large number of panoramic X-ray images every day. It is quite time consuming, tedious, and error-prone job. The examination quality is also directly related to the experience and the personal factors, i.e., stress, fatigue, etc., of the dental care providers. To assist them handling this problem, a residual network-based deep learning technique, i.e., faster R-CNN technique, is proposed in this study. Two kinds of residual networks, i.e., ResNet-50 and ResNet-101, are used as the base network of faster R-CNN separately. A modified version of Palmer notation (PN) system is proposed in this research for numbering the teeth. The modified Palmer notation (MPN) system does not use any notation like PN system. The MPN system is proposed for mainly three reasons: (i) teeth are divided into total eight categories, and to keep this similarity, a new numbering system is proposed that has the same number of category, (ii) 8-category MPN system is less complex to implement than 32-category universal tooth numbering (UTN) system, and with some preprocessing steps, MPN system can be converted into 32-category UTN system, and finally (iii) for the convenience of the dentist, i.e., it is more feasible to utilize 8-category MPN system than 32-category UTN system. Total 900 dental X-ray images were used as training data, while 100 images were used as test data. The method achieved 0.963 and 0.965 mean average precision (mAP) for ResNet-50 and ResNet-101, respectively. The obtained results demonstrate the effectiveness of the proposed method and satisfy the condition of clinical implementation. Therefore, the method can be considered as a useful and reliable tool to assist the dental care providers in dentistry.
Chapter
Recent decade witnesses the advancement in computing hardware and power. This advancement is the result of machine learning and artificial intelligence. One of the important advancements took place in development of autonomous cars. It was one of the long-standing points of research for scientists. The crucial components of any autonomous car are being able to recognize traffic signs. Traffic signs serve as the non-verbal communication channel on the road. Thus, the task of recognizing traffic signs is a challenge. Moreover, due to the rise in number of drivers on the road, the need for a system that can automatically recognize traffic signs has increased. As a result, this manuscript aims at solving the challenges of traffic sign recognition.
Chapter
In robot vision, it is generally required to recognize indoor objects with special colors and shapes quickly and accurately. However, the existing object detection algorithms based on deep learning generally require powerful computing resources, which leads to high power consumption and high cost of robot system. To solve this problem, this paper proposes a low power consumption, low cost indoor object detection algorithm based on color and shape information, with real-time and high accuracy. However, the performance of the indoor object detection with color and shape informations has a substantial decline under the complex indoor conditions. In this paper, we propose a faster and more robust algorithm. Our hierarchical detection scheme is composed of color thresholding and two-stage Maximally Stable Extremal Regions (MSER). Because of the complementarity of the two used algorithms, our method get the state-of-the-art in the field of object detection under the complex indoor conditions. Experiments are performed on our collected indoor object detection sets to validate the effectiveness of the proposed approach, and the experimental results show that this algorithm is superior to other color object detection algorithms.
Article
Traffic sign is the key aspect in road and also for the autonomous car. Detection and classification of these sign plays a vital role for the invention of driverless vehicles. Convolutional neural network (CNN) has the ability to learn local features using series of convolutional and pooling layer observing the image sequences. In this work, traffic sign detection and classification has been performed based on deep learning approach. The experiment conducted on Germen Traffic Sign Detection Benchmark (GTSDB) and Recognition Benchmark (GTSRB) for detection and recognition. For traffic sign detection a two-stage detector, Faster R-CNN with ResNet 50 backbone structure is used where the CNN layers extracted the features of traffic signs from the images and the region proposal network (RPN) filter the object from the image to create bounding box based on the extracted feature map. The classification network classifies the traffic signs and predict the proposal confidence score. A general deep learning model is transferred into a specific output with weights with transfer learning by tuning the pretrained model based on COCO image dataset. The performance is compared with ResNet 152, MobileNet v3 and RetinaNet based on the confidence score and mean average precision (mAP). Faster R-CNN with ResNet-50 shows better detection performance comparing with other backbone structure. In addition, a series of convolution layer with batch normalization followed by max pooling layer is used to build a classifier and softmax is used in the output for 43 class classification and 97.89% test accuracy has been obtained.
Article
Full-text available
Traffic sign recognition (TSR) is the basic technology of the Advanced Driving Assistance System (ADAS) and intelligent automobile, whileas high-qualified feature vector plays a key role in TSR. Therefore, the feature extraction of TSR has become an active research in the fields of computer vision and intelligent automobiles. Although deep learning features have made a breakthrough in image classification, it is difficult to apply to TSR because of its large scale of training dataset and high space-time complexity of model training. Considering visual characteristics of traffic signs and external factors such as weather, light, and blur in real scenes, an efficient method to extract high-qualified image features is proposed. As a result, the lower-dimension feature can accurately depict the visual feature of TSR due to powerful descriptive and discriminative ability. In addition, benefiting from a simple feature extraction method and lower time cost, our method is suitable to recognize traffic signs online in real-world applications scenarios. Extensive quantitative experimental results demonstrate the effectiveness and efficiency of our method.
Article
With the rapid development of intelligent transportation, more and more vehicles are equipped with intelligent traffic sign recognition systems, which can reduce the potential safety hazards caused by human cognitive errors. Therefore, a more safe and reliable traffic sign recognition system is the demand of drivers, and it is also the research hotspot of current automobile manufacturers. However, the pictures taken by the actual driving car are inevitably distorted and blurred. In addition, there are external uncontrollable factors, such as the impact of bad weather, which make the research of traffic sign recognition system face many difficulties, and the practical application is far from mature. In order to solve the above challenges, this paper proposes a Yolo model for traffic sign recognition. Firstly, the traffic signs are roughly divided into several categories and then preprocessed according to the characteristics of various types of signs. The processed pictures are input into the optimized convolutional neural network to subdivide the categories to obtain the specific categories. Finally, the proposed recognition algorithm is tested with the data set based on the German traffic sign recognition standard and compared with other baseline algorithms. The results show that the algorithm greatly improves the running speed on the basis of ensuring a high classification accuracy and is more suitable for traffic sign recognition system.
Article
Image segmentation is a very important topic in the field of computer vision. We present a method for semantic segmentation of selected stuff classes from a superset of classes. We show that in situations where only select stuff classes are required if we group them as per a strategy then it can attain much higher accuracy than the models trained on the original dataset with all classes intact. The COCO-Stuff Dataset is used for demonstrating the aforesaid strategy. For training purposes, the DeepLabv3+ with Mobilenet-v2 architecture is used. We have achieved an 80.2 percent mean Intersection over Union (mIoU) on these selected classes. We also refine the masks using Learning/Computer Vision (CV) methods and hence obtain better visualization results as compared to the existing DeepLabv3+ results.
Article
Cautionary traffic signs are of immense significance to traffic safety. In this study, a robust and optimal real-time approach to recognize the Indian cautionary traffic signs (ICTS) is proposed. ICTS are all triangles with a white backdrop, a red border, and a black pattern. A dataset of 34,000 real-time images has been acquired under various environmental conditions and categorized into 40 distinct classes. Pre-processing techniques are used to transform RGB images to gray-scale images and enhance contrast in images for superior performance. To find the ICTS, an optimized adaptive boosting cascade classifier is used. To classify the specific category of signs found by the optimized adaptive boosting cascade classifier, an 11-layer CNN model is built. Finally, using computer vision methods, this model is tested in real-time. Evaluation metrics such as precision, recall, F1 measure, error rate, and mAP are expressed on the ICTS, GTSDB, LISA, STSD, and DITS-based datasets to evaluate the proposed method and compare the results of predictions with other investigations. When compared to other state-of-the-art objects detection models such as SSD, YOLOv3, and Faster RCNN, the proposed model outperformed them all, with a precision of 97.15%, recall rate of 96.74%, an error rate of 3.26%, f1-score of 96.94%, and mAP@0.5IoU of 95.6%. (Dataset available: 10.21227/yy4h-rc98)
Article
Full-text available
In this paper we present a novel and robust approach for detection, categorization and recognition of road signs. It is known that the standard road signs contain few and easily distinguishable colors, such as red for prohibition, yellow for warnings, green, blue and white. We use a Bayesian approach for detecting road signs in the captured images based on their color information. At the same time, the results of the Bayes classifier categorize the detected road sign according to its color content. The SIFT transform is employed in order to extract a set of invariant features for the detected road sign label(s). Recognition is done by matching the extracted features with previously stored features of standard signs. We illustrate the accuracy and robustness of this approach.
Article
Full-text available
The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.
Article
Full-text available
This paper presents an automatic road sign detection and recognition system that is based on a computational model of human visual recognition processing. Road signs are typically placed either by the roadside or above roads. They provide important information for guiding, warning, or regulating the behaviors drivers in order to make driving safer and easier. The proposed recognition system is motivated by human recognition processing. The system consists of three major components: sensory, perceptual, and conceptual analyzers. The sensory analyzer extracts the spatial and temporal information of interest from video sequences. The extracted information then serves as the input stimuli to a spatiotemporal attentional (STA) neural network in the perceptual analyzer. If stimulation continues, focuses of attention will be established in the neural network. Potential features of road signs are then extracted from the image areas corresponding to the focuses of attention. The extracted features are next fed into the conceptual analyzer. The conceptual analyzer is composed of two modules: a category module and an object module. The former uses a configurable adaptive resonance theory (CART) neural network to determine the category of the input stimuli, whereas the later uses a configurable heteroassociative memory (CHAM) neural network to recognize an object in the determined category of objects. The proposed computational model has been used to develop a system for automatically detecting and recognizing road signs from sequences of traffic images. The experimental results revealed both the feasibility of the proposed computational model and the robustness of the developed road sign detection system.
Conference Paper
Full-text available
In this paper we have developed a new algorithm of artificial vision oriented to traffic sign shape classification. The classification method basically consists of a series of comparison between the FFT of the signature of a blob and the FFT of the signatures of the reference shapes used in traffic signs. The two major steps of the process are: the segmentation according to the color and the identification of the geometry of the candidate blob using its signature. The most important advances are its robustness against rotation and deformation due to camera projections.
Conference Paper
Full-text available
This paper deals with the detection and classification of traffic signs in outdoor environments. The information provided by traffic signs on roads is very important for the safety of drivers. However, in these situations the illumination conditions can not be predicted, the position and the orientation of signs in the scene are not known and other objects can block the vision of them. For these reasons we have developed an extensive test set which includes all kind of signs. In an artificial vision system, the key to recognize traffic signs is how to detect them and identify their geometric shapes. So, in this work we propose a method that uses a technique based on support vector machines (SVMs) for the classification. The patterns generated by the vectors represent the distances to borders (DtB) of the objects candidate to be traffic signs. Experimental results show the effectiveness of the proposed method.
Conference Paper
Full-text available
A new method is presented for detecting triangular, square and octagonal road signs efficiently and robustly. The method uses the symmetric nature of these shapes, together with the pattern of edge orientations exhibited by equiangular polygons with a known number of sides, to establish possible shape centroid locations in the image. This approach is invariant to in-plane rotation and returns the location and size of the shape detected. Results on still images show a detection rate of over 95%. The method is efficient enough for real-time applications, such as on-board-vehicle sign detection.
Conference Paper
Full-text available
Algorithms for classifying road signs have a high computational cost per pixel processed. A promising approach to real-time sign detection is to reduce the number of pixels to be classified as being a particular sign to a minimum by some form of sign detection on the image using less time expensive algorithms. In this paper, we adapt the fast radial symmetry detector to the image stream from a camera mounted in a car eliminate almost all non-sign pixels from the image stream. We then are able to apply normalised cross-correlation to classify the signs. This method is suitable for circular signs only; we apply it to Australian speed signs in this paper. Our results show that it is robust to a broad range of lighting conditions. Also, as the method is fast, there is no need to make unrealistic-ally strict assumptions about image structure.
Conference Paper
Full-text available
This paper presents an active vision system for real-time traffic sign recognition. The system is composed of two cameras, one is equipped with a wide-angle lens and the other with a telephoto-lens, and a PC with an image processing board. The system first detects candidates for traffic signs in the wide-angle image using color; intensity, and shape information. For each candidate, the telephoto-camera is directed to its predicted position to capture the candidate in a larger size in the image. The recognition algorithm is designed by intensively using built-in-functions of an off-the-shelf image processing board to realize both easy implementation and fast recognition. The results of on-road real-time experiments show the feasibility of the system
Article
Full-text available
At the Australian National University's Intelligent Vehicle Project, we are developing subsystems for: driver fatigue or inattention detection; pedestrian spotting; blind-spot checking and merging assistance to validate whether sufficient clearance exists between cars; driver feedback for lane keeping; computer-augmented vision (that is, lane boundary or vehicle highlighting on a head-up display); traffic sign detection and recognition; and human factors research aids Systems that perform such supporting tasks are generally called driver assistance systems (DAS). We believe that implementing DAS could prevent similar accidents or at least reduce their severity.
Article
Full-text available
This paper deals with the extraction of part of the visual information presented in streets, roads, and motorways. This information, provided by either traffic or road signs and route-guidance signs, is extremely important for safe and successful driving. An automatic system that is capable of extracting and identifying these signs automatically would help human drivers enormously; navigation would be easier and would allow him or her to concentrate on driving the vehicle. The system would indicate to the driver the presence of a sign in advance, so that some incorrect human decisions could be avoided. A deformable model scheme allows us to include the knowledge used while designing the signs in the algorithm and is used for their detection and identification. Two techniques to find the minimum in the energy function are shown: simulated annealing and genetic algorithms. Some problems are addressed, such as uncontrolled lighting conditions; occlusions; and variations in shape, size, and color.
Article
Full-text available
A vision-based vehicle guidance system for road vehicles can have three main roles: (1) road detection; (2) obstacle detection; and (3) sign recognition. The first two have been studied for many years and with many good results, but traffic sign recognition is a less-studied field. Traffic signs provide drivers with very valuable information about the road, in order to make driving safer and easier. The authors think that traffic signs most play the same role for autonomous vehicles. They are designed to be easily recognized by human drivers mainly because their color and shapes are very different from natural environments. The algorithm described in this paper takes advantage of these features. It has two main parts. The first one, for the detection, uses color thresholding to segment the image and shape analysis to detect the signs. The second one, for the classification, uses a neural network. Some results from natural scenes are shown
Article
Full-text available
In a visual driver-assistance system, road-sign detection and tracking is one of the major tasks. This study describes an approach to detecting and tracking road signs appearing in complex traffic scenes. In the detection phase, two neural networks are developed to extract color and shape features of traffic signs from the input scenes images. Traffic signs are then located in the images based on the extracted features. This process is primarily conceptualized in terms of fuzzy-set discipline. In the tracking phase, traffic signs located in the previous phase are tracked through image sequences using a Kalman filter. The experimental results demonstrate that the proposed method performs well in both detecting and tracking road signs present in complex scenes and in various weather and illumination conditions.
Article
The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.
Article
This paper presents an automatic road sign detection and recognition system that is based on a computational model of human visual recognition processing. Road signs are typically placed either by the roadside or above roads. They provide important information for guiding, warning, or regulating the behaviors drivers in order to make driving safer and easier. The proposed recognition system is motivated by human recognition processing. The system consists of three major components: sensory, perceptual, and conceptual analyzers. The sensory analyzer extracts the spatial and temporal information of interest from video sequences. The extracted information then serves as the input stimuli to a spatiotemporal attentional (STA) neural network in the perceptual analyzer. If stimulation continues, focuses of attention will be established in the neural network. Potential features of road signs are then extracted from the image areas corresponding to the focuses of attention. The extracted features are next fed into the conceptual analyzer. The conceptual analyzer is composed of two modules: a category module and an object module. The former uses a configurable adaptive resonance theory (CART) neural network to determine the category of the input stimuli, whereas the later uses a configurable heteroassociative memory (CHAM) neural network to recognize an object in the determined category of objects. The proposed computational model has been used to develop a system for automatically detecting and recognizing road signs from sequences of traffic images. The experimental results revealed both the feasibility of the proposed computational model and the robustness of the developed road sign detection system.
Book
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
Article
This study describes a novel method for characterizing the colorimetric and photometric properties of three-channel color imaging devices. The method is designed to overcome some undocumented aspects of the imager-characterization problem: The effective spectral sensitivity profiles of the imager's color channels depend on the level of radiant input energy, and these profiles must be known in order to determine the true intensity-response characteristics of the three channels. By fitting the response distributions of the three color channels explicitly with low-dimensional models, the method takes these dependencies into account, and may, therefore, offer several advantages over other imager-characterization methodologies, particularly where illuminant-independent characterization is required. An application of the technique is detailed, in which a CCD camera is characterized using only the Macbeth ColorChecker and a number of artificial illuminants. © 2001 John Wiley & Sons, Inc. Col Res Appl, 26, 442–449, 2001
Article
LIBSVM is a library for support vector machines (SVM). Its goal is to help users to easily use SVM as a tool. In this document, we present all its imple-mentation details. For the use of LIBSVM, the README file included in the package and the LIBSVM FAQ provide the information.
Article
This paper describes an automatic road sign recognition system by using matching pursuit (MP) filters. The system consists of two phases. In the detection phase, it finds the relative position of road sign in the original distant image by using a priori knowledge, shape and color information and captures a closer view image. Then it extracts the road sign image from the closer view image by using conventional template-matching. The recognition phase consists of two processes: training and testing. The training process finds a set of best MP filter bases for each road sign. The testing process projects the input unknown road sign to different set of MP filter bases (corresponding to different road signs) to find the best match. 2001 Elsevier Science B.V. All rights reserved.
Article
This paper deals with object recognition in outdoor environments. In this type of environments, lighting conditions cannot be controlled and predicted, objects can be partially occluded, and their position and orientation is not known a priori. The chosen type of objects is traffic or road signs, due to their usefulness for sign maintenance, inventory in highways and cities, Driver Support Systems and Intelligent Autonomous Vehicles. A genetic algorithm is used for the detection step, allowing an invariance localisation to changes in position, scale, rotation, weather conditions, partial occlusion, and the presence of other objects of the same colour. A neural network achieves the classification. The global system not only recognises the traffic sign but also provides information about its condition or state.
Article
From the publisher: This is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. SVMs deliver state-of-the-art performance in real-world applications such as text categorisation, hand-written character recognition, image classification, biosequences analysis, etc., and are now established as one of the standard tools for machine learning and data mining. Students will find the book both stimulating and accessible, while practitioners will be guided smoothly through the material required for a good grasp of the theory and its applications. The concepts are introduced gradually in accessible and self-contained stages, while the presentation is rigorous and thorough. Pointers to relevant literature and web sites containing software ensure that it forms an ideal starting point for further study. Equally, the book and its associated web site will guide practitioners to updated literature, new applications, and on-line software.
Conference Paper
In this paper, a new technology based on the genetic algorithm and new image filter for recognizing road traffic sign from motion image captured by a CCD camera in a car was developed. In order to realize a real-time position recognition, the step genetic algorithm with search region limits and image filter (denoted by SVF) were proposed. The genetic algorithm with search region limits was employed to detect the position and size of traffic signs in real-time as well as SVF was employed to extract specified colors. The feasibility and validity of the proposed scheme are demonstrated through road driving experiments.
Conference Paper
An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds
Conference Paper
The work presented is part of a CEC funded R&D Esprit project called AUTOCAT (Automatic Collection of road Attributes). The aim of the project was to demonstrate the automatic detection and localisation of traffic signs along the roads, in order to trigger the selective acquisition of a high resolution digital picture for sign inventory purpose. A Kalman tracker has been implemented to localise the signs (in 3D) from their apparent motion within the field of view of a wide angle camera, taking into account the camera motion. The whole image processing chain has been tuned on synthetic road-scene image sequences of a custom virtual circuit then validated in the field on the real vehicle demonstrator using the motion information of the embedded inertial navigation system. The results obtained show the successful coupling of an inertial navigation system, providing an accurate camera motion information, with a real time image processing system performing multi-target detection and tracking. It is shown how this application takes advantage of the Kalman filter implementation as an optimal estimator in the case of a moving camera analysing a static scene
Conference Paper
This research is concerned with a new technology for the recognition of traffic signs from a scene image using genetic algorithms (GA) and neural networks (NN). First, an original video image is obtained as a binary image by using the smoothing filter and the Laplacian filter. Second, the traffic sign in a scene image is searched using a proposed GA. Third, if there exist two traffic signs in a video image, the method to detect the second traffic sign is developed based on the fitness of individuals. Finally, it can be recognized using NN what kind of traffic sign is taken. Through these recognition experiments, it can be shown that the new technology proposed here is sufficiently valid for the recognition of a traffic sign from a scene image
Conference Paper
A visual control system for an unmanned vehicle is developed. The system uses dynamic image processing and fuzzy logic control. It quickly recognizes markers along a road and steers the vehicle. The markers are detected in real time by pipeline processing in the color identification processor and logical filter. The marker sequence is recognized by an improved Hough transform, then fuzzy theory decides the steering angle. To use the information on the movement of the vehicle, the authors constructed fuzzy inference rules on how position changes with time. The authors developed an LSI chip for the logical filter to make the system compact and practical (A4 size×10 cm). This system is mounted on a vehicle, and it steered the vehicle around a test track successfully
Article
A fast and robust framework for incrementally detecting text on road signs from video is presented in this paper. This new framework makes two main contributions. 1) The framework applies a divide-and-conquer strategy to decompose the original task into two subtasks, that is, the localization of road signs and the detection of text on the signs. The algorithms for the two subtasks are naturally incorporated into a unified framework through a feature-based tracking algorithm. 2) The framework provides a novel way to detect text from video by integrating two-dimensional (2-D) image features in each video frame (e.g., color, edges, texture) with the three-dimensional (3-D) geometric structure information of objects extracted from video sequence (such as the vertical plane property of road signs). The feasibility of the proposed framework has been evaluated using 22 video sequences captured from a moving vehicle. This new framework gives an overall text detection rate of 88.9% and a false hit rate of 9.2%. It can easily be applied to other tasks of text detection from video and potentially be embedded in a driver assistance system.
Article
In a road sign recognition task, many distortions of targets can occur at the same time. Scale invariance, tolerance to both in-plane and out-of-plane rotations and illumination invariance are examples of features that a road sign recognition system must possess. We propose a nonlinear correlator that performs several correlations between an input scene and different reference targets. Postprocessing of nonlinear correlation results permits attainment of a single output for the recognition system. The nonlinear filters provide invariance to. distortions of the target, noise robustness, and rejection of background noise. We combine a bank of nonlinear composite correlation filters to design a more versatile road sign recognition system. The bank of filters allows tolerance to changes in scale and tolerance to a certain degree of input-plane rotation. The synthesized nonlinear composite correlation filter permits tolerance to out-of-plane rotation of the target. The system is tested by analysis of real images, which include different distorted versions of stop signs. The processor can be designed for a variety of road signs in background scenes. The recognition results obtained for the proposed system show its robustness against the aforementioned distortions, any varying illumination conditions and partially occluded objects
Article
Proc. of the International Conference on Computer Vision, Corfu (Sept. 1999) An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest-neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low-residual least-squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially-occluded images with a computation time of under 2 seconds. 1.
An active vision system for real-time traffic sign recognitionTraffic signs localisation for highways inventory from a video camera on board a moving collection van
  • J Miura
  • T Kanda
  • Y Shirai
  • P Arnoul
  • M Viala
  • J Guerin
  • M Mergy
J. Miura, T. Kanda, and Y. Shirai, "An active vision system for real-time traffic sign recognition," in Proc. IEEE Intell. Transp. Syst., Oct. 2000, pp. 52–57. [7] P. Arnoul, M. Viala, J. Guerin, and M. Mergy, "Traffic signs localisation for highways inventory from a video camera on board a moving collection van," in Proc. IEEE Intell. Veh. Symp., Tokyo, Japan, Sep. 1996, pp. 141–146.