Automatic Number Plate Recognition Using Machine Learning

Full text

Automatic Number Plate Recognition Using
Machine Learning
A. M. Pujar(B)and Poornima B. Kulkarni
Department of CSE, Walchand Institute of Technology, Solapur, India
arkulkarni@witsolapur.org
Abstract. Automatic number plate recognition makes use of Image processing
and Machine learning technology to detect vehicle numbers irrespective of climatic
conditions. As the number of vehicles is increasing day by day, it’s necessary to
consider safety measures. It is used in various fields like traffic control, automatic
road tax collection at toll areas and vehicle parking systems in crowded areas. The
goal of the proposed system is to develop a capable automatic authorized vehicle
recognition system that makes use of the vehicle number plate. For detecting num-
ber plate, an infrared (IR) sensor is used, which helps in taking clear images from a
camera. Taking images of moving vehicles is the most important and difficult task.
Character segmentation is used to extract the vehicle number plate region from
an image by using R-CNN method. The Optical Character Recognition (OCR)
method is used for identifying the accurate character. The collected data is then
compared with the respective authority databases to investigate specific informa-
tion like vehicle owner, registration location, address and so on. If the vehicle
details match those in the database, then only gate barricade is opened. Through
this system, criminal activities can be minimized and road safety measures will
be considered. The system tries to promote the efficiency and accuracy of number
plate detection over climatic conditions.
Keywords: Machine Learning ·IR Sensors ·Character Segmentation ·
Character Recognition ·R-CNN ·OCR
1 Introduction
As per the statistics, the number of vehicles is increasing day by day, public safety
measures are taken into consideration. It is observed that the most of people violate
traffic rules, which lead to high road accidents and death rate also. Automatic Number
Plate Recognition (ANPR) is one of the favorable ways for controlling unfair activities
and takes public safety measures. The main goal of the proposed system is automatic
recognizing vehicle number plate from taken vehicle images.
In various fields, ANPR system can be applied like traffic control systems, automatic
toll tax collection at toll areas, vehicle parking system etc. This system can enhance
the accuracy and efficiency of identifying of the number plate over the environmental
conditions.
© The Author(s) 2023
S. Tamane et al. (Eds.): ICAMIDA 2022, ACSR 105, pp. 149–156, 2023.
https://doi.org/10.2991/978-94-6463-136-4_16

150 A. M. Pujar and P. B. Kulkarni
The vital part of system is taking images of moving vehicles and care to be taken that
none of the vehicles escaped from the camera. The vehicles’ movement is recognized by
the IR sensor. As soon as the vehicle is near to the camera, the IR sensor identifies the
vehicle and images are captured by the camera which is high resolution. Due to regular
climatic changes, it’s difficult to take clear images in rainy, dusty and cloudy circum-
stances. The core part the system is character segmentation and character recognition
[13]. By using the Machine learning method called region-convolution neural network
(R-CNN) classification of the number plate is done. R-CNN method is instructed before
detecting the alphabetic and numeric characters. The images taken by the camera are
converted into text format by the OCR algorithm. This process of conversion is called
as character segmentation. Character recognition is achieved by the OCR (Optical char-
acter recognition) algorithm. So that all characters of the number plate are recognized.
Once the vehicle number is detected, the vehicle details are matched with those in the
authorized database then vehicle related information is analyzed easily. By this mecha-
nism, if any traffic rules or criminal activities done by the vehicles then the respective
vehicles’ owner details can be identified using a number plate. So the system helps in
reducing the unfair activities and provides road safety measures for public [8].
The system can recognize the vehicle number and update to traffic authority if any
unfair work is done by the vehicle. From the number plate, all information about the
vehicle will be taken out for further investigation by the traffic authority. The proposed
can be used in most of the traffic areas for easy parking of vehicles, collecting toll
payment, traffic management system etc. [10].
2 Related Work
From so many years research is going on for detecting vehicle number plates. The
main job of the system was to reduce the road accident rate and to save public lives
by identifying the motorcyclists who were not wearing helmet during driving. Images
were processed with algorithms like the Circular Hough transform and Histogram of
oriented gradients descriptor [1] to know the details of the vehicle and it reported about
the presence of the helmet. The Circular Hough algorithm considers the top 1/5th of the
image and selects the circular region for predicting the presence of the helmet if the
driver did not wear a helmet, further actions would be taken on that person.
Another system was developed to monitor the car parking system efficiently using
ANPR camera to capture the images and it provided the parking location to the owner
by processing the images [5,12]. This System had given the idea of object detection
and also provided the parking details to the respective owners. As the car images are
taken by the ANPR camera, the number plate is extracted from the image. Through the
number plate respective car owner details are found so that system can update about
parking details of the car owner by locating the parking area. This system was mainly
used in a university campus, malls, hospitals, etc. This system was helpful for the public
to find their car easily as parking location was easily updated.
In one of the systems, CNN for classification and YOLO for object detection were
used but these methods have shown less accuracy and also had low performance [15]
YOLO was the first algorithm that was able to detect the object in real time. This
algorithm mainly works with three techniques:

Automatic Number Plate Recognition Using Machine Learning 151
1. Residual block 2. Bounding Box Regression 3. Intersection over Union
In the Residual block, the image was divided into many grid formats. Every grid had
the dimension of S * S, whatever image part is present in the grid would be detected.
The residual block contains a set of layers, where the output of one layer is input to next
layer. Each layer will be responsible for predicting the images of grid region.
Bounding box used to predict the image in the grid cell by comparing predicted
bounding box attributes and the actual bounding box region. A bounding box is a virtual
rectangle region that shows the point of reference to compare the similarity between the
actual and predicted objects. In the last step based on the overlapping of boxes result the
image prediction was carried out.
IoU is trained to fit the predicted object box within the real object box. If Intersection
over Union (IoU) value is 1, that implies the predicted box region is the same as the
actual box region. Through the IoU value model can detect the object from the image.
Even YOLO fails to detect objects which are very near because YOLO divides an
image into grids. Each grid proposes only two bounding boxes. Hence it gives error
localization during the detection of the number plate and YOLO is unable to detect
small objects because small objects will not be fixed in the grid cell. If object is not in
the grid cell it is impossible to detect the objects [3]. CNN method finds difficulty when
the image is rotated with a certain degree.
In another system, for detecting the number plate, an open-source computer vision
library called OpenCV was used with a machine learning library. It was able to detect
the vehicle images and the speed of the vehicles was also recorded. OpenCV sometimes
failed to detect the vehicle numbers due to seasonal variations [7].
CRF (Conditional random field) method was used in the earlier project to detect
the objects in the form of the probability distribution. It has constructed the spatial and
visual relationships among characters. So, it failed to achieve the maximum probability
to extract correct the characters on the number plate [11].
One of the systems was used to detect the license plate of Myanmar vehicles. These
number plates have Myanmar language numbers and characters. For character segmen-
tation, K-means and fuzzy K-means algorithms were used [9]. The system was able to
detect both Myanmar and normal English language number plates also.
3 Proposed System
Due to the rapid increase of transportation systems over the last few years, there is great
demand for effective monitoring and management of traffic rules to avoid accident and
death rates. In most areas people are unaware of traffic rules and in some places, the
traffic rules are not followed by the public. These two reasons lead to severe problems
in the future. Driving without a seat belt, not wearing a helmet and disobeying traffic
rules are the main causes of increasing the death rate as per the statistics. Real Time
Automatic Number Plate Recognition is a process where vehicles are detected or rec-
ognized using their number plate [9]. The proposed system uses image pre-processing
methodologies to convert digital images and vehicle number plate is extracted. A camera
is used to capture vehicle number plate images. This pixel-format images are converted
into readable characters by character recognition tools.

152 A. M. Pujar and P. B. Kulkarni
R-CNN and OCR methods are used for character segmentation and character recog-
nition purpose. R-CNN is trained to detect number plates from images and classifies
them as numerical and alphabetical characteristics [2]. This process is called as charac-
ter Segmentation. Through this system and collected data, it is easy to detect the vehicle
and its owner who is disobeying the traffic rules which can be forwarded to the traffic
police department for respective action to be taken on them. This creates awareness
among the public to follow traffic rules and to reduce accident rates or to minimize other
unfair activities.
This system can be used in the most areas such as vehicle tracking, traffic moni-
toring, road tax payment at highways, monitoring systems, toll collection points, and
parking management systems. The proposed system is to design Automatic number plate
detection using machine learning algorithms for more accuracy and better performance.
3.1 Objectives and Scope of the Proposed System
3.1.1 Objectives
This project will be designed to identify accurately number plates from the vehicle
image and to maintain the record of obtained characters of the number plate in the
database along with captured time. Machine Learning technology has suitable methods
for image processing. In the proposed system, R-CNN method is used for accurate
character classification. After the image is captured by the camera, R-CNN method
is trained to identify the alphabetic and numeric characteristics in the number plate
accurately which is called character segmentation.
OCR (Optical Character Recognition) is used for character recognition. Strings in
the digital images are converted to set of characters.
Finally, the number set is compared with the stored database to fetch the vehicle
details such as its owner, registration number, date of registration and past records. This
information is analyzed for further investigation and particular action to be taken against
vehicle owner.
3.1.2 Scope
The proposed system can be used by the traffic control system, tax payment at a toll,
parking system, etc. As the proposed system uses R-CNN method for character segmen-
tation and OCR method for character recognition, the accuracy of the system will be
enhanced. The system works for all types of vehicles.
4 Methodologies
The system works in the following steps
4.1 Vehicle Image Capturing
Capturing Vehicle image is vital part in this step. Vehicle images are captured using the
camera which is placed in a certain traffic location. Taking images looks simple but it is

Automatic Number Plate Recognition Using Machine Learning 153
a little hard to capture the moving vehicles’ images in reality. IR sensor is used to sense
the movement of vehicles near to camera which helps in capturing all vehicles’ images.
High-resolution camera is used to take clear images irrespective of manual conditions.
All parts of the vehicle are taken clearly in the image, especially the number plate. The
images are stored in the database and for analyzing purposes, images are to be processed.
The clarity of the image is to be good enough while the environmental conditions might
be rainy, dusty or clouded.
4.2 Extracting Number Plate
Number plate extraction ANPR does not classify the vehicle images as two-wheeler
or four wheelers, all types of vehicle images are taken by the camera. Characters are
extracted from captured vehicle images by following image pre-processing methods to
take out only the number plate from the images. The first step in image pre-processing
is cropping the images by removing the noisy content to remove unwanted background
parts from the image. Then colored RGB form of images is converted into gray-scale
images. After the pre-processing phase, the extracted number plate is stored in the
database for future processing usage.
4.3 Character Segmentation
Character segmentation R-CNN method is one of the favorable machine learning algo-
rithms used for the classification of characters in the number plate. The vehicle number
plate consists of numeric and alphabetical characters. R-CNN algorithm is used to clas-
sify the numerical and alphabetical characters easily. R-CNN method is trained before
to classify the number plate characters based on the labeled dataset so that it can eas-
ily classify the numeric and alphabetic characters. Nowadays, a standard font format is
followed for the vehicle number plates which make the classification process simple.
The first two characters in the number plate indicate respective the state code, and the
next two are numbers for the zonal code where the vehicle is registered. Classifying
these characters will increase the efficiency of the algorithm to get the accurate vehicle
number. From the segmentation process, accurate number plate characters are obtained
to analyze the vehicle details.
Figure 1explains the working of R-CNN algorithm. The vehicle number plate is
given as input to the system where each character in the number plate is fetched. The
characters are converted into binary format and it is given as input to R-CNN model.
The R-CNN is a supervised algorithm as it is trained to detect the numbers and alphabets
based on similarity with real values of characters. The classification of a character string
into particular alphabet or number by comparing features of real character, if the most
of features are matched then the algorithm predicts the character correctly. At the last
R-CNN gives recognized characters as the output.

154 A. M. Pujar and P. B. Kulkarni
Vehicle
number
plate
a
P
1001100…
R-
CNN
S S
2
1100111…
111010…
1010110…
a
P
2
Fig. 1. Workingof R-CNN (Source: https://www.semanticscholar.org/paper/Automatic-Number-
Plate-Detection).
OCR MH 12 DE1433
(Text format)
Fig. 2. Working of OCR (Source: https://www.egnyte.com/guides/governance/optical-character-
recognition)
4.4 Character Recognition
For character recognition process, OCR (Optical character recognition) method is used
that coverts strings in the image-to-text form [6]. The segmented characters are rec-
ognized and these characters are recorded for template matching to compare with the
existing template database.
Figure 2represents the working of OCR algorithm. It can automatically recognize
the characters in the image, scanned PDFs, handwritten notes and coverts into normal
text form. In the proposed system OCR takes the vehicle number plate and converts it
into text form. Already R-CNN algorithm classified characters as numerals and alpha-
bets [4]. Characters in the number plate are in the similar pattern which helps OCR to
recognize the characters correctly. All the recognized characters are stored in the tem-
plate database. Template matching characters are moved for accurate automatic number
plate recognition.
The Fig. 3represents the working of the entire ANPR system. For detecting the
number plate, the high resolution camera is used to capture the images of vehicles.
Image is given as input data to the system. This image goes under for pre-processing
phase. Here RGB images are converted into gray-scale and character binarization is done.
Extra noise content is omitted from the image to extract only the number plate out of the
taken images. This process is called as feature extraction. Later character segmentation
is done by the R-CNN algorithm which is one of the supervised classification methods
that is trained to classify the characters and numbers within the number plate. OCR

Automatic Number Plate Recognition Using Machine Learning 155
Fig. 3. Working of ANPR (Source: https://www.hindawi.com/journals/complexity/2021/559
7337)
algorithm is used for character detection that converts the image to text format. Once
the text form of the number plate is available then details regarding the vehicle number
will be analyzed further [14].
5 Conclusion
As per the previous studies, most of the models had less accuracy and less performance
for automatic vehicle number plate recognition. The proposed system uses high reso-
lution camera that helps to take clear images of vehicles irrespective of environmental
conditions. The system is capable to capture the vehicle images and image pre-processing
is done to extract the number plates. The accuracy of predicting the characters from num-
ber plate can be enhanced using machine learning algorithms: R-CNN and OCR. This
system would help to manage road safety issues and traffic rules to be followed by the
public as well illegal activities can be minimized by handing over the vehicle details to
traffic authorities for taking respective actions on the vehicles’ owner. In upcoming days
this system will be used in the most crowded traffic areas, for parking purpose and toll
tax collection, etc.
References
1. R. R. V. e. Silva, K. R. T. Aires and R. d. M. S. Veras, “Helmet Detection on Motorcyclists
Using Image Descriptors and Classifiers,” 2014 27th SIBGRAPI Conference on Graphics,
Patterns and Images, Rio de Janeiro, 2014, pp. 141-148.
2. S. Ren, K. He, R. Girshick et al., “Faster R-CNN: towards real-time object detection with
region proposal networks”, Int. Conf. on Neural Information Processing Systems, pp. 91-99,
2015.

156 A. M. Pujar and P. B. Kulkarni
3. J. Redmon, A. Farhadi, “YOLO9000: better faster stronger”, IEEE Conf. Computer Vision
and Pattern Recognition, pp. 6517-6525, 2017.
4. H. Li, C. Shen, “Reading car license plates using deep convolutional neural networks and
LSTMs”, 2016.
5. M. S. Mandi, B. Shibwabo, K. M. Raphael, “An automatic number plate recognition system
for car park management”, International Journal of Computer Applications, vol. 175, no. 7,
pp. 36-42, Oct. 2017.
6. L. Xie, T. Ahmad, L. Jin et al., “A new CNN-based method for multi-directional car license
plate detection”, IEEE Trans. Intell. Transp. Syst., vol. 19, no. 2, pp. 507-517, 2018.
7. Adrian Rosebrock, “Basic motion detection and tracking with Python and OpenCV”.
https://www.pyimagesearch.com/2015/05/25/basicmotion-detection-and-tracking-withpy
thon-And-opencv
8. Birgillito, G.; Rindone, C.; Vitetta, A. Passenger mobility in a discontinuous space: Modelling
access/egress to maritime barrier in a case study. J. Adv. Transp. 2018, 2018, 6518329.
9. Kyaw, N.N.; Sinha, G.; Mon, “K.L. License plate recognition of Myanmar vehicle number
plates a critical review”. In Proceedings of the 2018 IEEE 7th Global Conference on Consumer
Electronics (GCCE), Nara, Japan, 9–12 October 2018; pp. 771–774
10. Ferdowsi, A.; Challita, U.; Saad, W. Deep learning for reliable mobile edge analytics in
intelligent transportation systems: An overview. IEEE Veh. Technol. Mag. 2019, 14, 62–70.
11. L. Ladicky P. Sturgess, K. Alahari, C. Russell and P. H. S. Torr, “What, Where & how many?
Combining Object Detectors and CRFs”
12. Mohammed Y Aalsalem, Wazir Zada Khan, Khalid Mohammed Dhabbah, “An Automated
Vehicle Parking Monitoring and Management System Using ANPR Cameras” (2015) ISBN
978-89-968650-5-6
13. Zhongjie Mao.; Xi Chena.;Yucheng Wang.; Jia Yan.; “ROBUST TRACKING FOR MOTION
BLUR VIA CONTEXT ENHANCEMENT” 2021 IEEE | DOI: https://doi.org/10.1109/ICI
P42928.2021.9506594
14. Hossein Vahid Dastjerdi.; Vahid Rostami.; Farid Kheiri.; “Automatic License Plate Detection
System Based on the Point Weighting and Template Matching” (2015) IEEE 978-1-4673-
7485-9
15. Aman Kharwal; “Object Detection with Deep Learning using Yolo & Tensorflow” (2020)
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-
NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/),
which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any
medium or format, as long as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter’s Creative
Commons license, unless indicated otherwise in a credit line to the material. If material is not
included in the chapter’s Creative Commons license and your intended use is not permitted by
statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder.