Content uploaded by Wael Alqassas
Author content
All content in this area was uploaded by Wael Alqassas on Feb 23, 2021
Content may be subject to copyright.
World of Computer Science and Information Technology Journal (WCSIT)
ISSN: 2221-0741
Vol. 11, No. 1, 1-4, 2021
1
Recognition Impact on Rescaled Handwritten Digit
Images Using Support Vector Machine Classification
Wael Waheed AlQassas
Computer Science Department
Al Albayt university
Mafraq, Jordan
Mohammad Said El-Bashir
Computer Science Department
Al Albayt university
Mafraq, Jordan
Abstract— Handwritten Digit Recognition has been proposed using different techniques that were implemented over the available
datasets. Although existing systems reached high recognition accuracy, more efforts regarding speed and memory allocation is
required. In this research, we experiment the impact of image resolution reduction on recognition accuracy for handwritten digits. A
set of features were extracted, include histogram of pixels for horizontal, vertical, diagonal and inversed diagonal orientations.
Feature vector constructed by joining these features. Then, support vector machine is applied for classification. Rescaled
handwritten digit images were experimented against recognition accuracy, speed and memory. MNIST database of handwritten
digits is utilized for implementation. Results showed that the reduction of the size for the features vector due to image rescaling to
quarter of the original size had only about 1% accuracy degradation impact.
Keywords- Handwritten Digit Recognition; SVM; Histogram features; MNIST; Cubic Support Vector Machine and Recognition
Rate.
I. INTRODUCTION
In the last decades, the increasing demand of converting
massive amount of printed or handwritten documents into
digital form was the driver and motivator for character
recognition technology. This process was done in the past by
human operators which was an error prone and time-
consuming process.[1]
Handwritten Digit Recognition is known as the process of
transforming images of handwritten digits into numeric values
in a suitable format for the computer, for the purpose of
editing, searching, processing and a minimizing memory
storage size. In general, handwritten recognition systems has
effectiveness and importance in many fields such as the new
era of Online education due to COVID-19, processing bank
check amount automatically, mail zip codes recognition for
postal mail sorting purposes, electronic data indexing,
handwriting recognition on mobiles, numeric entries in the
form filled by hand and so on.
The handwritten recognition system can be divided into two
main domains on the basis of input method; online and offline,
each has its suitable processing technique. Online handwritten
recognition requires writing tool trail capturing instrument
(digital pen). Nowadays smart mobiles, smart boards,
electronic pad (digitizers) and digital personal assistant, have
imbedded online handwritten recognition system. Hence it is
important to improve and optimize the performance of the
recognition system, aiming for the improving the
processing speed and reducing the storage space.
The static representation of a digitized document is used in
the offline system of digit recognition, example of which are
check form, mail or document processing. Contrary to offline,
in online system depends on the information acquired during
the production of the handwriting [2],[3],[4].
Offline handwritten recognition process usually passes
through many phases, starting from image preprocessing and
data cleaning, then feature extraction from the dataset using a
suitable distinguishable feature, which usually uses histogram
technique, then pattern recognition depending on one of the
machines learning techniques and artificial inelegance. Such as
Support vector Machines, Neural networks, K-Nearest
Neighbor techniques.
Histogram as a feature extraction works mainly on
converting the image into useful digital values depending on
the arithmetic sum of parts of the processed image.
The rest of the paper is organized as follows: First, we give
an overview of the methods and techniques used in previous
literature, then we introduce the metrology followed in this
research, starting from image preprocessing and SVM
implementation for classification, then repeating the process
after image downscaling, and calculating the recognition
accuracy. The next part views results for this research, and
WCSIT 11 (1), 1-4, 2021
2
finally in the conclusion we discuss the impact of image
rescaling on the recognition accuracy, and propose focusing on
other feature extraction methods.
II. LITERATURE REVIEW
Machine learning support-vector networks or Support
Vector Machines (SVMs) are part of the machine learning tools
that are used in classification and regression analysis. Learning
algorithms associated with SVMs are supervised learning
models that recognize pattern by analyzing that data. Input
vectors are mapped to a higher dimension feature space.
Support vector network is a learning machine for two-group
classification problems. Conceptually Support vector network
implements non-linearly mapping of input vectors to a very
high-dimension feature space. A linear decision surface is
constructed in the mapped feature space. The data under
consideration, the Support Vector Machines are modelled to
categorize [3]. SVMs can implement a non-linear classification
using what is called the kernels, implicitly mapping their inputs
into high-dimensional feature spaces. The following steps are
generalization of modelling of hand written digit recognition.
Starting from modeling the SVM and then selecting the data set
to train this model. From the training of the SVM model the
hyperplane is generated. This hyperplane categorizes data into
set classes. After configuration of the hyperplane, testing
dataset is given as an input to the SVM model and calculating
the accuracy output compared with the proposed values.
In [5] feature extraction technique proposed was radial
histogram, and Euclidean distance for classification, the
research was implemented for Gujarati handwritten Characters.
Simple Normalization & Binarization pre-processing step was
also implemented. A 72 feature vectors of a 5-degree interval
was generated to build the radial histogram.
Another research presented handwritten recognition
algorithm based on histograms, structural features, and profiles.
Vertical and horizontal histograms were used, in
combination with the radial histogram, a 280-features
vector was built from the representation of 32x32 image
matrices for the characters using out-in radial and in-out radial
profiles. The recognition process has been supported by a
lexical component based on dynamic acyclic FSAs (Finite-
State-Automata). [6]
Using combination of multiple feature extraction methods
and an ensemble classifier system is can increase the model
accuracy. In [7] research paper Six feature set using different
approaches were extracted depending on zoning, projections,
edges, concavities and gradient to increase diversity to the
feature extraction phase of the model. The importance of
diversity in the methods enhanced the ability to recognize
certain types of images. The research shows that using a
combination of six features enhanced the recognition rate.
Therefore, these six methods are sufficient to achieve a very
high recognition rate. But on the other hand, that raised a
problem of finding the best scheme to combine those features.
III. METHODOLOGY
In this research we will find the impact of rescaling (down
scaling) handwritten digit images on recognition accuracy.
MNIST database [10] will be used as, this dataset contains
60000 training images and 10000 testing images, each image is
28x28 pixel 256 gray level. The images will be preprocessed
and converted into Monochrome 1bit depth images depended
on an experimented threshold as a Binarization stage, which
was implemented by experimentally selecting a threshold for
the gray value of the pixel color and convert it into binary
value. The next step is implementing histograms for four
proposed features to build features vector. The features vector
is processed using different SVM kernel models and
recognition of the system is evaluated, Fig.1 shows the
flowchart for the recognition process.
Figure 1. Recognition Process
The second phase in this paper will reduce the resolution of
the dataset images to be (14x14), this will reduce the size of the
data to be processed to one forth the original size, then repeat
the steps of the first phase.
In this research for four features were selected (Vertical,
Horizontal, Diagonals and Inverse-diagonals Histograms) for
the feature extraction stage. Vertical histogram as in (1)
generates 28 element, horizontal histogram as in (2) generates
also 28 elements, while Diagonal (3), (4) generates 55
element, and Inverse-diagonal (5), (6) generates 55 elements
too. All these elements were joined in one feature vector of 155
elements.
WCSIT 11 (1), 1-4, 2021
3
(1)
(2)
(3)
(4)
(5)
(6)
The next stage is applying the features vector to 3 different
SVM model types (Quadratic, Cubic and Medium Gaussian)
and calculating the recognition accuracy. This part was
implemented using MATLAB learner classification confusion
matrix.
The second phase of this research, worked on reducing the
resolution of the digit images to be 14x14 instead of 28x28.
This reduced the storage size of the dataset, and should reduce
the processing time for the SVM stage as a result.
The reduction was implemented depending on a voting
value for adjacent pixels vertically and horizontally to generate
one value for the new pixel, we use 1bit depth also for this
stage. This size of the new images was 196 pixels.
Figure 2. Inverse diagonal features maping 14x14 image
Implementing the proposed feature extraction method
mentioned before generated feature vector of 82 elements,
divided as follows, vertically and horizontally 14 feature each,
diagonal and inverse-diagonal 27 feature each.
IV. RESULTS
Through the experimentation stage, we initially implement
SVM on each feature separated, it was found that diagonal, and
inverse-diagonal histograms generated higher recognition
accuracy than vertical and horizontal histogram, but even with
a low recognition accuracy value (less than 75% for the
inverse-diagonal). While using a feature vector combined from
two histograms (diagonal and inverse-diagonal) gives
noticeable enhancement in recognition accuracy of 92.3%.
By implementing the SVM on all features as one feature
vector, on the original images that had 155 features, the
recognition accuracy reached up to 97.2% using Cubic SVM
model, 96.9% using Quadratic SVM model, and 96% using
Medium Gaussian SVM model. Total processing time for all
stages was 7700 second, and memory size of 89 Mbytes. Fig. 3
shows the confusion matrix for resulting from implementing
Cubic SVM model on the 155-features vector.
By implementing the SVM on all features as one feature
vector, on the reduced resolution images that had 82 features,
the recognition accuracy reached up to 96.2% using Cubic
SVM model, 95.8 using Quadratic SVM model, and 95% using
Medium Gaussian SVM model. Total Processing time for all
stages was 6299 second, and memory size of 46 Mbytes.
From the results, it was noticed that reducing the image
resolution from 28x28 pixels to quarter size the original 14x14
pixels, had only a small negative impact on the recognition
accuracy of about 1% only (from 97.2 to 96.2) while it had a
positive impact on the reduction of the processing time of
18.2% (from 7700 sec. to 6299 sec.).
WCSIT 11 (1), 1-4, 2021
4
Figure 3: Confusion matrix for Cubic SVM model of 155 features.
From comparing the three different SVM models applied
inn this research we noticed that cubic SVM model gives the
highest recognition accuracy 97.2% on the original images, and
96.2% on the downscaled images.
From feature extraction process it was noticed that
Diagonal and inverse-diagonal histogram features have higher
effect over vertical and horizontal.
V. CONCLUSION
Existing handwritten recognition systems reached high
recognition accuracy. In this research, we aim to reduce the
processing time and memory size by rescaling the digit images,
and find the impact of image resolution reduction on
recognition accuracy on handwritten digits. The selected set of
features depended on the evaluating histograms for Horizontal,
vertical, diagonal and inversed diagonal orientations of the
digit images. A features vector of 155 elements for the original
images was constructed, and another feature vector of 82
elements for the rescaled images was constructed.
Classification was implemented by applying SVM on each
feature vector. Results showed that the reduction of the size for
the features vector due to image rescaling to quarter of the
original size had only about 1% accuracy degradation impact.
The size of features-vector is still high, this will enhance the
SVM processing time. More studies are needed on enhancing
the feature extraction stage, looking for more suitable features
can enhance the recognition accuracy, and can minimize the
size of features-vector without impacting on the recognition
accuracy.
REFERENCES
[1] Cheriet, M., Kharma, N., Liu, C.L., Suen, C.: “Character Recognition
Systems”. Wiley, New Hersey(2007).
[2] R. Kambli1 ,Y. Ankurkar, A. Mane, “Handwritten Digit Recognition
with Improved SVM” ,JISET - International Journal of Innovative
Science, Engineering & Technology, www.ijiset.com ISSN 2348 –
7968. Vol. 1 Issue 4, June 2014.
[3] C. CORTES, V. VAPNIK, “Support-Vector Networks” , Machine
Learning, Kluwer Academic Publishers, Boston. Vol 20, pp. 273-297,
1995.
[4] B. El Kessab, C. Daoui, B. Bouikhalene, M. Fakir, K. Moro, “Extraction
Method of Handwritten Digit Recognition Tested on the MNIST
Database” International Journal of Advanced Science and Technology
Vol. 50, January, 2013.
[5] L. Shah, R. Patel, S. Patel, J. Maniar, “Handwritten Character
Recognition using Radial Histogram “, International Journal of Research
in Advent Technology, E-ISSN: 2321-9637 24 Electronics &
Telecommunication. Vol.2, No.4,April 2014.
[6] E. Kavallieratou, K. Sgarbas, N. Fakotakis and G. Kokkinakis,
“Handwritten Word Recognition based on Structural Characteristics
and Lexical Support”, Proceedings of the Seventh International
Conference on Document Analysis and Recognition (ICDAR) 2003.
[7] Rafae. Cruz, G. Cavalcanti and T. Ren, “Handwritten Digit Recognition
Using Multiple Feature Extraction Techniques and Classifier Ensemble”,
IWSSIP 2010 - 17th International Conference on Systems, Signals and
Image Processing, 2010
[8] Xiaoning Zhou, Jie Li, Changjun Yang, Jianming Hao1Study on
Handwritten Digit Recognition using Support vector machine, IOP Conf.
Series: Materials Science and Engineering 452 , 2018,
[9] A.Gil, C.Filho, M.Costa, “Handwritten Digit recognition using SVM
binary classifier and unbalanced decision trees”, Springer International
Publishing Switzerland , ICIAR 2014 ,part 1,LNCS 8814 ,pp245-255
2014.
[10] Yann LeCun, "THE MNIST DATABASE of handwritten digits".
Courant Institute, NYU Corinna Cortes, Google Labs, New York
Christopher J.C. Burges, Microsoft Research, Redmond.
http://yann.lecun.com/exdb/mnist/.
AUTHORS PROFILE
Wael Waheed AlQassas is a lecturer at Computer Science Department,Al
Albayt university,JORDAN. He received a Ph.D. in Computer
Information Systems from the Faculty of Computer Information
Systems, University of Banking and Financial Sciences. His research
interest in neural networks and artificial intelligence.
Mohammad Said El-Bashir is an Assistant Professor at the Department of
Computer Science faculty of computer science, Prince Hussein Bin
Abdullah College for Information Technology, Al Al-Bayt University
(Jordan). As he supervised several master students and been an
examiner to master thesis for several times. His research is in the field of
Machine Learning and Multimedia and has published several journal
papers in that field.
