ArticlePDF Available

A Framework for Recognition and Animation of Chess Moves Printed on a Chess Book

Authors:

Abstract and Figures

The work presented in this paper proposes a set of techniques to animate chess moves which are printed on a chess book. Those techniques include (1) extraction of chess moves from an image of printed page, (2) recognition of chess moves from the extracted image, and (3) displaying digitally encoded successive moves as an animation on a chessboard. Since all the moves are temporally related, temporal animations show change of spatial patterns in time. Moreover, it becomes easier to understand how the moves are played out and who leads the game. In this study, we animate chess moves printed in Figurine Algebraic Notation (FAN) notation. The proposed technique also eliminates false recognition by means of controlling possible moves in accordance with the rules of chess semantics.
Content may be subject to copyright.
A Framework for Recognition and Animation of
Chess Moves Printed on a Chess Book
1Süleyman Eken, 1Abdülkadir Karabaş, 1Hayrunnisa Sarı and 1Ahmet Sayar
1Department of Computer Engineering, Kocaeli University, 41380 Kocaeli, Turkey
Abstract: The work presented in this paper proposes a set of techniques to animate chess moves which are printed on a chess
book. Those techniques include (1) extraction of chess moves from an image of printed page, (2) recognition of chess moves
from the extracted image, and (3) displaying digitally encoded successive moves as an animation on a chessboard. Since all
the moves are temporally related, temporal animations show change of spatial patterns in time. Moreover, it becomes easier
to understand how the moves are played out and who leads the game. In this study, we animate chess moves printed in
Figurine Algebraic Notation (FAN) notation. The proposed technique also eliminates false recognition by means of
controlling possible moves in accordance with the rules of chess semantics.
Keywords: Animating chess moves, chess character recognition, chess readings, chess document image analysis
1. Introduction
Pattern recognition has been studied in many fields
including psychology, psychiatry, biometrics,
bioinformatics and gene expression analysis, cognitive
science, traffic flow, handwriting recognition in
criminology and banking, optical character recognition
(OCR), computational finance, stock markets and many
more. OCR systems are divided into two categories: (i)
task-specific systems and (ii) general purpose systems.
In task-specific systems, whole document is considered
to be text, and certain portions of a document are
digitized. This approach is widely used in bank check
readers, account processing systems, and airline
ticketing readers. In general purpose OCR systems, the
document is separated as text and non-text blocks. Text
blocks are split into lines, words, and characters. These
systems are known as document image analysis [1-3].
The proposed OCR system is applied on extraction
of chess moves recorded as text in Chess Informant
documents. Chess Informant is a publishing company
produces books of the same name periodically. Each
issue offers several hundred games or fragments of
games from master play, mostly annotated by the
players themselves. A Chess Informant page not only
includes game moves, but also many other texts and
images scattered around the page. So, the chess moves
need to be separated from the other parts of the page
[4]. Within this concept, the recognition process we are
dealing with falls in the second general class of OCR
system mentioned above.
Some of the related works on extraction and
recognition of chess characters are presented as
follows. Nabiyev [5] initially represented chess
readings turning several notations to common notation
called Figurine Algebraic Notation (FAN). After chess
readings analysis, the transformations of recognized
chess characters have been performed. Chess moves
have been animated on chessboard, so mistakes in
recognition stage have been eliminated by controlling
possible moves in accordance with the rules of chess.
For character recognition, Nabiyev utilized the
heuristic solution together with the figural information
related to the notations. Chen [6] developed a feature
comparison method for the Chinese-chess object by
calculating the distance between the contour of the
character and the center of the chess object. Basic
image analysis methods such as the noise filter, object
extraction, normalization, feature calculation (FC) and
maximum energy slop (MES) are used to achieve
robust Chinese-chess recognition. Zhu et al. [7]
presented a Chinese chessman pattern recognition
system based on rate-connectivity and concentric
circle algorithms. According to their experimental
results, concentric circle algorithm is more suitable
then rate-connectivity algorithm in terms of
complexity and accuracy. Wang et al. [8] proposed
Radial harmonic Fourier moments (RHFMs) based
algorithm to recognize Chinese chess characters. They
evaluated the efficiency of their method to experiment
on both toy images and real chess images with
recognition rate of 99.49% and 99.57% respectively.
Zhai [9] presented Chinese chessboard recognition
method considering colour recognition. Colour
recognition provides maintaining and restoring of state
matrix that can get the information of chessboard.
Peng et al. [10] proposed a technique recognizing
Chinese chess via computer vision methods. This
technique firstly utilize Rober operator to detect the
edge information of chess then mathematical
morphology and template circle method to detect and
segment of the chess. Finally, their technique use
projection histogram of polar coordinates image and
Fast Fourier Transform (FFT) to extract feature of
chess characters. All aforementioned studies focus on
recognition of Chinese chess characters. These
characters are all the same and represented with a
single image. So, the recognition process is not
complicated. On the other hand, the study presented in
this paper deals with the extraction of various chess
characters printed on a page of a chess book. We not
only deal with extraction and recognition of chess
characters but also extraction and recognition of printed
chess moves on the pages of the chess book. Moreover,
we deal with animating printed chess moves on the
user’s screen. The book we work on is called Chess
Informant book.
Baird and Tompson [11] proposed an empirical page
reader system performed top-down layout analysis -
divide a document image into smaller regions- for
identification of columns, lines, and characters through
skew-estimation technique. By analyzing the formal
rules of chess, the error rate was minimized
considerably. Baird and Thompson’s work guides us
about how a page reader for Chess Informant will be
constructed. In another work [12], a framework is
proposed to recognize chess moves in FAN notation.
To extract chess characters, feature vectors are formed.
The feature vector contains area, perimeter, and shape
properties such as thinness ratio, aspect ratio and
compactness. The recognition of chess characters is
achieved by using Multilayer Feed-forward (MLF)
neural network with back-propagation learning
algorithm and the feature vector.
There are some differences between this study and
the other previous works. The previous system works
on extracted chess moves and extracts and recognizes
characters of these moves. The previous work does not
care about extraction of these parts from machine
printed pages. The work presented in this paper
proposed a set of techniques to extract chess moves
printed on a double-column page of a Chess Informant
book, and generate animation video from the extracted
chess moves. The animation is played on a chess board.
The proposed system not only performs extraction and
animation but also checks if the moves are legal, in
accordance with the chess rules.
Figure 1. An example of chess moves from a page of Chess
Informant book
The remainder of this paper is organized as follows.
In Section 2, chess notations are presented for encoding
chess moves. Extraction of chess moves from a page,
recognition of the extracted chess characters, and
animating chess moves are explained in Section 3.
Moreover, performance evaluation for recognition of
chess characters and complexity analysis of the
proposed methodology are presented in same chapter.
Section 4 draws a conclusion and future works.
2. Notation for Encoding Chess Moves
In a Chess Informant book, each game consists of series
of moves, and each move is described by a move
number and two plies (half-moves), and each ply is
described in three characters on average (Figure 1).
Different types of chess notations are used for
symbolic representation of chess moves. There are
several systems developed to encode and record chess
moves (position of chess pieces). Each country has its
own symbols for chess pieces. The algebraic chess
notation is well-known and widely used notation
standard. There are a number of different types of
algebraic chess notations but popular types used today
are Short Algebraic Notation (SAN), Long Algebraic
Notation (LAN), and Figurine Algebraic Notation
(FAN).
The Chess Informant books use FAN notation (see
Fig. 1). To better understand the proposed system,
FAN is basically an algebraic notation in which chess
figures are represented with symbols. Pawn is the only
figure does not have piece identifier (symbol).
Chessboards are divided into ranks and files in the
algebraic notation. The ranks are known as the
horizontal rows of squares labelled 1 through 8. The
files are known as the vertical columns of squares
labelled a through h. After labelling, each square on
the chessboard can be described by a unique
combination of file-rank tuples. In this notation, black
moves come after white moves. There is a strict order
among them. Each move is sequentially numbered.
Many books and computer chess programs use FAN.
This is as closely as normal algebraic notation except
that identifiers of the piece are replaced with graphic
symbols of the pieces [13].
There are also various other symbols found in chess
informant books to present chess games, besides file-
rank tuples and chess figures. Some of them are used
by commentators to give evaluative comment on a
move like “!” (a particularly good move), “!!” (an
excellent move), “?” (a bad move). Some of them
indicates the strategic balance of the game position
like “∞” (unclear), “+-(white has a clear advantage),
“=/+” (black has slightly better chances), “=” (even
position). Some other symbols used in multilingual
publications like “□” (the player has counterplay), “
(the opponent’s plan this defends against), “Δ” (the
future plan this move supports). Characters, symbols,
letters, numbers, and figures are concerned in
recognition of chess text.
3. Extraction, Recognition and Animating
Chess Moves Printed on a Page
The proposed architecture is summarized in the
following steps:
Pre-processing of a page containing chess moves
(converting grayscale, smoothing, thresholding) to
improve the results of later processing steps
(Section 3.1)
Extracting chess moves from a page (Section 3.2)
Horizontal projection profiling to detect printed
lines of chess moves (Section 3.3)
Vertical projection profiling to segment characters
of each line (Section 3.3)
Recognition of segmented chess characters
(symbols, letters, numbers, and figures) (Section 3.4)
Animating digitized chess moves on a chessboard
and recording it as a video for educational purposes
(Section 3.5)
3.1. Pre-processing of an Image for Chess
Document
Preprocessing has three main stages: converting chess
image document to grayscale, smoothing and
binarization. Raw chess image document is firstly
converted into grayscale for simplifying the tasks of the
subsequent step (see Figure 2(a)). Then, smoothing
technique is applied. Most smoothing techniques
remove noise, but they blur edges. It is important to
maintain edges, because edges are critically important
to the visual appearance of images. We utilize median
filtering to reduce noise over the image for a chess
document (see Figure 2(b)). The median filtering
preserves certain edge shapes. Applying multiple
median filters lessens noise and brightness. To avoid
the computational expenses, the median filtering is used
with a small size (3x3) window [14].
a1a2a3
a4a5a6
a7a8a9
In the filter windows, a1a9 represent grey
levels of the pixels.
The aim of binarization is to reduce unwanted
information to increase the visibility of the desired
information. It helps with differentiating foreground
regions (characters for chess moves) from background
regions, which is a paper itself. Fixed threshold level
often does not provide accurate distinction between
foreground and background portions. This is because of
the fact that many scanned chess documents have
varying intensity values. So, the Otsu thresholding
technique [15] helps here to distinguish foreground
(objects) from the rest of the image (see Figure 2(c)).
The Otsu binarization takes into consideration of gray
level values distributions in image as well as the local
characteristics of the pixels. After pre-processing steps,
columns and text lines in chess image document can be
extracted now. Figure 2 shows results of the pre-
processing steps.
(a)
(b)
(c)
Figure 2. Pre-processing of image for a chess document (a)
Grayscale chess document image; (b) Smoothed chess document
image; (c) Binary chess document image
3.2 Extraction of Chess Moves
Throughout the paper, we assume text lines are
horizontally and vertically parallel to the respective
edges of the paper in a document. In other words, the
skew angle of a page is zero, even if a page is
manually scanned or photocopied.
Different approaches such as projection profiling,
Hough transform, nearest-neighbor clustering,
Fourier- transformation and moments have been used
to detect text line position by authors in the literature
[16]-[17]. Chess moves in a page of Chess Informant
book are written in bold font as shown in Figure 3(a).
The other parts in the pages are explanations of chess
moves and they are written in normal font. Using this
information, chess moves are extracted from a page
via the horizontal and vertical projection profiles. The
horizontal projection profile (HPP) is a sum of black
pixels along every row of the image. Vertical
projection profile (VPP) is the sum of black pixels
along every column of the image. In Eq.1; n and m
stand for column number and row number,
respectively. I[x, y] represents intensity value of chess
image.
HPPx=I[x, y],
n
y=1
VPP(x)= I[x, y]
m
x=1
(1)
The Chess Informant books are of two column
pages. These columns are distinguished by taking
vertical profiling. For multicolumn documents, the
vertical projection profile will have a plateau for each
column, separated by valleys for the between-column
and margin spacing. The HPP will have peaks whose
widths are equal to the character height and valleys
whose widths are equal to the spaces between lines.
Each column is distinguished by applying VPP, and
each line is distinguished and separated by applying
HPP. Moreover, since the chess moves are printed in
bold format, the HPP values for the lines of chess
moves are larger than the other parts’ HPP values.
This helps us to separate or extract the chess moves.
After applying HPP and VPP processes, the chess
moves are extracted from a page (see Figure 3(a)) as
shown in Figure 3(b). As shown in Figure 3(a), chess
moves are scattered around the pages (see bold texts).
These are collected as one set of moves as shown in
Figure 3(b).
(a)
Figure 3. (a) A sample of chess page from machine printed chess book (b) Extraction of chess moves portions from printed page
3.3 Segmentation of Chess Characters
After having extracted scripts of chess moves from the
page, segmentation is performed. To separate text lines
of extracted chess moves, the HPP is performed. White
spaces between the lines of chess moves are used to
segment the lines. HPP has zero height valleys between
the lines. Line segmentation is done at this zero height
points. To improve result of the HPP, smoothing
function (S(n)) is applied on HPP:
Sn=
0, n≤ N- k
2
0, n>N- k
2
1
kHn+i
k
2
i=-k
2
, otherwise
(2)
In Eq.2, H(n) is histogram function and N is the
number of elements in histogram function. k is a
smoothing factor. Figure 4 illustrates HPP of the
extracted chess moves and Figure 5 illustrates the line
segmentation point after applying smoothing function.
Figure 4. Text line extraction without smoothing function
Figure 5. Text line extraction with smoothing function
To extract each character from a segmented text
line, VPP is performed. VPP has valleys whose widths
are equal to the between-line spacing (or low peaks
between characters) (see Figure 6).
Figure 6. Running vertical projection profile on text lines
Chess characters are segmented according to points
found after VPP. To improve success of recognition
step, all background pixels (white pixels) are trimmed.
In other words, all characters are enclosed in full
frame (see Figure 7). Then, recognition process is
performed on each candidate character.
Figure 7. Segmented chess characters
3.4 Recognition of Segmented Characters
To recognize chess characters used for representing
chess moves, a data set is created.
3.4.1 Dataset Creation
To our knowledge no previous dataset for chess moves
from machine printed pages has been created. We have
created a dataset from the first thirty pages of Jacob
Aagaard’s Chess book [18]. This is done by means of
aforementioned character segmentation techniques. It is
very important to know that all characters in dataset are
gray level, not two-level. The dataset has 1674 number
of character samples for each one of 30 different
characters. Figure 8 shows a set of samples of images
belonging to bishop. As shown in Figure 8, some of
them are clear and some of them are noisy.
The dataset is divided into mutually exclusive
training and test sets. Training set contains 70% of the
total samples and test set contains 30% of the total
samples. The images in created dataset might be in
varying sizes, even if for the same character. Therefore,
characters need to be resized. All characters are
enlarged to a fixed size (20x20) by using a bilinear
interpolation algorithm [20].
Figure 8. Sample bishop images from the dataset
3.4.2 K-NN Classification based on Euclidian
Distance
In k-NN classification, training patterns are plotted
according to their observed feature values and are
labeled according to their known classes. An unlabeled
test pattern is predicted according to the most
frequently occurring class among its k-most similar
training patterns (majority voting); its nearest
neighbors. The most common used similarity measures
for k-NN classification are Euclidean, Minkowski,
Manhattan and Mahalanobis distances [19]. The
Euclidian distance metric is used for similarity
measure. k value is set to 1 to predict the same value,
or class, as the nearest instance in the training set.
In the training phase, trained characters and their
matrix forms are created. This is done for each
character in the training set. In other words, a trained
character is obtained by averaging all samples
belonging to same character (e.g. pawns) using Eq.3.
T[i, j]= 1
NCs[i, j]
N
s=1
(3)
In Eq.3, T[i, j] represents matrix form of trained
character and Cs[i, j] represents matrix of a sample
belonging to the same character. N stands for number
of samples. So, range value for elements of matrix
T[i, j], is between 0 and 255 (see Figure 9). In order
to minimize effect of too large or too small values in
the matrix, we use min-max normalization. So,
elements of trained matrix are normalized between 0
and 1. Suppose that minA and maxA are the minimum
and maximum values of an element of matrix T. Min-
max normalization maps a value, vi, of T to vi
' in the
range [new_minT, new_maxT] by computing
vi
'=vi-minT
maxT-minTnew_maxT-new_minT+ new_minT
(4)
Here, [minT-maxT] range represents [0-255] and
[new_minT-new_maxT] range represents [0-1].
Figure 9. A trained image for character “0”
In the test phase, the nearest instance in the training
set is predicted by means of Euclidean distance metric.
The smallest distance value (E) represents the greatest
similarity to testing segmented character I[i, j].
E= I[i, j]-
m
j
n
i
(5)
3.4.3 Performance Evaluation for Recognition
Chess Characters
Results for chess character recognition are evaluated
using sensitivity, specificity and accuracy as the
performance measures. The measures are described
below as:
Sensitivity = TP/(TP + FN)
(6)
Specificity = TN/(TN + FP)
(7)
Accuracy = (TP + TN)/(TP + TN + FP + FN)
(8)
In Eq.6, Eq.7 and Eq.8, TP = True Positive, TN =
True Negative, FP = False Positive and FN = False
Negative. An event is said to be TP if a character is
correctly classified and TN when a character is
correctly non-classified. In related development, an
event is said to be FN if a character is incorrectly non-
classified and a FP when a character is incorrectly
classified. Sensitivity measure indicates the ability of a
classification technique to recognize the character while
specificity measures the proportion of negatives which
are correctly classified. The accuracy measure however
indicates the degree of conformity of the correctly
classified characters to the ground truth. Table I shows
performance results for chess character recognition.
Table 1. Performance results for character recognition
Chess
Charact
ers
Avg.
Sensitivit
y
Avg.
Specificit
y
Avg.
Accura
cy
0
100%
100%
100%
1
97%
100%
97%
2
100%
100%
100%
3
100%
100%
100%
4
100%
100%
100%
5
96%
100%
99%
6
100%
99%
99%
7
100%
100%
100%
8
100%
100%
100%
9
100%
99%
99%
a
100%
100%
100%
b
100%
100%
100%
c
100%
99%
99%
d
100%
100%
100%
e
95%
100%
99%
f
100%
100%
100%
g
97%
100%
99%
h
100%
100%
100%
!
100%
99%
99%
-
100%
100%
100%
.
100%
100%
100%
1/2
100%
100%
100%
+
100%
100%
100%
x
100%
99%
99%
Knight
100%
100%
100%
Pawn
100%
100%
100%
King
100%
100%
100%
Queen
100%
100%
100%
Rook
100%
100%
100%
Bishop
100%
100%
100%
3.5 Animating Recognized Chess Characters on
Chessboard
Animation has been used in many real world
applications to recognize the patterns in spatially and
timely changing datasets [21]-[22]. Here, each move
results in one character to change its place on
chessboard. When we play those moves successively,
the game will be more easily understandable. If a
wrong recognition happens during the game extraction
phase, false recognitions are eliminated in accordance
with the chess semantics, by applying knowledge of
the rules of chess. Also, animated games are possibly
recorded as videos for educational purposes (see
Figure 10).
Figure 10. A sample screen for animating extracted and
recognized chess moves
3.6 Computational Complexity Analysis of
Each Step
Computational complexity of each steps of proposed
algorithm is analyzed as following.
In median filtering step, the median of pixel values
inside a filter window has to be calculated. In its
straight-forward implementation the pixel values
are sorted and the value located at (k2/2)th position
is takn here k is the filter size, i.e. the size of
one side of the square filter window. If quicksort is
used, the computational complexity of filter is
O(klgk) operations per ie of result image. Total
cost for entire image is O(n2klgk), but we can
eliminate klgk because k<<n. Finally, median
filtering step costs O(n2).
For binarization of chess image with L gray levels
using M-1 thresholds, Otsu’s exhaustive method
searches L
M-1 combinations of thresholds, which
can be approximated to O(LM-1) for (M-1) << L.
Here, the pixels in the chess image document to
two classes (black and white). So, total cost for
segmenting is O(L) [23].
HPP and VPP steps require operation of Eq.1 on
entire image. So its cost is O(n2).
We have n examples to be stored with d-dimension.
O(d) is necessary to compute distance to each
example and O(nd) to find one nearest neighbor.
O(knd) costs to find k closest examples. In this
study, k is chosen as 1. So, complexity is O(nd).
Total computational complexity of proposed steps to
extract and recognize chess characters from machine
printed pages is O(n2 + L + n2 + nd) = O(n2). Because,
O(L) and O(nd) are negligible in the complexity
analysis.
4. Conclusion and Future Works
It may be very hard to learn chess openings, endings
for chess players from machine printed chess books. In
this study, we propose some techniques to read in the
Chess Informant series which contain descriptions of
chess games selected by chess masters and published
by the International Chess Federation (FIDA).
Proposed techniques able to analyse the double-column
pages, extract game descriptions from machine printed
pages, recognize these chess characters and animate on
chessboard. To animate chess moves, the system also
check each move for its legality.
In the future, we aim at extracting and recognizing
chess moves in other alternative notations such as SAN
and LAN by using more robust classification
techniques. For example, we plan to use shape analysis
algorithms which are typical for such tasks and lead to
much better results even for more complicated tasks
than simple recognition of several symbols instead of
basic image analysis methods such as horizontal or
vertical projection or area estimation. Also, a chess
playing robotic arm system, capable of learning from
extracted and recognized chess moves in Chess
Informant series, can be developed.
References
[1] Tsujimoto S. and Asada H., Major Components
of a Complete Text Reading System,” In Proc. of
the IEEE, vol. 80, no. 7, pp.1133-1149, 1992.
[2] Nabiyev V.V., Yapay Zeka: Problemler,
Yöntemler, Algoritmalar, Chapter 2 and 3. Seçkin
Publishing, 2005. (in Turkish)
[3] Baird S.H., Document Recognition without
Strong Models. In Proc. of the International
Conference on Document Analysis and
Recognition (ICDAR), pp. 414-423, 2011.
[4] Chess Informant series,
http://www.chessinformant.rs/, Accessed Date:
January, 2015.
[5] Nabiyev V.V., Providing Harmony among
Different Notations through the Chess Readings,”
In Proc. of the IEEE 19th Signal Processing and
Communications Applications Conference, pp.
29-33, 2011.
[6] Chen W.Y., Chinese-Chess Image Recognition
by using Feature Comparison Techniques,”
Applied Mathematics & Information Sciences,
vol. 8, no. 5, pp. 2443-2453, 2014.
[7] Zhu H., Lei J., Tian, X., A Pattern Recognition
System Based on Computer VisionThe method
of Chinese chess recognition,” In Proc. of the
IEEE International Conference on Granular
Computing, pp. 8865-868, 2000.
[8] Wang K.., Zhang H., Ping Z., Hai Y., Chinese
Chess Character Recognition with Radial
Harmonic Fourier Moments,” In Proc. of the
International Conference on Document Analysis
and Recognition (ICDAR), pp. 1369-1373, 2011.
[9] Zhai N., Improved Chinese chessboard
recognition method,” Journal of Computer
Applications, vol. 30, no. 4, pp. 980-981, 2010.
[10] Peng H., Yangyu L., Chengrong L., Chinese
Chess Recognition based on Projection
Histogram of Polar Coordinates Image and
FFT,” In Proc. of the Chinese Conference on
Pattern Recognition, pp. 1-5, 2009.
[11] Baird S.H., Tompson K., Reading Chess,”
IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 12, no. 6, pp. 697
708, 1990.
[12] Eken S., Sayar A., Animating Chess Moves
Recorded on Chess Informant,” In Proc. Of the
3rd International Symposium on Computing in
Science & Engineering, pp. 35-40, 2013.
[13] Chesscrner, www.chesscorner.com, Accessed
Date: January, 2015.
[14] Schalkoff R.J., Digital Image Processing and
Computer Vision, Wiley, 1992.
[15] Otsu N., A threshold selection method from
gray-level histograms,” IEEE Transactions on
Systems, Man, and Cybernetics, vol. 9, no. 1, pp.
62-66, 1979.
[16] Kasturi R., Gorman L.O., Govindaraju V.,
Document image analysis: A primer,” Sadhana,
Part 1, no. 27, pp. 3-22, 2002.
[17] Gorman L.O., Kasturi R., Document Image
Analysis, IEEE Computer Society Press, 1995.
[18] Aagaard J., Excelling at Positional Chess,
Everyman Chess, 2003.
[19] Weinberger K.Q., Saul L.K., Distance Metric
Learning for Large Margin Nearest Neighbor
Classification,” Journal of Machine Learning
Research, vol. 10, pp. 207-244, 2009.
[20] Battiato S., Gallo G., Stanco F., A New Edge-
Adaptive Algorithm for Zooming of Digital
Images,” In Proc. of IASTED Signal Processing
and Communications, pp. 144-149, 2000.
[21] Sayar A., A distributed map animation
framework for spatiotemporal datasets,” Turkish
Journal of Electrical Engineering & Computer
Sciences, 2014.
[22] Sayar A., A Service Oriented Framework for
Animating Big Spatiotemporal Datasets,”
Lecture Notes in Computer Science, vol.
7652, pp. 238-250, 2013.
[23] Liu D., Yu J., Otsu method and K-means,” In
Proc. of Ninth International Conference on
Hybrid Intelligent Systems, pp. 344-349, 2009.
Süleyman Eken has received his
BEng. in Computer Engineering
from Karadeniz Technical
University in 2009 and ME. in
Computer Engineering from
Kocaeli University in 2012. He is
currently working towards Ph.D. degree in Computer
Engineering from Kocaeli University, Turkey. Also,
he is currently a Research Assistant of Computer
Engineering Department at Kocaeli University in
Turkey. His current research interests include
distributed file systems, data synchronization, Satellite
Image Processing, Remote Sensing, WEB-GIS
applications and Spatial Databases.
Abdülkadir Karabaş received the
B.S. degree from the Computer
Engineering Department, Kocaeli
University, Kocaeli, Turkey, in
2015. His research interests include
computer vision and pattern
recognition.
Hayrunnisa Sarı received the B.S.
degree from the Computer
Engineering Department, Kocaeli
University, Kocaeli, Turkey, in
2015. Her research interests include
computer vision and pattern
recognition.
Ahmet Sayar received M.S. in
Computer Science (2001) from
Syracuse University and Ph.D. in
Computer Science (2009) from
Indiana University, USA. He has
worked at Los Alamos National
Laboratory (New Mexico, USA) and Community
Grids Laboratory (Indiana, USA) as a researcher.
Since 2010, he is a professor at Computer Engineering
Department, Kocaeli University, Turkey. His current
research interests include Distributed Systems, Big
Data, Data-Intensive Computing, Remote Sensing,
GIS, and Spatial Data-Databases. He has co- authored
3 books and around 70 papers.
Article
Full-text available
With respect to the mass categorization that is central to most computer operations, there are two types of relevant data which affect speed of assimilation as well as information recall: structured data and unstructured data. Structured data refers to information with a high degree of organization, such that inclusion in a relational database. However, unstructured data may have its own internal structure, but does not conform neatly into a spreadsheet or database. CVs (Curriculum vitae) are this kind of data. Typically, CVs presented in PDF format can be structured using the PDF tagging feature, however most PDF data is untagged and unstructured. It is very difficult for non-technical business users and data analysts to deal with such closed boxes. Within the scope of this study, a web based smart resume designer was developed which will allow people gain time while creating their own resumes according to their own information in different accepted formats. The content structure of the PDF documents, the text data and the font and location information of this data were extracted and the information obtained was converted into certain structures in the order of reading and a predefined XML based resume template was created. Personal PDF documents are created using this template. PDF analysis and PDF creation was done directly by accessing the content stream of the PDF document with the help of the iText-pdf library, which is the Java library. Presentation templates is served to end-user on a desktop applicaiton with a GUI and users can select any metadata to create own document with select-and-apply approach. It is predicted that the template obtained from the PDF document will be saved in XML format and the templates in the ready-made XML format will be used for adaptation to different contents. The XML schema (XSD-xml schema definition) is defined for the automatic creation of templates in XML format and subsequent testing of their accuracy. With the application developed, automatic forms of resumes were recognized and different contents were adapted.
Thesis
Full-text available
Earth Observation Satellites survey the earth by taking pictures along their pre-defined orbit, and send those pictures to ground stations. After having completed their rotations around the earth, they create a set of pictures defining and depicting earth’s surface. These satellite images are called tiles or mosaics. Most of the time mosaics cannot represent objects (road, island, lake) as a whole, and they need to be stitched to get a picture of a specific object. Stitching of satellite mosaics is used in many science domains such as remote sensing, geomatics, information science, geophysics, map engineering and cartography, and in many application areas such as object recognition/extraction/tracing, spatial analysis, topologic analysis, civil and military simulation applications, augmented reality applications etc. When vision of the various parts of the same object falls in different mosaics, they need to be stitched together in order to obtain that object in one picture. This is inevitable in the applications of object recognition and extraction. For some other objects, we might need to stitch tens or even hundreds of mosaics. As the number and sizes of the mosaics to be stitched increases, the computation resources needed, such as CPU and memory, increases exponentially. This makes stitching in only one computer harder or even impossible. In central computations, we not only have resource shortage and poor performance problems but also some other problems stemming from the spatio-temporal characteristics of satellite images. That is, partially overlapping satellite images (mosaics) might be taken at different times (temporal) for the same location. This might cause misleading pixel values for the same object scattered on different mosaics, and as a result failure in image stitching and object extraction. In addition, it is not possible to determine the boundaries of an object presented as a satellite image. With today’s technologies, it does not have a fully automatic final solution. Aforementioned problems can be classified into two groups. These are (1) resource and performance problems in central computation, i.e. single machine, for stitching mosaics to extract an object and (2) defining both boundaries of an object and corresponding mosaics on a satellite image. For the second problem, we proposed a heuristic approach based on user interactions with reference maps (Google Maps). Boundaries of an object are roughly defined by users’ mouse actions. Regarding resource and performance problems, we proposed a distributed big data framework based on MapReduce approach to enable scalable image stitching and object extraction. Apache Hadoop was placed at the core of the framework. To realize such a system, we first need to define mapper and reducer functions, and then their input and output formats. Moreover, in case of having multiple levels of mapper and reducer functions, high level process and data flow needs to be defined. The first level mappers convert raster mosaics into vector counterparts and the following mappers and reducers are performed on the vector mosaics. Stitching of vector representations of the raster images is expected to give better performance in distributed computations by reducing the negative effects of bandwidth problem. Vectorised images were stitched with point set pattern matching based technique. Point set pattern matching is usually used computational geometry and image processing areas; it also looks for similarities between the pair of two point sets. After having done with stitching and extraction, extracted objects are made available for spatial and topological queries and analyzes in standard spatial databases (PostGIS and Oracle-Spatial) as point, line, line-sets and polygon in Open Geographical Spatial Consortium standards. The outcomes of this study present valuable knowledge about image processing with distributed scalable big data frameworks. The product obtained by this study can be used in applications requiring spatial and temporal analysis on big satellite map images. This study also shows that big data frameworks are not only used in applications of text-based data mining and machine learning algorithms, but also used in applications of algorithms in image processing. Feasibility and scalability tests of the proposed system were tested with real satellite images on real objects.
Article
Full-text available
Maps are an excellent way to present data that have spatial components. However, when the data being presented vary over time, a simple two-dimensional map ignores an important feature of the data. An animated map that shows a series of two-dimensional maps at successive points in time allows one to add a time dimension to the display of data. The current study proposes a distributed service-oriented architecture to create map animations from spatiotemporal datasets. We extend the open standards GIS web services definitions with a topic-based publish-subscribe paradigm, which best suits the animation requirements. The effectiveness of the technique is demonstrated in an exploratory data analysis of Turkey's earthquake seismic data records at the end of the paper.
Article
Full-text available
We propose a service oriented distributed system framework for animating big spatiotemporal vector datasets. The aim is exploiting the patterns in spatial data dynamically changing over time. Animation consists of successively played and temporally related still map images. Each map image in the animation is a satellite map enriched with (or plotted over) spatiotemporal datasets. The system components are designed as web services. We also extend open standards’ GIS web services definitions with topic-based publish-subscribe paradigm, which best suits to the animation requirements.
Conference Paper
Full-text available
In this study, we focus on animating chess games recorded on chess informant. This involves recognition of chess characters as well as moves and playing them on chessboard. The proposed technique eliminates false recognitions by means of controlling possible moves in accordance with the rules of chess (semantics). The paper produces solution for figurine algebraic notation (FAN). For character and figure recognition, we form feature vector including area, center of area, perimeter, thinness ratio, aspect ratio, compactness, Euler number, and projection. In the recognition stage, multi-layer feed-forward (MLF) neural network with back-propagation learning algorithm is utilized to recognize characters and figures of chess by using this feature vector. The results show that the proposed system is capable of contributing to the generation of robust game databases through digitizing of chess games recorded on chess informant
Article
Full-text available
Replication is an approach often used to speed up the execution of queries submitted to a large dataset. A compiletime/runtime approach is presented for minimizing the response time of 2dimensional range when a distributed replica of a dataset exists. The aim is to partition the query payload (and its range) into subsets and distribute those to the replica nodes in a way that minimizes a client’s response time. However, since query size and distribution characteristics of data (data dense/sparse regions) in varying ranges are not known a priori, performing efficient load balancing and parallel processing over the unpredictable workload is difficult. A technique based on the creation and manipulation of dynamic spatial indexes for query payload estimation in distributed queries was proposed. The effectiveness of this technique was demonstrated on queries for analysis of archived earthquake generated seismic data records.
Article
In this paper, we develop a feature comparison method for the Chinese-chess object by using the features comparison based on input image and database. Features are generated by calculating the distance between the contour of the character and the centre of the chess object. In this paper, the noise filter, object extraction, normalization, feature calculation (FC) and maximum energy slop (MES) method are used to achieve robust Chinese-chess recognition. There are two advantages when compared with other methods. 1) Our method is robust against the 40 incline degree attacks. 2) Our method can resist the 20% pepper and salt noise attacks. In order to demonstrate the effectiveness of the proposed scheme, simulations under all kinds of various conditions were conducted. The experimental results show that our proposed scheme can exactly identify chess images at 100% of accuracy under less than 20% noise added and 40 degree incline test environment condition.
Article
Maps are an excellent way to present data that have spatial components. However, when the data being presented vary over time, a simple two-dimensional map ignores an important feature of the data. An animated map that shows a series of two-dimensional maps at successive points in time allows one to add a time dimension to the display of data. The study presented in this paper proposes a distributed service-oriented architecture to create map animations from spatiotemporal datasets. we extend open standards'GIS web services definitions with topic-based publish-subscribe paradigm, which best suits to the animation requirements. The effectiveness of the technique is demonstrated on exploratory data analysis on Turkey's earthquake seismic data records at the end of the paper.
Article
The recognition of Chinese chess through computer vision includes chess detection and its character recognition, and the process should be fast and robust. In this paper, Robert operator is used to get the edge information of chess, then detection, localization and segmentation of the chess are realized through mathematical morphology and template circle method. A new algorithm based on projection histogram of polar coordinates image and Fast Fourier Transform is introduced to extract the rotation-invariant feature of chess characters. Experiments show that the algorithm can accurately recognize all chesses within 300 ms, and is robust to any rotation.