ArticlePDF Available

CCTV object detection with fuzzy classification and image enhancement

Authors:
  • Institute of Applied Mathematics and Mechanics

Abstract and Figures

In this paper we propose a novel approach for pattern recognition problems with non-uniform classes of images. The main concept of this classification method is to describe classes of images with their fuzzy portraits. This approach is a good generalization of the algorithm. The fuzzy set is calculated as a preliminary result of the algorithm before the final decision or rejection that solves the problem of uncertainty at the boundaries of classes. We use the method to solve the problem of knife detection in still images. The main aim of this paper is to test fuzzy classification with feature vectors in a real environment. We used selected MPEG-7 descriptor schemes as feature vectors. The method was experimentally validated on a dataset of over 12,000 images. The article describes the results of six experiments which confirm the accuracy of our method. In addition we conducted a test with enhanced images, achieved with two state-of-the-art super-resolution algorithms.
Content may be subject to copyright.
CCTV object detection with fuzzy classification and image
enhancement
Andrzej Matiolański
1
&Aleksandra Maksimova
2
&
Andrzej Dziech
1
Received: 1 October 2014 /Revised: 20 April 2015 / Accepted: 18 May 2015
#The Author(s) 2015. This article is published with open access at Springerlink.com
Abstract In this paper we propose a novel approach for pattern recognition problems with
non-uniform classes of images. The main concept of this classification method is to describe
classes of images with their fuzzy portraits. This approach is a good generalization of the
algorithm. The fuzzy set is calculated as a preliminary result of the algorithm before the final
decision or rejection that solves the problem of uncertainty at the boundaries of classes. We use
the method to solve the problem of knife detection in still images. The main aim of this paper
is to test fuzzy classification with feature vectors in a real environment. We used selected
MPEG-7 descriptor schemes as feature vectors. The method was experimentally validated on a
dataset of over 12,000 images. The article describes the results of six experiments which
confirm the accuracy of our method. In addition we conducted a test with enhanced images,
achieved with two state-of-the-art super-resolution algorithms.
Keywords Pattern recognition .Fuzzy classifier .Fuzzy inference .Data analysis .Knife
detection .Feature descriptor .Image enhancement
1 Introduction
The concept of automated image understanding is common in public safety applications and it
has been explored extensively in many domains. It is an active research topic not only in the
computer vision domain [16]. The next step is the detection of dangerous situations based on
Multimed Tools Appl
DOI 10.1007/s11042-015-2697-z
*Andrzej Matiolański
matiolanski@kt.agh.edu.pl
Aleksandra Maksimova
maximova.alexandra@mail.ru
Andrzej Dziech
dziech@kt.agh.edu.pl
1
Department of Telecommunication, AGH University of Science and Technology, Kraków, Poland
2
Institute of Applied Mathematics and Mechanics, National Academy of Science of Ukraine, Donetsk,
Ukraine
recordings from IP surveillance cameras. This paper deals with analyzing video footage
obtained using CCTV systems. There are various problems associated with analyzing poten-
tially dangerous situations. A knife held in the human hand is an example of a signal of danger.
Such scenes are generally dynamic and quick. Our aim is to solve the problem of knife
recognition in frames from camera video sequences.
There are several known approaches to knife detection. Żywicki et al. proposed a method
based on a simple wavelet classifier using Haar cascades [21]. Kmiećet al. presented an
algorithm involving the active appearance model (AAM) [8,9]. The AAM takes into account
the sharpness of the blade, and detects corners in images containing knives. The final results of
the method were presented in [4] on a small dataset. Maksimova used the geometrical
approach in her study published in [14]. The methods work with images pixel by pixel, which
is inefficient in many cases. The approach of representing the image as a set of feature vectors
using MPEG-7 descriptors is introduced in this paper.
In prior studies, an adequate algorithm accuracy was only achieved for simple examples
when the knife is clearly visible in the image. For more difficult situations, when the blade
reflects light reducing its visibility or the knife is turned edgewise to the frame, the quality of
the algorithm is poor. We used a single frame from the sequence to achieve conditions
approaching reality in which only some frames are of sufficiently high quality. Finally, we
tested the algorithm with artificially enhanced images from CCTV footage.
Methods of object identification in images are distinguished by high numbers of false
positives. Quality can be estimated more effectively by multi-valued truth-space used in fuzzy
logic theory [11]. In such case, the result of the classification algorithm is an information
vector with a degree of confidence for the object assigned to a particular class. Methods of
pattern recognition that use fuzzy sets are known as fuzzy classifiers [12]. Approaches to
creating fuzzy classifiers include tuning knowledge databases using evolutionary methods [6],
applying FuzzyLVQ (fuzzy learning vector quantization) and FSOM (fuzzy self-organizing
map) networks [3], and using fuzzy clustering methods [2]. We use the fuzzy clustering
approach extended for the pattern recognition problem.
The paper is organized as follows: Section 2 describes the feature vectors, Section 3
introduces the inference model based on the fuzzy classification method, Section 4 contains
experimental verification of the approach, and Section 5 is the conclusion.
2 MPEG-7 feature vectors
Cropped images were obtained from CCTV camera footage. They were scanned with a sliding
window of size W×H, so we solved the problem for these W×Hfragments of original images.
We treated the problem as a pattern recognition one. The database consists of two classes of
images: positive examples (PE) if the image features a knife (Fig. 1a), and negative examples
(NE) in all other cases (Fig. 1b). The images were taken indoors or through car windows,
since carrying knives in public is illegal in Poland.
Current literature describes many different visual descriptors with their advantages and
disadvantages [1]. We used visual descriptors from the MPEG-7 standard. Because of the
issues specific to recognizing knives in images, we chose two descriptors: edge histogram [19]
and homogeneous texture [18]. The former, containing information about various types of
edges in the image, is a numerical vector comprising 80 types of edges. The latter describes
specific image patterns: directionality, coarseness and regularity. The two descriptors provide
Multimed Tools Appl
us with information about features characteristic of knives. We avoid using color and shape
descriptors because of light reflections and the high number of knife types. Descriptors based
on keypoint matching (such as SIFT or SURF) also do not provide good results. The majority
of keypoints are detected in the background part of the image rather than in the knife itself. The
MPEG-7 feature vectors described are used to build the model presented in this work.
3 Fuzzy classification model
To create a model for knife detection, we considered the specifics of the problem and the
presentation of images using MPEG-7. Let us discuss solving the pattern recognition problem
in the face of uncertainty [20], where a real-world object (e O) is represented as a vector of
informative features:
x¼x1;x2;;xm
ðÞ;ð1Þ
where x
i
=f
i
(e), f
i
is the method for measuring the i-th feature of the object:
fi:OXi;ð2Þ
where X
i
is the assumed region for the feature, due to the nature of the object and its
measurement method, X
i
,whereis a set of real numbers.
Let Ωbe an alphabet of classes of images for the pattern recognition problem:
Ω¼ωj

k
j¼1ð3Þ
where ω
j
is the name of the class of images, jis the element index, and kis their number.
A finite set of samples is known:
Z¼ei;oi

n
i¼1ð4Þ
where e
i
is the real object, described by the feature vector x(1), iis the element index from the
set, n is the number of samples, and o
i
Ωis its label.
a) b)
Fig. 1 Example images: apositive example, bnegative example
Multimed Tools Appl
Let us construct a classifier as a mapping:
D:Xe
Ωð5Þ
where Ωis an alphabet of classes of images (3), e
Ωis t he set of fuzzy subsets over the alphabet
of classes, and X=X
1
×X
2
××X
m
is the region of admissible values in the feature vector space
of object x, specified in (1). The classification result in this situation will be the fuzzy set:
e
α¼Xk
i¼1αi=ωið6Þ
where α
i
is the degree of similarity between the object xand class of images ω
i
. To improve the
method, the final decision about an object belonging to a given class of images is performed by
analyzing the fuzzy set e
α,specified in (6).
3.1 The clustering algorithm with an unknown number of classes
We propose to carry out a preliminary analysis of the data in order to establish the intra-
structure for each class of images. We use the FCM-fuzzy (fuzzy C-means) clustering
algorithm [2]. The result of the algorithm is a fuzzy cpartition as matrix U=[u
ki
]
n×c
,where
u
ki
is the degree of membership x
k
to cluster i,nis the number of objects x,andсis the
number of clusters, which is a parameter of the algorithm. Two types of с-partition are used in
the work:
Mfcn ¼UkXc
i¼1uki ¼1
no
;ð7Þ
Mhcn ¼UMfcn
k;iu
ki0;1
fg
noð8Þ
where M
fcn
is a fuzzy partition and M
hcn
is a crisp c-partition.
Aside from the cpartition UM
fcn
, the results of the algorithm are geometrical centers of
clusters G={g
i
,g
2
,,g
c
}
m
. The FCM algorithm minimizes the Bezdek-Dann functional:
JFCM
γU;G;XðÞ¼
Xn
k¼1Xc
i¼1uγ
kid2xk;gi
ðÞmin
|{z}
U;G
fg ð9Þ
under the constraints:
Xc
i¼1uki ¼1;xk;k¼1;n;ð10Þ
where γfuzziness coefficient, and d
2
(x,g)square of the distance between the element xand
the center of the cluster g. Here, Euclidean distance is used. The altering-optimization method
Multimed Tools Appl
is used in the FCM algorithm. It is calculated at each step by the centers of the cluster
membership degrees u
ki
for object x
k
:
uki ¼Xc
j¼1
dxk;gi
ðÞ
dxk;gj

0
@1
A
2
γ1
0
B
@1
C
A
1
;ð11Þ
where 1ic,1kn, and then new centers of clusters by u
ki
:
gk¼Xn
k¼1uγ
ki xk

Xn
k¼1uγ
ki ð12Þ
To start with, the algorithm is used to determine the initial values of cluster prototypes.
Minimal p1fiand maximal p2fivalues are calculated for every feature f
i
,i¼1;m, specified in
(2) by samples Z, specified in (4):
gk¼p1þkp2p1
ðÞ
cþ1
ðÞ
;ð13Þ
where p1¼p1f1;p1f2;;p1fm

,andp2¼p2f1;p2f2;;p2fm

,1kc,cthe number
of clusters.
The main disadvantage of the FCM algorithm is the requirement to set as a parameter of the
algorithm the number of clusters that in the study of data structures is unknown in advance. To
solve the problem of finding the optimal number of clusters, the criterion of cluster adequacy is
used. Informally, it can be described as the most appropriate cluster structure, which is different
for each task. It also implies that the choice of criteria will be different.
Certain criteria of cluster validity exist, with the following considered for the proposed
fuzzy model. Bezdeks partition coefficient is [18]:
vPC UðÞ¼
Xn
k¼1Xc
i¼1u2
ki
n;ð14Þ
where Ufuzzy с-partition, сthe number of clusters, and nthe number of samples
elements. The next property is v
PC
:
vPC ¼1UMhcn;ð15Þ
vPC ¼1
cU¼1
c
¼U;ð16Þ
where M
hcn
is specified in (8), and ¯
Uis the fuzziest partition available, since it assigns every
point in Xwith equal membership values 1
cto all cclasses. Bezdeks partition coefficient
Multimed Tools Appl
belongs to the class that uses information about Uonly, but not information about the data
itself, such as Gand X.
The Xie-Beni index v
XB
[18] belongs to the class of validity criteria using the total
information (U,G;X):
vXB U;G;XðÞ¼
Xc
i¼1Xn
k¼1u2
kijjxkgijj2
nðmin
|{z}
ijjjgigjjj2
no
Þ¼
σ
n

sep GðÞ
2
43
5;ð17Þ
where σis the ratio of the total variation of (U,G), and the separation sep(G)ofthe
vectors G:
σU;G;XðÞ¼
Xc
i¼1Xn
k¼1u2
kijjxkgijj2

;
sep GðÞ¼min
|{z}
ijjjgigjjj2
no
:ð18Þ
The lower value v
XB
specified in (17) indicates a better partition on X, which is right for γ=
2[17]. Studies of the influence of the fuzziness coefficient on the Xi-Beni index point to its
instability for high values of γ[17].
To find the optimal number of classes, a method based on the scheme presented in
Fig. 2is proposed in [15]. The result is presented as a fuzzy model FP
ω
=<U,G,X,c>
which will be referred to as a fuzzy portrait of a class of images [7] within the
general concept proposed by the author, where ωis the class of images from alphabet Ω,
specifiedin(3).
Fig. 2 Generalschemeoffuzzymodelcreation
Multimed Tools Appl
In the scheme, X
ω
m
corresponds to the part of input data Z, specified in (4), for
elements with o
i
=ω;Ν(X
ω
):
m
[0,1]
m
is the normalization procedure for the sample; C
FCM
is the clustering algorithm presented as function C
FCM
(X;c,):
n×m
M
fcn
,wherenis the
number of sample elements, M
fcn
is specified in (7), and сis the parameter of the algorithm that
specifies the number of clusters and other options of the FCM algorithm [2,17]; v*isthe
validity criterion such as (14) or (17); and U1¼1;1;;1
|fflfflfflfflffl{zfflfflfflfflffl}n

T
is the unit vector G
1
={g
1
},
where g
1
is calculated by (12) providing that U=U
1
. The range of values is с=2,3,,
c
max
, with cmax ffiffi
n
p,wherenis the cardinality of the set X
ω
. The validity criteria are
analyzed during the fuzzy model creation step. For criterion (17) the best partition is
at the minimal value.
The proposed scheme of intra-class analysis and fuzzy model FP
ω
creation will be used as
part of the classification method (5). It is the set of samples Z(4) whose elements are
grouped by membership to classes of Ωsuch as Xωi¼x1;x2;;xm
ðÞjx1x2;;xm;ωi
ðÞZ
fg
,
i¼1;K. For every class of images a local fuzzy model is created. The method to combine all
local models into a single fuzzy model is:
FM ¼<FPωi
fg
K
i¼1;D>; ð19Þ
where Dis the decision making algorithm (5). Next, we define D and show how the problem of
pattern recognition is solved under uncertainty using only the information from the model (19).
3.2 Decision making method
As a result of algorithm Dfor object xunder conditions of uncertainty is made. According to
the statement of the problem considered is a fuzzy set e
αe
Ω, specified in (6).
The main idea for this method is as follows. The tuning of the classification model is carried
out for each class of images separately; however, decision making uses combined data for all
classes of images. The advantages of this approach are the transparency of the model and the
adaptability of the method to new types of data. It will be considered as the method for
decision making Dx
*;FM;Λ;ε

¼e
α,whereFM is the model (19), e
αis the formula (6),
Λ¼p1fi;p2fi
no
m
i¼1is the set of pairs including the minimal p1fiand maximal p2fivalues,
used for normalization of data, f
i
is the feature of the object, i¼1;m,mis the number of
features, εis the threshold value, Kis the number of classes, and x* is the new recognizable
object. Let us denote the set of cluster centers corresponding to model FPωias Gωiand their
number as cωi.
Step 1 Transform x* using Λwith formula:
x¼x*p1
p2p1
;ð20Þ
where p
1
and p
2
as in formula (13).
Multimed Tools Appl
Step 2 Calculate α
i
for every ωi;i¼1;Kto have e
αin formula (6). Set ω=ω
i
,α=α
i
,FP ¼FPωi.
Calculate the distance from xto the nearest center of cluster for class ωby model FP:
l¼min gGωdx;gðÞðÞ;ð21Þ
where dis the distance.
Determine B
ω
, which consists of the centers of whole clusters of the model
except class ω:
Bω¼[k
j¼1
ji
Gωk;ð22Þ
where iis the index of class ω.Calculateαwith the formula:
α¼1þXgBω
l2
d2x;gðÞ

1
;ð23Þ
where lis calculated by (21), B
ω
is determined in (22), and dis the distance.
The formula is the result ~
α̃from formula (6) from α(23) for all classes.
Step 3 Change ~
α̃by excluding classes with a low membership α
i
replacing it with the formula:
αi¼0;if α<ε
1;if αε
;ð24Þ
where εis the parameter of the classification method corresponding to the threshold.
Step 4 Calculate the resulting class of images:
Hðe
αÞ¼ ωi;if !i:αi¼1;
ω0;otherwise:
;ð25Þ
where ω
0
is an unknown class of images, affiliation to which denotes the rejection option.
Fig. 3 Class histograms in the fuzzy classification model for the HT descriptor during the decision making
process. The algorithm cannot determine the classes
Multimed Tools Appl
4 Experimental verification
We developed standalone computer software as a practical implementation of the method
proposed in the article. The specially designed library for fuzzy classification was developed in
C++ and used in this application to create a classification algorithm D, specified as in (5). The
application supports learning and test modes of operation. Two sets of samples are used in the
experiments:
learning samples used to create the model in learning mode,
test samples used to estimate the quality of classification in test mode.
A dataset [10] of 12,899 examples of images, including 9340 NE and 3559 PE,
was prepared. The alphabet of classes of images is Ω={NE,PE}. The edge histogram
(EH) and homogeneous textures (HT) descriptors were calculated for the whole
dataset.
We estimated the classification ability of the descriptors using a pattern recognition
method based on histograms [13]. The histograms form a part of the fuzzy classifi-
cation model (described in detail in Section 3). Example histograms of four features
in FCM model are presented for the HT descriptor in Fig. 3and for the EH descriptor
in Fig. 4. The red line corresponds to PE and the green line corresponds to NE; the
range of permissible values of features follows the axis of abscissa, and the normal-
ized degree of membership follows the ordinate. The example features originate from
our fuzzy classification model. Statistical analysis of the set of samples presented with
the HT descriptor shows that this descriptor cannot be used to distinguish classes of
images (Fig. 3). The classification was carried out with the EH descriptor since it
provides good results of statistical analysis of the images (Fig. 4). A comparison of
Fig. 4 Class histograms in the fuzzy classification model for the EH descriptor during the decision making
process. Classes are clearly visible and found by the algorithm
Tab l e 1 Confusion matrix
Method model ω
0
ω
1
ω
2
ω
1
q
10
q
11
q
12
ω
2
q
20
q
21
q
22
Multimed Tools Appl
Figs. 3and 4clearly shows the differences between the two descriptors. HT is not
able to provide visible classes. As a result, the algorithm based on the fuzzy
classification model frequently returns no answers, since it does not have sufficient
data to make a decision. The same problem does not apply to EH, and the classes are
easily determined as separate pieces of data. This shows that the EH descriptor is
suitable for addressing the problem described in this paper, in contrast to HT.
Following analysis of all such histograms, we excluded the HT descriptor from further
consideration. In the remainder of the article, we present the results of experiments for
the EH descriptor which demonstrate the high quality of the algorithm.
Let us consider the confusion matrix Qto evaluate the quality of algorithm D.Cellq
ij
of the
confusion matrix contains the number of elements from the test sample for which ω
i
is the right
class; the classification result is ω
j
.Table1shows an example of the confusion matrix for the
problem with two classes. An artificial class ω
0
is used as the reject option, as in step 4 of the
decision making algorithm.
Let p
i
be the a priori probability for class ω
i
,i¼1;K,K
i¼1pi¼1. To calculate the
probability of an event, the algorithm classifies the object belonging to class ω
i
as an object
of class ω
j
,i¼1;K;j¼0;Kusing the formula:
pij ¼qij
XK
k¼0qik
;ð26Þ
where q
ij
is an element of the confusion matrix, and Kis the number of classes.
a) b) c) d)
Fig. 5 Example images: agood positive example, bbad positive example, cgood negative example, dbad
negative example
Tab l e 2 Learning and test sets
Exp. 1 Exp. 2 Exp. 3 Exp. 4 Exp. 5
Num Desc. Num Desc. Num Desc. Num Desc. Num Desc.
Learning samples PE 378 Good 378 Good 573 Good 573 Good 2348 All
NE 6164 All 284 All 6164 All 431 Good 6164 All
Test samples PE 195 Good 195 Good 3026 All 3026 All 1211 All
NE 3176 All 147 All 3176 All 8909 All 3176 All
Multimed Tools Appl
Two types of error for the algorithm with the reject function are considered. The first type of
error is calculated as:
AI¼XK
i¼1piXK
j¼1
ji
pij
0
B
@1
C
A;ð27Þ
where p
i
is the probability for class ω
i
and p
ij
is calculated by (26). To calculate the probability
of an event:
R¼XK
i¼1pipi0;ð28Þ
where p
i
,p
i0
are as in (13). The second type of error includes the first type of error and a
rejection, and is calculated as:
AII ¼AIþRð29Þ
We conducted a series of experiments on the prepared set of samples. The dataset contains
examples of varying quality. The quality of negative examples has a lower impact on the
results (2629). For positive images, poor quality has a significant impact on the results.
Tab l e 3 Experiment results
Knife detection Exp1 Exp2 Exp3 Exp4 Exp5
Learn A
I
27.74 % 25.47 % 14.30%13,90% 17.16 %
R1.23 % 1.53 % 0.49 % 0.53 % 0.87 %
A
II
26.50 % 27.00 % 14.78 % 14.42 % 17.92 %
Tes t A
I
27.02 % 26.76 % 14.32%14.59% 17.46 %
R1.53 % 0.70 % 0.64 % 0.66 % 0.84 %
A
II
28.56 % 27.46 % 14.96 % 15.25 % 18.30 %
a) b)
Fig. 6 Resized images: awithout quality enhancement, bwith quality enhancement
Multimed Tools Appl
Each image was analyzed by an operator in order to select good examples. As a result, 573
good positive examples and 431 good negative examples were selected. Figure 5shows some
example images.
Five sets of samples were selected for further tests. Their numerical descriptions are
presentedinTable2. Examples are marked as goodif the selected examples are used, and
allif all available examples are used. For every example, the set of samples is divided at a
ratio of two to one.
Results of experiments in Table 3show that the best characteristic has fuzzy models created
on sets which contain good positive examples only. This is experiments 3 and 4, with 14 % of
errors on test samples. This shows that the model should be trained on good positive examples
only. The quality of negative examples has a lower impact on the result.
When compared to the state-of-the-art algorithm, our results are satisfactory. We conducted
additional experiments to check the accuracy of other methods on the same dataset. The
algorithm presented by Żywicki et al. returns 29 % of errors, while the algorithm presented by
Masimova et al. returns 23 % of errors. Results presented by Kmiećet al. are impressive (92 %
accuracy on 40 images); however, the best outcome achieved on our dataset was 21 % of
errors.
In addition, we conducted some experiments with super-resolution techniques. We created a
new dataset with enhanced images from the original database of knife images. We used two
known algorithms for a single image at super-resolution. The first is a learning-based method
introduced by Kim and Kwon [7], while the second is a method based on iterative Weiner
filtering originally presented by Hung and Siu [5]. During initial tests, we were not able to train
Kims algorithm, therefore we did not achieve a similar (or close to similar) outcome with our
dataset in terms of image quality. The second algorithm gave an outcome similar to that
reported in the original paper. The average PSNR achieved for the image database was 0.41 dB
lower. We assume that an appropriate level of image enhancement was achieved. Figure 6
Tab l e 4 Learning and test sets for the experiment with enhanced images
Exp.6
Num Desc.
Learning samples PE 2348 All
NE 6164 All
Test samples PE 1211 All
NE 3176 All
Tab l e 5 Results for the experiment with enhanced images
Knife detection Exp6 Exp5
Learning A
I
19.74% 17.16 %
R1.01 % 0.87 %
A
II
20.71 % 17.92 %
Tes t A
I
20.47% 17.46 %
R0.91 % 0.84 %
A
II
21.07 % 18.30 %
Multimed Tools Appl
shows enhanced and non-enhanced images from the dataset, with the improved quality clearly
visible. The image looks smooth and does not show distortions near the edges. All images
from the dataset were enlarged by a magnitude of two (from a resolution of 100×100 pixels to
200× 200 pixels).
The next step was to apply our method to a new, high resolution (HR) dataset. Experiment 6
was conducted using the HR database under the same conditions as experiment 5 (Table 4).
This allowed us to directly compare results from both experiments. A further experiment was
conducted on the entire dataset, since it was difficult to determine which images qualify as
goodfollowing enhancement.
Results of experiment 6 are shown in Table 5. The experiments were conducted with a
cross-validation technique to avoid significant distortions. The results achieved in experiment
6 are worse than those from experiment 5, which means that image enhancement does not
improve classification accuracy. However, the images appear to be visually better to the human
eye, while those treated with the algorithm do not. This is due to the EH descriptors, since the
enhancement process smooths the image and some information is lost from near the edges. EH
descriptors contain less data, therefore the classification algorithm cannot provide results with
the same or higher level of accuracy (Table 6).
5 Conclusions
The article presents a fuzzy model for knife detection. Selected elements of the MPEG-7
descriptor are used as a feature for the pattern recognition problem. Experimental verification
shows that the method can be successfully used in difficult situations. The result of less than
15 % of images being mislabeled for the entire dataset is good for the problem at hand,
although it would need to be improved for real systems. The method can be used simulta-
neously with other image processing techniques. Further research will be aimed at assessing a
set of sequential images from video, and using a combination of our method with other
approaches.
The approach can be used for different pattern recognition problems with non-uniform
classes where the object has a specific form, such as the knife in our example. However, image
descriptors should be selected according to the features of the objects under consideration. The
fuzzy model must be trained using good examples only. Subsequently, such a model can be
applied to examples with less clear images. We also intend to generalize our algorithm to solve
the detection problem for a wide range of objects in an automatic way.
A further conclusion concerns processing images following quality enhancement. The
experiment shows that in some cases images should be analyzed before any manipulation is
applied, even if enhancement improves the image quality to the human eye (e.g., CCTV camera
operator). This is significant in creating architectures for modern, intelligent CCTV systems.
Tab l e 6 Results comparison for state-of-the-art and proposed algorithms
Algorithm Żywicki [21]Kmieć[4] Maksimova [14]Proposed Proposed (with image
enhancement)
Error rate (achieved
on dataset [10])
29 % 21 % 23 % 14 % 20 %
Multimed Tools Appl
Acknowledgments This research has been financed by the European Regional Development Fund under the
Innovative Economy Operational Programme, INSIGMA project No. POIG.01.01.02-00-062/09.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International
License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and repro-
duction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a
link to the Creative Commons license, and indicate if changes were made.
References
1. Baran R, Glowacz A, Matiolanski A (2013) The efficient real- and non-real-time make and model
recognition of cars, Multimedia Tools and Applications. Springer, USA
2. Bezdek JC, Keller J, Krisnapuram R, Pal R (2005) Fuzzy models and algorithms for pattern recognition and
image processing. Springer Verlag, New York
3. Chen N (2004) Fuzzy classification using self-organizing map and learning vector quantization. In: CASD
MKM 2004. Lecture notes in artificial intelligence, vol. 3327, 4150. Springer-Verlag
4. Glowacz A, KmiećM, Dziech A (2013) Visual detection of knives in security applications using active
appearance models, multimedia tools and applications. Springer, USA
5. Hung K-W, Siu W-C (2012) Single image super-resolution using iterative Wiener filter, acoustics, speech
and signal processing (ICASSP), 2012 I.E. International Conference on, vol., no., 1269, 1272, 2530. doi:
10.1109/ICASSP.2012.6288120
6. Ishibuchi H, Nakashima T, Nii M (2005) Classification and modeling with linguistic information granules.
Springer
7. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior,
IEEE transactions on pattern analysis & machine intelligence. vol. 32, no. 6, 11271133. doi:10.1109/
TPAMI.2010.25
8. KmiećM, Glowacz A (2011) An approach to robust visual knife detection. Mach Graph Vis
20(2):215227
9. KmiećM, Głowacz A, Dziech A (2012) Towards robust visual knife detection in images: Active appearance
models initialised with shape-specific interest points. In: Dziech A, Czyżewski A (eds.), Multimedia
communications, services and security, Vol. 287 of communications in computer and information science.
Springer Berlin Heidelberg, 148{158. doi:10.1007/978-3-642-30721-8_15
10. Knives images database, available for download, http://kt.agh.edu.pl/~matiolanski/KnivesImagesDatabase/
11. Konor A (2005) Computational intelligence: Principles, techniques and applications. Springer, Heidelberg
12. Kuncheva LI (2005) Fuzzy classifier design. Fhysica-Verlag, Heidelberg
13. Maksimova A (1995) The model of data presentation with fuzzy portraits for pattern recognition. In: Int. J.
Comput, vol. 11, issue 1, 1995, 1724
14. Maksimova A (2013) Knife detection scheme based on possibilistic schell clustering. In:
Proceedings of 6th International Conference Multimedia communications, services and security,
143152. Krakow, Poland
15. Maksimova A (2013) Decision making method for classifying models based on intra-class clustering on
FCM-algorithm. In: Artificial Intelligent J., vol. 3(61), 171178 (In Russian)
16. Mery D, Riffo V, Zuccar I, Pieringer C (2013) Automated x-ray object recognition using an efficient search
algorithm in multiple views. In: Computer vision and pattern recognition workshops (CVPRW), 2013 I.E.
Conference on, 368{374. doi:10.1109/CVPRW.2013.62
17. Pal NR, Bezdek JC (1995) On cluster validity for the fuzzy c-means model. In: J. IEEE transactions on fuzzy
systems, vol.3, no. 3, August 1995, 370379
18. Ro YM, Kim M, Kang HK, Manjunath BS, Kim J (2001) MPEG-7 homogeneous texture descriptor. ETRI J
23(2):4151
19. Won CS, Park DK, Park S-J (2002) Information: efficient use of MPEG-7 edge histogramm. In: ETRI J., vol.
24, No. 1, Februar 2002, 2330
20. Yu MA, Kozlovskii VA (2011) Algorithm of pattern recognition with intra-class clustering. In:
Proceedings of 11th International Conference Pattern recognition and processing,5457,
Minsk
21. Żywicki M, Matiolański A, Orzechowski TM, Dziech A (2011) Knife detection as a subset of object
detection approach based on Haar cascades. In: Proceedings of 11th International Conference Pattern
recognition and information processing,139
142.Minsk,Belarus
Multimed Tools Appl
M. Sc. Eng Andrzej Matiolanski He is a Ph.D. student and Teaching Assistant at the Department of
Telecommunications of AGH University of Science and Technology. He has received his M.Sc. degree from
the Faculty of Physics and Applied Computer Science of AGH University of Science and Technology in 2010.
His research interests include computer vision, image processing and super-resolution algorithms. He has actively
participated in both national and international research projects like INDECT, INSIGMA, MAYDAY EURO
2012 and IMCOP. Email: matiolanski@kt.agh.edu.pl
Aleksandra Maksimova received MSc degree (2004) of Computer Science from the Donetsk National
University and PhD of Information technology (2014) in Ukraine. Currently she is researcher in the Institute
of Applied Mathematics and Mechanics of National Academy of Science of Ukraine. Primary research interests
are pattern recognition, data mining, fuzzy theory, fuzzy classification methods.
Multimed Tools Appl
Andrzej Dziech holds the position of a full professor at the Department of Telecommunications of AGH
University of Science and Technology in Krakow, Poland. He received his M.Sc. and Ph.D. degrees from the
Institute of Electrical Engineering in Saint Petersburg in 1970 and 1973, respectively, and the D.Sc. from
Technical University of Poznan in 1978. He is an author of 6 books and nearly 180 publications. He was a
supervisor of 18 Ph.D. students. His fields of interest are related to digital communication, image and data
processing, data compression, information and coding theory, random signals, computer communications
networks and signal processing. He was awarded 4 times for research achievements by the Ministry of Education
of Poland. Professor Dziech actively participated in numerous international research projects, e.g., Tempus,
Knixmas, Calibrate. Currently, he is coordinating a European Union FP7 integrated project INDECT.
Multimed Tools Appl
... Silah tanıma için Ganada üniversitesinin araştırma grubuna ait veri kümesi tercih edilmiştir (Int Kyn. 6) . Yine bıçak tanıma için 'Knive Image Database' veritabanı kullanılmıştır( Matiolański et al. 2016). ...
Article
This study presents an effective and innovative solution to overcome the security issues/problems in public places. As an alternative video surveillance system, the proposed method detects and localizes gun and knife objects from videos in real-time. In connection with Convolutional Neural Network (CNN) based object detection, the Faster R-CNN structure was applied to detect gun and knife objects with the highest performance. After conducting a simulation on test images, we have found that the F1-score performance of the developed system is about 70% recognition rates. The trained Faster R-CNN model can be utilized for different public places, including airplanes, bus stations, stadiums, and public vehicles, where the security is an important factor. Moreover, the developed method can be embedded in the local surveillance system in terms of reporting dangerous objects as well as minimizing the risks caused by such objects.
... In this study, a data set suitable for the purposes was created by collecting images from open access data sets [9,[12][13][14], which have repeatedly been the source of many previous studies in the literature and original internet browsers and video pages [4,15]. This dataset has 16000 images containing 9500 knives, 3500 guns, and 3000 ordinary pictures. ...
... Image enhancement [1] refers to the process of heightening a particular piece of information in an image based on a specific requirement [2] while weakening or eliminating unnecessary information [3] and improving the image quality [4]. Image enhancement [5] aims to make certain image features more distinct and prominent, in order that the processed image is more propitious to human visual characteristics and machine analysis, and more advanced image processing and analysis can be achieved [6]. Traditional medical image enhancement technologies [7][8][9][10][11] are generally divided into three categories, namely the spatial domain methods, frequency domain methods, and deep learning methods. ...
Article
Full-text available
Medical image quality requirements have been increasingly stringent with the recent developments of medical technology. To meet clinical diagnosis needs, an effective medical image enhancement method based on convolutional neural networks (CNNs) and frequency band broadening (FBB) is proposed. Curvelet transform is used to deal with medical data by obtaining the curvelet coefficient in each scale and direction, and the generalised cross-validation is implemented to select the optimal threshold for performing denoising processing. Meanwhile, the cycle spinning scheme is used to wipe off the visible ringing effects along the edges of medical images. Then, FBB and a new CNN model based on the retinex model are used to improve the processed image resolution. Eventually, pixel-level fusion is made between two enhanced medical images from CNN and FBB. In the authors' study, 50 groups of medical magnetic resonance imaging, X-ray, and computed tomography images in total have been studied. The experimental results indicate that the final enhanced image using the proposed method outperforms other methods. The resolution and the edge details of the processed image are significantly enhanced, providing a more effective and accurate basis for medical workers to diagnose diseases. © 2019 Institution of Engineering and Technology. All rights reserved.
... The Fisher vector formulations are described in brief as follows. Moreover, detailed discussion about fisher vector formulation and detailed analysis can be found in the references [43] and [15]. The local features are modelled using GMM model which is defined as: ...
Article
Full-text available
Animal biometrics is an emerging research field that develops quantified methodologies for representing and detecting the visual appearances of animal based on generic features and primary biometric characteristics. The identification of individual cattle is an important issue for classification of different breads, their registration, traceability, health management, and verification of false insurance claim throughout the world. To solve these major problems, the muzzle (nose) point image pattern of cattle is a suitable and primary biometric characteristic for the recognition of cattle. The recognition of muzzle point images is similar to the recognition of minutiae points in the human fingerprints. In this paper, we propose a hybrid feature extraction approach for the automatic recognition and classification of cattle breeds based on captured muzzle point image pattern features using low- cost camera. The major contributions of this research is based on following aspects: (1) preparation of muzzle point image dataset, (2) extraction of salient set of features and (3) K-nearest neighbour (K-NN), Fuzzy-KNN, Decision Tree (DT), Gaussian Mixture Model (GMM), Probabilistic Neural Network (PNN), Multilayer Perceptron (MLP) and Naive Bayes classification models are applied to recognition, and classification of individual cattle using their muzzle point pattern. This paper, therefore, demonstrates the automatic recognition and classification of livestock using the set of extracted muzzle point image features. The experimental results show that proposed hybrid feature extraction and recognition approach outperforms the current state-of- the art method for the identification of individual cattle using their muzzle point image pattern.
Article
The ability to detect gun and gun held in hand or other body parts is a typical human skill. The same problem presents an imperative task for computer vision system. Automatic observer independent detection of hand held gun or gun held in the other body part, whether it is visible or concealed, provides enhance security in vulnerable places and initiates appropriate action there. Compare to the automatic object detection systems, automatic detection of gun has very few successful attempts. In the present scope of this paper, we present an extensive survey on automatic detection of gun and define a taxonomy for this particular detection system. We also describe the inherent difficulties related with this problem. In this survey of published papers, we examine different approaches used in state-of-the-art attempts and compare performances of these approaches. Finally, this paper concludes pointing to the possible research gaps in related fields.
Article
Full-text available
As the security threat and crime rate have been increased all over the globe, the video surveillance system using closed-circuit television (CCTV) has become an essential tool for many security-related applications and is widely used in many areas as a monitoring system. However, most of the data collected by the video surveillance system is used as evidence of objective data after crime and disaster have occurred. And, often time, video surveillance systems tend to be used in a passive manner due to the high cost and human resources. The video surveillance system should actively respond to detect crime and accidents in advance through real-time monitoring and immediately transmit data in case of an accident. This study proposes developing an intelligent video surveillance system that can actively monitor in real-time without human input. In solving the problems of the existing video surveillance system, deep learning technology will be carried through the data processing model design to visualize data for crime detection after building an artificial intelligence server and video surveillance camera. In addition, this design proposes an intelligent surveillance system to quickly and effectively detect crimes by sending a video image and notification message to the web through real-time processing.
Preprint
Full-text available
Despite rapid advances in image-based machine learning, the threat identification of a knife wielding attacker has not garnered substantial academic attention. This relative research gap appears less understandable given the high knife assault rate (>100,000 annually) and the increasing availability of public video surveillance to analyze and forensically document. We present three complementary methods for scoring automated threat identification using multiple knife image datasets, each with the goal of narrowing down possible assault intentions while minimizing misidentifying false positives and risky false negatives. To alert an observer to the knife-wielding threat, we test and deploy classification built around MobileNet in a sparse and pruned neural network with a small memory requirement (< 2.2 megabytes) and 95% test accuracy. We secondly train a detection algorithm (MaskRCNN) to segment the hand from the knife in a single image and assign probable certainty to their relative location. This segmentation accomplishes both localization with bounding boxes but also relative positions to infer overhand threats. A final model built on the PoseNet architecture assigns anatomical waypoints or skeletal features to narrow the threat characteristics and reduce misunderstood intentions. We further identify and supplement existing data gaps that might blind a deployed knife threat detector such as collecting innocuous hand and fist images as important negative training sets. When automated on commodity hardware and software solutions one original research contribution is this systematic survey of timely and readily available image-based alerts to task and prioritize crime prevention countermeasures prior to a tragic outcome.
Conference Paper
Full-text available
In this paper, we propose an iterative Wiener filter which can simultaneously perform interpolation and restoration by using non-local means to directly model the correlation between the desired high-resolution image and observed low-resolution image. A novel mechanism is proposed to control the decay speed of the correlation function while iteratively updating both estimated correlation and high-resolution image. During the iterations, the image is decomposed into patches with similar intensities at initial iterations and the patches are connected naturally with good convergence. Experimental results show that the proposed algorithm is able to produce natural image structures, and provides better PSNR and visual quality than the state-of-the-art algorithms using the sparse representation and natural image priors.
Article
Full-text available
Make and Model recognition of cars (MMR) has become an important element of automatic vision based systems. Nowadays, MMR utility is commonly added to traffic monitoring (e.g. Licence Plate Recognition) or law enforcement surveillance systems. Facing the growing significance of Make and Model Recognition of cars we have designed and implemented two different MMR approaches. According to their disparate assumption data of these implementations one is obligated to estimate different car models in milliseconds (with a bit less emphasis placed on its accuracy) while the other is aimed first of all to reach higher classification accuracy. Both the implemented MMR approaches, called Real-Time and Visual Content Classification, respectively, are described in this paper in detail and with reference to other MMR methods presented in the literature. Analyses of their performance with respect to classification accuracy and, in case of the Real-Time approach, to its response time are also presented, discussed and finally concluded.
Article
Full-text available
In this paper, a novel application of Active Appearance Models to detecting knives in images is presented. In contrast to its popular applications in face segmentation and medical image analysis, we not only use this computer vision algorithm to locate an object that is known to exist in an analysed image, but–using an interest point typical of knives–also try to identify whether or not a knife exists in the image in question. We propose an entire detection scheme and examine its performance on a sample test set. The work presented in this paper aims to create a robust visual knife-detector to be used in security applications.
Article
Computerised monitoring of CCTV images is attracting a lot of attention both from potential end-users seeking to increase the effectiveness of their video surveillance systems and as a popular research topic as new methods and algorithms are being developed. In this paper an approach to detecting knives in images is presented. It is based on the use of Histograms of Oriented Gradients (HOG), feature descriptors invariant to geometric and photometric transformations except for rotation. We introduce a dataset containing images of knives in different backgrounds and in varying lighting conditions and evaluate the performance of an HOG-based SVM classifier. We study the question of creating a detector based on knife blade colour and discuss the use of GPU parallel computing as a method of speeding up the detection process.
Conference Paper
This paper deals with a novel method for knife detection in images. The special feature of any knife is its simple geometric form. The proposed knife detection scheme is based on searching object with corresponding form. Fuzzy and possibilistic shell clustering methods are used to find contours of objects in the picture. These methods give as a result a set of second-degree curves offered in analytical form. To answer the question whether there is a knife in the picture, the angle between two such curves is calculated and its value is estimated. A C++ program was developed for experiments allowing to confirm the suitability of the proposed scheme for knife detection. Robustness of possibilistic c quadric shell clustering to outlier points let us use this algorithm in real-life situations. The objective of research is to use the results for developing a system for monitoring dangerous situation in urban environment using public CCTV systems.
Conference Paper
In this paper a novel application of Active Appearance Models to detecting knives in images is presented. Contrary to popular applications of this computer vision algorithm such as face segmentation or medical image analysis, we use it not only to locate an instance of an object that is known to exist in the analysed image. Using an interest point typical to knives we try to answer the question, whether a knife is or is non-existent in the image in question. We propose an entire detection scheme and examine its performance on a sample test set. The work presented in this paper aims at creating a robust visual knife detector that will find application in computerised monitoring of the public using CCTV.
Article
This book focuses on various techniques of computational intelligence, both single ones and those which form hybrid methods. Those techniques are today commonly applied issues of artificial intelligence, e.g. to process speech and natural language, build expert systems and robots. The first part of the book presents methods of knowledge representation using different techniques, namely the rough sets, type-1 fuzzy sets and type-2 fuzzy sets. Next various neural network architectures are presented and their learning algorithms are derived. Moreover, the family of evolutionary algorithms is discussed, in particular the classical genetic algorithm, evolutionary strategies and genetic programming, including connections between these techniques and neural networks and fuzzy systems. In the last part of the book, various methods of data partitioning and algorithms of automatic data clustering are given and new neuro-fuzzy architectures are studied and compared. This well-organized modern approach to methods and techniques of intelligent calculations includes examples and exercises in each chapter and a preface by Jacek Zurada, president of IEEE Computational Intelligence Society (2004-05).
Conference Paper
In order to reduce the security risk of a commercial aircraft, passengers are not allowed to take certain items in their carry-on baggage. For this reason, human operators are trained to detect prohibited items using a manually controlled baggage screening process. In this paper, we propose the use of an automated method based on multiple X-ray views to recognize certain regular objects with highly defined shapes and sizes. The method consists of two steps: 'monocular analysis', to obtain possible detections in each view of a sequence, and 'multiple view analysis', to recognize the objects of interest using matchings in all views. The search for matching candidates is efficiently performed using a lookup table that is computed off-line. In order to illustrate the effectiveness of the proposed method, experimental results on recognizing regular objects --clips, springs and razor blades-- in pen cases are shown achieving around 93% accuracy for 120 objects. We believe that it would be possible to design an automated aid in a target detection task using the proposed algorithm.