Download full-text PDF

A Novel Fuzzy-Based Smoke Detection System Using Dynamic and Static Smoke Features

Conference Paper · May 2015with109 Reads
DOI: 10.1109/IranianCEE.2015.7146309
Conference: Conference: 23rd Iranian Conference on Electrical Engineering (ICEE), At Sharif University of Technology, Tehran, Iran
Abstract
Automatic fire surveillance is an important task for providing emergency response in the event of unexpected fire hazards. Early detection of fire can substantially mitigate the ecological or economical costs associated with a fire disaster. In this regard, as smoke usually always precedes fire, an intelligent smoke detection system is proposed that exploits a Fuzzy Inference System (FIS) in order to aggregate the features of smoke. In addition, robust smoke feature detection algorithms are implemented that take into account both dynamic and static characteristics of smoke. The smoke features include motion, motion orientation (estimated by using the accumulation of motion) for the former and texture for the latter. Experimental results on different video frames show that the proposed smoke detection system has robust performance on detecting the existence of smoke which shows the effectiveness of the proposed smoke detection system.
Figures
A Novel Fuzzy-Based Smoke Detection System Using
Dynamic and Static Smoke Features
Yashar Deldjoo
1
, Fatemeh Nazary
2
and Ali M. Fotouhi
3
1
Department of Signal and Systems, Chalmers University of Technology, Gothenburg, Sweden
Email: deldjoo@alumni.chalmers.se
2
Department of Electrical Engineering, Science and Research branch, Islamic Azad University, Qazvin, Iran
Email: atena.nazary@gmail.com
3
Department of Electrical Engineering, Tafresh University, Tafresh, Iran
Email: fotouhi@tafreshu.ac.ir
Abstract —Automatic fire surveillance is an impor-
tant task for providing emergency response in the event
of unexpected fire hazards. Early detection of fire can
substantially mitigate the ecological or economical costs
associated with a fire disaster. In this regard, as smoke
usually always precedes fire, an intelligent smoke detec-
tion system is proposed that exploits a Fuzzy Inference
System (FIS) in order to aggregate the features of
smoke. In addition, robust smoke feature detection
algorithms are implemented that take into account both
dynamic and static characteristics of smoke. The smoke
features include motion, motion orientation (estimated
by using the accumulation of motion) for the former
and texture for the latter. Experimental results on
different video frames show that the proposed smoke
detection system has robust performance on detecting
the existence of smoke which shows the effectiveness of
the proposed smoke detection system.
Keywordssmoke detection, smoke features, fuzzy
inference system, motion block
I. Introduction
It is well-known that smoke precedes flame in a fire
accident. In addition, the range of smoke spread is much
larger than that of flame. Therefore, smoke detection is
not only more possible but also of vital importance as
early detection of smoke can prevent economical, ecological
and human-life endangering threats which can arise as the
result of a fire disaster [1], [2].
Conventional fire detection techniques are based on
physical properties of smoke and fire such as humidity of
the air, temperature and so on [3]. Despite the low cost
and simplicity, these non-visual fire detection mechanisms
share a major bottleneck in the sense that they all need
to be in close proximity of the fire area [3]. The latter
could be efficiently and effectively managed by using real-
time visual surveillance and traffic monitoring technologies
which do not require a minimum-length distance and
also provide strong smoke feature analysis capabilities due
to significant advances in the field of image and video
analysis [
4].
Robust visual smoke detection heavily depends on
good description of features of smoke. The main (visually
discernible) features of smoke can be noted as: (1) color,
(2) motion, (3) motion orientation, (4) texture and so
on. Based on these characteristics of smoke, many smoke
detection methods have been proposed in the literature [1],
[5], [6], [7]. For example in [1], an elegant and working
model for smoke detection is proposed that takes into
account the main aforementioned smoke features. The
proposed method includes first detection of motion based
on block-wise subtraction in every two successive frames.
This results in a number of candidate regions for smoke,
known as Motion Blocks (MBs). In addition, assuming that
smoke color resides solely in the white-bluish or black-
grayish part of the color spectrum, the resulted MBs were
further analyzed for their color. Furthermore, noting the
inclination of smoke to spread in the upward direction
due to hot air flow, the author proposed to discard the
candidate MBs that did not conform with this principle,
assuming them not to be Smoke Blocks (SBs). The method
proposed in [1], though promising, has some major side
effects as well. For example, it does not exploit many other
static or dynamic features of smoke such as wavelet [8],
[9]ortexture[6] which can bring high discrimination
to the system. In addition and more importantly, the
author does not explicitly describe how the features in the
image are combined to detect SBs, e.g. in a cascade or
parallel manner. In this line, while it seems the features
are combined in a cascade manner, even this approach
is not guaranteed to produce the optimal result because
a MB requires a "‘pass"’ value from all previous steps
in the serial interconnection to be considered as a SB.
This can be a hard condition for SB-detection and many
other potential SBs may be discarded via this approach.
Several other works have addressed the problem of smoke
detection from other perspectives. For instance in [7], the
authors proposed a charming fire surveillance method that
deals with fire detection and smoke detection problems
separately. To this end, in the fire detection stage, various
color spaces are considered as a major cue for recognizing
flames. A set of Gabor wavelets are also employed to model
the spectral, spatial and temporal characteristics of fire.
As for smoke detection, MBs are detected using Motion
Histogram Image (MHI) approach [10]. Smoke detection
is realized by employing block-wise flicker detection and
smoke texture models in the subsequent stages. Readers
are referred to [11], [12] for details on other approaches for
smoke detection.
2015 23
rd
Iranian Conference on Electrical Engineering (ICEE)
978-1-4799-1972-7/15/$31.00
c
2015 IEEE 729
Figure 1. The flowchart of our proposed algorithm
II. Proposed method
Figure 1 illustrates the main steps used in our pro-
posed smoke detection algorithm. Based on the presented
flowchart, our proposed algorithm consists of two main
feature analysis steps followed by a Fuzzy Inference System
(FIS) and a decision-making strategy to decide when to
signal a fire alarm. In the dynamic feature analysis section,
smoke main dynamic features are computed aiming to sep-
arate moving objects from the background. The resulting
image after this step is an image with a number of detected
and annotated Motion Blocks (MBs) standing for smoke or
other particles in the image with motion traits similar to
smoke. Static feature analysis techniques are also employed
to extract smoke static features such as color and texture.
We provide a detailed description of the feature analysis
techniques exploited in this work in sections II-A and II-B.
The obtained results from feature analysis are aggregated
via a Fuzzy Inference System (FIS) followed by a decision-
making stage for deciding when to signal a fire alarm.
A. Dynamic feature extraction
In dynamic analysis of smoke [1], motion of the smoke
is one of the most important features that is exploited in
three main steps as explained in the following:
1) Motion detection: For detecting motion in video,
every two successive frames are subtracted. The difference
image is binarized by comparing the pixel values with T
1
a predefined threshold. The difference between two frames
at time t t and t is given by d(x, y, t t)as:
d(x, y, t t)
=
1if|I(x, y, t t) I(x, y, t)| >T
1
0 otherwise
(1)
The above approach is simple and computationally
efficient. However, considering the great amount of noise
that can be present in images (e.g. due to intrinsic
electronic noises and quantification), pixel-by-pixel image
comparison is susceptible to false segmentation. To rem-
edy, Yuan [1] proposed block-based frame differencing in
which pixels within a block are summed up and then the
difference of blocks are calculated. The difference between
two blocks at time t t and t is given by b(i, j, t t)
as :
d(x, y, t t)
=
1if
x,yb
ij
[I(x, y, t t) I(x, y, t)]
>T
2
0 otherwise
(2)
In the difference frame b(i, j, t t), the blocks with
value "1" as shown in Figure 2 are considered as MBs
where T
2
is a predefined suitably chosen threshold and
is a function of lightening conditions in the image. The
relationship between the width and height of the image
and the block is established via W
i
= W
b
N
c
and
H
i
= H
b
N
r
in which N
c
and N
r
are the number of
motion blocks in rows and columns of the original image.
Figure 2. Block-wise processing (a)The size of a block (b)Motion
and non-motion blocks
2) Motion orientation: One of the important features of
smoke is its motion orientation which points upward in the
air due to the hot airflow. In order to exploit this feature,
the orientation of motion is segregated into 8 directions
which include: 0, 45, 90, 135, 180, 225, 270, 315 degrees.
For every MB in frame t, we preform a search in the frame
t t as the following error function:
argmin
b
ij
x,yb
ij
I(x, y, t)
x,yb
ij
I(x, y, t t)
(3)
where b
ij
is a candidate MB in frame t and b
ij
is the
searching block in frame t t. The searching block in
frame t t includes blocks in 8 main directions of 0, 45,
90, 135, 180, 225, 270, 315 degrees around the location of
the candidate MB in frame t t with distances equal
to dis =1, 2, 3 pixels away from its center. Therefore, b
ij
contains a total of 8*3=24 candidate MBs in the frame
t t for the sake of comparison.
3) Motion accumulation: The block-wise orientation
analysis is not quite precise and can be susceptible to false
calculation. In order to reduce this effect, for every MB
in frame t, a waiting time of W
t
is considered in which
we accumulate the estimated motion orientations from
previous step and report the orientation with maximum of
occurrence as the final orientation of the interest of that
particular MB. The process is repeated for all MBs in the
current frame t.
2015 23
rd
Iranian Conference on Electrical Engineering (ICEE)
730
B. Static feature extraction
The color of smoke changes around the gray part of the
color spectrum depending on the combustible materials.
This feature is exploited by obtaining an RGB-contrast
image [5] from a candidate MB which is more robust than
conventional color analysis methods as such approaches
usually rely on experimental thresholds to discriminate
colors. The result of RGB-contrast evaluation is provided
to a texture extraction method to extract the texture
property on the RGB-contrast image for classification.
1) RGB-contrast Evaluation: Smoke is usually gray and
for a gray color, the three components of R, G and B
are similar in values. Other pure colors have dissimilar R,
G and B values. The concept of RGB-contrast image [5]
provides a robust approach for giving discrimination to
gray valued objects compared with other colors and there-
fore enhancing detection of this color. For a neighborhood
of 2*2 of pixels , an RGB-contrast image is computed as
follows:
RGBc(i, j)
=
max (R, G, B)(i + j)mod3=0
median (R, G, B)(i + j)mod3=1
min (R, G, B)(i + j)mod3=2
(4)
where i and j represent the coordinates in x and y
directions. For grayish colors, the values of max, median
and min are approximately in a similar range. For other
colors, the three values are very different. Therefore for
any color except in the gray range, the 2*2 block colors
are dissimilar which is of interest.
Figure 3. The result of applying an RGB-contrast feature on four
uniform colors. Note that only in the gray image, the RGB-contrast
results in distinct texture pattern.
2) Texture analysis: The result of evaluating an RGB-
contrast feature on a gray-colored image (similar to smoke)
is one with texture patterns quite different compared to
non-gray colors (e.g. pure green, red or blue). We show
this effect in figure 3. The property of texture is exploited
by computing the roughness of the texture according to
Eq. 5
R =(1
1
1+σ
2
)(5)
where σ is low for colors with similar (R, G, B) values
(i.e. gray colors), therefore R is low for smoky-like gray
colors.
C. Fuzzy inference:
A Fuzzy Inference System (FIS) is developed to model
the qualitative aspects of human knowledge and reasoning
on detecting a fire event, without quantitative knowledge
available [13]. FIS is a popular system modeling technique
which is capable of inferring complex non-linear relation-
ships between the inputs and outputs of a system. The
latter is realized based on concepts in fuzzy set theory,
fuzzy if-then rules and fuzzy reasoning [14], [15], [16]. We
employed a Mamdani-based FIS as apposed to Sugeno
because the inputs and outputs in our model are all fuzzy
concepts.
1) Mamdani’s rule: The Mamdani’s rule for a Multi-
Input Single-Output (MISO) system is defined as follows.
If (X
1
is A
1
)and(X
2
is A
2
) ... and (X
p
is A
p
)then(Y
is C)whereA
j
and C are linguistic variables that define
partitioning of the input and output space respectively and
are characterized by membership functions M
j
and M
o
for
the input and the output variables respectively.
Figure 4. Architecture of Mamdani FIS model for our proposed
algorithm
2) The structure of the proposed Mamdani-based FIS:
The architecture of the Mamdani FIS used in our system
for smoke classification is illustrated in Figure 4.The
inputs to the system are (θ, R) the motion orientation of
a MB and the texture roughness computed on each MB.
We apply FIS on each MB individually and give each
MB a score based on fusion of the inputs in a fuzzified
manner. The output of the FIS is p, a membership value
that represents the likelihood of an MB to be Smoke
Block (SB). This procedure is repeated for all MBs in the
image. Finally, a decision making strategy is implemented
to decide when to alarm for fire based on the proportion of
the blocks in the image detected as MB and their likelihood
value. The FIS in our proposed model is composed of five
functional layers as follows:
Layer 1 "The Fuzzy Layer": The inputs to this layer
are a number of MBs selected from the original image,
2015 23
rd
Iranian Conference on Electrical Engineering (ICEE)
731
each containing a 2-tuple (θ
i
,R
i
) representing the
orientation and roughness of ith MB. For each input
in the 2-tuple, we obtain a membership value in the
corresponding fuzzy sets, in a process also known as
fuzzification.
Figure 5. The membership functions of the inputs (a)orientation
(b) texture and (c) output. Note that "low" texture and "upward"
orientation should result in "danger" mode in the output.
Layer 2 "The Product Layer": Through a specific t-
norm operator in this layer a weight is evaluated on
each rule that represents the firing strength of the rule.
We use the classical "min" operator. A total of 12 rules
are contemplated in our FIS as listed in Table I;
Layer 3 "The Implication Layer": Based on the firing
strength in the product layer, 12 weights are evaluated
that are used to compute a fuzzy output for each rule.
This would require a MF for the output layer.
Layer 4 "The Aggregation Layer": The overall fuzzy
output is derived by applying max operator on each
of the 12 output.
Layer 5 "The Defuzzification Layer": The result of in-
ference is transformed from a fuzzy set to a crisp value
based on an appropriate scheme, which is centroid of
area in our model.
Table I. 12 rules based on mamdani FIS model
1. If (θ is M
sl
) and (R is M
low
)then(Z
1
= M
cldanger
)
2. If (θ is M
sl
) and (R is M
mid
)then(Z
2
= M
clsaf e
)
3. If (θ is M
sl
) and (R is M
high
)then(Z
3
= M
saf e
)
4. If (θ is M
up
) and (R is M
low
)then(Z
4
= M
danger
)
5. If (θ is M
up
) and (R is M
mid
)then(Z
5
= M
cldanger
)
6. If (θ is M
up
) and (R is M
high
)then(Z
6
= M
clsaf e
)
7. If (θ is M
sr
) and (R is M
low
)then(Z
7
= M
cldanger
)
8. If (θ is M
sr
) and (R is M
mid
)then(Z
8
= M
clsaf e
)
9. If (θ is M
sr
) and (R is M
high
)then(Z
9
= M
saf e
)
10. If (θ is M
d
) and (R is M
low
)then(Z
10
= M
clsaf e
)
11. If (θ is M
d
) and (R is M
mid
)then(Z
11
= M
saf e
)
12. If (θ is M
d
) and (R is M
high
)then(Z
12
= M
saf e
)
D. Decision Making:
The aim of decision making is to decide when to signal
a fire alarm. This depends on the area of image (volume
of the air) covered by smoke. The greater number of pixels
affected by the smoke, the greater number of SBs are likely
to be detected by the algorithm.
In general if the image contains N blocks out of which
N
s
= a N blocks are SBs, the proposed FIS can produce
an ideal value of "1" for each SB which will give us a
theoretical threshold T = a N as for threshold for fire
signal. However, in reality based on the features of the SBs
a value between [0-1] is computed. Therefore, we reach a
practical threshold of T =
aN
b
as for firing threshold in
which
a
b
<< 1.
III. Results
A total of three video streams from standard smoke
detection data sets were collected as for our experimental
results. Figure 6 shows the qualitative results of applying
our proposed smoke detection algorithm on these video
sequences. Each frame is classified via the algorithm as
Smoke Frame (SF) or Non-Smoke Frame (NSF). In the
Smoky Video, the result of smoke detection for frames
305, 340, 410 and 425 are shown. Frames 305 and 410
and 425 are classified as SF and frame number 340 as
NSF. This result is an accordance with the corresponding
video streams. Because in frame 340, the smoke density
has mitigated so much so that it can be considered as
NSF. In other frames, the smoke is visible to the eye.
In frame 425 however, though the density of smoke is
low the algorithm has considered it as SF because the
sensitivity of the algorithm has been adjusted to detect
low-densed video frames as SF or NSF. The video sequence
Waste Basket is commonly tested video by researchers.
In this video, Frames 20 and 85 are NSF and frames 310
and 675 as SF. As can be seen, in this video in the first
twoframesthesmokeisgraduallyaccumulatingtheair
however its density is NOT high enough to activate our
smoke detection algorithm. As soon as the smoke density
and volume reaches a boundary level in the subsequent
two video frames, the smoke detection system labels the
frames as SF and an alarm is signaled. In the last video,
named Behind the Face, smoke can be seen from a long
distance from the back of the fences. This is a partial
occlusion and can cause problem for visual sensors. Our
smoke detection system classifies frame 30 and 80 as NSF
and frames 185 and 280 as SF. This is an attractive result
as well and shows the strength of our proposed smoke
detection algorithm that is able to adjust itself to various
indoor or outdoor unexpected fire hazards.
IV. Conclusion
A new approach for intelligent real-time detection of
smoke has been proposed based on video processing. The
method employs robust smoke feature evaluation methods
in order to discriminate smoke from non-smoke moving
objects. These features include both dynamic and static
features of smoke which are motion, motion orientation
and texture. For a natural and realistic aggregation of
these features a Fuzzy Inference System (FIS) is proposed
to combine the smoke features in a fuzzified manner.
The end-result is provided to a decision making stage for
final evaluation and deciding when to signal a fire alarm.
The proposed solution is implemented in Matlab and has
been tested on various standard smoke detection video
sequences. The results are promising and show the effec-
tiveness of our proposed solution on detecting the existence
of smoke in both indoor and outdoor environments.
2015 23
rd
Iranian Conference on Electrical Engineering (ICEE)
732
Figure 6. Smoke detection results. (1) The Smoky Video: (a) Fr 305 - SF (b) Fr 340 - NSF (c) Fr 410 - SF (d) Fr 425 - SF (2) The Waste
Basket Video: (a) Fr 20 - NSF (b) Fr 85 - NSF (c) Fr 310 - SF (d) Fr 675 - SF (3) The Behind the Face Video (a) Fr 30 - NSF (b) Fr 80 - NSF
(c) Fr 185 - SF (d) Fr 280 - SF (SF = Smoke Frame, NSF = Non-Smoke Frame)
References
[1] F. Yuan, “A fast accumulative motion orientation model based
on integral image for video smoke detection,” Pattern Recogni-
tion Letters, vol. 29, no. 7, pp. 925–932, 2008. 1, 2
[2] Y. Wei, Y. Chunyu, and Z. Yongming, “Based on wavelet
transformation fire smoke detection method,” in Electronic
Measurement & Instruments, 2009. ICEMI’09. 9th Interna-
tional Conference on. IEEE, 2009, pp. 2–872. 1
[3] T.-H. Chen, C.-L. Kao, and S.-M. Chang, “An intelligent real-
time fire-detection method based on video processing,” in Se-
curity Technology, 2003. Proceedings. IEEE 37th Annual 2003
International Carnahan Conference on. IEEE, 2003, pp. 104–
111. 1
[4] O. Marques, Practical Image and Video Processing Using MAT-
LAB, 1st ed. Wiley-IEEE Press, 2011. 1
[5] J. Chen, Y. Wang, Y. Tian, and T. Huang, “Wavelet based
smoke detection method with rgb contrast-image and shape
constrain,” in Visual Communications and Image Processing
(VCIP), 2013. IEEE, 2013, pp. 1–6. 1, 3
[6] H. Tian, W. Li, L. Wang, and P. Ogunbona, “A novel video-
based smoke detection method using image separation,” in
Multimedia and Expo (ICME), 2012 IEEE International Con-
ference on. IEEE, 2012, pp. 532–537. 1
[7] Z. Yu, Y. Xu, and X. Yang, “Fire surveillance method based
on quaternionic wavelet features,” in Advances in Multimedia
Modeling. Springer, 2010, pp. 477–488. 1
[8] J. Gubbi, S. Marusic, and M. Palaniswami, “Smoke detection in
video using wavelets and support vector machines,” Fire Safety
Journal, vol. 44, no. 8, pp. 1110–1115, 2009. 1
[9] B. U. Toreyin, Y. Dedeoglu, A. E. Cetin et al.,“Wavelet
based real-time smoke detection in video,” in European Signal
Processing Conference, 2005, pp. 4–8. 1
[10] J. Davis, “Recognizing movement using motion histograms,”
Technial Report 487, MIT Media Lab, vol. 1, no. 487, 1999. 1
[11] T. X. Tung and J.-M. Kim, “An effective four-stage smoke-
detection algorithm using video images for early fire-alarm
systems,” Fire Safety Journal, vol. 46, no. 5, pp. 276–282, 2011.
1
[12] Ko, “Spatiotemporal bag-of-features for early wildfire smoke
detection,” Fire Safety Journal, vol. 44, no. 8, pp. 1110–1115,
2009. 1
[13] J. shing Roger Jang, “Anfis: Adaptive-network-based fuzzy
inference system,” IEEE Transactions on Systems, Man, and
Cybernetics, vol. 23, pp. 665–685, 1993. 3
[14] S. Guillaume, “Designing fuzzy inference systems from data: an
interpretability-oriented review,” IEEE Trans. Fuzzy Systems,
pp. 426–443. 3
[15] T. TAKAGI, “Fuzzy identification of systems and its
applications to modeling and control,” IEEE Trans. SMC,
vol. 15, no. 1, pp. 116–132, 1985. [Online]. Available:
http://ci.nii.ac.jp/naid/80002450824/en/ 3
[16] E. Mamdani and S. Assilian, “An experiment in linguistic
synthesis with a fuzzy logic controller,” International
Journal of Man-Machine Studies, vol. 7, no. 1, pp. 1
13, 1975. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/S0020737375800022 3
2015 23
rd
Iranian Conference on Electrical Engineering (ICEE)
733
Project
Recommendations are traditionally generated on the basis of user’s implicit and explicit preferences on movies’ attributes, such as genre, director, and actors. However, user’s preferences can be b…" [more]
Project
The main goal of this project is to develop strong key management and data communication security in IoT platforms.
Conference Paper
June 2017
    Video summarization is a powerful tool for video understanding and browsing and is considered as an enabler for many video analysis tasks. While the effect of video summarization models has been largely studied in video retrieval and indexing applications over the last decade, its impact has not been well investigated in content-based video recommendation systems (RSs) based on low-level... [Show full abstract]
    Conference Paper
    September 2015
      Recommender Systems (RSs) play an increasingly important role in video-on-demand web applications -- such as YouTube and Netflix -- characterized by a very large catalogs of videos and movies. Their goal is to filter information and to recommend to users only the videos that are likely of interest to them. Recommendations are traditionally generated on the basis of user's preferences on... [Show full abstract]
      Conference Paper
      September 2015
        One of the challenges in video recommendation systems is the New Item problem, which happens when the system is unable to recommend video items, that no information is available about them. For example, in the popular movie-sharing websites, such as Youtube, everyday, hundred millions of hours of videos are uploaded and big portion of these videos may not contain any meta-data, to be used by... [Show full abstract]
        Article
        January 2016 · Journal on Data Semantics
          This paper investigates the use of automatically extracted visual features of videos in the context of recommender systems and brings some novel contributions in the domain of video recommendations. We propose a new content-based recommender system that encompasses a technique to automatically analyze video contents and to extract a set of representative stylistic features (lighting, color,... [Show full abstract]
          Discover more