Conference PaperPDF Available

Speed up temporal median filter and its application in background estimation

Authors:
  • Ho Chi Minh City University of Foreign Languages - Information Technology (HUFLIT), Vetnam, Ho Chi Minh City

Abstract and Figures

Median filter is usefiil in image and video processing. It can be used for filtering noises, for estimating background images, and so on. However, median filter is time-consuming. This paper presents a new method for realizing the median filter in GPU (Graphics Processing Unit) for background estimation problem. The proposed method can perform the filtering in just several milliseconds for each frame in high resolution videos. The experimental results show that the proposed method is even faster than the fastest GPU-based algorithms deployed in OpenCV for background estimation problem.
Content may be subject to copyright.
The
2016
IEEE
RIVF
International Coriference on Computing & Communication Technologies, Research, Innovation,
and
Vision
for
the Future
Speed Up Temporal Median Filter
and
Its
Application in Background Estimation
Thanh-Sach
LE
Nhu-Tai DO Kazuhiko HAMAMOTO
Tokai University HCM City Univ. University
of
Technology HCM City Univ.
of
Foreign Languages-
Email:
Ltsach@hcmut.edu.vnInformation
Technology Email: hama@keyaki.cc.u-tokai.ac.jp
Email: donhutai@huflit.edu.Vll
Abstract-Median
filter is useful
in
image
and
video processing.
It
can be used for filtering noises, for estimating background
images,
and
so on. However, median filter is time-consuming.
This
paper
presents a new method for realizing the median filter
in
GPU (Graphics Processing Unit) for background estimation
problem. The proposed method can perform the filtering
in
just
several milliseconds for each frame
in
high resolution videos. The
experimental results show
that
the proposed method is even faster
than
the fastest GPU-based algorithms deployed
in
OpenCV for
background estimation problem.
Index
Terms-fast
median filter; background estimation; video
surveillance; histogram; median; GPU
I.
INTRODUCTION
Median filter is well-known in image and video processing
[1]. This filter can be applied in space domain (i.e., X-Y
plane
of
images) for smoothing data
[2]
for a long time.
It
is especially effective for Pepper and salt. Recently, the
median filter has been also demonstrated
as
a robust method
for estimating background images in video surveillance [3].
The estimation can be done by attaching a filter at every pixel
in video frame. The kernel
of
filters moves on time axis and
occupies K video frames
(K
is the kernel size).
Despite it's effectiveness, median filter has high compu-
tational complexity compared to linear filters, for example,
Gaussian. Thank to the fast Fourier transform algorithm and
its relationship to convolution, linear filtering can be computed
quickly. The complexity is also independent to the filter
kernel's size. Conversely, data elements occupied by median
filter must be ordered and the element lying at the middle
point is returned
as
the filter's response. Due to this ordering
process, the median filter's response can not be determined
as
fast
as
linear filters'. Therefore, speeding up the computation
of
median filter is an open problem in filtering.
In video surveillance, there are many tasks that need to
be performed in order to detect, to recognize and to track
objects; some systems also need to analyze the behavior
of
objects. Compared to those tasks, background estimation is
just an initial step. Therefore, the computation
of
all those
tasks, including the background estimation, should be done in
real-time. This is a motivation to speed up median filter for
background estimation problem.
A brute-force method for computing the median filter's
response is to sort and then pick-up the median value. The
computational complexity for that approach is known
as
O(K
log2
K); where, K is the number
of
elements in the
filter's kernel. Another approach for fast finding the median
value was proposed by Jensen
[4]
(2005). He used two heaps
(one MAX and one MIN heap) to maintain the elements in the
kernel. The median value was defined
as
the average
of
the
two elements on the top
of
the heaps. The complexity
of
the
whole process is still
0(Klog2
K).
However, after initializing
the two heaps with
O(K
log2
K),
Jensen's approach needs
only 0(log2
K)
to update the filter's response. Compared
with other proposals in image processing for median filtering,
Jensen's algorithm has a higher complexity; however, Jensen's
algorithm does not require 8-bit constraint.
One
of
the earliest effort to speed up the median filter's was
proposed by Huang et al. (1979)
[5].
Their important idea is to
use a histogram having 256 bins (each pixel value is used
as
the index to the histogram) to fast obtain the count
of
elements
under the filter's kernel. The median value is searched by
scanning the histogram from left to right and accumulating
the count in each bin. The median value is defined as the first
bin's index that the accumulated number is greater than a
half
of
the filter kernel's size, i.e.,
K/2.
In the case
of
2D-
kernel, this approach has a complexity
of
O(r)
for finding the
median value by using (2r + 1) histograms. Where, r be the
2D-kernel's radius.
Weiss
[6]
(2006) and Simon
[7]
(2007) try to reduce
complexity to determine the median value under the kernel
to 0(1). However, both
of
the methods need 256 iterations
for adding or subtracting histogram
of
256 bins. So, they
need SIMD (Single Instruction, Multiple Data) architecture
to compute those operations parallel.
Hung et al., in
[8]
(2010) and [11] (2014), proposed an
algorithm which takes several operations for testing whether
the median value changing or not when adding/removing
new/old data.
If
the median value has been changed, Hung's
algorithm finds it by a scanning process proposed by Huang
[5]. Otherwise, his algorithm can save the time
of
scanning
the histogram. However, Hung's proposal still needs many
comparisons and needs to scan the histogram
as
in Huangs
method when the median is changed.
In
[10]
(2013), Sach et al. proposed a method for fast
determining the median value, which is further optimized
in this paper by using
GPu.
Therefore, we will present a
summary
of
[10]
in Section II before presenting new ideas
978-1-5090-4134-3/16/$31.00 ©2016 IEEE 175
Fig.
1.
A balanced median structure
for deploying
it
to GPU.
II. PREVIOUSLY
PROPOSED
MEDIAN
STRUCTURE
The main contribution in [10] is to create a new algorithm
to avoid the searching process for finding the median value
when moving the filter kernel forward on time axis. Similar to
Huang
[5]
and Hung [8], Sach et al. preserves a histogram
of
elements under the filter kernel. However, instead
of
searching
the median value, a structure
!R,
!R
~
{LW,
MW,
RW},
was
proposed, it is referred to as the median structure. Where, L W
(Left Weight) is the number
of
elements that are less than the
current median value;
RW
(Right Weight) is the number
of
elements that are greater than the current median value; and
MW
Median
Weight is the number
of
elements that are equal
to the current median value. A typical histogram and its values
in the median structured is shown in Fig.
1.
A median structure can belong to one
of
the following
situations at a time.
a) Un-balanced: This situation happens
if
the structure sat-
isfies either LH-Condition (Left-Heavier) or RH-Condition
(Right-Heavier). These conditions are given in Inequality (1)
and (2) respectively.
LW>
(MW+RW)
RW~
(LW+MW)
(1)
(2)
b) Balanced:
if
the structure is neither LH-condition nor
RH-condition then
it
is balanced.
In
the case
of
unbalanced situation, the median value is
invalid. The structure needs to be rebalanced to update the
median value.
The unbalanced condition only happens when the filter
move its kernel forward on time, because this action will add a
new element to the kernel and remove the oldest one from the
kernel. Therefore, the median structure needs to be checked
for an unbalanced condition when moving the kernel forward.
It
has to be re-balanced
if
needed according to the following
procedure.
In
order to re-balance the median structure,
median
indi-
cator, as shown in Fig.
1,
needs to be shifted to left
or
to
right (i.e., move to the nearest non-empty
bin
on the left or
on the right). New weight for
MW
can be picked up from the
histogram at the median indicator. L W and
RW
are updated
according to Eq. (3) to (5) or from (6) to (8).
A. Re-balancing LH-condition
RW+-RW+MW
MW
+-
newweight
LW+-LW-MW
B.
Re-balancing RH-condition
LW+-LW+MW
MW
+- newweight
RW+-RW-MW
(3)
(4)
(5)
(6)
(7)
(8)
III.
SPEED-UP
MEDIAN
ALGORITHM
FOR
BACKGROUND
ESTIMATION
To
estimate background images, we install a filter at each
pixel on video frame. The kernel
of
those filters moves
forward on time axis to perform the processing. Therefore,
each incoming frame causes the kernel moving to include
pixels on this new video frame. At the same time, the filters
also remove pixels on the oldest frame.
The filters work independently on separated data (pixels in
video frames). This characteristic implies that the estimation
can done in parallel and that it is well-suited with GPU (Graph-
ics Processing Unit). The core
of
this paper is to implement the
algorithm given in [10] using GPU for background estimation
problem.
The underlying ideas
of
the proposed algorithm can be
summarized as follows:
a)
There are several kinds
of
memory (global, shared, and
constant) inside
of
GPU with different amounts
of
accessing
time. Our proposed algorithm needs to store many parameters
and data.
We
carefully select a right kind
of
memory for each
of
our data. The data needed for the proposed algorithm will
be explained in Section III-A.
b) As presented in Section
n,
Sach's algorithm [10] needs
to store L
W,
MW,
and
RW
in a median structure. This
paper derives another way for storing the structure and for re-
balancing it. This modified way (explained in Section III-C)
can reduce the demand
of
memory and computational time.
c)
In
computation (true for both
of
CPU and GPU) ,
it
is
important to increase cache hit in order to lower memory
accessing time. The histogram in the proposed algorithm is
examined carefully (shown in Section Ill-D) to obtain a good
data arrangement to achieve this target.
d) Sach's algorithm [10] builds histogram using intensities
in video frames. This paper presents another way for trans-
forming intensities (shown in Section Ill-E). This transformed
way, in combination with the histogram rearrangement given
above, can further reduce
of
memory accessing time.
176
A. Memory layout
When a filter moves forward to occupy new coming data
(on new video frame), it needs to remove pixels on oldest
video frame too.
To
memorize which data needed for removing
from filter's kernel, the proposed algorithm utilizes a
circular
buffer (of size
K,
kernel size) to keep K video frames. Much
of
data in those K frames are not used during the filtering, so
the buffer is selected to store on host, i.e., CPU's size.
Similar to our previous algorithm, we need to store the
following data in memory and use them during the filtering.
a) llistogram: There are W x H filters for video frames
of
size W x H. Each
of
the filters needs a histogram
of
size
K.
Therefore, the proposed method needs a block
of
W x H x K
in
GPu.
This block is referred to
as
h.
In order to access to a bin in a histogram, we need
3 indexes: (r,
c,
k). Where, r E
[1,
H],
c E
[1,
W],
and
k E
[1,
K].
Conversion from these indexes to the bin's address
in histogram block are important, discussed in Section ill-D.
b) Median
structure
R: Different to Sach's algorithm [10],
the median structure in this paper consists
of
only
LW.
We
have W x H filters, so we need W x H values
of
L W in
GPU.
c) Median: This is the median value
of
the kernel.
It
is the
median indicator
as
shown in Fig.
1.
It
is also used to access
to the histogram to obtain
MW.
We
have to store W x H
median values for all the filters.
d)
Data
in
and
out: New coming data (data
new
) and the
data that has to be removed from the kernel (dataremoved)
need to be stored temporarily. The proposed method needs
W x H locations for each
of
data
new
and dataremoved
of
W x H filters. These data will be uploaded to GPU whenever
a new coming video frame is available.
Data blocks from
a)
to
d)
explained above are selected to
store on global memory
of
GPU because they are large and
frequently accessed. However, parameters
W,
H and K are
stored in constant memory.
B.
Outline
of
the proposed algorithm
The proposed algorithm is shown in Table I. The proposal
consists
of
codes that are executed on host (CPU) and device
(GPU).
At the beginning, the proposal needs to allocate and ini-
tialize memory in both
of
host and device. Those data have
been presented in Section ill-A. For data on device, the
proposed algorithm needs to perform two actions "upload"
and "download" to initialize and to get data back to host after
processing, e.g.,
as
shown in Line 10 and 12 in Table
I.
The proposed algorithm attaches one thread to a filter, and
it needs W x H filters for video sequence with the size
of
W x
H.
In GPU computing, threads are organized into blocks.
According to [15], blocks
of
size 32 x 8 threads are selected
to achieve good performance. These parameters are expressed
in Block and
Grid
in Line
11
in Table I.
TABLE I
THE
PROPOSED ALGORITHM
Running
on
host
Input: video sequence (live or off-line)
Output: background image at each video frame
o.
Allocate and initialize host's memory
1.
Allocate and initialize device's memory
2. while (has new frame) do
3.
pN
ewFrame
+--
get new video frame;
4.
pOldFrame
+--
remove oldest frame from the video buffer;
5. add
pN
ewFrame
to video buffer;
6.
pM
edian
+--
doFiltering(pOldFrame,
pN
ewFrame);
7.
end
while
8.
Free host's and device's memory
Updating
the
kernel
9.
median_frame
+--
doFiltering(oldest frame, newest frame) {
10.
upload video frame (oldest and newest) to device;
11.
call doFilteringOnDevice< Grid, Block
>(all
data on device);
12.
download median frame from device and return it;
13.
} Running
on
device
14.
void doFilteringOnDevice(data on device) {
15.
(r,
c)
+--
map thread-index to data's location;
16.
med
+--
get median at (r, c);
17.
dataremoved
+--
get oldest data at (r, c);
18.
datanew
+--
get newest data at (r, c);
19.
rebalance(data on device,
r,
c,
med,
dataremoved, datanew
);
20. }
21. void rebalance( data on device, parameters) {
22.
her, c, dataremoved) = her, c, dataremoved) -1;
23.
her, c, datanew ) = her, c, datanew ) + 1;
24. update L W
if
needed;
25. }
C.
Revised median structure and updating algorithm
In [10], median structure consists
of
three values
LW,
MW,
and
RW.
However, we have
LW
+
MW
+
RW
=
K.
Furthermore,
MW
can be obtained from histogram by
h[r,
c,
med]; where,
med
is obtained from Line 16 in Table I
by M edian[r,
c],
with
Median
is the map
of
median values,
explained in Section ill-A. Therefore, we need not to keep
RW
and
MW
in the structure for every filter. This idea
reduces the redundancy in using memory and in computation
time.
By adding
LW
into Inequality
1,
LH-condition is now 2 *
LW
>
K.
Replacing
RW
= K -
LW
-
MW
in Inequality 2
and applying some calculations, RH-condition is now K
>=
2 *
(LW
+
MW),
with
MW
= h[r,
c,
med]. Furthermore,
re-balancing the median structure for LH-condition and RH-
condition are reduced from Eq. (3) to (8) into Eq. (9) and (10)
respectively; where,
med
o1d
(med
new
) is the median indicator
in Fig. 1 before (after) it moves left or right.
LW(r,
c)
+--
LW(r,
c)
-h(r,
c,
med
new
)
(9)
LW(r,
c)
+--
LW(r,
c)
-h(r,
c,
med
o1d
) (10)
Careful examination new added data (P) and old removed
data (q) and their relationships with the median structure,
as
shown in Table II. The table shows that there are 5 cases
(55.6%) the proposed algorithm needs not to re-balance the
structure, e.g., on Row 1,5, and 7-9 . The other 4 cases
177
TABLE II
CASES
OF UPDATING MEDIAN STRUCTURE
Addp
Remove q LW
MW
RW Op
1
<median
< median
1-1
= 0 none
2
<median
> median
+1
-1
+-
3
<median
-median
+1
-1
+-
4 1 > median 1 < median 1
-1
1
+1
-+1
5 > median > median +1-1=0 none
6 1 > median 1 - median 1 1
-1
+1
-+1
7 1 = median 1 < median 1
-1
1
+1
none 1
8 1 - median 1 > median 1 1
+1
-1
none 1
9 1 - median 1 - median 1 1
+1-1-0
none 1
Fig.
2.
Histogram arrangement
need to re-balance the structure depending on LH-
or
RH-
condition. Cutting-off branches in this way also helps the
proposed algorithm saving time.
D.
Histogram arrangement
The operations on histogram in the proposed algorithm, as
indicated in Line 22 ans 23 in Table I, are quite simple. The
total cost for these operations consisting
of
accessing time to
the data element in the histogram and the computation time
(increasing
or
deceasing). Because histogram block is quite
large and is stored in global memory
of
GPU, the proposed
algorithm bases on the two following heuristics to arrange data
elements in the histogram block in a way so that it can increase
cache hit.
a) Data elements (intensity
of
pixels) coming in and out
nearby filters have similar values, i.e., small differences.
Therefore, bins that have similar indexes
of
nearby filters
should be loaded into the same cache. The proposed algorithm
achieves this by arrange bins in a histogram block into a small
region.
b) Threads
of
the same block (of execution) are scheduled
to be executed parallel, and they can share the same cache.
Based on the above heuristics, the proposed algorithm
arranges the whole histogram block into
nw
x
nh
x
nk
block
along columns, rows and bins respectively.
nw
x
nh
are the
number
of
threads in a block
of
execution. They are selected
as 32 x 8, as mentioned in Section IlI-B.
nk
is selected
empirically to 32. The arrangement is shown in Fig. 2.
E.
Transformation
of
histogram
The arrangement
of
histogram, as shown in Section III-D,
can further achieve better cache hit
if
we have a way to
transform intensities into new values that vary in smaller a
range.
Assume that we know background model in advance, we
can take the difference between incoming frame with the back-
ground model. Thereby, all pixels belonging to the background
produce a small absolute values. Therefore, instead
of
creat-
ing histogram with input intensities, the proposed algorithm
builds the histogram using this kind
of
differences. For some
beginning video frames, the proposed algorithm uses the first
frame as the background. After that, the proposed algorithm
updates the background frequently to take differences.
IV.
EXPERIMENTAL
RESULT AND COMPARISON
A. Environments and Data-sets
1) Programming Language: The proposed algorithm for
temporal median filter was implemented in C++.
We
do not
have source code
of
Huang's
[5]
and Hung's [8], [11] method,
so we fairly re-implement these with C++. Other methods
for median filters, e.g.
[6]
and [7], were not used, because
they were not proposed for processing temporal data. For
other referred methods in background estimation comparison
(GPU_GMG
[16],
GPU_MOG
[17], and
GPU_MOG2
[18]),
we use C++ implementation in OpenCV [14].
2) Machine & system: The experiment was done with
ASUS TP500 laptop having the following features. Processor:
Intel(R) Core(TM) i3-4030U, 1.9Ghz; RAM: 8GB; and OS:
Windows 8.1 Pro 64-bit. One GPU card is included: GeForce
820M with CUDA Capability 2.1, Multiprocessors; CUDA
Cores: 48, GPU Clock: 1.25 GHz
3) Data-sets: The comparison is performed in both
of
time and accuracy. For time comparison, this paper uses 5
videos from PETS2009 [9], listed in Table III from 1 to
5.
Those have the resolution
of
768 x 576 pixels. We also
performed the evaluation with one video captured by ourself
for transportation in Ho Chi Minh City, Vietnam. This video
has the resolution
of
720 x 576 pixels.
For accuracy comparison, we need data-sets that have
ground-truth (Le., background model). We evaluate the accu-
racy with 9 videos captured for real scene and 20 ones synthe-
sized to simulate a large variation
of
weather conditions, like
sunny, foggy, and windy. All
of
29 videos and accompanied
benchmark for background estimation process can be obtained
from [12] and [13] respectively.
B.
Result and Comparison
1) Computation time: According to [10], by using median
structure as shown in Section III -C, the computation time
is independent to the filter kernel' size, Le.,
K.
Therefore,
K is chosen in this paper for obtaining good estimation
of
background image for input video sequences.
178
TABLE III
LIST
OF
VIDEO
SEQUENCES
Index Name Name
in
PETS2009 #Frames
Video 1 S1.L1.Time_13-S7.View_00l 221
2 Video 2 S1.L3.Time_14-33.View_001 344
3 Video 3 S2.L1.Time_12-34.View_001 79S
4 Video 4 S2.L2.Time_14-SS.View_001 436
S Video S S2.L3.Time_14-4l.View_001 240
6 Video 6 Traffic
in
HCM
City, Vietnam 2711
To
show the effectiveness
of
the proposed method, this
section divides results into three groups in the following.
Group
1:
This group is to present the contribution
of
underlying ideas to the reduction
of
computation time. Results
are given in Table IV. In this group, Version 0 means that:
(a) the implementation uses the previously proposed median
structure [10], (b) bins in the histogram
of
every filter are
organized continuously in memory, and (c) the histogram
of
filters are constructed from the pixels' intensity, instead
of
using the proposal in Section III-E.
If
ideas proposed in
Section III-C to III-E are added to Version 0 gradually, we
obtain other versions.
Table IV indicates that the combination
of
all the ideas
from Section llI-C to llI-E can produce a better performance.
Especially, the arrangement
of
bins in histograms into blocks
as
shown in Section III-D (Version 2) can dramatically
reduce the computation time. This effect is the consequence
of
increasing cache hit.
Group
2:
This group compares the optimization (using
GPU) in this paper with the original (using CPU) proposed in
[10]. Other methods that use histogram for determining median
value are also included. As seen from Table V, GPU-based
optimization outperforms other CPU-based implementation
even they also uses histogram.
Group
3: This group compares our proposal with some
fastest methods in background estimation for video surveil-
lance. All
of
the referred methods
GPU_MOG
[17],
GPU_MOG2 [18], and
GPU_GMG
[16]
are optimized by
GPU and included in OpenCV [14]. GPU_MOG2 is currently
the fastest method in background estimation benchmark [13].
As shown in Fig.
3,
the proposed method is even faster than
GPU_MOG2 for all videos in the test. Actually, by using
the revised median structure (proposed in Section III-C), the
proposed method needs just some simple operations to main-
tain the median value for each filter. However, the proposed
method requires more memory compared to GPU_MOG2.
Fortunately, almost the required memory is located on GPU.
2) Accuracy: This section presents a comparison about
the accuracy
of
background estimation process between
GPU_MOG2 and the proposed method. The results are shown
in Fig.
4.
It
can be seen that temporal median filter (as
optimized in this paper) can produce the result accurately
compared to GPU_MOG2 for almost
of
videos in the test.
V.
CONCLUSION
Several ideas for using GPU to optimize the histogram-
based temporal median filter has been presented in this paper.
In combination with the fast method for determining median
values given in [10], the proposed method in this paper can
perform background estimation really fast, farther satisfaction
than real-time requirement. The result can be compared to
fastest method in the field, i.e., GPU_MOG2, in computation
time and accuracy.
ACKNOWLEDGMENT
This research work is sponsored by Dept.
of
Science
and Technology in Ho Chi Minh City (Vietnam) on Project
"Vision-based Traffic
Parameter
Estimation
and
Visual-
ization", Contract No. 42120151HD-SKHCN dated June 29,
2015.
REFERENCES
[1] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd edition,
Prentice Hall, 2002.
[2] J.
W.
Tukey, Exploratory Data Analysis, Preliminary ed. Reading, MA:
Addison-Wesley, 1971.
[3] M. Piccardi, Background subtraction techniques: a review, in Proc.
of
IEEE International Conference
on
Systems, Man and Cybernetics, Vol.
4, pp.3099-3104, 2004.
[4] Steven
L.
Jensen, Implantable Medical Device Fast Median Filter, US
Patent, 200S.
[S]
T.
S. Huang, G. J. Yang and G.
Y.
Tang, A Fast Two-Dimensional
Median Filtering Algorithm, IEEE Trans.
on
Acoustics, Speech,
And
Signal Processing,
Vol.
Asp-27,
No.1,
pp. 13-18, 1979.
[6]
Ben
Weiss, Fast Median
and
Bilateral Filtering, ACM Transactions
on
Graphics (TOG), vol. 2S, no. 3, pp. S19-S26, 2006.
[7] Simon Perreault and
P.
Hebert, Median Filtering in Constant TIme,IEEE
Trans. on Image Processing,
Vol.
16,
No.9,
pp. 2389-2394, 2007.
[8] M. Hung, J. Pan, C. Hsieh, Speed
Up
Temporal Median Filter
for
Back-
ground Subtraction,in Proc.
of
IEEE International Conference on Per-
vasive Computing, Signal Processing and Applications, pages 297-300,
2010.
[9] PET2009: http://www.cvg.rdg.ac.ukJPETS2009!
[10] Thanh-Sach
LE
and Trung-Hieu LUU, A Fast Temporal Median Filter
and Its Applications
for
Background Estimation in Video Surveillance,
in
Proc. 2013
IEEE
Int'l
Conf. on Soft Computing and Pattern Recognition,
pp.390-39S, 2013.
[11] Mao-Hsiung Hung,Jeng-Shyang Pan et al. A Fast Algorithm
of
Temporal
Median Filter
for
Background Subtraction, in Journal
of
Information
Hiding and Multimedia Signal Processing, Vol.S, No.1, pp.33-40, 2014.
[12] A. Vacavant,
T.
Chateau, and A. Wilhelm A Benchmark Dataset
for
Out-
door Foreground Background Extraction,
in
Lecture Notes
in
Computer
Science (Springer), Vo.7728, pp.291-300, 2013.
[13] A. Sobral and A. Vacavant A comprehensive review
of
background
subtraction algorithms evaluated with synthetic
and
real videos, in Journal
of
Comput. Vis. Image Underst., vol. 122, pp.421, 2014.
[14] OpenCV website, http://www.opencv.org.
[IS] Shane Cook, CUDA Programming -A Developers Guide to Parallel
Computing with GPUs, Elsevier Inc., 2013.
[16] Andrew B. Godbehere, Akihiro Matsukawa, and Ken Goldberg Visual
tracking
of
human visitors under variable-lighting conditions
for
a
responsive audio art installation,
in
Proc.
of
2012 IEEE American Control
Conference, pp.430S-4312, 2012.
[17]
P.
KaewTraKulPong and R. Bowden
An
Improved Adaptive Background
Mixture Model
for
Real-time Tracking with Shadow Detection, in Proc.
of
2
nd
European Workshop
on
Advanced Video Based Surveillance
Systems, pp.,
200l.
[18] PZivkovic, Z. Improved adaptive Gaussian mixture model
for
back-
ground subtraction, in Proc.
of
17th
Int'l
Conf. on Pattern Recognition,
pp.28-31,2004.
179
TABLE
IV
PROCESSING TIME (IN
MSEC.)
OF VARIOUS KINDS OF OPTIMIZATION,
KERNEL'S
SIZE
= 81
Method
Version
0:
without any idea proposed
in
Section III-C to III-E
Version
1:
applying idea proposed
in
Section III-C
Version
2:
applying ideas proposed
in
Section III-C to III-D
Version
3:
applying ideas proposed
in
Section III-C to III-E
Version 3
~
"Proposed method", hereafter
Video 1
10,07
8,14
4,24
3,66
Averaged time for Video sequences
Video 2 Video 3 Video 4 Video 5
10,56 10,24 11,08 10,03
9,08 7,86 9,33 8,06
4,36 3,93 4,47
4,21
3,72 3,10 4,09 3,54
TABLE
V
Video 6
8,37
6,46
3,79
3,83
Summary
Averaged fps
10,1
99
8,2
123
4,2 240
3,7
273
COMPARISON OF PROCESSING TIME (IN
MSEC.)
BETWEEN GPU AND CPU,
KERNEL'S
SIZE
= 81
Method
CPU's version of HUANG's proposal
[5]
CPU's version of HUNG's proposal
[8]
CPU's version of SACH's proposal
[10]
Proposed method
Video 1
872,60
194,55
28,61
3,66
Averaged time for Video sequences
Video 2 Video 3 Video 4 Video 5
882,83 844,17 936,04 829,25
108,83 74,25 134,08 157,65
26,73 25,77 27,14 26,70
3,72 3,10 4,09 3,54
Fig.
3.
Comparison between various algorithms
in
background estimation
Fig.
4.
Comparison of PSNR for several videos
in
the benchmark
[12]
180
Video 6
312,55
56,64
20,39
3,83
Summary
Averaged fps
779,6 1
121,0 8
25,9
39
3,7 273
... In this method, a fixed scene is required to avoid noises when modeling the background. Compared with linear filters, it has higher computational complexity, and its processing speed is not as fast as linear filters [22][23][24]. ...
Article
Full-text available
Background modeling plays an important role in the application of intelligent video surveillance. Researchers have presented diverse approaches to support the development of dynamic background modeling. However, in the case of pumping unit surveillance, traditional background modeling methods often mistakenly detect the periodic rotational pumping unit as the foreground object. To address this problem here, we propose a novel background modeling method for foreground segmentation, particularly in dynamic scenes that include a rotational pumping unit. In the proposed method, the ViBe method is employed to extract possible foreground pixels from the sequence frames and then segment the video image into dynamic and static regions. Subsequently, the kernel density estimation (KDE) method is used to build a background model with dynamic samples of each pixel. The bandwidth and threshold of the KDE model are calculated according to the sample distribution and extremum of each dynamic pixel. In addition, the strategy of sample adjustment combines regular and real-time updates. The performance of the proposed method is evaluated against several state-of-the-art methods applied to complex dynamic scenes consisting of a rotational pumping unit. Experimental results show that the proposed method is available for periodic object motion scenario monitoring applications.
Article
This paper presents a hardware design for an energy-efficient, high-speed, one-dimensional median filter. Existing architectures focus on operating speeds, thus resulting in redundant power dissipation. This paper presents an algorithm and mathematical model for controlling the clock signals attached to circuit by analyzing the behavior of the filter, which immobilizes the data in registers and reduces not only signal transitions but also switching activities, thereby reducing the total dynamic power consumption. Furthermore, the proposed architecture provides high-speed computation. A median result can be produced in each clock cycle, and the maximum operating frequency performance is nearly independent of the filter size. The proposed architecture uses 90-nm process technology and experimental results show that the proposed method is more energy efficient than existing designs. The power consumption is reduced by 25% on average.
Conference Paper
Full-text available
For a responsive audio art installation in a skylit atrium, we introduce a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian segmentation, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and Gale-Shapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. We demonstrate that our algorithm has improved recall and F2-score over existing methods in OpenCV 2.1 in a variety of situations. We further demonstrate that feedback between the tracking and the segmentation systems improves recall and F2-score. The system described operated effectively for 5-8 hours per day for 4 months; algorithms are evaluated on video from the camera installed in the atrium. Source code and sample data is open source and available in OpenCV.
Conference Paper
Full-text available
Most of video-surveillance based applications use a foreground extraction algorithm to detect interest objects from videos provided by static cameras. This paper presents a benchmark dataset and evaluation process built from both synthetic and real videos, used in the BMC work-shop (Background Models Challenge). This dataset focuses on outdoor situations with weather variations such as wind, sun or rain. Moreover, we propose some evaluation criteria and an associated free software to compute them from several challenging testing videos. The evaluation pro-cess has been applied for several state of the art algorithms like gaussian mixture models or codebooks.
Article
Temporal median filter is one of most popular background subtraction methods. However, median operation is very time-consuming which limits its applications. This paper presents a fast algorithm to reduce the computation time of the temporal median operation. By utilizing the characteristics of high correlation of adjacent frames, the fast algorithm designs simple mechanism to check whether the median of the current frame is equal to that of the previous frame. The proposed algorithm reduces the computing frequency of median operations significantly, and the experimental results indicate it is much faster than the existing algorithms.
Article
Median filter is well-known for researchers in many fields. In image and video processing, it can be used for filtering noises, for estimating background images, and so on. Its effectiveness is undoubtful. However, median filter is time-consuming. This paper presents a new method for computing median filter's response. The proposed method is O(l) and better than existing methods analytically and experimentally. The experimental results with background estimation problem consolidate the finding in this paper.
Article
Temporal median filter is one of most popular background subtraction methods. However, median operation is very time-consuming which limits its applications. This paper presents a fast algorithm to reduce the computation time of the temporal median operation. By utilizing the characteristics of high correlation of adjacent frames, the fast algorithm designs a simple mechanism to check whether the median of the current frame is equal to that of the previous frame. The proposed algorithm reduces the computing frequency of median operations significantly, and the experimental results indicate it is much faster than the existing algorithms.
Article
Median filtering is a cornerstone of modern image processing and is used extensively in smoothing and de-noising applications. The fastest commercial implementations (e.g. in Adobe ® Pho- toshop ® CS2) exhibit O(r) runtime in the radius of the filter, which limits their usefulness in realtime or resolution-independent contexts. We introduce a CPU-based, vectorizable O(log r) algo- rithm for median filtering, to our knowledge the most efficient yet developed. Our algorithm extends to images of any bit-depth, and can also be adapted to perform bilateral filtering. On 8-bit data our median filter outperforms Photoshop's implementation by up to a factor of fifty.