Rotation invariant local phase quantization for blur insensitive texture analysis

Abstract

This paper introduces a rotation invariant extension to the blur insensitive local phase quantization texture descriptor. The new method consists of two stages, the first of which estimates the local characteristic orientation, and the second one extracts a binary descriptor vector. Both steps of the algorithm apply the phase of the locally computed Fourier transform coefficients, which can be shown to be insensitive to centrally symmetric image blurring. The new descriptors are assessed in comparison with the well known texture descriptors, local binary patterns (LBP) and Gabor filtering. The results illustrate that the proposed method has superior performance in those cases where the image contains blur and is slightly better even with sharp images.

Full text

Page 1

Rotation Invariant Local Phase Quantization
for Blur Insensitive Texture Analysis
Ville Ojansivu, Esa Rahtu and Janne Heikkil¨ a
Machine Vision Group, University of Oulu, PO Box 4500, 90014, Finland
{vpo,erahtu,jth}@ee.oulu.fi
Abstract
This paper introduces a rotation invariant extension
to the blur insensitive local phase quantization texture
descriptor. The new method consists of two stages, the
first of which estimates the local characteristic orien-
tation, and the second one extracts a binary descrip-
tor vector. Both steps of the algorithm apply the phase
of the locally computed Fourier transform coefficients,
which can be shown to be insensitive to centrally sym-
metric image blurring. The new descriptors are as-
sessed in comparison with the well known texture de-
scriptors, local binary patterns (LBP) and Gabor filter-
ing. The results illustrate that the proposed method has
superior performance in those cases where the image
contains blur and is slightly better even with sharp im-
ages.
1. Introduction
Natural surfaces usually exhibit some repetitive in-
tensity variations or patterns that are in general referred
to as texture. Analysis of texture information is impor-
tant in machine vision, and it has numerous applica-
tions, includingsurface inspection, medical image anal-
ysis, and remote sensing [6]. In some applications, im-
age degradation may limit the applicability of the tex-
ture information. One class of degradation is blur, due
to motion, out of focus, or atmospheric turbulence. Be-
cause image deblurring is very difficult and introduces
new artifacts, it is desirable that the texture analysis
could be done in such a way that it is insensitive to blur.
If the orientation of the textures changes, also rotation
invariance is a desired property.
The focus of this paper is on the blur and rotation
insensitive texture classification. There are not many
texture analysis methods that claim to be insensitive to
blurring. One method, although not invariant to ro-
tation, based on short term Fourier transform (STFT),
called local phase quantization (LPQ), was proposed in
[5]. Another blur robust descriptor based on color con-
stancy was proposed in [7]. Also, blur invariant mo-
ments or modified Fourier phase [1] could be used in
principle, but they are mainly targeted at global object
recognition, not local texture analysis.
In this paper, we propose a new blur and rotation in-
sensitive texture descriptor for gray scale images. The
approach is based on the application of STFT. The ad-
vantage in STFT is that the phase of the low frequency
coefficients is insensitive to centrally symmetric blur,
which is commonly encountered in real images; for ex-
ample, in the case of out of focus, motion and atmo-
spheric turbulence.
Local frequency analysis has been used for texture
analysis also previously. One of the best known meth-
ods uses Gabor filters and is based on the magnitude
information [2]. Phase information has been used in
[8] and histograms together with spectral information
in [9]. Nevertheless, blur sensitivity has not been con-
sidered as a design criterion in these operators.
2. LPQ texture descriptor
In this section, we shortly present the LPQ method.
For a comprehensive discussion, one can refer to [5],
where also a decorrelation scheme is presented.
Spatial blurring is represented by a convolution be-
tween the image intensity and a point spread function
(PSF). In the frequency domain, this results in a multi-
plication G = F ·H, whereG, F and H are the discrete
Fourier transforms (DFT) of the blurred image, original
image, and the PSF respectively. Further considering
only the phase of the spectrum the relation turns into a
sum? G =? F +? H. When the PSF of the blur is
centrally symmetric its Fourier transform H is always
real valued i.e.? H ∈ {0,π}. Furthermore, the shape
of H for a regular PSF is close to a Gaussian or a sinc-
function ensuring that at least the low frequency values
978-1-4244-2175-6/08/$25.00 ©2008 IEEE

Page 2

of H are positive. At these frequencies,? H = 0 caus-
ing? F to be a blur invariant property. Because LPQ
uses finite size 2-D discrete STFT computed locally,
this invariance is in part disturbed but is still pertinent.
In LPQ, the phase is examined in local neighbor-
hoods Nxat each pixel position x = [x1,x2]Tof the
image f(x). These local spectra are computed using a
discrete STFT defined by
F(u,x) =
?
y
f(y)wR(y − x)e−j2πuTy,
(1)
where u is the frequency, and w(x) is a window func-
tion defining the neighborhood Nx.
regular LPQ, wR is a NR-by-NR rectangle given as
wR(x) = 1 if |x1|,|x2| < NR/2 and 0 otherwise.
The local Fourier coefficients are computed at four
frequency points u1 = [a,0]T, u2 = [0,a]T, u3 =
[a,a]T, and u4 = [a,−a]T, where a is a sufficiently
small scalar to satisfy H(ui) > 0. For each pixel posi-
tion this results in a vector
In the case of
F(x) = [F(u1,x),F(u2,x),F(u3,x),F(u4,x)].
(2)
The phase information in the Fourier coefficients is
recorded by observing the signs of the real and imagi-
nary parts of each component in F(x). This is done by
using a simple scalar quantization
qj=
?
1,
0,
if gj≥ 0
otherwise,
(3)
where gjis the j-th component of the vector G(x) =
[Re{F(x)},Im{F(x)}].
The resulting eight binary coefficients qjare repre-
sented as integer values between 0-255 using coding
fLPQ(x) =
?8
these values from all positions is composed, and used
as a 256-dimensionalfeature vector in classification.
j=1qj2j−1.Finally, a histogram of
3. Rotation invariant LPQ
The rotation invariant local phase quantization (RI-
LPQ) method is composed of two stages: character-
istic orientation estimation and directed descriptor ex-
traction. We start by introducing the first of these and
present two alternative approaches to it.
3.1. Estimation of characteristic orientation
Let Rθ be a 2-D rotation matrix corresponding to
angle θ, and let f(x)′= f(R−1
image. Itis aknownfactthattheFouriertransformoff′
is simply the Fourier transformof f rotated by Rθ. The
θx) denote the rotated
0pi/2pi 3pi/2 2pi
−1
0
1


(a)
0pi/2 pi 3pi/2 2pi
−1
0
1


(b)
Figure 1. Example of C(x) (solid), C(x) >
0 (dotted), and sine approx. (dashed). (a)
Typical case (98%), (b) Rare case (2 %).
same will apply to the circular local neighborhoodsNx,
whose position will in addition change to x′= Rθx.
Taking advantage of this phenomenon, we estimate
the coefficients (1) on a circle of radius r at frequen-
cies vi = r[cos(φi) sin(φi)]T, where φi = 2πi/M,
and i = 0,...,M − 1. We further replace the rect-
angular window wRin (1) with a circular Gaussian one
definedas wG(x) = 1/(2πσ2)exp{−(x2
if |x1|,|x2| < NG/2 and 0 otherwise.
Basedonthe above
case of rotation the resulting vector V(x)
[F(v0,x),...,F(vM−1,x)]
x′, and it undergoes a circular shift corresponding to
the rotation angle θ, within the discretization accuracy
2π/M.Note also that due to separability, V(x) is
efficiently evaluated for all image positions x using
simply 1-D convolutions for the rows and columns
successively.
In order to also achieve blur insensitivity, we con-
sider only the phase of V(x). This is performed by
observing the signs of the imaginary part C(x) =
Im{V(x)} similarly to (3) for G(x).
The characteristic orientation is extracted from the
quantized coefficients using a complex moment as
1+x2
2)/(2σ2)}
arguments, inthe
=
will be relocated to
b(x) =
M−1
?
i=0
ciejφi,
(4)
where ciis the i-th quantized component of C(x). The
characteristic orientation is defined for each pixel posi-
tion as ξ(x) =? b(x).
Now for a neighborhood Nxfrom f′, the character-
istic orientation ξ(x)′≈ ξ(R−1
the characteristic orientation of Ny from f. The ap-
proximation in the equation is due to the discretization.
Thedirect evaluationofC(x) requiresfrequencyco-
efficients to be computed at M points. In the experi-
ments with real textures, we however noticed that in
most cases (approx.98%) the shape of C(x) had a form
θx) + θ, where ξ(y) is

Page 3

ofone periodof sinusoidshownin Figure1(a). Because
of this, the direct approach is perhaps not the most effi-
cient way of evaluating the characteristic orientation.
Hence, we consider also an approximation scheme,
where the form of C(x) is interpolated using a cosine
ˆC = A(x)cos(φ + τ(x)), with φ ∈ [0,2π], and the
parametersA(x) and τ(x) are estimated using only two
sample points [F(v0,x),F(vM/4−1,x)]. The charac-
teristic orientation is then derived as the position of the
maximumvalueofˆC(x) i.e. ξ(x) = −τ(x), whichcor-
responds to the definition of the complex moment (4).
Some error to the approximation scheme is caused
by those C(x) that do not have the aforementioned
shape. However, most of these still have a sinusoidal
form for just more than one period as shown in Figure
1(b). Inmost ofthese cases, the approximationstill pro-
duces adequately stable orientation.
3.2. Computation of oriented LPQ
In the second stage of the method, we extract the bi-
nary descriptor vector. The procedure here is similar
to the original LPQ, but the neighborhoodat each loca-
tion is rotated to the direction of the characteristic ori-
entation. The operation can be formulated by defining
oriented frequency coefficients as
Fξ(u,x) =
?
y
f(y)wR(R−1
ξ(x)(y−x))e−j2πuTR−1
ξ(x)y.
For the rotated image f′this becomes
Fξ(u,x)′=
?
y
f(y)′wR(R−1
ξ(x)′(y − x))e−j2πuTR−1
ξ(x)′y
=
?
t
f(t)wR(R−1
ξ(R−1
θ
x)(t − R−1
θx))e
−j2πuTR−1
ξ(R−1
θ
x)t
= Fξ(u,R−1
θx),
(5)
where we have used the fact that Rφ+γ = RφRγand
substituted t = R−1
θy. For RI-LPQ, we define Fξ(x)
by replacing the F(ui,x) in (2) with the oriented co-
efficients Fξ(ui,x). We then proceed similarly to in
LPQ, only using Fξ(x) instead of F(x), to produce
256-dimensional descriptor vector.
Equation (5) shows that the rotation of f only relo-
cates the coefficients Fξ(u,x), which does not affect
the histogram constructed later in the process. Hence
due to the same argumentsas for LPQ the resulting fea-
tures are insensitive to both rotation and blur.
Furthermore, we quantized the orientations in
Fξ(u,x) to K possible values, and precomputed the
correspondingset ofwindowfunctionsandcomplexex-
ponentials. In this way, the computational complexity
became close to that of the standard LPQ.
Figure 2. Examples of the textures: (left)
sharp and (right) blurred (blur radius is
one pixel).
4. Experiments
We performed three experiments to show the appli-
cability of the proposed method for classification of
blurred as well as sharp textures. As test material, we
used the three applicable test suites of the Outex tex-
ture image database1[3]. For comparison, we per-
formed the experiments also using two well known tex-
ture classification methods, namelythe local binarypat-
tern (LBP) descriptor2[4], and the Gabor filter bank
based method3[2]. The LBP has also a rotation in-
variant version which was taken into the experiments.
According to [3], the Gabor method had the best per-
formance in the Outex TC 00000 test and the rotation
invariant LBPriu2
P,Rfor the other two, Outex TC 00010
and Outex TC 00012 containing rotated textures.
In the experiments, we used a 3-nearest neighbor
classifier, which was trained and tested using the appro-
priate image sets defined in the Outex tests. The blur
was generated by convolving the test images with a cir-
cular PSF beforethe classification. Figure 2 contains an
example of a sharp and a blurred texture.
For the RI-LPQ method,we used two approachesfor
the characteristic orientation estimation, as described in
Section 3. These two variants are referred to as RI-LPQ
and RI-LPQa respectively. The parameter values for
both were r = 1/5, NG= 5, M = 36, σ = 2, NR= 9,
a = 1/9, and K = 36. The comparison methods LBP
and standard LPQ are computed with standard parame-
ters in 7-by-7 neighborhoods,as explained in [5].
In the first experiment, we used the test suite Ou-
tex TC 00000, which contains 480 images from 24
classes without rotation.
compare the rotation invariant methods, RI-LPQ and
LBPriu2
P,R, to non-invariant ones. The results with dif-
ferent blur sizes are shown in Figure 3(a). It seems that
all of the methods manage well when no blur is present,
but as the blur strength increases the performance of all
The purpose here is to
1http://www.outex.oulu.fi/
2http://www.ee.oulu.fi/mvg/page/lbp matlab/
3http://vision.ece.ucsb.edu/texture/software/

Page 4

0 0.250.50.751
20
40
60
80
100
Circular blur radius [pixels]
Classification accuracy [%]


(a) Blur
00.25 0.50.751
20
40
60
80
100
Circular blur radius [pixels]
Classification accuracy [%]
(b) Blur and rotation
0 0.250.5 0.751
20
40
60
80
100
Circular blur radius [pixels]
Classification accuracy [%]
(c) Blur, rotation and diff. illumin.
Figure 3. Classifications results for test suites (a) Outex TC 00000 (blur), (b) Outex TC 00010 (ro-
tation and blur) and (c) Outex TC 00012 (Rotation, illumination changes and blur)
otherthan LPQ, RI-LPQ, andRI-LPQa start to decrease
soon. It can be also noticed that standard LPQ performs
slightly better than RI-LPQ when no rotation is present,
but the difference is rather small. On the other hand,
comparing the rotation invariant LBP to standard LBP,
the performance difference is significant already with
small blur sizes.
In the second and third experiment, we used test
suites Outex TC 00010 and Outex TC 00012, which
contain 4320 and 9120 images from 24 classes with
rotation. The latter contains also images with differ-
ent illuminations. The results for these experiments,
are shown in Figures 3(b) and 3(c), respectively. As
can be seen, both RI-LPQ methods are insensitive to
blur, while the performance of the LBPriu2
creases rapidly when the blur strength increases. RI-
LPQ achieves also better or similar results for sharp im-
ages in both experiments. The results for RI-LPQ seem
to be better for a less blurred image, but RI-LPQa per-
forms equally well with high blur radiuses. The non-
rotation invariant methods achieved only slightly over
50 % accuracy even without any blur.
P,Rmethod de-
5. Conclusions
In this paper, we proposed a new rotation and blur
insensitive texture analysis method. The approach in-
troduced uses the local low frequency Fourier phase in-
formation in two stages: first to estimate the local char-
acteristic orientations, and then to compute the directed
binary descriptors. Both steps are performed in a way
thattheyareinsensitivetocentrallysymmetricblur. Be-
cause onlyphaseinformationis used, the methodis also
invariant to uniform illumination changes.
In the experiments, the new method was shown to
outperformLBP and Gabordescriptors,with both sharp
and blurred textures. Also in the comparison to the
standard LPQ, the new method performed clearly bet-
ter in the case of rotated patterns, and with comparable
accuracy when no rotation was present. The proposed
method can be also implemented in such a way that the
computation load is close to that of the standard LPQ.
References
[1] J. Flusser and T. Suk. Degraded image analysis: An in-
variant approach. IEEE Trans. Pattern Anal. Machine In-
tell., 20(6):590–603, 1998.
[2] B. S. Manjunathi and W. Y. Ma. Texture features for
browsing and retrieval of image data. IEEE Trans. Pat-
tern Anal. Machine Intell., 18(8):837–842, 1996.
[3] T. Ojala, T. M¨ aenp¨ a¨ a, M. Pietik¨ ainen, J. Viertola,
J. Kyll¨ onen, and S. Huovinen. Outex - new framework
for empirical evaluation of textureanalysis algorithms. In
Proc. 16th Int. Conf. on Pattern Recognition (ICPR’02),
pages 701–706, 2002.
[4] T. Ojala, M. Pietik¨ ainen, and T. M¨ aenp¨ a¨ a. Multireso-
lution gray-scale and rotation invariant texture classifica-
tion withlocal binary patterns. IEEE Trans. Pattern Anal.
Machine Intell., 24(7):971–987, 2002.
[5] V. Ojansivu and J. Heikkil¨ a. Blur insensitive texture clas-
sification using local phase quantization. In Proc. Int.
Conf. on Image and Signal Processing (ICISP’08), pages
236–243, 2008.
[6] M. Tuceryan and A. K. Jain. Texture analysis. In C. H.
Chen, L.F.Pau, and P.S.P.Wang, editors, TheHandbook
of Pattern Recognition and Computer Vision, pages 207–
248. World Scientific Publishing Co., 1998.
[7] J. van de Weijer and C. Schmid. Blur robust and color
constant image decription. In Proc. IEEE Int. Conf. on
Image Processing (ICIP’06), pages 993–996, 2006.
[8] A. P.Vo, S.Oraintara, and T. T. Nguyen. Usingphase and
magnitude information of the complex directional filter
bank for texture image retrieval. In Proc. IEEE Int. Conf.
on Image Processing (ICIP’07), pages 61–64, 2007.
[9] L. Xiuwen and W. DeLiang. Texture classification us-
ing spectral histograms. IEEE Trans. Image Processing,
12(6):661–670, 2003.