A Time-Varying Subband Transform with Projection-Based Reconstruction

Abstract

this paper a subband transform implemented by an N-channel maximally decimated uniform filter bank, which consists of a set of N filters {h N-1 n=0 followed by the downsampler of factor N. We also assume here that they have the same length KN, where K is an integer. Let E = [E K-1 , . . . , E 0 ] = [h 0 , . . . , h N-1 ] , (1) where E k , k = 0, . . . , K-1 is an analysis transform matrix of size N N, which leads to a polyphase matrix of the analysis matrix described by E(z) = k=0 E k z -k . Let f (t)} t#T be a set of blocked signals (or vectors at time instance t) consisting of N consecutive samples, where describes a set of time indexes. The analysis subband transform generates Manuscript received November 22, 2002

Full text

Page 1

IEICE TRANS. FUNDAMENTALS, VOL.E86–A, NO.8 AUGUST 2003
1
PAPER
A Time-Varying Subband Transform with Projection-Based
Reconstruction
Special Section on Digital Signal Processing
Toshihisa TANAKA†a), Regular Member, Takateru SAITO††, Student Member,
and Yukihiko YAMASHITA†††b), Regular Member
SUMMARY
application in image coding are introduced. We construct the framework
in which an analysis subband transform is chosen from a given set of anal-
ysis transforms and perfect reconstruction is achieved. To this end, we
suggest the reconstruction method from the transformed coefficients by ap-
plying the theory of convex projections (POCS). We introduce convex sets
for perfect reconstruction from the transformed vector. We further pro-
pose other convex sets which assure smoothness of plane regions for image
coding applications. We show several image coding examples. The pro-
posed coding method is an extension of conventional one with multiple
block-based transforms. At each time instance (block), the transform to
be applied is chosen from a given set of orthogonal subband transforms
according to a certain criterion. Experimental results show that the use of
multiple subband transforms leads to the improvement of coding perfor-
mance compared to the use of single transform even though there exists
side information.
key words: Subband Transform, Filter Bank, POCS, Image Coding
A novel class of time-varying subband transforms and its
1.Introduction
Subband transforms are widely used in the area of signal
processing and subband coding [1]. The subband trans-
form that decomposes an original image with a set of filters
and downsamplers is generally regarded as a filter bank [2].
We consider in this paper a subband transform implemented
by an N-channel maximally decimated uniform filter bank,
which consists of a set of N filters {hn}N−1
downsampler of factor N. We also assume here that they
have the same length KN, where K is an integer. Let
n=0followed by the
E = [EK−1,..., E0] = [h0,..., hN−1]T,
(1)
where Ek, k = 0,...,K−1 is ananalysis transform matrixof
size N × N, which leads to a polyphase matrix of the analy-
sis matrix described by E(z) =?K−1
a set of blocked signals (or vectors at time instance t) con-
sisting of N consecutive samples, where T describes a set
of time indexes. The analysis subband transform generates
k=0Ekz−k. Let {f(t)}t∈Tbe
Manuscript received November 22, 2002.
Manuscript revised March 3, 2003.
†The author is with the Laboratory for Advanced Brain Signal
Processing, Brain Science Institute, RIKEN, Saitama 351-0198,
Japan
††The author was with the Department of Electrical and Elec-
tronic Engineering, Tokyo Institute of Technology, Tokyo, 152-
8552, Japan
†††The author is with the Department of International Devel-
opment Engineering, Tokyo Institute of Technology, Tokyo, 152-
8552, Japan
a)E-mail: tanaka@bsp.brain.riken.go.jp
b)E-mail: yamasita@ide.titech.ac.jp
a sequence of the transformed vectors {g(t)}t∈Tas
g(t) =
K−1
?
k=0
Ekf(t − k) = E¯f(t),
(2)
where ¯f(t) = [f(t − (K − 1))T,..., f(t)T]T. In most of
applications, it is desired that the original sequence f(t) is
perfectly reconstructed from the transformed sequence g(t).
Such an invertible subband transform is studied well and is
called the perfect reconstruction filter bank [1,3,4]. In the
image coding community, moreover, this class of subband
transforms is still promising techniques, even if wavelets
have been adopted in the JPEG 2000 image coding stan-
dard [5], because several kinds of filter banks, which pro-
vide better coding efficiency than wavelets do, have been
reported [6–8].
We can consider here a more general setting:
g(t) = E(t)¯f(t),
(3)
where E(t) is a time-varying matrix of size N × KN. The
most possible application of this class of transforms will be
image coding. Let us consider a natural image, which is
generally a non-stationary process. This can be intuitively
understood fromthe fact that a natural imagecan be roughly
classified into plane (smooth) and edge regions, whose cor-
relation functions can be different from each other. There-
fore, there is a doubt that a single transform is applied to
a whole image. From this point of view, the use of various
subband transforms fordecomposing an image will improve
performance in coding. This strategy seems like adaptive
image coding such as JPEG-LS [5]. It should be, however,
noted that in this adaptive coding, adaptation is performed
in the quantization and/or encoding step. In most lossy cod-
ing methods, the transform step is followed by the quanti-
zation/coding step. The strategy mentioned above is quite
different from the adaptive coding method in the sense that
the adaptation is applied in the transform step.
Several studies of time-varying subband transforms (or
equivalently filter banks) have been conducted. De Queiroz
and Rao proposed the so-called time-varying lapped trans-
form [9]. In this transform, a subband transform matrix is
decomposed to orthogonal matrices, which vary depending
on time. This strategy has been applied to image coding ap-
plications [10]. In this class, however, every analysis trans-
form E(t)isdetermineddependingonthepast2(K−1)trans-
forms E(t −k), k = 1,...,2(K −1). As a result, the analysis

Page 2

2
IEICE TRANS. FUNDAMENTALS, VOL.E86–A, NO.8 AUGUST 2003
Analysis Subband
Transform 1
Original
Image
Quantizer/
Coder
Analysis Subband
Transform 2
Fig.1
posed in this paper
An example of the time-varying subband transform structure pro-
transform E(t) has the corresponding explicit inverse R(t)
and the reconstructed signal obtained by
ˆf(t) = R(t)¯ g(t),
(4)
where ¯ g(t) = [g(t − (K − 1))T,..., g(t)T]T, equals to the
original signal, i.e.,ˆf(t) = f(t). Tanaka and Yamashita pro-
posed a lapped transform consisting of overlapping filters
which do not change and non-overlapping filters which can
vary depending on time [11]. Since this transform also has
an explicitinverse, the original signal canbe obtainedby the
reconstruction formula (4).
As we have seen, these conventional time-varying sub-
band transforms have very restricted structures. Therefore,
we will deal in this paper with a more general class of
time-varyingsubband transforms in whichananalysis trans-
form can be determined independently of the past ones, and
therefore the original signal can not generally obtained by
the straightforward reconstruction as in (4). Specifically,
we consider the case where we are given multiple subband
transforms and each transform E(t) corresponds to one of
the subband transforms (see Fig. 1). We further assume
that given analysis subband transforms are paraunitary filter
banks [2]. Therefore, a subband transform havingthe parau-
nitary property also has the perfect reconstruction property.
The problem here is, however, that we generally cannot re-
construct the original signal f(t) from the transformed vec-
tor g(t) generated in the model given as in (3) by simply
applying the synthesis subband transform corresponding to
the analysis one E(t). In Section 3, we address this problem
and show a general reconstruction method by using the pro-
jection onto convex sets (POCS) [12] which ensures perfect
reconstruction. The result leads to a wide range of applica-
tion, since we only impose paranunitariness on each anal-
ysis subband transform E(t). Any paraunitary filter banks
such as lapped orthogonal transforms [3,4,6,13], parauni-
tary cosine-modulated filter banks/modulated lapped trans-
form [3, 14–16] can be adaptively applied in terms of the
proposed framework.
In the remainder of the paper, we concentrate our ar-
guments upon an application in image coding with lapped
orthogonal transforms. In Section 4, we attempt to improve
subjective quality of decoded images by introducing addi-
tional convex sets. In Section 5, we show a criterion for
choosing the best analysis transform in image coding appli-
cations and also illustrate an image coding method. More-
over, we exhibit results of image coding experiments, which
tell us that the use of time-varying subband transforms im-
proves the quality of decoded images.
2.Preliminaries
2.1Notations
The following conventions are adopted in terms of notation:
Bold-faced characters are used to denote vectors and matri-
ces. Inand 0ndenote the n × n identity matrix and the n × n
null matrix, respectively. ATdenotes the transposition of
A. I denotes the identity operator. A∗denotes the adjoint
operator of A.
2.2 Theory of POCS
The theory of POCS was introduced by Youla and Webb
[12]. Let H be a Hilbert space. Then, for arbitrary element
f ∈ H, its projection Pf onto a closed convex set C ⊂ H is
defined as the closest element to f in C:
?f − Pf? = min
g∈C?f − g?.
(5)
Note that Pf is uniquely determined by f and C.
Assume that M closed convex sets Ci, i = 0,..., M−1,
in H, are available and let Pibe the projector onto Ci. It
was proven in [12] that the sequence {fk} generated by the
iteration given as fk+1= PM−1···P1P0fkweakly converges
to a point f∗∈ C∗=?M−1
The following important collorary was also proved by
Youla [17]. Let Hi be a closed subspace of H such that
?M−1
an orthogonal projector onto Hi. Then, if a convex set Ci
corresponds to a closed linear manifold defined as
i=0Ci, if C∗? ∅.
i=0Hi = {0} and H = H0+ ··· + HM−1, and let Pibe
Ci= f(i)+ H⊥
i,
(6)
where f(i)= Pif, theintersectionC∗hasonlyoneelement f,
that is, C∗= {f}. This iterative signal restoration method is
calledthe methodof alternating orthogonalprojections [17].
3.Time-Varying Subband Transform and Reconstruc-
tion Method
It is shown in this section that by introducing appropriate
closed linear manifolds, we can exactlyreconstruct the orig-
inal image from the transformed vector.
We obtain the transformed vector by the transforma-
tion described in (3). Let E be a set of L orthogonal subband
analysis transform matrices E(l): E = {E(l),l = 0,...,L−1}.
We assume here that each E(t) corresponds one of elements
of the analysis transform set E, that is, E(t) ∈ E. Figure
1 depicts an simple example of the time-varying subband
transform when L = 2. The conventional time-varying sub-
band transforms belong this class.
Therefore, we can categorize the sequence {g(t)} ac-
cording to which subband transform is selected. We write
by Gla set of transformed vectors g(t) that is generated by
E(l).
Now, we can rewrite (3) as

Page 3

TANAKA et al.: A TIME-VARYING SUBBAND TRANSFORM WITH PROJECTION-BASED RECONSTRUCTION
3
g(t) =
L−1
?
l=0
∆l(t)E(l)¯f(t),
(7)
where
∆l(t) =
?
IM,
0M,
if E(t) = E(l)
otherwise.
(8)
It is clear that?L−1
l=0∆l(t) = IM. Let
˜Pl =

...
0
∆l(t)
∆l(t + 1)
0
...

,
(9)
and
E(l)=

...
0
E(l)
K−1
···
E(l)
K−1
E(l)
···
0
E(l)
0
0
...

.
(10)
Then,˜Plrepresents an orthogonal projector with diagonal
entries 0 or 1, and E(l)describes a subband transform oper-
ator which is not time-varying. Notice that?L−1
can further modify (3) as
l=0˜Pl= I. We
g = Af,
(11)
where
A =
L−1
?
l=0
˜PlE(l),
(12)
and f = [..., f(t−1)T, f(t)T, f(t+1)T,...]T, g = [..., g(t−
1)T, g(t)T, g(t + 1)T,...]T∈ H (e.g. H = ℓ2(Z)). Now, the
problem is simplified: Find the inverse of the operator A. In
the following, we show the method for finding the inverse
by the theory of POCS.
From (11), we obtain
˜f = A∗g = A∗Af =

L−1
?
l=0
Pl
f,
(13)
where
Pl= E(l)∗˜PlE(l).
(14)
We hereobtain the problem to find (A∗A)−1beingequivalent
to the one to find A−1. The following proposition enables us
to apply the theory of POCS to finding the inverse.
Proposition 1: If the subband transform operator E(l)is
unitary, i.e., E(l)∗E(l)= I, the operator Plis an orthogonal
projector.
Proof: It can be easily checked that P2
Proposition 1 implies that Pl defines the closed sub-
space denoted by Hlas its range as follows. For an element
l= Pland P∗
l= Pl. 2
defined as f(l)= Plf, we can define the set
Cl= f(l)+ H⊥
l.
(15)
It is verified thatClis a closed convex set, or rather, a closed
linear manifold, and the projection onto Clis given as
PClx = f(l)+ (I − Pl)x,
(16)
for any element x in H. Finally, the original signal is ob-
tained as the limit of the recursion as
fi+1= PCL−1···PC1PC0fi, f0=˜f.
(17)
As seen, the reconstruction problem from the transformed
coefficients yields the image recovery problem by the
method of alternating orthogonal projections.
4.Improved Convex Sets for Enhancement of Decoded
Images
The reconstruction based on alternating orthogonal projec-
tions that is used in the previous section guarantees that the
original imageis exactlyreconstructedfromthetransformed
vectors that are not corrupted by quantization or truncation.
In the actual situation of image coding, however, only the
transformed vectors where all the coefficients are corrupted
by quantization are available.
As an effect of quantization, it is well known that
in block-based coding, the blocking effect appears as very
sharp square edges because of block partitioning. Although
subbandtransformscanreducetheblockingeffect,apercep-
tual distortion like blocking still remains. In order to reduce
this artifact, we extend the convex sets Clthat assure perfect
reconstruction introduced in the previous section.
Before the formulation, we will define two terms for
indicating a set of transformed coefficients. Assume that the
filters hn, n = 0,...,N −1 are bandpass filters ordered from
low to high passbands. The set {[g(t)]0}t∈T is a collection
of coefficients generated by the lowpass filter h0, which is
called the DC subband. Similarly, the sets {[g(t)]n}t∈T, n =
1,...,N −1 are called the AC subbands. Therefore, there is
one DC subband and N − 1 AC subbands in a transformed
image.
We first modify the convex sets Cl for the original
image f to those only for the AC subband of f. Specif-
ically, we reconstruct only AC subbands from the trans-
formed vectors g(t). Let˜P−and˜P∼be orthogonal projec-
tors which extract the DC subband and the AC subbands,
respectively. Moreover, we define the orthogonal projectors
as P−l= E(l)∗˜P−˜PlE(l)and P∼l= E(l)∗˜P∼˜PlE(l). It is evident
that P−l+ P∼l= Pl. Then, the modified closed convex sets
are defined as follows:
C∼l= f(l)
∼+ H∼⊥
l,
(18)
where f(l)
range of P∼l. These convex sets guarantees that AC sub-
bands are perfectly reconstructed from the transformed vec-
tors without quantization.
∼
= P∼lf and H∼lis the subspace defined as the

Page 4

4
IEICE TRANS. FUNDAMENTALS, VOL.E86–A, NO.8 AUGUST 2003
(a) LOT16, {t|g(t) ∈ G0}
(b) LOT48, {t|g(t) ∈ G1}
Fig.2
Classification maps at rate 0.50 bpp
We further introduced a “relaxed” convex set for the
DC subband. We adopt the convex set such that the coeffi-
cients in the DC subband are band-limited:
C−l= {f|˜P−E(l)f is band-limited.}.
(19)
However, we can only give approximated projectors onto
these convex sets since the corresponding convex projectors
are given by ideal band-pass filters.
In the decoder, additionally, we know the step sizes of
quantizer. Therefore,we canintroducethefollowingconvex
sets:
Cq= {f|ci− qi/2 ≤ ci≤ ci+ qi/2,i ∈ I}.
(20)
where I is a set of indexes for the DC coefficients such that
(˜P−˜Pl)i,i= 1, ciis an ith coefficient of the DC subband, that
is, ci= (˜P−˜PlE(l)f)i, and qiis the step size of the quantizer
for ci. Actually, this approach is similar to the deblocking
method for DCT-coded image proposed by Zakhor [18] and
modified by Yang et al [19]. However, the main difference
lies in defining convex set only for the DC subband in the
transform domain.
5. Image Coding Applications
In this section, we clarify the efficiency of the proposed
time-varying transform in image coding through simulation
studies.
5.1 Transform Selection Criteria
We have to definethe criteria to choose the most appropriate
transform at the encoder. At eachtime instance (block) t, we
canobtain L different transformedvectors denoted by g(l)(t),
l = 0,...,L − 1, as g(l)(t) = E(l)¯f(t). Then, we choose one
vector g(l∗(t))(t) from the L vectors according to the criterion
described as follows. Let R(g(l)(t)) be the code length of
g(l)(t) encoded by an entropy coder followed by a quantizer.
1. Let R∗= minl=0,...,L−1R(g(l)(t)). If the minimizer l∗(t) is
uniquely determined, we choose it, that is,
l∗(t) = argmin
l=0,...,L−1R(g(l)(t)),
(21)
and we have that g(t) = g(l∗(t))(t).
2. Otherwise, let L∗be a set of indexes l such that g(l)(t)
minimizes R(g(l)(t)), that is, L∗= {l|R∗= R(g(l)(t))}.
Then, we find g(l)(t), l ∈ L∗that minimizes the squared
norm ?g(l)(t)− ˆ g(l)(t)?2, where ˆ g(l)(t) is the transformed
vector consisting of the quantized coefficients. The
minimizer is given by
l∗(t) = argmin
l∈L∗
?g(l)(t) − ˆ g(l)(t)?2,
(22)
where the minimizer is not always determined
uniquely. In this case, we choose any minimizer here,
and we have that g(t) = g(l∗(t))(t).
We now have a set of the selected vectors {g(l∗(t))(t)}K−1
decoding, the encoder has to encode side information with
respect to l∗(t) as well.
k=0. For
5.2 Experimental Results and Discussion
The time-varying subband transform used here consists of
two subband transforms (L = 2) listed in Table 1. Each
subband transform is paraunitary, and therefore we can per-
fectly reconstruct the original signal by using the alternating
projections.
We can utilize the same coder as the baseline JPEG
[20] to encode the transformed coefficients. That is, the
transformed coefficients are uniformly quantized, zig-zag
scanned, and Huffman coded with run-length. Because we
adopt two transforms to construct the time-varying trans-
form, the amount of bits for side information equals to the
number of blocks of the image. For example, the bit rate of
side information without compression requisite for coding
an image of size 512 × 512 with the time-varying transform
is (512/8)2/5122= 0.015625 bits per pixel (bpp).
The transforms we compare are:
• DCT8: Discrete cosine transform of block size eight,
which is extensively used for image/video coding stan-
dards [21,22]
• PLOT16/48: Time-varying subband transform with the
reconstruction method based on POCS introduced in
Section 3, in which LOT16 and LOT48 are selectively
used.
• LOT16: Lapped orthogonal transform (LOT) [3]
• LOT48:Generalized lapped orthogonal transform
(GenLOT) [4]
Moreover, we compare two proposed reconstruction meth-
ods with POCS:
• Method I†, which can achieve perfect reconstruction,
described in Section 3.
• Method II††, which can achieve perfect reconstruction
†The reconstruction based on POCS is only used for the time-
varying subband transform PLOT16/48. For LOT16 and LOT48,
Method I means that we just apply synthesis transform to the trans-
formed coefficients.
††LOT16 and LOT48 are not time-varying, and therefore we do
not need the reconstruction for AC subbands in this case

Page 5

TANAKA et al.: A TIME-VARYING SUBBAND TRANSFORM WITH PROJECTION-BASED RECONSTRUCTION
5
(a) Original(b) DCT8
(c) PLOT16/48 (I)(d) LOT16 (I)(e) LOT48 (I)
(f) PLOT16/48 (II) (g) LOT16 (II) (h) LOT48 (II)
Fig.3
Decoded images of 512 × 512 Barbara at 0.25 bpp.
for the AC components and assures smoothness in the
DC subband, described in Section 4.
Imagesused for the comparison are Barbara, Lena, and Pep-
per all of size 512 × 512. For an approximated convex pro-
jector onto C−las in (19), a simple separable lowpass filter