Blocking artifact detection and reduction in compressed data
ABSTRACT A novel frequency-domain technique for image blocking artifact detection and reduction is presented. The algorithm first detects the regions of the image which present visible blocking artifacts. This detection is performed in the frequency domain and uses the estimated relative quantization error calculated when the discrete cosine transform (DCT) coefficients are modeled by a Laplacian probability function. Then, for each block affected by blocking artifacts, its DC and AC coefficients are recalculated for artifact reduction. To achieve this, a closed-form representation of the optimal correction of the DCT coefficients is produced by minimizing a novel enhanced form of the mean squared difference of slope for every frequency separately. This correction of each DCT coefficient depends on the eight neighboring coefficients in the subband-like representation of the DCT transform and is constrained by the quantization upper and lower bound. Experimental results illustrating the performance of the proposed method are presented and evaluated.
- SourceAvailable from: microsoft.com[show abstract] [hide abstract]
ABSTRACT: An exact derivation of an optimal lapped orthogonal transform (LOT) is presented. The optimal LOT is related to the discrete cosine transform (DCT) in such a way that a fast algorithm for a nearly optimal LOT is derived. Compared to the DCT, the fast LOT requires about 20-30% more computations, mostly additions. An image coding example demonstrates the effectiveness of the LOT in reducing blocking effects; the LOT actually leads to slightly smaller signal reconstruction errors than does the DCTIEEE Transactions on Acoustics Speech and Signal Processing 05/1989;
- IEEE Transactions on Signal Processing 01/1998; 46:1043-1053. · 2.81 Impact Factor
Conference Proceeding: Post-Filtering Methods for Reducing Blocking Effects from Coded ImagesConsumer Electronics, 1994. Digest of Technical Papers., IEEE 1994 International Conference on; 07/1994
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002 877
Blocking Artifact Detection and Reduction
in Compressed Data
George A. Triantafyllidis, Student Member, IEEE, Dimitrios Tzovaras, and
Michael Gerassimos Strintzis, Senior Member, IEEE
Abstract—A novel frequency-domain technique for image
blocking artifact detection and reduction is presented in this
paper. The algorithm first detects the regions of the image which
present visible blocking artifacts. This detection is performed in
the frequency domain and uses the estimated relative quantization
error calculated when the discrete cosine transform (DCT)
coefficients are modeled by a Laplacian probability function.
Then, for each block affected by blocking artifacts, its dc and
ac coefficients are recalculated for artifact reduction. To achieve
this, a closed-form representation of the optimal correction of the
DCT coefficients is produced by minimizing a novel enhanced
form of the mean squared difference of slope for every frequency
separately. This correction of each DCT coefficient depends on the
eight neighboring coefficients in the subband-like representation
of the DCT transform and is constrained by the quantization
upper and lower bound. Experimental results illustrating the
performance of the proposed method are presented and evaluated.
Index Terms—Blocking artifacts, compressed domain, MSDS
and video compression standards including JPEG , ,
H.263 , MPEG-1, MPEG-2, MPEG-4 , and others, used
in a wide range of applications. The B-DCT scheme takes
advantage of the local spatial correlation property of the images
by dividing the image into 8
8 blocks of pixels, transforming
each block from the spatial domain to the frequency domain
using the discrete cosine transform (DCT) and quantizing the
DCT coefficients. Since blocks of pixels are treated as single
entities and coded separately, correlation among spatially adja-
cent blocks is not taken into account in coding, which results
in block boundaries being visible when the decoded image is
reconstructed. For example, a smooth change of luminance
across a border can result in a step in the decoded image if
neighboring samples fall into different quantization intervals.
HE BLOCK-based discrete cosine transform (B-DCT)
scheme is a fundamental component of many image
Manuscript received May 11, 2000; revised March 20, 2002. This work was
supported by the IST European Project OTELO. This paper was recommended
by Associate Editor O. K. Al-Shaykh.
G. A. Triantafyllidis is with the Information Processing Laboratory,
Aristotle University of Thessaloniki, Thessaloniki 54006, Greece (e-mail:
D. Tzovaras is with the Informatics and Telematics Institute, Thessaloniki
546 39, Greece (e-mail: Dimitrios.Tzovaras@iti.gr).
M. G. Strintzis is with the Information Processing Laboratory, Aristotle
University of Thessaloniki, Thessaloniki 54006, Greece, and also with the
Informatics and Telematics Institute, Thessaloniki 546 39, Greece (e-mail:
Digital Object Identifier 10.1109/TCSVT.2002.804880
Such so-called “blocking” artifacts are often very disturbing,
especially when the transform coefficients are subject to coarse
Subjective picture quality can be significantly improved by
decreasing the blocking artifacts. Increasing the bandwidth or
bit rate to obtain better quality images is often not possible or is
too costly. Other approaches to improve the subjective quality
of the degraded images have been published. Techniques which
do not require changes to existing standards appear to be the
most practical solution, and with the fast increase of available
computing power, more sophisticated methods can be imple-
mented. If the blocking effects can be significantly reduced, a
higher compression ratio can be achieved.
The detection scheme is applied in the subband-like represen-
tation of the modified DCT coefficients which are produced, if
we assume that theDCT coefficients follow the Laplacian prob-
ability model  (hereafter, Laplacian corrected DCT coeffi-
cients). Specifically, the presence of visual blocking artifacts of
the B-DCT reconstructed image is inferred from data in the fre-
difference of two neighboring Laplacian corrected DCT coeffi-
cients in thesubband-like domain is greaterthan a threshold.
frequency DCT coefficients are recalculated by minimizing a
novel enhanced form of the mean squared difference of slope
(MSDS) , which involves all eight neighboring blocks. The
minimization is constrained by the quantization bounds and is
performed for every frequency separately, in the subband-like
representation of the DCT transform. Thus, a closed-form
terms of the eight neighboring coefficients in the subband-like
A first major advantage of the proposed algorithm is that it is
applied entirely in the compressed domain. This is in contrast
to the large majority of the deblocking algorithms which are
applied in the spatial domain.
Compared to other methods of deblocking in the frequency
• theproposedalgorithm introducesthenovelandenhanced
form of MSDS which involves all neighboring blocks,
including the diagonally located neighboring blocks;
• our algorithm minimizes MSDS in the frequency do-
main, in order to recalculate the DCT coefficients for
blockiness removal. This minimization is performed
1051-8215/02$17.00 © 2002 IEEE
878IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
for each frequency separately, producing better results
than global minimization. This is intuitively expected
because B-DCT schemes (such as JPEG) use scalar rather
than vector quantization, and also, the DCT transform
(which is an approximation of the Karhuene–Loeve (KL)
transform) produces almost uncorrelated coefficients’;
• furthermore, this paper contains the novelproposition that
the Laplacian corrected DCT coefficients should be used
in place of the simple DCT reconstructed coefficients in
the formula which provides the correction of the DCT
coefficients, because they provide statistically better es-
timates of the original DCT coefficients. This proposition
is supported by the experimental results;
• finally, this paper presents not only a method for the
removal of the blocking artifacts, but also proposes a
novel blockiness detection method which reduces the
time and the computational load of deblocking algorithms
by having the deblocking algorithm applied only where
needed (where there exist disturbing blocking artifacts).
The rest of this paper is organized as follows. In Section II, a
review and discussion of various techniques that have been pro-
posed in the past for the removal of blocking artifacts are given.
Section III describes the mathematical analysis underlying the
concept of blocking artifact detection in the subband-like trans-
form domain, under the assumption that the DCT transform fol-
lows a Laplacian probability density function (pdf). Section IV
strained minimization. Experimental results given in Section V
evaluate visually and quantitatively the performance of the pro-
posed methods. Finally, conclusionsare drawninSection VI.
Many techniques have been proposed in the literature for the
reduction of blocking artifacts. Two general approaches have
been followed. In the first approach, the blocking effect is dealt
with at the encoding side , . The second approach pro-
poses postprocessing at the decoding side, aiming to improve
the visual quality of the reconstructed image without any mod-
ification on the encoding or decoding procedures. Due to this
advantage, most of the recently proposed algorithms follow the
The majority of the postprocessing techniques in the litera-
ture are applied in the spatial domain. These techniques may be
restoration approaches –.
In the first category, Jarske et al.  test several filters to
conclude that the Gaussian low-pass filter with a high-pass fre-
quency emphasis gives the best performance. Reeves and Lim
 apply the 3
3 Gaussian filter only to those pixels along
block boundaries. A similar technique by Tzou  applies a
separable anisotropic Gaussian filter, such that the primary axis
of the filter is always perpendicular to the block boundary. A
space-variant filter that adapts to local characteristics of the
signal is proposed by Ramamurthi and Gersho in . The al-
gorithm distinguishes edge pixels from nonedge pixels via a
neighborhood testing and then switches between a one–dimen-
sional (1-D) and a two–dimensional (2-D) filter accordingly to
reduce blocking effects. In , an adaptive filtering scheme
is reported, progressively transforming a median filter within
blocks to a low-pass filter when it approaches the block bound-
aries. An adaptive filtering process is also employed in .
Theshape and thepositionof theGaussian filteringare adjusted
based on an estimation of the local characteristics of the coded
image. A region-based method is presented in , where the
degraded image is segmented by a region growing algorithm,
and each region obtained by the segmentation is enhanced sep-
arately bya Gaussian low-passfilter. Lee etal. in  propose a
2-D signal-adaptivefilteringand Chouet al.  removeblock-
iness by performing a simple nonlinear smoothing of pixels.
In , Apostolopoulos et al. propose to identify the blocks
that potentially exhibit blockiness by calculating the number of
nonzero DCT coefficients in a coded block and comparing it to
a threshold. Then, a filter is applied along the boundaries but
only updating the pixels within the distorted block. However,
the above filtering approaches frequently result in overblurred
recovered images, especially at low bit rates. Another approach
is proposed in –, where the optimal filters for subband
coding of the quantized image are efficiently determined for the
reduction of quantization effects in low bit rates.
In the second category, Xiong et al.  use an overcom-
plete wavelet representation to reduce the quantization effects
of block based DCT. Other approaches using wavelet represen-
tation are presented in , , and . In  the wavelet
efficient image deblocking.
In the third category, O’ Rourke and Stevenson  propose
a postprocessor that can remove blockiness in block encoded
images. To achieve this, they maximize the a posteriori proba-
the decompressed image is modeled by an MRF, and the Huber
minimax function is chosen as a potential function. A similar
approach is followed by Luo et al. . In , Meier et al. re-
move blocking artifacts by first segmenting the degraded image
into regions by an MRF segmentation algorithm, and then each
region is enhanced separately using an MRF model.
Finally, in the fourth category, iterative image recovery
methods using the theory of projections onto convex sets
(POCS) are proposed in , , . In the POCS-based
method, closed convex constraint sets are first defined that
represent all of the available data on the original uncoded
image. Then, alternating projections onto these convex sets
are iteratively computed to recover the original image from
the coded image. POCS is effective in eliminating blocking
artifacts but less practical for real time applications, since the
iterative procedure adopted increases the computation com-
plexity. In  and , the constrained least-square method is
proposed, which aims to reconstruct the image by minimizing
an objective function reflecting a smoothness property.
Very few approaches in the literature have tackled the
problem of blocking artifact reduction in the transform domain
, . In the JPEG standard , a method for suppressing
the block to block discontinuities in smooth areas of the image
is introduced. It uses dc values from current and neighboring
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA879
blocks for interpolating the first few ac coefficients into the
current block. In , Minami and Zakhor present a new
approach for reducing the blocking effect. A new criterion,
the mean squared difference of slope (MSDS)—a measure of
the impact of blocking effects—is introduced. It is shown that
the expected value of the MSDS increases after quantizing the
DCT coefficients. This approach removes the blocking effect
by minimizing the MSDS, while imposing linear constraints
corresponding to quantization bounds. To minimize the MSDS,
a quadratic programming (QP) problem is formulated and
solved using a gradient projection method. The solution is
obtained in the form of the optimized value of the three lowest
DCT coefficients. The blocking effect due to the quantization
of low frequency coefficients is reduced if the quantized DCT is
replaced by the optimized values during the decoding phase. To
remove the high-frequency blocking effect, low-pass filtering
of the decoded image is proposed. In , Lakhani and Zhong
follow the approach proposed in  for reducing blocking
effects using, however, a different solution of the optimization
problem, minimizing the MSDS globally and predicting the
four lowest DCT coefficients.
Our proposed method for the reduction of blocking artifacts
also adopts thecriterion of MSDS. However,the form of MSDS
which is now used has been enhanced by also involving the
diagonal neighboring pixels. Furthermore, the optimization is
performed in the subband-like domain for each frequency sep-
arately using the Laplacian corrected DCT coefficients. Before
applying the blockiness reduction algorithm, a method is used
for the detection of the most disturbing blocking artifacts in
the reconstructed image. This method of blockiness detection
is elaborated in the next section.
III. DETECTION OF BLOCKING ARTIFACTS USING THE DCT
LAPLACIAN MODEL IN THE SUBBAND-LIKE DOMAIN
In the classical B-DCT formulation, the input image is first
divided into 8
8 blocks, and the 2-D DCT of each block is
determined. The 2-D DCT can be obtained by performing a 1-D
DCT on the columns and a 1-D DCT on the rows. The DCT
coefficients of the spatial block
are then determined by the
are the DCT coefficients of the
are the dimensions of the image, and
The transformed output from the 2-D DCT is ordered so that
the dc coefficient
is in the upper-left corner and the
Fig. 1.Subband-like domain of DCT coefficients.
higher frequency coefficients follow, depending on their dis-
tance from the dc coefficient. The higher vertical frequencies
frequencies are represented by higher column numbers.
A typicalquantization-reconstruction process of the DCT co-
efficients as described in JPEG  is given by
structed quantized coefficient. Then, the reconstructed pixel in-
tensity is obtained from the inverse DCT.
DCT coefficients with the same frequency index
all DCT transformed blocks can be scanned and grouped to-
gether, starting from the dc coefficients (
transforming an image with an 8
band transform of 64 frequency bands. Fig. 1 shows the scheme
coefficients reallocated to form a subband-like transform of the
We shall assume that for typical input image statistics, the
DCT coefficients may be reasonably modeled by a Laplacian
pdf as 
indicates the bin index in which the co-
falls, andrepresents the recon-
8 2-D DCT can be seen to
which is a zero-mean pdf with variance
If the Laplacian-modeled variable is quantized using uniform
step sizes, the only information available to the receiver is that
the original DCT coefficient is in the interval
880IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
Fig. 2.Subband-like domain of DCT coefficients of “Lena” image.
ficientin thecenterof theintervalas
plifies implementation. The optimal reconstruction (minimum
mean squared error) lies in the centroid of the distribution for
, thus, under the assumption of Lapla-
cian statistics , we have the following:
Note that this implies a bias
toward the origin
For different coefficients, we have different step sizes and vari-
ances . Therefore, considering the DCT coefficient of block
and variance. Then, the bias
be found from (9) and (7)
toward the origin can
Given the coefficient variances, we can estimate the
eters using (6), thus
can be easily estimated by
estimation is only used for the calculation of
the reconstructed DCT coefficients in place of the original DCT
coefficients, since only the former coefficients are available.
in , which estimates the variances of the DCT coefficients
from the variances of the pixel values using the form
. The variances
of the adopted method, described earlier. However, we did not
choose to employ the method in  because it is more compu-
Therefore, we can use (9) to precalculate all
tain the optimal estimation
coefficient (Laplacian corrected DCT coefficients)
so as to ob-
of the reconstructed DCT
positive and negative values and
The above concepts are now extended and applied to define
a method for blockiness detection. First, we define the newly
introduced relative theoretical quantization error
function is appropriately used to handle both
blocks. Occurrences of large values of this difference indi-
cate that very different levels were used to quantize the
blocks, producing a blocking artifact between these blocks.
Thus, we shall infer the presence of an artifact between blocks
is an adaptive threshold defined by
For a given compressed image, the detection criterion is
applied on each coefficient in all of the 64 bands of the DCT
subband-like domain, in order to locate the most disturbing
blocking artifacts. More specifically, for each band in the
subband-like domain, we scan the coefficients vertically,
horizontally and diagonally, and apply criterion (14). We
assume that a blocking artifact between the
block is disturbing when (14) is satisfied for
more than two frequencies in the subband-like domain, e.g.,
and frequencies. Thus, we introduce the following
Artifact between neighboring blocks
block and the
This criterion was tested in a large number of pictures and
was found to be very efficient in detecting the most disturbing
IV. REDUCTION OF BLOCKING ARTIFACT IN THE
As noted, blocking effects result in discontinuities across
block boundaries. Based on this observation, a metric called
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA881
MSDS was introduced in , involving the intensity gradient
(slope) of the pixels close to the boundary of two blocks.
Specifically, it is based on the empirical observation that
quantization of the DCT coefficients of two neighboring blocks
increases the MSDS between the neighboring pixels on their
To better understand this metric, consider an 8
the input image and a block
horizontally adjacent to . If the
coefficients of the adjacent blocks are coarsely quantized, a dif-
boundaries of the original unquantized image is rather unlikely,
because most parts of most natural images can be considered to
be smoothly varying and their edges are unlikely to line up with
block boundaries. From the above, it is clear that a reasonable
method for the removal of the blocking effects is to minimize
the MSDS, which is defined by
is the intensity slope across the boundary between
blocks, defined byand
is the average between the intensity slope of
blocks close to their boundaries, defined by
The ideas in the above discussion are applicable to both hori-
zontal and vertical neighboring blocks. Specifically, if blocks
,denote the blocks horizontally adjacent to , and blocks ,
present the blocks vertically adjacent to
which involves both horizontal and vertical adjacent blocks
(hereafter, MSDS ) is given by
, then, the MSDS
We now extend the definition of MSDS by involving the four
diagonally adjacent blocks. If
, then we define
, andare defined similarly to (17)–(19).
is a block diagonally adjacent
(hereafter, MSDS ) is
,,, and are the four blocks diagonally adjacent
; the MSDS involving only the diagonally adjacent blocks
and (22). Thus, the total MSDS (hereafter, MSDS ) considered
,, and are defined in a manner similar to (21)
Fig. 3.Vertical and horizontal slope (MSDS ) and diagonal slope (MSDS ).
in this paper, involving the intensity slopes of all the adjacent
Fig. 3 shows the pixels involved in the calculation of MSDS
and MSDS .
The form of MSDS used in the proposed methods of  and
 is MSDS which, as mentioned above, involves only the
horizontal and vertical adjacent blocks for its computation and
cent blocks. This implies that their methods cannot remove the
specific type of blocking artifact called “corner outlier” ,
which may appear in a corner point of the 8
over, even if we ignore the reduction of the corner outliers, the
introduction of the MSDS yields better results than the simple
form of MSDS , since more neighboring pixels (i.e., the neigh-
boring diagonal pixels) are used, providing a better estimation
for the DCT recalculation.
In , a global minimization of the MSDS is proposed
for the reduction of blocking effects. However, since B-DCT
schemes (such as JPEG) use scalar quantization (i.e., quanti-
zation of individual samples) for each frequency separately, a
separate minimization of the contribution of the quantization
of each particular coefficient to the blocking artifact is more
appropriate than a global minimization. Global minimization
would be more suitable if vector quantization (i.e., quantiza-
tion of groups of samples or vectors) of the DCT coefficients
were used, which is, however, not the case in B-DCT coding
schemes. Consider also that, since the DCT transform is very
close to the KL transform, the DCT coefficients are almost un-
correlated . Thus, the modification of each DCT coefficient
based on the minimization of MSDS which includes values of
the low-, middle-, and high-pass frequency coefficients is obvi-
ously not the best solution, and the minimization of MSDS for
each frequency separately is the appropriate procedure.
The new enhanced form of the MSDS
eight neighboring blocks is used in this paper, and its local
constrained minimization for each frequency, produces a
closed-form representation for the correction of the DCT
coefficients in the subband-like domain of the DCT transform.
To achieve this, the form of MSDS in the frequency domain
is obtained, and all other frequencies apart from the one
under consideration are set to zero. It was observed that only
8 block. More-
882IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
the first sixteen DCT coefficients (i.e.,
be recalculated by MSDS minimization, since the modification
of the remaining coefficients does not improve significantly
the reduction of the blocking artifacts (because of their poor
contribution to MSDS ), while requiring noneligible extra
computational load. In the sequel, the MSDS is calculated and
minimized in the frequency domain.
) need to
A. Calculation of MSDS in the Frequency Domain
denote a 8
denote its forward DCT, where
denote the eight blocks adjacent to
vertical, and diagonal directions, and
, anddenote their corresponding forward DCTs.
Following (18) and (19), the expression
which is used for the calculation of
8 block of the input image and
, , , ,
in the horizontal,
in (17) is
matrix [where the
th row of
is the basis vector
denote its transpose.
transform as follows:
block can be derived from the inverse DCT
crete cosine transformation matrix
easily seen to equal
can also be expressed in the frequency domain and (25) can be
expressed as follows:
anddenote the th row and th column of the dis-
. Using (26),
. Likewise, the other terms of (25)
(27) reduces to
denotes the row number and , expression
is the identity matrix. Thus,
according to (17), the
and blocks is produced
adding the squares of (29) for all
the frequencies ??? ?? ? ?? ?.
Frequency response of the filter derived by the proposed method for
The sum of the MSDS terms of the
the four horizontally and vertically adjacent blocks can now be
expressed as 
block corresponding to
B. Calculation of MSDS in the Frequency Domain
Using (22), the expression
calculation of the MSDS term
, which is used for the
in (21), is found by
The above may be expressed in the frequency domain, using
Using (28), (33) reduces to
Using (21), the MSDS term
wise, similar expressions are found for
from (23) the expression of the MSDS in the frequency do-
main is immediately obtained.
is now easily computed. Like-
,, and, and
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA883
512 ? 512 original “Boat” image. (d) 512 ? 512 original “Crowd” image. (e) 256 ? 256 original “Moon” image. (f) 256 ? 256 original “Couple” image. (g) 256
? 256 original “Girl” image. (h) 256 ? 256 original “Pentagon” image. (i) 128 ? 128 original “Claire” image. (k) First frame of the 352 ? 240 original “Tennis”
image sequence. (l) First frame of the 176 ? 144 original “Foreman” image sequence.
Images used for the experimental evaluation of the proposed method. (a) 512 ? 512 original “Lena” image. (b) 512 ? 512 original “Peppers” image. (c)
C. Local Minimization of MSDS for Each Frequency
We now set to zero all frequencies apart from frequency
. This implies that we set to zero all elements of the DCT
matrices involved in the expressions of MSDS and MSDS in
the frequency domain, apart form the specific
Thus, for the computation of MSDS using (31), we set to zero
all elements with frequencies
,, and. If is the
of the matrices,
th element of the vector
, the MSDS for the specific frequencyis
now easily derived from (31) as follows:
884IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
where the subscripts
For MSDS , we also set to zero allfrequencies apart from the
. Then, if
(21) and (34),we obtainfor the MSDS term
frequency the following expression:
indicate the th element of each
, , and using
For all four diagonal blocks, the MSDS
for the specific fre-
Setting the gradient of MSDS
tain the representation corresponding to the minimum MSDS .
Therefore, the imposition of
and MSDS to zero, we ob-
expression of the DCT coefficient at frequency
in (40), at the bottom of the page, subject to
. Thus, (39) provides the
, as shown
Note that the proposed algorithm is developed in the fre-
quency domain using DCT coefficients of the image. We chose
however to use the Laplacian corrected DCT coefficients in-
stead of the simple DCT reconstructed coefficients in the for-
mula which provides the correction of the DCT coefficients,be-
cause the former are statistically better estimates of the original
Therefore, (40)—subject to the constraint of—(41) provides
the correction of the
DCT coefficient for the reduction of
the blocking effect in B-DCT coded images (e.g., JPEG coded
images), in terms of its eight neighboring Laplacian corrected
DCT coefficients in the subband-like domain. Fig. 4 shows the
frequency response of the filter of (40) for the example frequen-
andare the quantization upper and lower limit,
V. EXPERIMENTAL RESULTS
In this section, simulation results demonstrating the perfor-
mance of the proposed technique are presented. For this pur-
of blocking artifacts at 0.4096 bpp. (c) Portion of JPEG coded “Lena” image
at 0.2989 bpp. (d) Detection of blocking artifacts at 0.2989 bpp. (e) Portion of
JPEG coded “Lena” image at 0.1942 bpp. (f) Detection of blocking artifacts at
(a) Portion of JPEG coded “Lena” image at 0.4096 bpp. (b) Detection
pose, several images (as shown in Fig. 5) of different character-
istics were chosen and compressed using a JPEG and MPEG-1
intra-picture. The same algorithm can be also applied for the
case of MPEG inter-coding with no extra modifications.
The blocking artifact detection algorithm, presented in Sec-
tion III, was applied to the test images, in order to locate the
blocks affected by artifacts in a JPEG coded image. Figs. 6–8
show the disturbing blocking artifacts (indicated with a white
pixel value) pointed out by the criterion (16) in the JPEG coded
images at three different bit rates for the images of “Lena,”
“Peppers,” and “Claire.” In Figs. 6 and 7, magnified portions
of the “Lena” and “Peppers” images are shown, so as to better
illustrate the detection of the blocking artifacts. The portions
used are identified by a white line in the “Lena” and “Peppers”
original images (see Fig. 5).
In order to measure and evaluate the performance of our ap-
proach for blocking artifact reduction, the proposed constrained
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA885
Detection of blocking artifacts at 0.4211 bpp. (c) Portion of JPEG coded
“Peppers” image at 0.3137 bpp, (d) Detection of blocking artifacts at 0.3137
bpp. (e) Portion of JPEG coded “Peppers” image at 0.1989 bpp. (f) Detection
of blocking artifacts at 0.1989 bpp.
(a) Portion of JPEG coded “Peppers” image at 0.4211 bpp. (b)
monly used metrics, such as the mean square error or signal-to-
noise ratio were not employed, since they involve pixels of the
entire image and not just the pixels near the block boundaries.
Rather, thevalue of the MSDS per blockis preferred to be used
for the evaluation of the proposed technique.
Recall that the recalculation of the DCT coefficients for
blockiness removal is given in (40) and uses the Laplacian
corrected DCT coefficients
duced after the minimization of the newly introduced formula
of MSDS for each frequency separately. Furthermore, the
blockiness reduction algorithm is applied only when criterion
(16) is valid. This implies that the algorithm is performed only
where disturbing blocking artifacts are expected to be present.
. This formula is pro-
blocking artifacts at 0.4907 bpp. (c) JPEG coded “Claire” image at 0.3779 bpp.
(d) Detection of blocking artifacts at 0.3779 bpp. (e) JPEG coded “Claire”
image at 0.2968 bpp. (f) Detection of blocking artifacts at 0.2968 bpp.
(a) JPEG coded “Claire” image at 0.4907 bpp. (b) Detection of
Table I shows the image name, its size, the MSDS of the
original image (all in the first column), the coding rate (bits per
pixel), and the MSDS per image block for the cases of: 1) the
nonsmoothed reconstructed image; 2) the reconstructed image
processed by method of ; and 3) the reconstructed image
processed by the proposed algorithm. As expected, in B-DCT
coded images, the value of MSDS per block increases com-
pared to the original images, due to quantization. Our approach
shows a significant reduction of the MSDS and clearly outper-
forms the method proposed in . A visual illustration of the
performance of our method, showing the JPEG reconstructed
magnified portions of “Lena,” “Peppers” and “Claire” images
and thecorrespondingreconstructedportions oftheimages pro-
cessed by the proposed method is shown in Fig. 9 and in more
886IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
(b) Reduction of blocking artifacts with the proposed method at 0.2989 bpp.
(c) Portion of the JPEG coded “Peppers” image at 0.3137 bpp. (d) Reduction
of blocking artifacts with the proposed method at 0.3137 bpp. (e) JPEG coded
“Claire” image at 0.3779 bpp. (f) Reduction of blocking artifacts with the
proposed method at 0.3779 bpp.
(a) Portion of the JPEG coded “Lena” image at 0.2989 bpp.
detail in Fig. 10. These figures illustrate the efficiency of the
proposed method. Moreover, to better illustrate the comparison
of the proposed method to the method in , the JPEG coded
image “Lena” is used in Fig. 11.
Table II shows the results found when comparing the
proposed method with the method of  using the metric of
MSDS (this metric is used in ). The metric of MSDS
(instead of MSDS ) does not take into account the differences
between the diagonal pixels. Thus, this metric is not suitable for
evaluating the corner outliers reduction. However, the results
indicate that the proposed algorithm continues to outperform
the method of  since it uses the local optimization for every
frequency separately, employs the corrected Laplacian DCT
coefficients instead of the simple reconstructed DCT coeffi-
cients and involves more neighboring pixels (i.e., the diagonal
pixels), producing better estimation of the recalculation of the
As stated earlier, the newly introduced form of MSDS in the
paper serves so as to remove the disturbing corner outliers from
(b) Reduction of blocking artifacts with the proposed method at 0.2989 bpp.
(c) Portion of the JPEG coded “Peppers” image at 0.3137 bpp. (d) Reduction
of blocking artifacts with the proposed method at 0.3137 bpp. (e) Portion of the
JPEG coded “Claire” image at 0.3779 bpp. (f) Reduction of blocking artifacts
with the proposed method at 0.3779 bpp.
(a) Portion of the JPEG coded “Lena” image at 0.2989 bpp.
the reconstructed images. In order to better illustrate the effec-
blocking artifacts and the corner outliers.
Tables III–V indicate the improvement achieved by the var-
ious innovations introduced in the present paper. This improve-
mentis evaluatedusing threetest imagescompressedattwodif-
First, Table III compares the MSDS of the proposed method
which uses the Laplacian corrected DCT coefficients for the
recalculation of the DCT coefficients and the MSDS of the
method which employs the simple reconstructed DCT coeffi-
cients instead. Both methods employ the local minimization of
the MSDS form and use the prior blockiness detection method.
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA887
processed by the proposed method.
(a) Detail of the JPEG coded image Lena at 0.2989 bpp. (b) Same detail as the image processed by the method of . (c) Same detail as the image
MSDS PER BLOCK FOR VARIOUS TEST IMAGES
Table IV also compares the MSDS of two methods. The first
method uses the local minimization of the MSDS form while
the second uses the local minimization of the MSDS form.
for the DCT recalculation and the prior blockiness detection
method. Results show that the use of MSDS for the DCT recal-
culation provides better results than the use of the simple form
of MSDS .
imization of MSDS . In both methods, the corrected Laplacian
corrected DCT coefficients and the prior blockiness detection
method are used. Results clearly support and justify our choice
to use the separate for each frequency minimization of MSDS
for the DCT recalculation.
888IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
detail of the image processed by the proposed method.
(a) Detail of the JPEG coded image Lena at 0.2989 bpp with corner outliers. (b) Same detail of the image processed by the method of . (c) Same
PER BLOCK FOR THREE TEST IMAGES
PROPOSED METHOD WITH OR WITHOUT USING THE CORRECTED
LAPLACIAN DCT COEFFICIENTS
PER BLOCK FOR THREE TEST IMAGES WHEN WE APPLY THE
The blocking artifact reduction algorithm is somewhat
slower than the algorithm in , since it employs the Lapla-
cian corrected DCT coefficients and involves the diagonal
adjacent pixels. However, the overall proposed algorithm was
found to be faster than that of  because of the use of the
blockiness detection algorithm which excludes the blocks
where the blocking artifacts are not disturbing (especially at
high bit rates). Table VI shows the time needed in order to
apply our algorithm compared to the algorithm of . Note
that larger values of the coding rate indicate that the blockiness
reduction algorithm will be applied in fewer blocks, since there
will not be many disturbing blocking artifacts and as a result
the proposed algorithm will be faster.
Furthermore, the proposed algorithm is applied on the
received/reconstructed DCT coefficients and, therefore, does
not need extra time for pixel postprocessing needed by methods
PER BLOCK FOR THREE TEST IMAGES WHEN WE APPLY THE
PROPOSED METHOD USING THE LOCAL MINIMIZATION OF MSDS
MSDS PER BLOCK FOR THREE TEST IMAGES WHEN WE APPLY THE THREE
PROPOSED METHOD USING THE GLOBAL MINIMIZATION OF MSDS
OR THE LOCAL MINIMIZATION OF MSDS
TIME NEEDED TO APPLY THE PROPOSED ALGORITHM COMPARED TO THE TIME
NEEDED TO APPLY THE ALGORITHM OF  AND THE MPEG-4 DEBLOCKING
FILTER. THE VALUE OF ? FOR A 512 ? 512 IMAGE IS 0.29 MS ON A
PENTIUM III BASED PC. THE PROPOSED ALGORITHM FIRST EMPLOYS THE
PROPOSED BLOCKINESS DETECTION SCHEME
TRIANTAFYLLIDIS et al.: BLOCKING ARTIFACT DETECTION AND REDUCTION IN COMPRESSED DATA889
applied in the spatial domain. Thus, it is much faster than
methods working in the spatial domain. Compared, for ex-
ample, with the well-known spatial domain deblocking method
of the MPEG-4 postfilter , our algorithm is about nine
times faster (see Table VI).
When images are compressed using B-DCT transforms, the
decompressed images often contain bothersome blocking arti-
facts. This paper presented a novel algorithm applied entirely
in the compressed domain, in order to detect and reduce these
blocking artifacts. In our approach, the Laplacian statistical
model is adopted for the DCT coefficients and a better esti-
mation of the DCT reconstructed coefficients is produced, in
order to calculate the relative theoretical quantization error.
This error is used in a newly introduced criterion, in order
to efficiently detect the blocking artifacts of coded images.
Thus, the time and the computational load of the deblocking
algorithm is reduced compared to other deblocking methods,
since it is applied only where is needed. A novel form of the
criterion of MSDS (i.e., the MSDS form) is also introduced
involving all eight neighboring blocks, instead of the simple
form of MSDS which  uses. MSDS is then minimized
for each frequency separately, producing a closed form for
the correction terms for the DCT coefficients so as to achieve
reduction of the blocking effect of coded images. This local
minimization is shown to achieve better results than the global
minimization adopted in . Experimental evaluation of the
performance of the proposed technique showed its ability to
detect and alleviate blocking artifacts effectively.
 W. B. Pennebaker and J. L. Mitchel, JPEG Still Image Data Compres-
sion Standard. New York: Van Nostrand, 1993.
 G. K. Wallace, “The JPEG still-picture compression standard,”
Commun. ACM, vol. 34, pp. 30–44, Apr. 1991.
 Video Coding for Low Bit Rate Communications, ITU-T Telecommu-
nications Standardization Sector, Recommendation H.263, ver. 2, Jan.
 K. R. Rao and J. J. Hwang, Techniques and Standards for Image, Video,
and Audio Coding, NJ: Prentice Hall, 1996.
 H. S. Malvar and D. H. Staelin, “The LOT: Transform coding without
37, pp. 553–559, 1989.
 H. S. Malvar, “Biorthogonal and nonuniform lapped transforms for
transform coding with reduced blocking and ringing artifacts,” IEEE
Trans. Signal Processing, vol. 46, pp. 1043–1053, Apr. 1998.
blocking effects from coded images,” IEEE Trans. Consumer Electron.,
pp. 521–526, Aug. 1994.
 H. C. Reeve and J. S Lim, “Reduction of blocking artifacts in image
coding,” Opt. Eng., vol. 23, pp. 34–37, Jan./Feb. 1984.
 T. Meier, K. N. Ngan, and G. Crebbin, “A region-based algorithm for
enhancement of images degraded by blocking effects,” in Proc. IEEE
Tencon’96, vol. 1, Perth, Australia, Nov. 1996, pp. 405–408.
 J. G. Apostolopoulos and N. S. Jayant, “Postprocessing for low
bit-rate video compression,” IEEE Trans. Image Processing, vol. 8, pp.
1125–1129, Aug. 1999.
 R. Molina, A. K. Katsaggelos,and J. Abad, “Bayesian imagerestoration
using a wavelet-basedsubband decomposition,” in Proc. 1999Int. Conf.
Acoust., Speech, Signal Processing, Phoenix, AZ, Mar. 15–19, 1999.
 Z. Xiong, M. T. Orchard, and Y. Q. Zhang, “A deblocking algorithm for
JPEG compressed images using overcomplete wavelet representations,”
IEEE Trans. Circuits Syst. Video Technology, vol. 7, pp. 692–695, Aug.
 N. C. Kim, I. H. Jang, D. H. Kim, and W. H. Hong, “Reduction of
blocking artifact in block-coded images using wavelet transform,”
IEEE Trans. Circuits Syst. Video Technology, vol. 8, pp. 253–257, June
 T.-C. Hsung, D. P. K. Lun, and W.-C. Siu, “A deblocking technique for
block-transform compressed image using wavelet transform modulus
maxima,” IEEE Trans. Image Processing, vol. 7, pp. 1488–1496, Oct.
 T. P Rourke and R. L Stevenson, “Improved image decompression for
reduced transform coding artifacts,” IEEE Trans. Circuits Syst. Video
Technol., vol. 5, pp. 490–499, Dec. 1995.
 J. Luo, C. W. Chen, K. J. Parker, and T. S. Huang, “Artifact reduction in
low bit rate DCT-based image compression,” IEEE Trans. Image Pro-
cessing, vol. 5, pp. 1363–1368, Sept. 1996.
 T. Meier, K. N. Ngan, and G. Crebbin, “Reduction of blocking artifacts
in image and video coding,” IEEE Trans. Circuits Syst. Video Tech-
nology, vol. 9, pp. 490–500, Apr. 1999.
 Y. Yang, N. P Galatsanos, and A. K. Katsaggelos, “Regularized recon-
struction to reduce blocking artifacts of block discrete cosine transform
compressed images,” IEEE Trans. Circuits Syst. Video Technol., vol. 3,
pp. 421–423, Dec. 1993.
block-transform compressed images,” IEEE Trans. Image Pro-
cessing, vol. 4, pp. 896–908, July 1995.
determination of local bound constraints in iterative image restoration,”
in Proc. 1998 IEEE Int. Conf. Image Processing, vol. 2, Chicago, IL,
Oct. 5–7, 1998, pp. 833–837.
 A. Zakhor, “Iterative procedures for reduction of blocking effects in
2, pp. 91–95, Mar. 1992.
 K. H. Tzou, “Post-filtering of transform-coded images,” in Proc. SPIE
 B. Ramamurthi and A. Gersho, “Nonlinear space-variant postpro-
cessing of block coded images,” IEEE Trans. Accoust., Speech, Signal
Processing, vol. ASSP-34, pp. 1258–1268, Oct. 1986.
 Y.-F. Hsu and Y.-C. Chen, “A new adaptive separable median filter
for removing blocking effects,” IEEE Trans. Consumer Electron., pp.
510–513, Aug. 1993.
 D. G. Sampson, D. V. Papadimitriou, and C. Chamzas, “Post-processing
of block-coded Images at Low Bitrates,” in Proc. 1996 IEEE Int. Conf.
Image Processing, Lausanne, Switzerland, Sept. 1996.
 Y. L. Lee, H. C. Kim, and H. W. Park, “Blocking effect reduction
of JPEG images by signal adaptive filtering,” IEEE Trans. Image
Processing, vol. 7, pp. 229–234, Feb. 1998.
 J. Chou, M. Crouse, and K. Ramchadran, “A simple algorithm for
removing blocking artifacts in block transform coded images,” IEEE
Signal Processing Lett., vol. 5, pp. 33–35, Feb. 1998.
of quantized signals,” SignalProcessing Mag.,vol. 62, pp. 15–36, 1997.
, “Optimal pyramidal and subband decompositions for hierarchical
coding of noisy and quantized images,” IEEE Trans. Image Processing,
vol. 7, pp. 155–167, Feb. 1988.
 M. G. Strintzis and D. Tzovaras, “Optimal pyramidal decomposition
for progressive multi-dimensional signal coding using optimal
quantizers,” IEEE Trans. Signal Processing, vol. 46, pp. 1054–1069,
technique to reduce the blocking artifacts in transform coded images,”
IEEE Trans. Circuits Syst. Video Technol., vol. 8, pp. 358–367, June
, “A DCT-based spatially adaptive post processing technique to re-
cuits Syst. Video Technol., vol. 10, pp. 36–41, Feb. 2000.
 S. Minami and A. Zakhor, “An optimization approach for removing
blocking effects in transform coding,” IEEE Trans. Circuits Syst. Video
Technol., vol. 5, pp. 74–82, Apr. 1995.
 G. Lakhani and N. Zhong, “Derivation of prediction equations for
blocking effect reduction,” IEEE Trans. Circuits Syst. Video Technol.,
vol. 9, pp. 415–418, Apr. 1999.
 K. R. Rao and P. Yip, Discrete Cosine Transform: Algorithms, Advan-
tages, Applications.New York: Academic, 1990.
 A. Gersho and R. M. Gray, Vector Quantization and Signal Compres-
sion.Boston, MA: Kluwer, 1992.
 R. L. de Queiroz, “Processing JPEG-compressed images and docu-
ments,” IEEE Trans. Image Processing, vol. 7, pp. 1661–1672, Dec.
890IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 10, OCTOBER 2002
 N. M. Namazi, P. Penafiel, and C. M. Fan, “Nonuniform image motion
estimation using the Kalman filtering,” IEEE Trans. Image Processing,
vol. 3, pp. 678–683, Sept. 1994.
 K. E. Matthews and N. M. Namazi, “A Bayes decision test for detecting
uncovered-background and moving pixels in image sequence,” IEEE
Trans. Image Processing, vol. 7, pp. 720–728, May 1998.
 N. M. I-Ming Pao and N. M. Ming-Ting Sun, “Modeling DCT co-
efficients for fast video encoding,” IEEE Trans. Circuits Syst. Video
Technol., vol. 9, pp. 608–616, June 1999.
 A. K. Jain, Fundamentals of Digital Image Processing.
Cliffs, NJ: Prentice-Hall, 1989.
 ISO/IEC/SC29/WG11, Information Technology—Generic Coding of
Audio-Visual Objects—Part 2: Visual ISO/IEC 14496, July 1999.
George A. Triantafyllidis (S’96) was born in Thes-
saloniki, Greece, in 1975. He received the diploma
degree in 1997 from the Electrical Engineering De-
partment, Aristotle University of Thessaloniki, Thes-
saloniki, Greece, where he is currently working to-
ward the Ph.D. degree in the Information Processing
He is a Research Associate with Aristotle Univer-
sity of Thessaloniki. He has participated in several
research projects funded by the European Union and
the Greek Secretariat of Research and Technology.
Since 2000, he has served as a Teaching Assistant at Aristotle University of
Thessaloniki. His research interests include image compression and analysis, as
well as monoscopic and stereoscopic image sequence coding and processing.
Dimitrios Tzovaras received the diploma degree in
electrical engineering and the Ph.D. degree in 2-D
and 3-D image compression from Aristotle Univer-
sity of Thessaloniki, Thessaloniki, Greece, in 1992
and 1997, respectively.
He is a now Researcher in the Informatics and
Telematics Institute of Thessaloniki. Previously,
he was a Leading Researcher on 3-D imaging at
Aristotle University of Thessaloniki. His main
research interests include image compression, 3-D
data processing, virtual reality, medical image
communication, 3-D motion estimation, and stereo and multiview image
sequence coding. His involvement with those research areas has led to the
co-authoring of more than 20 papers in refereed journals and more than 50
papers in international conferences. Since 1992, he has been involved in more
than 20 projects in Greece, funded by the European Commission, and the
Greek Ministry of Research and Technology.
Dr. Tzovaras has served as a regular reviewer for a number of international
journals and conferences. He is a member of the Technical Chamber of Greece.
SM’79) received the diploma degree in electrical
engineering from the National Technical University
of Athens, Athens, Greece, in 1967, and the M.A.
and Ph.D. degrees in electrical engineering from
Princeton University, Princeton, NJ, in 1969 and
He then joined the Electrical Engineering De-
partment, University of Pittsburgh, Pittsburgh, PA,
where he served as Assistant Professor (1970–1976)
and Associate Professor (1976–1980). Since 1980,
he has been Professor of Electrical and Computer Engineering at the University
of Thessaloniki, Thessaloniki, Greece, and, since 1999, Director of the Infor-
matics and Telematics Research Institute, Thessaloniki. His current research
interests include 2-D and 3-D image coding, image processing, biomedical
signal and image processing, and DVD and Internet data authentication and
Dr. Strintzis has served as Associate Editor for the IEEE TRANSACTIONS ON
CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY since 1999. In 1984, he was
awarded a Centennial Medal of the IEEE.