Space-Color Quantization of Multispectral Images in Hierarchy of Scales

SPACE-COLOR QUANTIZATION OF MULTISPECTRAL IMAGES IN HIERARCHY OF SCALES
Milan Jovovic
University of Montenegro, Podgorica, Yugoslavia, email: milanj@cg.ac.yu
ABSTRACT
In this paper a novel model for multiscale space-color
quantization of multispectral images, is described. The approach
is based on the hierarchical clustering technique, derived from
the statistical physics model of free energy >3, 4@. The group
vectors for image color are computed on the adaptively selected
windows of computation, as contrasted to the block-size
windows, optimizing the accuracy of the computation of the
group vectors with the density of sampling an image by the
group windows. The algorithm is suitable for the implementation
in parallel computer architectures.
The results of quantization of color images by our algorithm
are compared with 3 image compression techniques: 1) wavelets,
2) discrete cosine transform (DCT), and, 3) quad tree (QT).
Contextual information of spatial coherency of the data is used
in the segmentation process, in our algorithm. As a result, much
better spatial resolution and small size of compressed images are
obtained by our algorithm, as compared to the other techniques,
for any error level of compression selected. Major spatial
features are optimally color-coded along the hierarchy of scales
of computation. The images quantized with our algorithm are
suitable for the run-length encoding scheme of the hierarchy of
binary images.
1. INTRODUCTION
Clustering techniques are applied in many problems (like
pattern recognition, learning, source coding, image and signal
processing >1-7@) where a priori knowledge about the
distribution of the data is not available. This tool is widely used
for analyzing multidimensional data in diverse disciplines such
as engineering, biology, social science, and astronomy.
The probabilistic inference in clustering is used in this work
based on the maximum entropy principle. Various approaches to
the probabilistic and fuzzy inference in clustering are presented
in literature >2, 7@. We propose the algorithm, described in [3, 4,
5], with the structure suitable for the implementation in parallel
computer architectures, by adaptively segmenting the input data
space. The computation of the clustering parameters becomes
more effective on the adaptively selected, local windows of
computation in our clustering algorithm, as compared to the
other techniques. The maximum entropy inference in the
estimation of the clustering parameters is used in this work, as
contrasted to the method in [6].
Distinct image features become segmented on a certain scale,
in our algorithm, and the probability distribution of the data is
restructured in accordance to the new structure of the image
features, for which the lower scales of the computation are
performed. Computation of the space-feature vector parameters
can be used in coding, analysis and segmentation of
multidimensional data, such as multispectral images, which are
considered in this work, as well as the motion information from
image sequences in [5].
The spatial segmentation of images is performed while using
the Green's function, parameterized with the scale parameter, as
the integration function, in the segmentation process. On a larger
spatial scale, a larger extent of spatial integration of distortion
vectors is performed, while it becomes effectively more local in
space as we decrease the spatial scale of segmentation. As a
result, much better spatial resolution and small size of
compressed images are obtained by our algorithm, as compared
to the other image compression techniques.
The multiscale processing of multispectral images enable us
to adaptively segment the spectral content of an image among
the selected group windows. In those regions where the spectral
content varies more we need to select smaller group windows
than in relatively flat regions of the image. Optimal quantization
of multispectral images is important in image archiving, color
palette design as well as for the spectral signature recognition.
The group vectors for image color are computed on the
adaptively selected windows of computation, as contrasted to the
block-size windows. We base our algorithm on the adjustable
size windows to suffice for the estimation of the group vectors.
The goal is to minimize the variation in the robustness of the
estimate of group vectors across the selected group windows.
2. METHOD
We define a cluster here with its computed group vector
representative y, and the selected group window of computation,
W. Let d(x, y) denotes a distortion measure introduced by a data
point x to the representation y. The distortion energy, or variance
V of a cluster is defined by: � � � �.,¦ W xPyxdV
It can be shown that the probability density function that
maximizes the entropy: � � � �^ ` ,logmax ¦� W xPxP
subject to: � � � � � � ,1and, ¦¦ WW xPxPyxdV
is the Gibbs distribution: � � � � � �� � ,,
,,
¦ �
��

W
yxd
yxdyxd
e
e
Z
exP E
EE
where Z is the partition function, and E is a Lagrange multiplier.
2.1. Hierarchical Clustering of Color Vectors on Spatial
Windows of Computation
The dynamics of the process of clustering, in our algorithm,
is derived from the model of “free energy”, originally used in
statistical physics to model different complex systems. In this
section we shall describe the mathematical model of the
computation of the group vectors, as well as the points of
discontinuities, when new clusters emerge from the previously
selected clusters, optimizing the distortion energy among the
clusters. The free energy describes the state of a cluster for a
given parameter E,

� � .log1 ZF EE �
The parameter E is inversely proportional to temperature (E=1/T)
in physical analogy. At the equilibrium, the cluster settles in the
state that minimizes its free energy.
Derivation of the algorithm is analogous to that given in >4@
for the time domain signals. We shall give, here, only the key
points of the algorithm. The distortion measure, applied in the
algorithm, is chosen to be the squared distance of an image point
spectral vector to the space-color group vector, > @bgrc G :
2222 )()()( bIgIrIz bgr �����
where > @bgr IIII G defines the spectral content of an image at
the spatial point (x, y). Computation is defined by the equations
of motion:
� �
.0
,2
!w
w� c
� w
w� ¦
EE
F
PIcc
Fc W
GGG�G (1)
On the scale of the whole image, W (the largest scale), we
start with the estimation of the group vector from the initial
point, E=0. For a given parameter E, this map is stable if the
Hessian of the free energy is positive definite. We “cool” the
system of equations (increasing E) up to the point when it
becomes unstable. The point in scale Ec, when some nonconvex
component becomes dominant in the estimation process
indicates the point of instability of the map (1). This is the point
of discontinuity in the algorithm, which is followed with the
procedure of phase transition - splitting of the cluster.
The cooling procedure is defined by, .0!w
w c EE
F At the
equilibrium point 0 w
w
c
FG , if the Hessian of the free energy is
positive definite, we compute: c
Vc GG w
w� G , and update:
.02
1
2
2
!w
w ' Tc
c
Fc GG
G GGE These equations enable the
equilibrium point to escapes potential barrier in the free energy
landscape, such that the local minimum is avoided.
The map in Eq. (1) for the space-color group vector
computation results in the scatter matrix,
� � � �� � � �� �� �� � � � � �� �
� �� � � �� � � � � �22 2
2
2
2
2
»»
»»
¼
º
««
««
¬
ª
�����
�����
�����
w
w�
¦¦¦
¦¦¦
¦¦¦
W bW gbW rb
W bgW gW rg
W brW grW r
PbIPgIbIPrIbI
PbIgIPgIPrIgI
PbIrIPgIrIPrI
c
FI EGS
to be equal to the cluster covariance matrix multiplied by the
scale parameter. If the Hessian of the free energy is negative
definite for some of the clusters, at the critical value of the scale
parameter, Ec, the condition of phase transition is reached and
we split that cluster along the principal component vector
corresponding to the maximal singular value of the scatter
matrix, according to:
� � � � � � � �> @> @ � �3),(0 ),(0 21,
22 ®¯­ �
! ���¦ �� Wyx
WyxEEEbIgIrIe
ji
T
bgrbgr
jiE
where the vector >Er Eg Eb@ is the principal component vector
corresponding to the maximal singular value of the scatter
matrix, and >er eg eb@=>(Ir - r) (Ig - g) (Ib - b)@ is the point
distortion vector.
The integration is obtained by summing up the projections of
the distortion vectors multiplied by the Green's function in
equation (3). The parameter E here plays the role of the spatial
extent of integration and it is used consistently in the
discriminant function. On a lower value of the scale parameter E,
the group windows are formed by using a larger extent of the
spatial integration of the projections of distortion vectors. As we
gradually increase E, the integration becomes effectively more
local in space.
The cooling procedure makes an adaptive multiscale
algorithm for processing of multidimensional data. The
algorithm produces a tree of splitting clusters, that gives a
representation of data in the hierarchy of scales. The estimation
of the code vectors for every node of the tree is obtained with
separately defined maps, as in Eq. (1), and on the selected group
windows of computation, what makes this algorithm suitable for
the implementation in parallel computer architectures. This data
structure, also, enables image coding to be carried out by the
hierarchy of binary images.
3. RESULTS
A colorful image, “Flowers” (362x500), from the Matlab’s
library of images is chosen for the purpose of illustrating the
multispectral still image quantization by our algorithm. In Figure
1, we show the results of the hierarchical clustering of image
data for the 4, 5, 10, and 90 clusters computed (Figures 1.(a)-
(d)). The original image is given in Figure 1.(e). The covariation
of the spectral data is used in the algorithm for the selection of
the space-color clusters distribution, in contrast to processing of
different colors of the image data independently, what has been
used in the other techniques (see below).
In Figures 1.(a) and (b) are shown the results of the
hierarchical clustering of image data at the point in scale when
the brownish cluster in Figure 1.(a, a') reaches the critical value
of the computation, and passes through the process of phase
transition, and the formation of the two new group windows of
computation out of the joint one. When the two new clusters
settle at the equilibrium points of the maps, as in Eq. (1), the
resulting color vectors of the clusters give them more reddish
color, to the first, and more yellowish color, to the second
cluster, what can be seen in Figure 1.(b, b'). The other clusters
haven’t noticeably changed the appearance of colors with the
change of the scale of computation.
The distortion energy of the whole image is used to describe
the error of computation. In Figure 1.(f) is given the error of
computation for the space-color clustering vs. the scale
parameter E.
The smoothing effects are shown in Figure 2. The space-
color quantization is shown in Figures 2.(a)-(b) for 20 clusters
computed with the pixel based segmentation, corresponding to
the discriminant function with the scale parameter E = f in the
Green's function. In Figures 2.(c)-(d) the smoothing effects are
present where the segmentation is performed matching the scale
parameter of computation E, in the Green's function. The
distortion energy of the image in Figure 2.(a) is V = 648.41 with
the scale parameter E = 7.6x10-8, corresponding to the final point
on the graphic in Figure 2.(b). The distortion energy of the
image in Figure 2.(c) is V = 967.59, and is obtained with the
computation up to the scale parameter E =5.2x10-8 (Figure 2.(d)).

(a) (a') (b) (b')
(c) (d) (e) (f)
Figure 1: Multispectral image quantization. The space-color clustering is shown for: (a) 4 clusters, (b) 5 clusters, (c) 10 clusters, and (d)
90 clusters. (e) The original image. (f) The error of space-color clustering vs. the scale parameter E of computation.
(a) (b) (c) (d)
Figure 2: Smoothing effects. The space-color clustering is shown for 20 clusters computed without smoothing (a-b), and with
smoothing (c-d).
For the purpose of comparing different image compression
techniques, the summed square distance function is used to
indicate the error level of compressed images. The error level
of the image with 10 space-color clusters computed (Figure
1.(c)) is E1=230x106, and this value is matched with the other
techniques. This image is shown again in Figure 3.(d).
The Matlab’s image processing tool is used for analyzing
the data. The wavelets’ functions wavedec2 and wdencmp are
used for wavelets decomposition and compression of the data.
The wavelets’ coefficients are globally, hard thresholded with
the level thr=227 for the R, G, and B, components of the
image, separately, to match the error level of compression, E1.
As a result, the number of nonzero wavelets’ coefficients
obtained is 3358, 3310, and 3272, for the R, G, and B
components of the image, respectively. The resulting
compressed image is shown in Figure 3.(a).
The DCT’s coefficients of the image with the amplitude
lower than a threshold value, thr=146.5, are set to zero for the
R, G, and B components of the image, separately, to match the
error level of compression, E1. The number of nonzero
coefficients obtained is 2012, 1875, and 1548, for the R, G,
and B components of the image, respectively. The resulting
compressed image is shown in Figure 3.(b).
The QT technique requires a dimension of an image to be
of the power of 2. From the image with 10 space-color clusters
computed, the middle portion of the image (256x256) is used,
and the error level is found to be E2=90x106. This error level is
matched with the QT decomposition of the original image. The
intensity image is decomposed with the Matlab’s function
qtdecomp with the threshold value thr=109 to match the error
level, E2. The number of quadratic windows obtained is 4132
for which the average values of the R, G, and B components
are computed. The resulting image is shown in Figure 3.(c).
4. DISCUSSION
In the method of hierarchical clustering of image data,
developed in this work, convex optimization of the free energy
is achieved by the adaptation in the cooling schedule and with
the adaptive selection of spatial windows of computation. The
error of the space-color quantization becomes relatively higher
in the regions where the entries of the scatter matrix show
relatively larger covariation of data than in flat regions of an
image. As a result, selected group windows are relatively
bigger in flat regions than in the regions of the stronger
variation of the data.
The spatial resolution of the group windows increases with
the depth of the tree structure, representing the hierarchy of
clusters. On every resolution level of clustering the background
color is represented with a relatively larger spatial window,
what can be seen in Figures 1.(a)-(d), as a result of the
relatively lower variation of the spectral data in the
background.
The colors of the newly formed windows can significantly
differ from that of the joint window, since the color vectors are
computed in the domains of the newly created windows of
0 1 2 3 4 5 6 7
x 10-8
0
2000
4000
6000
8000
10000
12000
Th
e
Err
or
of
Co
m
pu
tati
on
The Scale
0 1 2 3 4 5 6 7 8
x 10 -8
0
2000
4000
6000
8000
10000
12000
The
Erro
r of
Co
mpu
tatio
n
The Scale Parameter
0 1 2 3 4 5 6 7
x 10 -8
0
2000
4000
6000
8000
10000
12000
The
Erro
r of
Com
puta
tion
The Scale Parameter

(a) (b) (c) (d)
Figure 3: Comparison of the techniques for color image compression. In all of the techniques the error level of compression is matched
with the space-color clustered image. The number of compression parameters is obtained for: (a) wavelets: p_r=3358, p_g=3310,
p_b=3272, (b) DCT: p_r=2012, p_g=1875, p_b=1548, and (c) QT: p=4132. (d) P=10 space-color clusters quantization.
computation. The original group window is divided along the
principal component vector corresponding to the maximal
singular value of the scatter matrix. In the discriminant
function (3) the zero point on the direction of the maximal
principal component, [Er Eg Eb], corresponds to the group
vector of the joint cluster, [r g b]. The projections of the color
vectors, [Ir Ig Ib], that belong to the newly created windows are
located on the opposite sides of the color vector of the joint
cluster, along the principal component vector. The Green's
function ensures appropriate spatial integration of the
distortion vectors at the given point in scale, E. This explains
the formation of the reddish and the yellowish windows in
Figure 1.(b, b'), out of the brownish window in Figure 1.(a, a').
The distortion energy of the whole image is used to show
the error of computation in the applications of the space-color
quantization (Figure 1.(f)). The regions of the linear decrease
of the error of computation correspond to the process of
cooling of the free energy of the system of equations (1). The
points of the abrupt decrease of the error correspond to the
points of phase transitions. When two smaller group windows
are created out of the joint window, the corresponding group
vector maps become more effective on the new windows of
computation, what is reflected in the abrupt decrease of the
error of computation.
Major spatial features are optimally color-coded with our
algorithm along the hierarchy of scales of computation. The
image with 90 space-color clusters computed, in Figure 1.(d),
appears almost visibly indistinguishable from the original
image, although it contains about 1,000 times less colors than
in the original image.
The smoothing function in discriminating the clusters
makes a better connected cluster-windows in the image. This
process also helps smooth out the noisy portion in the image,
what can be seen by observing the lover left corner of the
images in Figures 2.(a) and 2.(c). Our approach makes the
clustered image data suitable for the run-length encoding
scheme of the hierarchy of binary images. Computational time
and the overall size of the binary images are generally lower as
a result of using the smoothing function in the segmentation
process, while the distortion energy is larger as compared to
the image with the same number of clusters selected without
the smoothing function. The color resolution increases linearly
in our algorithm of hierarchical space-color clustering.
For a larger level of the error of compression selected,
much better spatial resolution of the space-color features in the
image are obtained by our algorithm, what can be seen in
Figure 3. The visual appearance of the images compressed on a
lower resolution by our algorithm can be improved by adding
an error metric based on the data from human psychophysics
[6], to our error function.
5. CONCLUDING REMARKS
In this paper, we have described a novel model for the
multiscale quantization of multidimensional images by
hierarchical clustering. The application of the algorithm has
been described with respect to the space-color quantization of
multispectral images. The group vectors are computed on the
adaptively selected spatial windows of computation. We have
used the group windows to trade off between the density of the
measurement windows and the accuracy of the estimate of the
group vectors.
For every value of the distortion energy, or the scale
parameter, the minimal number of the space-color clusters is
obtained, optimizing the accuracy of the computation of the
group vectors with the density of sampling an image by the
group windows. The minimax optimization is achieved along
the hierarchy of the scales of computation, progressively
always decreasing the uncertainty of the estimation process.
Spatial segmentation is performed while using the Green's
function, parameterized with the scale parameter, as a
smoothing function in the segmentation process. As a
consequence, much better spatial resolution and small size of
compressed images are obtained by our algorithm, as compared
to the other image compression techniques.
6. REFERENCES
[1] A.Gersho, and R.M.Gray, Vector Quantization and Signal
Compression, Norwell, MA:Kulwer, 1992.
[2] A.K.Jain, and R.C.Dubes, Algorithms for Clustering Data,
Englewood Cliffs, New Jersey, 1988.
[3] M.Jovovic, “Image segmentation for feature selection from
motion and photometric information by clustering,” SPIE
Symp. on Visual Information Processing V, Orlando, 1996.
[4] M.Jovovic, S.Jonic, and D.Popovic, “Automatic synthesis of
synergies for control of reaching - hierarchical clustering,” Med.
Eng. and Physics, 21/5:325-337, 1999.
[5] M.Jovovic, “Space-Feature Vector Quantization of
Multidimensional Images in Hierarchy of Scales,” IEEE Trans.
Image Processing, in preparation, 2001.
[6] M.T.Orchard, and C.A.Bouman, “Color Quantization of
Images,” IEEE Trans. Signal Processing, 39:2677-2690, 1991.
[7] K.Rose, E.Gurewitz, and G.C.Fox, “A Deterministic Annealing
Approach to Clustering,” Pattern Recog. Lett. 11:589-594, 1990.