Hierarchical Scale Decomposition of Images: Singular Feature Analysis

Hierarchical Scale Decomposition of Images:
Singular Feature Analysis
– Report for Thalweg ARC. –
M. Jovovic
AIR project
INRIA
78153 Le Chesnay Cedex, France
Abstract
In this paper we propose a method for image analysis, pro-
cessing and coding, based on physical computation of sig-
nal distortion. A binary tree data structure of coupled sys-
tem of data sets was initially proposed in [9, 10], derived
from the statistical physics model of free energy. We assess
the scale invariance, in the method, by hierarchically clus-
tering data. Theoretical model of error propagation is given
in such a computational scheme. This decomposition of im-
age information is analyzed by multifractal model formal-
ism. We study how it correlates with the convective struc-
ture in clouds, that is associated with rain. The results are
shown for MeteoSat IR images, provided by Thalweg ARC.
project.
The regularity constraints of data are used in the hier-
archical scale decomposition of images. Accordingly, the
reconstruction formula is derived based on the Laplacian
system of diffusion of the residual information from the most
singular sets. This gives us an effective way of compressing
and progressive coding of information in image sequences.
The proposed algorithm, also, is suitable for the implemen-
tation in parallel computer architectures.
1. Introduction
Information theoretical approach to statistical mechanics
was introduced in the work of Jaynes [8], and the princi-
ple of maximum entropy was proposed as an inference pro-
cedure. Clustering techniques are applied in many prob-
lems (like pattern recognition, learning, source coding, im-
age and signal processing [7, 9, 10, 11, 12, 14]) where a
priori knowledge about the distribution of the data is not
available. Clustering of data is widely used tool for ana-
lyzing multidimensional data in diverse disciplines such as
engineering, biology, social science, and astronomy.
Various approaches to the probabilistic and fuzzy infer-
ence in clustering are presented in literature [7, 14]. The
computation of the clustering parameters becomes more ef-
fective on the adaptively selected, local windows of compu-
tation in our clustering algorithm, as compared to the other
techniques. We treat the propagation of signal distortion in
such an approach globally, in space and in scale, that re-
sults in clustering spatial and temporal features in image
data as distinct singular sets. The hierarchical decompo-
sition of image data enables progressive coding of image
sequences, as well as, the implementation of the algorithm
in parallel computer architectures.
A multifractal model formalism is derived during my
posdoc stage at “Thalweg ARC.” project, to explain the de-
composition of image sequences into the singular data sets.
The partition function describes the probabilistic model of
data clusters and is analyzed as a multifractal measure in
the method. Singularity analysis of computational maps of
clustering vectors is derived to describe the computational
means of decomposing the image information into different
singular sets. We show also that the propagation of infor-
mation in image sequences is governed by the scale-space
wave equation, therefore enabling us to treat singular fre-
quencies of data clusters in an unified way, both in space
and in time.
Contextual information of the spatial coherency of data
is used in the segmentation process in the hierarchical scale
computation of feature vectors. The spatial segmentation
of images is performed while using the Green’s function,
parameterized with the scale parameter, as the integration
function in the segmentation process. The scale information
is evaluated by conjoining the two parameters: the scale pa-
rameter � of the signal distortion, and the spatial scale pa-
rameter � . A larger extent of spatial integration of the mo-
tion information is used on a larger scale, while it becomes
effectively more local in space as we decrease the scale of
segmentation.
Distinct singular features are segmented on a certain
scale and the least singular feature become segmented in
two spatial windows with the Laplacian system regularity
constraints, in the hierarchical scale computation. Accord-
ingly, the reconstruction formula is derived based on the
1

Laplacian system of the diffusion of the residual informa-
tion from the most singular sets. This gives us an effective
way of compressing and progressive coding of the informa-
tion in image sequences. The binary tree data structure of
the clustering parameters is suitable in the coding schemes
that use the hierarchical structure of the binary images of
the spatial distribution of cluster windows, along with the
feature vectors and residual image information that make
up for the point feature vector estimation.
Image motion has been an important problem in com-
puter vision and image analysis since the results can be ap-
plied in the analysis of scene segmentation, coding, shape
recovery, target tracking, or as a module in parallel algo-
rithms for the recovery of the information about scene, inte-
grating several visual cues [6, 9, 13, 15]. Accurate compu-
tation of displacement parameters from one image frame to
the next is, however, a very difficult task. We use the cluster
windows, in the algorithm, to address the question of the
appropriate integration of the information.
The multiscale approach in the estimation of the motion
vectors in image sequences enables us, also, to solve the
so-called aperture problem. Sufficient texture content is re-
quired in order to robustly compute motion vector for a win-
dow of pixels. Two questions have to be answered before
we select these cluster windows: what is the criteria of hav-
ing sufficient texture for a window of pixels, and what size
of the window we choose. To answer these questions we
start off by intuitive reasoning of what constitute good im-
age invariants to be traced from one frame to the next with a
reasonable accuracy. Due to the aperture problem we know
that we can not trace the window of pixels with the uniform
intensity, while if we have a straight edge, we can only de-
termine the motion component orthogonal to that edge. On
the other hand if a window contains strong surface mark-
ings, like a corner for example, we can uniquely find the dis-
placement parameters for that window and therefore solve
the aperture problem. However, we still have to address
the question of the first-order deformation of intensity due
to the noise in the system, resulting in shortening/dilation
in the image points sampling introduced by the rotation of
camera. An appropriate window size is required to suffice
for the estimation of introduced parameters.
Tomasi and Kanade [15], in their algorithm, used a fixed
size windows to estimate motion vectors. We base our algo-
rithm on adjustable size windows to suffice for the estima-
tion of the cluster vectors. In our approach a limit constraint
on the robustness of the estimate of cluster vectors is used
across the selected cluster windows, that cover the whole
image space.
The method of singular features decomposition of image
sequences by hierarchical clustering is derived in Section 2.
The results of the application of the algorithm are shown
in Section 3. The scale invariance of the feature vectors
mapping is evaluated for the still images in space, and for
a longer image sequences, both in space and in time. The
results are discussed in Section 4. Concluding remarks are
given in Section 5.
2. Method
2.1. Maximum entropy inference
The principle of maximum entropy inference states the fol-
lowing: among all the probability density functions (pdf),
that satisfy a given set of constraints, choose that that max-
imizes the entropy. The chosen pdf is agreeable with all the
knowledge available (a priori knowledge, or that obtained
by the estimation), and at the same time keeps the maximal
uncertainty towards anything else (a posteriori knowledge,
or the future results of the estimation). This, also, means
that such a chosen pdf is maximally unbiased toward any
future solution that includes the future knowledge obtained
about the problem. Any other pdf is biased toward some of
the possible solutions.
We define a cluster here with its computed cluster vector
representative � , and the selected cluster window of com-
putation, � . Let �����
��
� denotes a distortion measure in-
troduced to a data point � by the representation � . The
distortion energy, or variance � of a cluster is defined by:
���
���
�����
��
���������
It can be shown [8] that the probability density function that
maximizes the entropy: ���fiffffifl �"!
�
�������$#&%(')�������+* sub-
ject to: �,�-!
�
�.���
/�
�0������� , and !
�
�������1�,2 , is the
Gibbs distribution:
�������3�54(6�7 8:9<;$= > ?@ � 4(6.7 8:9<;$= >$?
!
�
4
6�7 8:9<;$= > ?
where @ is the partition function, and � is a Lagrange mul-
tiplier.
2.2. Clustering motion information in image
sequences
If we limit our attention to an image sequence of small inter-
frame displacements we derive first the necessary condition
for computing the common motion vector for a window of
pixels. Then, we shall describe the derivation of the algo-
rithm for space-motion quantization of image sequences.
Let A��CBD �E�� denote the image brightness of some scene
point BD �F� D /G.� at the time E . Also, let assume that the
scene point � D /G.� projects onto a new point � DIH � D /G H
�JG.� at the time ��E H �JE�� . The brightness change model for
computing the small image point displacements, based on
the gradient method [5], can be written as:
AK�LB
D
/E
HNM
E��
�OAK�LB
D
/E��
�QPKRS
2

and for a feature point corresponding to a window of pixels
� , we can write,
T
�
���
��A��LB
D
�E��
�UPKRV�UA��CB
D
�E
HNM
E��/�0W:XY�
B
D

or in a more compact form,
T
�
���
�[Z1�UPKR
��W\X1�]B
D
(1)
where Z5�^A��CBD �E��_�`AK�LBD /E H �(E��+ aPb�dc0eLf
eLg
ehf
eLi$j
�Rk�
c �
D
�JG
jml
, and X is a weighting factor. For a unique time
frame period of image sampling we can conveniently sub-
stitute displacement vector with an image motion vector
R"�nB
o
��c p
o
jml
where p and o correspond to the D and G
components of the motion vector, respectively.
The residue T in the equation 1 can be locally minimized
by setting the result of differentiation equal to zero:
q
�
�
�
�rZY�QP)B
o
��PsX1�
B
D�t
By assuming Bo to be constant within the selected window
� , we have,
�
�
�
P�P
l
XY� uv�
B
o
�
�
�
Z�PwXY� u
t
Or, equivalently, it can be written as a xSy_x matrix equation:
z
B
o
�`{� (2)
where z � !
�
PKP
l
XY�(u| and {�� !
�
Z�P�X�� u
t
As we can see, in order to solve the equation 2, the matrix
z must be nonsingular. Also, we want to select a robust
feature in scale, particularly if it is to be tracked through
the sequence of images. We propose an algorithm for the
selection of space-motion information features across the
hierarchy of scales, as described below.
2.3. Hierarchical scale decomposition
The nonlinear dynamics of clustering, in this work is de-
rived from the model of “free energy”, originally used in
statistical physics to model different complex systems. In
this section we shall describe the mathematical model of
clustering the motion information, as well as the points of
discontinuities, when new clusters emerge from the existing
clusters, described by the cooling and melting procedures.
The free energy describes the state of a cluster for a given
parameter � , }
�
�
�3�~�
2
�
#&%('(
@
t
The parameter � is inversely proportional to temperature
( � �F2fi€C ), in physical analogy. At the equilibrium, the
cluster settles in the state that minimizes its free energy.
The distortion measure, applied in the algorithm, is cho-
sen to be the constraint equation on the motion vector Bo , also
known as the extended optical flow constraint equation:
�‚�„ƒJWw����A:…
HN†
A‡B
oˆH
A �(‰
o
�ŁB
o
�/��W(
which provides the mass conservation principle [1]. In this
work the coherency of data is estimated with its Green’s
function, to control the smoothness of the optical flow in
the computational scheme, adaptively in scale.
The largest scale computation is performed on the whole
window � , starting from initial point �ŁBo � �‹�-� qffiŒ a Œ � ,
and the initial equation of motion:
Ž
B
o
��� eC
e�
‘
�’!
�
x(ƒ ehf
e.g
�“
�‡”
�
H
eC
e
7O•

t
(3)
The nonlinear map in 3 exhibits no chaotic behavior [17].
This gives a fixed-point iteration:
B
o
�nB
oˆH
!
�
ƒ eLf
e.g
�
!
�
�
ehf
e.g
�
W
�
�`P��ŁB
o

�
�
t (4)
For a given parameter � , this map is stable if the Hessian
of the free energy, eC–�
e�
‘
–
is positive definite. The system of
equation 3 can be adaptively “cooled” (increasing � ) up to
the point when it becomes unstable.
The point in scale �˜— , when some nonconvex component
becomes dominant in the estimation process indicates the
point of instability of the map. This is the point of disconti-
nuity in the algorithm, which is followed with the procedure
of phase transition - splitting of the cluster.
The cooling procedure is defined by the equation:
�
”
�
HI™
}
™
�
•

t
At the equilibrium point,
™
}
™
B
o
�`�
if the Hessian of the free energy is positive definite, we com-
pute:
M
B
o
���
™
�
™
B
o
(5)
and update, š
�
�œ›
2
x
M
B
o
™
W
}
™
B
o
W
M
B
o
l
•

t (6)
Note that this way we keep the integral:
�.
�Ÿž’�
��
™
}
™
�
�‡B
oˆHœ™
�
™
B
o
�
�
�`� (7)
3

 ¡β¢β >
β £
¤
¥
¦ §
¨
¥ ©rª¬«�­
=
 
§¯®
£
§ °
£
Figure 1: Adaptation in the cooling schedule
and,
š
}1±

‘:²
—�³�´Jµ
…r¶
�
š
}1±
7
²
—�³/´(µ
…r¶
if we neglect the higher order terms. The same potential
levels difference the equilibrium point moves away by the
change of the parameter � (6), as with the change of the di-
rection of computation (5), what is expressed by the equa-
tion of continuation (7). These equations enables the equi-
librium point to escapes potential barrier in the free energy
landscape, such that the local minimum is avoided, as it is
shown on figure 1.
If the Hessian of the free energy is negative definite for
some of the clusters, at the critical value of the scale pa-
rameter, �ffi— , the condition of phase transition is reached and
that cluster is splitted along the principal component vector
corresponding to the maximal singular value of the scatter
matrix: ·
�
™
W
}
™
B
o
W

according to:
¸º¹
�LB
D
�»� !
i
¼
�LB
D
�~BG��:c
4
g
4
i
j
c ½
g
½
i
j¾l
� ¿
•
 B
DÁÀ
�
¹

Â
 B
DÁÀ
�
W
t
(8)
¸ˆ¹
�LB
D
� is the point residual motion information of the pro-
jections of the distortion vectors, along the principal compo-
nent vector, c ½
g
½
i
j
, corresponding to the maximal sin-
gular value of the scatter matrix. From equation 4 we write
the point distortion vector as:
c
4
g
4
i
j
�„ƒKc
A
g
��A
W
g
�/Ã
‘+Ä
¶
A
i
��A
W
i
�0Ã
‘+Ä
¶
j
t
The integration is obtained by summing up the projections
of the distortion vectors multiplied by the Green’s function
¼
�LB
D
�œBG�� in the equation 8. The Gaussian function is used
in this work
¼
�LB
D
�ÅBG.�Æ�
4(6�7�Ç
g
6
i
Ç
–
. The parameter �
here plays the role of the spatial extent of integration. On
a lower value of the scale parameter � , the cluster windows
are formed by using a larger extent of the spatial integration
of the projections of distortion vectors. As we gradually in-
crease � , the integration becomes effectively more local in
space.
Let’s analyze now the entries of the scatter matrix of the
map, as written in equation 4. After few lines of derivation
we have,
e
Ä�È
eLÉ
�`x
�
!fiÊÌË
–/f\–
ÍŁÎ
!LÊ
f
–
Í Î

e
Ä�È
e
‘
�`x
�
!CÊVË
–0f
Í
frÏ
Î
!CÊ
f
Í
f Ï
Î

e
Ä�Ð
e
‘
�`x
�
!CÊVË
–0f\–
Ï
Î
!LÊ
f
–Ï
Î
t
And by the symmetry of the partial derivatives,
™.Ñ
‘
™
p
�
™�Ñ
É
™
o
We can further simplify the notation by deriving the nor-
malized sensitivity coefficients. We have,
Ò
Ë
É
�Óe
Ë
eLÉ
�OA
g

Ò
Ë
‘
�be
Ë
e
‘
�ÔA
i

and we write,
Ò
Ë
É
=
É
�
!
Ê
Ë
–
9
 Õ
È
?
–
Î
!LÊ
9

Õ
È
?
–
Î

Ò
Ë
É
=
‘
�
!
Ê
Ë
–
 Õ
È
 Õ
Ð
Î
!CÊ

Õ
È

Õ
Ð
Î

Ò
Ë
‘
=
‘
�
!
Ê
Ë
–
9
(Õ
Ð
?
–
Î
!CÊ
9

Õ
Ð
?
–
Î
And by the symmetry,
Ò
Ë
‘
=
É
�
Ò
Ë
É
=
‘
t
We can now write the scatter matrix in the form of the nor-
malized sensitivity coefficients as:
Ö
�×x
�×Ø
Ò
Ë
É
=
É
Ò
Ë
É
=
‘
Ò
Ë
‘
=
É
Ò
Ë
‘
=
‘ Ù
t
2.4. The multifractal model formalism
Singularity of data clusters is evaluated by the means of the
scalability of the maps of feature vector representatives in
scale-space. For a given maximal value of the distortion
energy the minimal number of singularity manifolds is ob-
tained in the hierarchical scale computation.
For a given cluster the partition function @ � � ŁÚJ� de-
scribes the distribution of the data points with respect to the
cluster vector representative. The scale information is eval-
uated by conjoining the two parameters: the signal energy
distortion scale parameter � , and the spatial scale parameter
� , which equals to the number of data points inside the spa-
tial window of computation � . For a given point distortion
measure of the signal, ƒ
W
������
��
� , the partition function
is written by:
@
�
�
��J�3�
�
�
�
6�7
Ë
–
4

and a data point belongs to the cluster in probability, with
the probability density function:
�~�
�
6�7
Ë
–
@
t
The partition function can be conveniently written as:
@
�Ô�
6.7

��
¹
6�7$Ü
t
This function is a multifractal measure, giving a way of de-
composing the signal into feature vector clusters ordered by
the singularity exponents, ÝÞ�Fx � � Â 2 , and singular
frequency factors,
}
€J�
Â
2 , as written in:
@
�O�
6wß
–Là
9há
â
?
t
The nonlinear dynamics of clustering is governed by the
two energy functions. For a given cluster its free energy,
and the distortion energy is defined by:
}
�ŁB
o

�
�
���
2
�
#&%('$ã
@
���
�.�
�����
��
���
t
We relate the mechanism of the multifractal decomposition
of the signal to the stability analysis of the map, as written in
the equation 3. The stability condition of this map is given
by the relation: x � � Â 2 . We limit the singularity exponent
of the clusters with the maximal value Ý , by splitting that
cluster in two for which the condition is reached: 2 �˜— �ä�
Ý , at the critical value of the scale parameter �˜— .
The cooling and melting procedures describe an adaptive
multiscale method for processing of multidimensional data.
A binary tree of splitting clusters gives a representation of
data in the hierarchy of scales. On figure 2 is shown a tree
structure with 5 clusters, corresponding to the leaves of the
tree. The cluster with the group window equal to the whole
image frame, �"å , corresponds to the root of the tree. At the
critical value of the scale parameter � å— , the computed clus-
ter vector representative is � å . The root cluster is splitted
in two at the critical value of the scale parameter � å— . The
structure of the tree is formed at the increasing values of the
scale parameter � å— Â �
¹
—
Â
�
W
—
Â
��æ
— , splitting one of the
clusters, that reaches the critical value of computation, in
two. The depth of the tree and the number of the clusters,
corresponding to the leaves of the tree, are determined by
the error value of the coding scheme.
The estimation of the code vectors for every node of the
tree is obtained with separately defined maps and on the
selected group windows of computation, what makes this
algorithm suitable for the implementation in parallel com-
puter architectures. This data structure enables, also, the
coding of the spatial distribution of cluster windows to be
carried out by the hierarchy of binary images.
ç è é ê$ë é ìç èîí�ê ëïí�ì
ç èîð�ê ëïðñìç è ò ê$ë ò ìç èJóñê$ëwómì
β ô
í
ç è
í
ê$ë
í
ì
β ô
ð
ç è
ð
ê ë
ð
ì
β ô
ó
ç è
ó
ê ë
ó
ì
β ô
ò
ç è
ò
ê$ë
ò
ì
Figure 2: The binary tree structure of the distribution of
clusters.
2.5. Temporal aspects
For a longer sequence of images we propose an alternative
scheme, by considering the splitted clusters with the con-
strained equations of motion:
Ž
B
o
¹
���
eC
ß
eŸ
‘
ßöõ
�
eC
–
eŸ
‘
–
Ž
B
o
W
��� eC
–
eŸ
‘
–
õ
�
eC
ß
eŸ
‘
ß
�‡”
�
õ
eC
e
7where the plus sign corresponds to the cooling part and mi-
nus to the melting part of the algorithm. This system of
equations can be analysed by the series expansion of the
system’ free energies:š
}
�Å÷
š
¹
š
W"ø‚ù
}
¹3ú
�
}
W
}
W
ú
�
}
¹üû
�þý
eC
ß
e�
‘
ß
M
B
o
¹
H
eC
–
eŸ
‘
–
M
B
o
Whß
H
ý
eC
ß
e
7
ß
M
�
¹
H
eC
–
e
7
–
M
�
Whß
H
¹
W
ý
eC–a
ß
e
7
–
M
�
W
¹
H
eC–a
–
e
7
–
M
�
W
W
ß
H
¹
W
�
M
B
o
¹
M
B
o
W
�
Ø
eC–/
ß
e“
‘
ß
–
ú
�
eC–/
–
e“
‘
–
–
ú
�
eC–/
ß
e“
‘
ß
–
eC–/
–
e“
‘
–
–
Ù
�
M
B
o
¹
M
B
o
W
�
l
H��
�[x �
t
This gives an update formula for the parameter � :š
�
�
H
� �

‘
��
–
á
ß
���
Ð
ß
–
�

‘
ß �
6�7
�

‘
ß
�
–
á
–
���
Ð
–
–
�

‘
–
�
¹
6
ß
–
�
–
á
ß
���
–
�
H
� �

‘
–
�
–
á
–
�
�
Ð
–
–
�

‘
–
�
6.7
�

‘
ß
�
–
á
ß
�
�
Ð
ß
–
�

‘
–
�
¹
6
ß
–
�
–
á
–
� �
–

for the cooling part of the algorithm, andš
�
���
� �

‘
��
–
á
ß
�
�
Ð
ß
–
�

‘
ß
�
Œ
7
�

‘
��
–
á
–
���
Ð
–
–
�

‘
–
�
6
¹
Œ
ß
–
�
–
á
ß
� �
–
�~�
� �

‘
–
�
–
á
–
���
Ð
–
–
�

‘
–
�
Œ
7
�

‘
��
–
á
ß
�
�
Ð
ß
–
�

‘
–
�
6
¹
Œ
ß
–
�
–
á
–
� �
–

5

(a) (b) (c) (d)
(e) (f) (g) (h)
Figure 3: Test image sequence. The test image pattern in (a) is expanded radially and deformed toward lower right corner.
The motion information singular decomposition and corresponding residual information along the major and complement
principal components are shown for the 2(b-d), 4(e-g), and 8(h) clusters computed
for the melting part.
The above system of equations results in the determinant
of the map:
�
���‚W
H
�
™
W
}
¹
™
B
o
W
¹
H�™
W
}
W
™
B
o
W
W
���
H
�/2ï�
�
Wfi�
™
W
}
¹
™
B
o
W
¹
™
W
}
W
™
B
o
W
W
t
For �� �  2 the eigenvalues of the determinant of the
map have negative values if the Hessians of the free ener-
gies,
}
¹ and
}
W
, are positive definite, giving the conditions
of the numerical stability of the coupled system of equa-
tions.
The use of the constrained equations of motion 2.5 can
be seen as to enhance the cluster independence in the cool-
ing part, while it, also, enhances melting of the similar mo-
tion components of the splitted clusters, in the melting part
of the algorithm.
Interesting points of observation become the “signal en-
ergy levels”:
}
� �������ff��fihE
t
��u
t
2fi�
and its propagation in time. The time derivative of this equa-
tion becomes:fl�ffi

Ü
� 2ff
e
e
7
!
�
Ñ
�
H
e
e
7
ffi

Ü
�"
Ž
�
�~� ÷
ffi

Ü
� 2# Qe
e�
‘
!
�
Ñ
�
H
e
eŸ
‘
ffi

Ü

ø
Ž
B
o
��u
t
x(�
The Green’s formula:
�

™
}
™
�
�‡B
oˆHœ™
�
™
B
o
�
�
�׍.
as well as, the accompanying wave equation:
™
W
}
™
�
W
�
†
W
�3
are used in the computational scheme for the estimation of
the clustering parameters, and are still part of my ongoing
research.
At the “signal scale equilibrium” eC
e
7
�- , an isolated
cluster can be modeled by the equations:
™
�
™
�
�
�
†
WL�S
and,
�

™
�
™
B
o
�
�
�
�

±<±
†
�
±<±
W
�
†
W
�
�‡B
o
�O.
The Green’s function gives a model of spatial coherency of
information for a data cluster:
¼
�
�
JB
o
�
�
±<±
†
�
±<±
W
�
†
W
�

and is applied locally in the segmentation process.
6

3. Results
Gray scale sequences of images of small inter frame dis-
placements are used to show the singularity decomposi-
tion of image sequences, for the cooling part of the algo-
rithm, only. We shall explain how the selection of spatial
windows along with the motion information in the space-
motion quantization algorithm depend on the spatial content
of image gradients ��A
g
/A
i
� , in an image, and on the robust-
ness of the estimate of the quantum image motion signal
��p

o
� .
Test image sequence is designed with the test image pat-
tern A ¹ � D /G.�Y�%$ &(' �*) D �,+-$ &.'˜�*)˜G.�+ h��/� D /G0�k2C� . This
gives a 2D half periode originating in the upper left cor-
ner and having the maximum brightness in the middle of
the image, as shown on figure 3 (a). This test pattern is
expanded radially, and deformed toward lower right corner
with radially graded motion originating in the upper left cor-
ner. The disparity vectors R ¹ �Ó½ c x D � 2ÞxîG �’2
j
and
R
W
�21üc
D
G
j
, expand and deform the test pattern in the
first image into the image pattern A
W
�
D
�G.�"�3$ &('˜�4)
�
D
�
�
g
¹
�U�
g
W
�/�5+6$ &('˜�4)
��G‹�U�
i
¹
�Q�
i
W
��� .
The computation of image motion vectors starts with the
value of the scale parameter � �’ , computing the average
displacement parameter ��p å o å � of the whole image frame
( � å - the root window, in figure 2). The value of the scale
parameter is monotonically increased up to the point when
the process reaches the critical value of the scale parameter
�
å
— , and the point of the formation of new group windows of
computation, by using the discriminant function, given in
equation 8. We continue next with the convex minimization
of the free energy and the estimation of the group vectors
�
¹
and �
W
on separately defined maps, as in equation 3, on
the selected group windows of computation, �
¹
and �
W
.
The hierarchical clustering algorithm is applied here for
the formation of the tree structure of the distribution of clus-
ters. The gray level values used for labeling of the group
windows, indicate the singularity level of the corresponding
cluster of data points. The gray scale code is used uniformly
to label more singular clusters with the brighter value. The
residual motion information along the major and comple-
ment principal components, ¸ö¹ �LBD � and ¸
W
�LB
D
� , are shown
with the bluish shade corresponding to the positive, and the
reddish to the negative values of the functions. The results
in figure 3 are shown for up to 8 clusters computed to de-
scribe the dependencies of the feature vectors on the com-
plex image motion pattern in the test image and the image
gradients distribution in the image.
The “vortex” sequence of 6 images is used to evaluate
the spatial distribution of the singular sets in scale, both
in spectral and time domain. For a still image decompo-
sition the distortion measure �U�
± 7
�/8
±
W
is applied as in
[12]. The 5th image in sequence is shown in figure 4 (a),
for which the spectral singular sets are shown in 4 (b). The
distribution of the singular sets for the motion information
is shown in 4 (c), evaluating the scale behavior of this im-
age sequence in the time domain. A distortion measure
�‚�
± 7
�98
±
W
H;:
��A
¹
�ÌA
W
�IA
g
p‚�IA
i
o
�
W
is used in mapping
the joint motion and spectral information feature vectors.
The joint motion and spectral information singular sets of
the image sequence are shown in figure 4 (d).
The MeteoSat infrared image displays the IR data acqui-
sition where the gray level intensities correspond to cloud
temperature. We decompose the images into the scale sin-
gular feature sets and study its relation to the pluiometry.
The image in figure 5 (a) is decomposed into the singular
sets with up to the 4 data clusters computed. The distribu-
tion of the temperature fronts is gray scale coded in accor-
dance with spectral scalalability data clusters. The resulting
images is shown in figure 5 (b-f).
4. Discussion
In the method of hierarchical decomposition of image data,
developed in this work, convex optimization of the free en-
ergy of data clusters is achieved by the adaptation in the
cooling schedule and with the adaptive selection of spatial
windows of computation. The cooling process is imple-
mented to evaluate the scale invariance of the group vector
representative for a cluster of data points. In the process
of phase transition, the least singular cluster of data points,
for which the condition of phase transition is reached, is di-
vided in two newly created windows, according to the rule
given in equation 8. This way we achieve a minimax opti-
mization of the error function. For every value of the scale
parameter, a minimal number of the feature vector data clus-
ters is obtained for a given limit constraint of the maximal
scaling exponent Ý , in the data clusters decomposition.
The symmetry of the space-motion features can be ob-
served for the first 4 clusters computed, as shown on fig-
ure 3. The cluster windows are splitted along the diago-
nal line in figures 3 (b and e), since the largest variance in
the estimate the motion vector goes along the direction of
distortion of the test pattern. The major principal compo-
nent vector of the data scatter matrix is orthogonal to the
diagonal line of segmentation. The expansion motion of the
test pattern determines dominantly the shape of the resid-
ual information along the complement principal component
vector, for the first 4 clusters computed, as shown on fig-
ures 3 (d and g).
A disk form structure is formed around the vanishing
point of image gradients in the middle of figure 3 (b). Also,
due to the image gradients distribution, a circular structure
in the cluster windows distribution is shown in figure 3 (e).
The motion field probability distribution is restructured in
the newly formed cluster windows, and the average motion
vectors define the circular structure in the equation ƒÌ�ۍ .
7

(a) (b) (c) (d)
Figure 4: The “vortex” sequence. For the still image in (a) the distribution of the singular sets is shown in (b). In the
sequence of 6 images the distribution of the singular sets for the motion information is shown in (c), and for the joint motion
and spectral information in (d).
pt2.jpg
(a)
p2.jpg
(b)
p4.jpg
(c)
p6.jpg
(d)
p8.jpg
(e)
p10.jpg
(f)
Figure 5: The MeteoSat infrared image of the convective clouds (a), is decomposed in temperature fronts (b-f). The temper-
ature fronts in figures (b-f) are gray scale coded according to their singularity factors. The rain process is shown associated
with the perturbation propagating through the fusion of convective clouds
8

The image in figure 3 (h) shows the refinement of the space-
motion features, dominated by the distortion motion in the
test image. The newly formed spatial features can be ob-
served in the lower right quadrant of the image, where the
distortion motion is the strongest.
The results of processing the MeteoSat infrared images
are shown in figures 4 and 5. Decomposition of the still
image in figure 4 (a) shows that the most singular man-
ifolds distribute along the strongest temperature fronts of
the cloud, as it is also observed in the work [16]. For the IR
image of the clouds in figure 5 (a) we find however an alter-
nation of the singular sets along the edges of the clouds, as
shown in figures 5 (b-f). This can be explained by the mul-
tipolar character of those edges, that is associated, in our
view, to the turbulent exchange of energy along the vertical
layers inside the clouds.
The space-motion clusters decomposition of the image
sequence, is shown in figure 4 (c). In the observation of the
motion in the image sequence we find that the elongation
of the most singular sets goes along the direction orthogo-
nal to the major component of motion vector as well as to
the stronger image gradient content. This effect can be also
observed in figure 3 (h). The top of the cloud forming the
vortex movement shows greater stability in scale as com-
pared to the bottom of the cloud. This can be also observed
in figure 4 (d), where joint motion and spectral information
is evaluated in scale for the sequence of 6 images. The most
singular manifold for the joint motion and spectral informa-
tion features decomposition is formed more in the cloud like
shape on top of the cloud, as compared to the bottom part
where it is formed dominantly by the scale invariant motion
information features.
The estimation of the feature vector data clusters, by our
algorithm, can be effectively used in compressing the image
motion information in video sequences. The spatial features
of the group windows distribution are arranged in the hier-
archical structure of binary images. Based on the Laplacian
diffusion system of equations:
†
W
¸
�׍ B
DVÀ9<
¸
�`Z‡�LB
D
� B
DVÀ>=

(9)
the reconstruction formula: ¸ �LBD �w�b�"!@? effA
9
g
=

B
?
e
´
Z �
B
C
�$�
Ò
B
gives us a way of diffusing the residual information from the
most singular sets, denoted by = , into the smoother regions,
denoted by < .
The image motion vectors and the residual motion in-
formation are estimated, in this work, from the linear con-
straint equation on the motion vector components. The
smoothing in the optical flow field estimation is controlled
adaptively by the hierarchical scale computation. A larger
order constraint equation, that uses the information of an
image stimulus, results in spatial clustering of features
along the object stimulus line as shown in [9].
5. Concluding remarks
In this paper, we have described a new method for analysis,
processing and coding of image information in image se-
quences. The hierarchical scale decomposition of image in-
formation into different singular sets is obtained by evaluat-
ing scale-space scalability of singular features for the clus-
ters of data points. The information content is evaluated by
the scale-space frequencies of respective data cluster.
For every level of signal distortion, the minimal data
clusters decomposition is obtained in hierarchy of scales.
A limit constraint of the scaling exponents is used in
the decomposition scheme. The minimax optimization is
achieved in the hierarchy of scales of computation, progres-
sively always decreasing the uncertainty in the estimation.
The Green’s function expresses spatial coherency of
data clusters and is used in the segmentation process as a
smoothing function. In the method proposed, harmonic de-
composition of images is achieved in herarchy of scales. A
better spatio-temporal resolution and small code size of data
features is obtained, that can be used effectively in com-
pressing image information by our method. The image re-
construction formula is based on the Laplacian system of
the diffusion of the residual information from the most sin-
gular sets. We intend to investigate the reconstruction part
of our method in our future work. We also intend to inves-
tigate a parallel computer implementation of the algorithm
along with the refinement of our computational scheme, in
our future work.
Acknowledgments
I am grateful to Hussein Yahia and Isabel Herlin, as well as
to all the people that I met in the AIR Lab and the INRIA
for providing a group support and inspiring environment to
work.
References
[1] D.Bereziat, and J-P.Berroir, “Motion Estimation on Metere-
ological Infrared Data using Total Brightness Invariance Hy-
pothesis,” Environment Modeling System, 2000.
[2] R.E.Blahut, Principles and Practice of Information Theory,
Addison-Wesley, 1987.
[3] R.Courant, and D.Hilbert, Methods of Mathematical Physics,
John Wiley & Sons, New York, 1962.
[4] A.Gersho, and R.M.Gray, Vector Quantization and Signal
Compression, Norwell, MA:Kulwer, 1992.
9

[5] B.K.P.Horn, and B.G.Schunk, “Determining Optical Flow,”
Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[6] B.K.P.Horn, Robot Vision, The MIT Press, Cambridge, 1986.
[7] A.K.Jain, and R.C.Dubes, Algorithms for Clustering Data,
Englewood Cliffs, New Jersey, 1988.
[8] E.T.Jaynes, “Information theory and statistical mechanics,” in
Papers on probability, statistics and statistical physics (R.D.
Rosenkrantz, ed.), Kulwer Academic Publishers, 1989.
[9] M. Jovovic, “Image segmentation for feature selection from
motion and photometric information by clustering,” SPIE
Symp. on Visual Inform. Processing V, Orlando, 1996.
[10] M. Jovovic, “ A multiscale processing of data streams,” (in
English), Med. Eng. and Physics, Vol. 21/5, pp. 325-337,
1999.
[11] M. Jovovic, S. Jonic, and D. Popovic, “Automatic synthesis
of synergies for control of reaching - hierarchical clustering,”
Informacione Tehnologije IV, Zabljak, Yugoslavia 1999.
[12] M. Jovovic, “Space-Color Quantization of Multispectral Im-
ages in Hierarchy of Scales,” ICIP’01, Vol. I, pp. 914-917,
Thessaloniki, Greece, 2001.
[13] T.Poggio, E.B.Gamble, and J.J.Little, “Parallel Integration of
Vision Modules,” Science, vol. 242, pp. 436-440, 1988.
[14] K.Rose, E.Gurewitz, and G.C.Fox, “A Deterministic Anneal-
ing Approach to Clustering,” Pattern Recog. Lett. vol. 11, pp.
589-594, 1990.
[15] C.Tomase, and T.Kanade, “Shape and Motion from Image
Streams: a Factorization Method-2. Point Features in 3D Mo-
tion,” TR-CMU-CS-91, Pittsburgh, 1991.
[16] A. Turiel, and A. del Pozo, “Reconstructing Images From
Their Most Singular Fractal Manifold,” IEEE Trans. Image
Processing, vol. 11/4, pp. 345-350, 2002.
[17] S.Wiggins, Introduction to Applied Nonlinear Dynamical
Systems and Chaos, Springer-Verlag, New York, 1990.
10