Local Scale Measure from the Topographic Map and Application to Remote Sensing Images.

Full text

Page 1

MULTISCALE MODEL. SIMUL.
Vol. 0, No. 0, pp. 000–000
c ? 20XX Society for Industrial and Applied Mathematics
LOCAL SCALE MEASURE FROM THE TOPOGRAPHIC MAP AND
APPLICATION TO REMOTE SENSING IMAGES∗
BIN LUO†, JEAN-FRANC ¸OIS AUJOL‡, AND YANN GOUSSEAU§
Abstract. In this paper, we propose a spatially accurate definition of scale for images. The
proposed definition of local scale relies on the hierarchical structure of the topographic map, the
set of all level lines of an image. Namely, a connected component of a level line is associated with
each point of the image and the scale at this point is then computed from the corresponding shape.
Level lines are selected under the assumption that the blur is uniform over the image, a realistic
assumption for satellite images. First, we discuss the links between the proposed approach and
recent scale definitions making use of the total variation flow or the Rudin–Osher–Fatemi model. This
comparison sheds some light on the relationships between morphological and variational approaches
to scale definitions. Then we perform several experiments on synthetic and satellite images and
numerically compare our local scale definition with results using the linear scale space, variational,
or morphological approaches. These experiments suggest that the proposed method enables a robust
and spatially accurate computation of local scales, without the need for complex parameter tuning.
Key words. local scale, topographic map, remote sensing images
AMS subject classifications. AUTHOR: PLEASE PROVIDE
DOI. 10.1137/080730627
1. Introduction. The notion of scale is of primary importance in digital image
analysis. On the one hand, it is crucial information to tune the spatial extent of
analysis tools. In remote sensing imaging, the best observation scale is closely related
to the concept of analysis scale in geography [39]. The NATO standard STANAG
3769 [40] gives some examples of the best scales for interpreting certain objects in
remote sensing images. In computer vision, most object recognition or detection
methods include a scale computation while or before extracting significant features.
As a side product, this also permits one to achieve scale invariance, which is necessary
considering the arbitrariness of object distances to the camera.
On the other hand, scale itself is also useful for the classification of images. In
particular, remote sensing images are often made of several regions showing some
texture-like homogeneity. These regions usually exhibit different size distributions of
their constituents. It is tempting to characterize each pixel of such images by the size
of the object(s) (in a sense to be defined) it belongs to. The purpose of this paper is
to show that this idea is feasible and yields spatially accurate features for describing
remote sensing images.
A classical approach to estimate the scale of structures in an image has been
introduced by [25, 26]. Local scales are computed by studying extrema of differential
operators in the linear scale space. This method has been widely applied in computer
∗Received by the editors July 18, 2008; accepted for publication (in revised form) January 20, 2009;
published electronically DATE. This work was supported by the CNES-DLR-ENST Competence
Centre.
http://www.siam.org/journals/mms/x-x/73062.html
†TELECOM ParisTech, CNES-DLR-ENST Competence Centre, Paris, France, and DIS Gipsa-
Lab, Grenoble, Saint Martin d’Heres 38402, France (Bin.luo@gipsa-lab.inpg.fr). This author’s re-
search was supported by the COC and TELECOM ParisTech.
‡CMLA, ENS Cachan, CNRS, UniverSud, Cachan 94235, France (Jean-Francois.Aujol@cmla.
ens-cachan.fr).
§TELECOM ParisTech, LTCI CNRS, Paris 75014, France (gousseau@enst.fr).
1

Page 2

2
BIN LUO, JEAN-FRANC ¸OIS AUJOL, AND YANN GOUSSEAU
vision to select the optimal analysis scale of structures; see, e.g., [28, 35]. Similar to
this approach, the methods presented in [20, 46, 47, 52] propose to estimate the salient
scale of an image by considering extrema of various information theoretic operators
in the linear scale space. For remote sensing images, it is also proposed in [30] to
rely on a linear scale space to estimate a resolution invariant characteristic scale. The
invariance is achieved by studying the effect of the image acquisition process on the
linear scale space decomposition. Indeed, the linear scale space is very convenient
since it enables one to easily model scale (see, e.g., [35]) or resolution [30] changes.
However, using the linear scale space prevents such methods from achieving spatial
accuracy. It is well known that the positions of structures that are detected by such
methods are relatively erratic and very sensitive to contrast changes, especially for
large structures. Therefore, one can expect some difficulty in building up spatially
accurate indexing of images from these methods. Moreover, these methods disregard
structures without significant gradients, for which no scale is computed.
In order to obtain spatially accurate scale measures, it is quite natural to look
toward nonlinear approaches. In [21, 22] it is proposed to measure the significant
scales of structures by computing the entropy of the joint distribution of pixels in a
fixed neighborhood. It has been shown (see [22]) that such an approach yields better
repeatability and is less sensitive to image perturbation than methods making use
of Gaussian filtering. Other methods enabling the computation of spatially accu-
rate scales are mainly based on nonlinear scale spaces, either in a morphological or
variational framework. Recently, several scale measures based on nonlinear partial
differential equations have been proposed. In [51, 50], a local definition of scale based
on total variation regularization is introduced. In [8], it is proposed to estimate the
local scales of structures from their evolution speed under the total variation flow.
In both works, the idea is that the evolution time of structures is related to their
scale. Independently, the mathematical morphology school has long ago proposed to
characterize materials by looking at the size distribution of grains, through the use
of the so-called granulometry [18, 43]. Following this idea, it is proposed in [31] to
use the pattern spectrum of images (roughly, the derivative of granulometry) to index
grayscale images. In the framework of remote sensing imaging, [5, 13] have proposed,
in view of the classification of satellite images, to compute size distributions (called
derivative morphological profiles) at each pixel. A closely related approach to the ex-
traction of urban area was previously proposed in [27]. In these works, the proposed
local features contain some information about scale. This information is well localized
thanks to the use of connected morphological filtering of features.
In this paper, our purpose is twofold. First, we introduce a method to compute
a local scale measure following the general approach of granulometries. Contrarily
to previous morphological approaches, the proposed method is autodual (is invariant
to contrast reversal) and does not necessitate any structuring element. Second, we
establish theoretical links between the variational and morphological approaches to
scale computation.
The first contribution is a method to compute a local scale measure (defining a
characteristic scale at each pixel of a digital image) by using the topographic map [11]
of the image. The main idea is that, for each pixel, we associate the scale of the
most significant structure containing it. The definition of this structure relies on the
topographic map, which is made of the connected components of the boundaries of
level sets of the image. More precisely, we make use of the digital image transform
presented in [38], an efficient tool to compute the topographic map, representing an
image by a hierarchical structure (an inclusion tree) of shapes. From this tree we

Page 3

LOCAL SCALE MEASURE FROM THE TOPOGRAPHIC MAP
3
search, for a given pixel, the most contrasted shape containing it and associate the
scale of this shape with the pixel. In order to deal with optical blur that is inherent
in satellite images, a new definition of shape is proposed. The underlying idea of this
definition is to group level lines corresponding to the same structure in the image.
This grouping is made possible under the assumption that the optical blur is constant
over the image, an assumption that makes sense for remote sensing images but would
be wrong for natural, everyday life images. Experimental results suggest that the scale
measure is a pertinent feature for remote sensing image classification. In particular,
the use of the topographic map yields very satisfactory spatial accuracy of the scale
measure, especially when compared with linear methods (see section 6). Our approach
is also compared with variational and morphological methods to estimate local scales.
The second contribution of the paper is to shed light on the relationship be-
tween morphological and variational approaches to the computation of local scales.
In particular, it is shown that, under some hypotheses, approaches relying on the
total variation flow or on the Rudin–Osher–Fatemi model amount to defining scale
as a weighted average of the size of shapes containing each pixel. In this framework,
the term “size” refers to the ratio of the area to the perimeter. This result is also
investigated numerically on synthetic and real images.
The paper is organized as follows. In section 2, we present the topographic map
of an image, the fundamental tool on which our work is based. Then, in section 3,
we explain how an accurate scale definition can be given by using the topographic
map. In section 4, alternative variational definitions of scale are given, and the link
between these approaches and ours is investigated. In section 5, we illustrate the
method with numerical examples, both on synthetic and real images. In section 6, we
provide experimental comparisons between the proposed local scale measure, linear
approaches, variational methods, a possible use of the so-called maximally stable
extremal regions, and methods relying on the pattern spectrum. In section 7, we
propose some extensions to the proposed definition of scale.
2. Topographic map. In this section, we introduce the main tool to be used
in this paper, the topographic map of an image as introduced and studied in [11,
12, 37, 38]. It is made of the connected components of the topological boundaries
of level sets, thereafter called level lines. It gives a complete representation of the
image, invariant to local contrast changes. The topographic map has a tree structure,
inherited from inclusion properties of level sets.
For an image u : Ω ⊂ R2→ R, its upper and lower level sets are defined,
respectively, as
Ψλ= {x ∈ Ω, u(x) ≥ λ}
andΨλ= {x ∈ Ω, u(x) ≤ λ}
for λ ∈ R. Observe that u can be reconstructed using the upper level sets by the
following formula:
u(x) = sup{λ ∈ R : x ∈ Ψλ}.
Lower level sets also permit us to reconstruct u. Moreover, these sets are globally
invariant with respect to contrast changes. Each of these family, upper sets on the one
hand and lower sets on the other hand has a tree structure with respect to inclusion.
Several authors [42, 11, 12, 19] have proposed the connected components of level sets
as an efficient way to represent images. This representation enjoys the same contrast
invariant and hierarchical structure as level sets representations while being local,

Page 4

4
BIN LUO, JEAN-FRANC ¸OIS AUJOL, AND YANN GOUSSEAU
(a)(b)
Fig. 1. Example of FLST: (a) Synthetic image; (b) inclusion tree obtained with FLST.
thus in particular robust to occlusions or local contrast changes. Again, connected
components of upper sets have a tree structure, as well as connected components of
lower sets. In order to merge both trees, Monasse and Guichard [37, 38] introduced
the concept of tree of shapes. It relies on the concept of shape. A shape is defined
as the union of a connected component of an upper or lower set and its holes. The
holes of a set A are defined as the connected components of the complementary set
of A which do not intersect with the boundary of Ω. It can be shown that these
shapes have a tree structure. Under some regularity conditions, this tree of shapes
is the same as the tree made of the connected components of topological boundaries
of upper (or lower) sets of the image, the topographic map. In what follows, we use
the term topographic map for this tree. An important property of this representation
is its autoduality, that is, its invariance with respect to the operation u → −u (or,
equivalently, to u → C − u, with C a constant). This implies in practice that light
and dark objects are treated in the same way. As we will see in the following, it
is important for the purpose of this paper to deal with a single tree instead of two
separated ones, since this will enable us to associate a unique contrasted shape with
each pixel.
In order to apply the preceding definition to a discrete image I, it is conve-
nient to consider a continuous image (defined at each point of its domain) associated
with I. Two straightforward choices are piecewise constant interpolation (each pixel
corresponds to a constant value) and bilinear interpolation. Of course, bilinear in-
terpolation is preferable in order to obtain regular lines, free of pixelization effects.
However, this also implies longer computation time. Moreover, we observed that for
the purpose of this work, namely local scale computation from the topographic map,
bilinear interpolation was unnecessarily complex. In the remainder of the paper, level
lines are computed on piecewise constant interpolations of discrete images. Thanks
to its tree structure, the topographic map of an image can be efficiently computed
using the fast level set transform (FLST) algorithm presented in [38]. Figure 1 shows
the result obtained with this algorithm on a synthetic image. Notice that another
implementation has also been proposed in [45].
We end this section by giving some notation for the attributes of shapes that are
used in what follows. For a pixel x of an image u, we denote by {fi(x)}i=1,...,N(x)the
set of shapes that contain x, N(x) being the number of shapes containing x and the
shapes being ordered so that fi(x) ⊂ fi+1(x). For the sake of clarity, we will omit
the x dependency when it is not necessary. For each shape fi, we define |fi| its area,

Page 5

LOCAL SCALE MEASURE FROM THE TOPOGRAPHIC MAP
5
P(fi) its perimeter, and I(fi) its gray level value, that is, the gray level of the upper
or lower level set defining fi. The contrast of the shape fi is then defined as the
absolute value of the difference between the gray level values associated, respectively,
with fiand fi+1:
(2.1)
C(fi) = |I(fi+1) − I(fi)|.
3. Scales of an image. The basic idea of this paper is to associate with each
pixel a shape (i.e., a node in the FLST) from which its scale can be computed.
Such shapes are obtained by filtering the topographic map. Basically, shapes are
recursively grouped in order to account for structures present in the image and the
most contrasted groups are kept, as detailed in section 3.1. Shape grouping is defined
by taking advantage of the particular structure of satellite images, for which the blur
is constant over the image and depends only on the (usually known) point spread
function of the acquisition device. This fact is investigated in section 3.2. Section 3.3
then defines the local scale associated with each pixel.
3.1. Most contrasted shape. The aim of this section is to propose a method
to extract the most contrasted shapes from the topographic map. A notion of mean-
ingful shape was introduced in [14] and consists roughly in picking the shapes whose
perimeters and contrasts are very unlikely in a noise image, the decision being made
using an a contrario method. In our case, the objective is different. First, we intend
to associate a shape with each pixel. Second, the selection criterion should not rely
on the size of shapes because such a process would bias the following definition of
characteristic scale. In this section, shapes are selected using solely some contrast in-
formation. In section 7.1, we briefly investigate the use of both contrast and regularity
in the shape selection process.
Using the definition of contrast given by (2.1), the simplest definition of the most
contrasted shape at a pixel would be to define
(3.1)
ˆf(x) = farg maxi=0,...,N(x)C(fi).
For instance, in Figure 1 the most contrasted shape associated with pixels of shape C
is C, and the most contrasted shape associated with pixels of shape E is B. However,
this definition is not satisfactory for real images. Indeed, the definition of contrast
by formula (2.1) corresponds to the contrast of a given binary structure under the
assumption that only one line is associated with the boundary of this structure. This
is, for instance, the case for the synthetic image of Figure 1. Now, in a natural
image, the contours of objects are always blurred by the acquisition process. As a
consequence, a discrete image being quantified, a contour is in fact associated with a
set of level lines. In practice, the contrast of each line is often equal to one. Therefore,
the choice of the most contrasted shape using (3.1) can be ambiguous at best or even
completely meaningless in the presence of strong noise, as will be illustrated in the
experimental section.
A solution to this problem would be to compute the contrast of a line in a neigh-
borhood and to select the most meaningful line along a monotone branch of the tree
(see [14]). In the present work, we choose to group lines corresponding to a single
structure by using a simple model of blur. To do so, we recursively sum up the
contrasts of shapes fiand fi+1such that
(3.2)
|fi+1| − |fi| < λP(fi),