Page 1

Determining Multiscale Image Feature Angles

from Complex Wavelet Phases

Ryan Anderson, Nick Kingsbury, Julien Fauqueur?

Signal Processing Group, Dept. of Engineering, University of Cambridge, UK

Abstract. In this paper, we introduce a new multiscale representation for 2-D images named

the Inter-Coefficient Product (ICP). The ICP is a decimated pyramid of complex values based

on the Dual-Tree Complex Wavelet Transform (DT-CWT). The complex phases of its coeffi-

cients correspond to the angles of dominant directional features in their support regions. As

a sparse representation of this information, the ICP is relatively simple to calculate and is a

computationally efficient representation for subsequent analysis in computer vision activities

or large data set analysis. Examples of ICP decomposition show its ability to provide an in-

tuitive representation of multiscale features (such as edges and ridges). Its potential uses are

then discussed.

1Introduction

Wavelets, once used primarily for compression, have found new uses for image content analysis.

The ability of the wavelet transform to isolate image energy concisely into spatial, directional, and

scalar components have allowed it to characterize the multiscale profile of non-stationary signals,

including 2-D images, very effectively. In particular, complex wavelets have shown a strong ability

to consistently represent object structures in 2-D images for object recognition and computer vision

activities.

In this paper, we explore methods of building upon the phase information of complex wavelets

to yield intuitive image representations. To date, complex wavelet magnitudes have typically been

used in place of real wavelets to improve the consistency of segmentation, denoising, etc. However,

phase information, which indicates the offset of directional features within the support region of a

wavelet coefficient, has found less application to date in analysis and coding applications (although

stereo matching and motion estimation are two examples of its use). Recently, in [3], Romberg et

al. have described a probabilistic model, the Geometric Hidden Markov Tree (GHMT), which uses

phase as well as magnitude information to infer the angle and offset of contour segments in the

vicinity of a complex wavelet coefficient. In this paper, we introduce a faster method to calculate the

angle of directional energy in the vicinity of a coefficient. This method, which we have named the

Inter-Coefficient Product (ICP), may find use in large-scale image analysis or real-time computer

vision, where computational complexity must be minimized. We introduce the ICP as a complement

to another phase-based transform, the Interlevel Product (ILP) [1]. Upon describing the background

of complex wavelets (section 2), we will develop the ICP transform in section 3 and show example

ICP decompositions in section 4. We conclude in section 5 with a discussion of potential uses of the

ICP and its relationship to the DT-CWT and the ILP.

?This work has been carried out with the support of the UK Data & Information Fusion Defence Technology

Centre.

Page 2

2The Dual-Tree Complex Wavelet Transform

Standard real wavelets, such as the Haar and Daubechies wavelets, suffer from shift dependence.

Shift dependence implies that the decomposition of image energy between levels of a multiscalar

decomposition can vary significantly, if the original image is shifted prior to decomposition. This

variation limits the effectiveness of the real wavelet transform to consistently represent an image

object at multiple scales.

Complex wavelets, including the linearly-separable Dual-Tree Complex Wavelet [2] have been

created to address the problems of shift dependence. A complex wavelet is a set of two real wavelets

with a 90◦phase difference. For 2-D image analysis, the DT-CWT produces d = 1...6 directional

subbands at approximately

labelled with equally-spaced angles of 15◦, 45◦, 75◦, 105◦, 135◦, and 165◦respectively). The impulse

responses in each of these subbands are shown in figure 1. We note that the magnitude responses

of each these subbands can be used to infer feature orientations. However, the lack of precision in

these methods is the primary motivation for us to seek a superior representations through the use

of complex phase information.

π

10,

π

4,

2π

5,

3π

5,

3π

4, and

9π

10(for convenience, these subbands are often

DT CWT real part

DT CWT imaginary part

π/10π/4 2π/53π/53π/49π/10

Fig.1. The real and imaginary impulse responses of the DT-CWT for each of the six subbands.

Figure 2 shows both the phase (fig. 2a) and magnitude (fig. 2b) responses of a DT-CWT coefficient

to a shifting step response in 1-D. In particular, we observe that this phase response is consistently

linear with respect to the feature offset, in the vicinity of the wavelet coefficient. If we define DW

as the distance between adjacent coefficients, as indicated by the vertical lines in figure 2a, then we

have experimentally determined the relationship between coefficient phase and feature offset to be

−4.49/DW radians per unit length. With this ratio, we can convert DT-CWT phase to a spatial

offset of an edge or impulse, or vice-versa.

In the 2-D case, the phase and magnitude relationships described above apply to edges and ridges

oriented in the direction of the subband. The ratios for 2-D subbands differ from the 1-D example;

for subbands 1, 3, 4, and 6, the ratio is −4.49/?DWcosπ

10

or −6.35/DW radians per unit length. for

?or −4.72/DW radians per unit length.

For subbands 2 and 5, the ratio is −4.49/?DWcosπ

4

?

Page 3

−25−20−15 −10−50510152025

0

50

100

150

200

250

300

350

a) The complex phase of a decimated level 3

DT-CWT coefficient (located at the central

dotted line), in the presence of a step edge

at all possible offsets (the x axis). Note that

when an edge or ridge occurs anywhere be-

tween the coefficient and its immediate neigh-

bours (shown as the vertical dashed lines),

the phase response is linear. This linearity

will be used to infer the offset of the edge, rel-

ative to the coefficient location, at this scale.

−25−20−15 −10−5 0510 152025

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

Real

Imaginary

Magnitude

b) The magnitude response of the same DT-

CWT coefficient under the same conditions.

The overall magnitude is calculated from real

and imaginary components, as shown.

Fig.2. Illustration in the 1-D case of the behaviour of the phase and magnitude of a DT-CWT coefficient

(level 3, in this case) in the presence of a step edge, at the indicated x coordinate, relative to the coefficient.

these 2-D cases, the subband offset of a feature is defined in the direction normal to the subband, as

shown in figure 3.

dA= Subband Offset

π/10

A

Step Edge

Subband Direction

Fig.3. Definition of the subband offset dA in the 2-D case between a coefficient location A and a step edge.

Page 4

3The Inter-Coefficient Product

In this section, we introduce the Inter-Coefficient Product (ICP). We begin in sections 3.1 and 3.2

by showing how the orientation of a 2-D feature (such as an edge or ridge) can be determined from

the phase difference of two adjacent DT-CWT coefficients. This derivation leads us naturally to the

definition of the ICP in section 3.3.

3.1Determination of feature orientation from neighbour coefficients

Consider Figure 4, which shows a feature (a step edge, in this case) that spans the support regions

of two horizontally adjacent DT-CWT coefficients at locations A=(x,y) and B=(x+1,y), at some

arbitrary level. This figure illustrates the trigonometric relationship between the angle of the feature

and its subband offsets with respect to these coefficients.

α

θ1

π/10

dB

Dw sin π/10

dA

dA- dB- Dw sin π/10

π/10

Dw

A

dB

Step Edge

Subband Direction

Dw cos π/10

B

Fig.4. Trigonometric relationship between the angle θ1 = α +

its subband offsets dA and dB to two horizontally adjacent

A=(x,y) and B=(x + 1,y).

π

10of a feature (step edge, in this case) and

10subband DT-CWT coefficients located at

π

As this step edge is closest to the d = 1,

Wl(x + 1,y,1) will correspondingly have large magnitudes in this subband only. From figure 4, we

can see that the angle of the feature, θ1, can be calculated as

π

10subband, the DT-CWT coefficients Wl(x,y,1) and

θ1=

π

10+ α =

π

10+ tan−1dA− dB− Dwsinπ

10

Dwcosπ

10

(1)

where dA and dB are the subband offsets of the edge to the two coefficients as defined in figure

3, using subband 1. These two offset lengths are equal to

respectively1, according to our phase/offset relationships described at the end of section 2. Thus, we

DW

4.72? Wl(x,y,1) and

DW

4.72? Wl(x + 1,y,1)

1We denote the phase argument of a complex number c as? c = arg(c).

Page 5

can rewrite Equation (1) as

θ1=

π

10+ tan−1

DW

4.72? Wl(x,y,1) −DW

?? Wl(x,y,1) −? Wl(x + 1,y,1)

4.72? Wl(x+1,y,1) − Dwsinπ

Dwcosπ

10

− tanπ

10

=

π

10+ tan−1

4.72cosπ

10

10

?

(2)

We note that for −π

to an individual subband, we can assume that α ≈ tanα. Applying this approximation twice to

Equation (2), we can simplify this expression to

5< α <π

5, which is the approximate range of feature angles that will contribute

θ1=

π

10+

1

4.49[? Wl(x,y,1) −? Wl(x + 1,y,1)]

? Wl(x,y,1) −? Wl(x + 1,y,1)

4.72cosπ

10

−π

10

(3)

⇒ θ1=(4)

Thus, for subband 1, we merely divide the phase difference between two horizontally adjacent DT-

CWT coefficients by 4.49 to obtain the angle of the dominant feature in their vicinity.

3.2Feature orientation calculations for all subbands

In the previous section example, we considered the

particular example of step edge. In the general case, to detect a feature at any orientation, the same

type of calculation is achieved for all six subbands based on phase difference between appropriate

neighbour DT-CWT coefficients: we compare horizontal neighbours for the

(d = 1,6), vertical neighbours for the2π

theπ

subband. We have already determined θ1(see Equation (4)).

By symmetry, the geometric relationship in the

subbands d = 3,4, and 6, where feature angles in subbands 3 and 4 are measured relative to the

vertical axis. For these two subbands, therefore, we would therefore modify equation (4) to addπ

to the angle. However, in theπ

with the angles of dominant features with two diagonally adjacent coefficients. Using subband 2, the

angle of the feature is related to coefficient offsets by the following simpler equation:

π

10subband (d = 1) given the orientation of our

π

10and

9π

10subbands

5and3π

5subbands (d = 3,4), and diagonal neighbours for

4and3π

4subbands (d = 2,5). Thus six orientation angles are calculated θ1,...,θ6, one for each

π

10subband d = 1 can be equally applied to

2

4,3π

4subbands 2 and 5, the Wlvalues possess a different relationship

θ2=π

4+ tan−1dA− dB

√2Dw

(5)

Performing the same substitutions, assumptions, and simplifications as with our previous exam-

ple, we establish the linear relationship between θ2and the Wlcoefficient phases to be

θ2=π

4+

1

8.98[? Wl(x,y + 1,2) −? Wl(x + 1,y,2)](6)

As with the previous example, this relationship is identical for subband 5, except that3π

be added to the phase of θ5in the operation above.

4would

Page 6

3.3 Definition of ICP

In the previous section, we established the linear relationship formulae between a feature orientation

and the difference of phase between adjacent DT-CWT complex coefficients. We calculate these phase

differences by means of conjugate products of the adjacent pairs of DT-CWT coefficients as detailed

in the previous section. The complex products are a natural way to represent the feature orientation

(through the complex phase) and also the feature strength (through the complex magnitude). Thus,

if we consider Wl(x,y,d) to be the complex DT-CWT coefficient at spatial location x,y (numbered

from the top left corner), subband d, and level l, then we introduce the constant-phase complex

values Wl∆:

Wl∆(x,y,1) = Wl(x,y,1)

Wl∆(x,y,2) = Wl(x,y + 1,2) × Wl(x + 1,y,2)∗

Wl∆(x,y,3) = Wl(x,y,3)

Wl∆(x,y,4) = Wl(x,y,4)∗

Wl∆(x,y,5) = Wl(x,y,5)∗

Wl∆(x,y,6) = Wl(x,y,6)∗

From this definition, the feature orientation θd calculated for each subband d in the previous

section, can be expressed with respect to? Wl∆(x,y,d). For instance, from equation (4), θ1 =

1

4.49? Wl∆(x,y,1).

The magnitudes of Wl∆are the product of the magnitudes of the two adjacent DT-CWT coef-

ficients, and the phases of Wl∆are their shift-invariant phase differences. Note that we can divide

the magnitudes of Wl∆by

Using the conjugate products Wl∆and the expressions for feature orientations θd, we now define

the Inter-Coefficient Product.

× Wl(x + 1,y,1)∗

× Wl(x,y + 1,3)∗

× Wl(x,y + 1,4)

× Wl(x + 1,y + 1,5)

× Wl(x + 1,y,6)

(7)

?| Wl∆| to mitigate the non-linear product effect of this operation.

Definition 1 (Inter-Coefficient Product) Given a DT-CWT decomposition of an image with

coefficients Wl(x,y,d) for levels l and subbands d = 1,...,6, we define the Inter-Coefficient Product

(ICP) for each subband d, level l and decimated location (x,y) as the following set of complex

coefficients {ψl(x,y,d),d = 1,...,6}:

ψl(x,y,1) =

| Wl∆(x,y,1) | × ei(

ψl(x,y,2) =

| Wl∆(x,y,2) | × ei(π

ψl(x,y,3) =

| Wl∆(x,y,3) | × ei(π

ψl(x,y,4) =

| Wl∆(x,y,4) | × ei(π

ψl(x,y,5) =

| Wl∆(x,y,5) | × ei(3π

ψl(x,y,6) =

| Wl∆(x,y,6) | × ei(

where i =√−1.

We will consider the contribution of a feature to the subband which is the closest to its orienta-

tion, since it is where the DT-CWT coefficient response is linear and the strongest. The coefficient

magnitudes automatically reveal the dominant orientation of the feature across subbands.

At each location (x,y) and each level l, the orientation of a potential feature (such as an edge or

a ridge) in the vicinity of (x,y) will be given by the phase of an ICP coefficient. The magnitude of

the ICP coefficient will reflect the strength of this feature.

?

?

?

1

4.49?

Wl∆(x,y,1))

?

?

?

4+

1

8.98?

Wl∆(x,y,2))

2+

1

4.49?

Wl∆(x,y,3))

2+

1

4.49?

Wl∆(x,y,4))

4+

1

8.98?

Wl∆(x,y,5))

1

4.49?

Wl∆(x,y,6))

Page 7

4Results and Interpretation

Figure 5 shows an ICP decomposition for the “Lenna” image, for level l=3,9π

5a, and level l=4 (coarser),

can see the ability of the ICP to follow coarse and fine image contours.

If we shift the original image half the current subband coefficient spacing prior to ICP transform,

we apply the worst possible offset in multiscale misalignment (described in [1]) that may occur if

one was to compare two separate instances of an image object. Figure 6 shows this offset; note that,

relative to figure 5b, the coefficients make small, predictable changes in direction and magnitude

to reflect changing support regions, but dominant edge features in the image keep the coarse-level

representation relatively invariant to multiscale misalignment, which shows the shift independence

of ICP.

Note also that the ICP is a reversible transform; with all the ICP coefficients and the last row

and column of DT-CWT coefficients, one can divide out all of the original DT-CWT coefficients

one row/column at a time (and thence reconstruct the original image with a reverse DT-CWT

transform). However, as the ICP itself acts in the manner of a differential operator, modifications

to the ICP coefficients can propagate throughout the image, far beyond the support range of the

original coefficient.

10subband in figure

π

4subband in figure 5b. When overlayed upon the original image, we

(a)(b)

Fig.5. ICP Coefficients for “Lenna” picture, at a) Level 3, Subband 6; and b) Level 4, Subband 2. Note the

ability of the ICP to follow, for example, the fine edge of the top of the hat in a), and the coarse-scale

rim of the hat on the left in b).

π

4

Page 8

Fig.6. ICP shift independence: the input image of figure 5b has been shifted by half a sample in each

direction prior to ICP transform. Although this shift corresponds to the worst alignment case, we observe

minor changes in the ICP coefficients.

5Conclusions

The ICP transform extracts phase information from the DT-CWT transform into an intuitive, sparse

format that reveals the orientation of directional features with finer precision than with the sole use

of magnitudes of real or complex wavelets coefficients. The entire process from pixel domain to ICP

domain is efficient to implement: the DT-CWT is linearly separable into row and column operations,

and the subsequent ICP operation performs simple operations on decimated coefficients. Thus, we

believe ICP coefficients to be appropriate for multiscale image processing activities such as contour

tracking, registration, and rotation- and scale-invariant object recognition, in large images or real-

time systems where computational complexity is a strong factor in system design.

In particular, we note that the ICP and the ILP [1] transforms complement one another very elo-

quently in the description of multiscale features. The ICP is highly informative as to small rotations

and relatively insensitive to feature structure. By contrast, the ILP is indicative of the nature of the

feature itself and is insensitive to small rotations. Between these two pyramidal image representation

transforms, we can build a highly informative hybrid representation of image objects that can be

detected at various scales and rotations. Our future research will pursue such models with these

coefficients.

References

1. R. Anderson, N. Kingsbury, and J. Fauqueur. Coarse level object recognition using interlevel products

of complex wavelets. In International Conference on Image Processing (ICIP), September 2005.

2. N.G. Kingsbury. Complex wavelets for shift invariant analysis and filtering of signals. Journal of Applied

and Computational Harmonic Analysis, (3):234–253, 2001.

3. J. Romberg, M. Wakin, H. Choi, and R. Baraniuk. A Geometric Hidden Markov Tree Wavelet Model.

In SPIE Wavelets X, San Diego, CA, August 2003.