Content uploaded by Cicero Mota
Author content
All content in this area was uploaded by Cicero Mota on Jan 17, 2013
Content may be subject to copyright.
Submitted to International Journal of Computer & Information Science 1
Categorization of Transparent-Motion Patterns Using the Projective Plane
Cicero Mota0, Michael Dorr0, Ingo Stuke1, and Erhardt Barth0
University of L¨ubeck, Germany
Abstract
Based on a new framework for the description of Ntrans-
parent motions we categorize different types of transparent-
motion patterns. Confidence measures for the presence of all
these classes of patterns are defined in terms of the ranks of
the generalized structure tensor. To resolve the correspon-
dence between the ranks of the tensors and the motion pat-
terns, we introduce the projective plane as a new way of
describing motion patterns. Transparent motions can occur
in video sequences and are relevant for problems in human
and computer vision. We show a few examples for how
our framework can be applied to explain the perception of
multiple-motion patterns and demonstrate a new illusion.
Keywords: Human and computer vision, multiple transpar-
ent motions, generalized structure tensor.
1. Introduction
Motion estimation has many applications in computer vi-
sion, e.g., video coding, image tracking, image enhancement,
depth recovery, etc. Accordingly, various algorithms for mo-
tion estimation are known, see [3, 11] for reviews. However,
the problem of motion estimation is always linked to the prob-
lem of motion detection and the selection of the appropriate
motion model. This is because the assumptions under which
the motion parameters can be estimated correctly are rarely
fulfilled in real dynamic scenes. In fact, motion estimation
is an ill-posed problem [6] and algorithms rely on some sort
of local or global regularization of the motion field in order
to produce meaningful results. The so-called aperture prob-
lem, noise, occlusions, appearing objects, and transparencies
in image sequences create situations where motion estimation
becomes difficult. Therefore, a correct decision on what local
or global motion model to use is as important as the estima-
tion of the motion parameters.
Motion selectivity is also a key feature of biological vi-
sual processing and has been studied by recordings of neural
responses and by psychophysical experiments. Human ob-
servers are able to see and distinguish multiple transparent
0Institute for Neuro- and Bioinformatics, University of L ¨ubeck, Ratze-
burger Allee 160, 23538 L ¨ubeck, Germany, {mota, dorr, barth}@inb.uni-
luebeck.de
1Institute for Signal Processing, University of L¨ubeck, Ratzeburger Allee
160, 23538 L¨ubeck, Germany, stuke@isip.uni-luebeck.de
motions. A special case is that of overlaid 1D motions, i.e.,
the case of moving straight patterns. Of particular interest is
how human observers resolve the ambiguities that are inher-
ent in this type of patterns [1, 26, 14] and how visual neurons
respond to such patterns [19].
Transparent motions are additive or multiplicative super-
positions of moving patterns and occur due to reflections,
semi-transparencies, and partial occlusions. Different ap-
proaches for the estimation of motion vectors for the case
of multiple transparent motions are known [21, 8, 9, 13, 25,
27, 18, 24, 23] and meanwhile the non-linear transparent-
motions equations introduced by Shizawa and Mase [21, 22]
have been solved for an arbitrary number of motions [18].
Nevertheless, the issue of confidence for multiple-motion
models has, to our knowledge, only briefly been addressed
in [4, 16, 17].
This paper provides a framework for the analysis of im-
age sequences with the occurrence of transparent moving pat-
terns, such that, for example, the motion of two overlaid 1D
patterns (e.g. two gratings) can be distinguished from the
motion of one 2D pattern (these patterns remain equivalent
within traditional theories of only one motion). First, we es-
tablish a correspondence between moving patterns and sub-
sets of the projective plane. This is done such that 2D moving
spatial patterns correspond to points and 1D spatial patterns
correspond to lines of the projective plane. This correspon-
dence is then used to show that different motion patterns cor-
respond to different ranks of the generalized structure tensor
JN, see Table 2.
The purpose of our paper can be understood by analogy
with the case of only one motion. Obviously, in case of no
image structure, no motion can be determined. In case of 1D
spatial structure (e.g. straight edges) the motion is still not
defined and this is either solved by not estimating motion at
1D patterns or, in most cases, by estimating only a component
of the motion vector that is orthogonalto the orientation of the
1D spatial pattern. For more than one motion, we encounter
many more situations that are similar to the aperture problem
in the sense that not all motion parameters can be estimated.
This generalized aperture problem is therefore more complex.
1.1 Single Motion Estimation Using the Structure Ten-
sor
We first review a methodfor model selection based on sim-
ple confidence measures for the case of only a single motion.
This will make our extensions to multiple motions more com-
prehensible.
Submitted to International Journal of Computer & Information Science 2
The structure tensor method [7, 15] is a local approach
for the estimation of motion vectors. This method relies on
the assumption that the intensity or color of a point does not
change when the point moves [12]. This assumption leads to
the well known Constant Brightness Constraint Equation
vxfx+vyfy+ft= 0,(1)
where f(x, t)represents the image sequence and v=
(vx, vy)Tthe motion field. Eq. (1) does not fully constrain
vat a given position xand therefore vis estimated under
the assumption of being constant in a spatio-temporal region
Ω. This assumption is equivalent to the assumption that, for
(x, t)in Ω,the gradient of flies in a plane whose normal is
parallel to (vx, vy,1)T.Estimation is performed by looking
for a unitary vector n= (nx, ny, nt)Tthat best represents
the normal of such a plane in a least square sense, i.e., a min-
imal point of the functional
E1(n) = ZΩ
[∇f·n]2dΩ.(2)
Such a vector nis given, up to a scaling factor, by an eigen-
vector associated to the smallest, and ideally zero, eigenvalue
of the so-called structure tensor:
J1=ZΩ
f2
xfxfyfxft
fxfyf2
yfyft
fxftfyftf2
t
dΩ (3)
For dΩ(x, t) = ω(x, t) dxdt,J1can be simply computed as
J1=ω∗ ∇f⊗ ∇f=ω∗∇f∇fT(4)
Note that since E1is homogeneous, both nand −nare mini-
mal points of E1. Actually, λnminimizes E1when the argu-
ments of E1are vectors with norm λ. Therefore, we can think
of nas homogeneous coordinates for vand simply write
v= [vx, vy,1]T= [nx, ny, nt]T(5)
It follows that the estimation of n, and therefore v, is reliable
only if rank J1is two. Therefore the goodness of fit for the
estimator can be assessed based on the eigenvalues of J1.
Note however that even for the ideal case, rank J1= 2,the
vector ndoes represent motion only if nt6= 0.
An interesting property of the structure tensor is that, be-
sides allowing for motion estimation, it encodes a local de-
scription of the image sequence f(x, t).Under constant mo-
tion v,the sequence fcan be described by
f(x, t) = g(x−tv)(6)
within Ω. Therefore, a rank J1= 0 corresponds to the mo-
tion of regions with constant intensity (◦) and any motion vec-
tor is admissible in this region; rank J1= 1 corresponds to
the motion of a straight pattern (|), in this case admissible
motion vectors are constrained by a line; other moving pat-
terns (•) correspond to the rank J1= 2; and non-coherent
motion like noise, popping up objects, etc. correspond to
rank J1= 3.Table 1 summarizes these correspondences.
Moving Patterns rank J1
◦0
|1
•2
others 3
Table 1. Different moving patterns and the
ranks of the structure tensor: (◦) constant in-
tensity pattern; (|) 1D pattern; (•) 2D patterns.
2. The Generalized Structure Tensor
Our approach is based on the framework for estimating
multiple motions as introduced in [18] that we will briefly
summarize here. An image sequence consisting of two trans-
parent layers is modeled as
f(x, t) = g1(x−tu) + g2(x−tv),(7)
where u= (ux, uy)and v= (vx, vy)are the velocities of
the respective layers. In homogeneous coordinates, the basic
constraint equation is
cxxfxx +cxy fxy +cyy fyy +cxtfxt +cy tfyt +ctt ftt = 0,(8)
where c= (cij )Tis given by
cij =(ujvjif i=j
uivj+ujviotherwise. (9)
with ut=vt= 1.As in the single motion case, Eq. (8)
implies that the Hessian of flies in a hyperplane of a six-
dimensional space (the space of 3×3symmetric matrices)
whose normal is the symmetric matrix Cwith entries cij if
i=jand cij /2if i6=j. Proceeding in a way similar to the
single motion case, cis estimated as the eigenvector srelated
to the smallest eigenvalue of the tensor
J2=ZΩ
f2
xx fxxfxy ··· fxxftt
fxxfxy f2
xy ··· fxyftt
.
.
..
.
..
.
.
fxxftt fxy ftt ··· f2
tt
dΩ.(10)
or in short notation
J2=ω∗d2
f⊗d2
f=ω∗d2
fd2
fT,(11)
where d2
f= (fxx, fxy , fyy , fxt, fy t, ftt )T.Therefore a reli-
able estimation of cis possible only if rank J2= 5.Note
however that srepresents the motion vectors of a transparent
image sequence only if its last coordinate is different from
zero. This condition is necessary but not sufficient. A suffi-
cient condition for sto represent transparent motion is given
in Appendix A.
Submitted to International Journal of Computer & Information Science 3
Moving Pattern Projective Representation rank J1rank J2rank J3
◦the empty set 0 0 0
|a point 1 1 1
|+|2 points 2 2 2
|+|+|3 points 3 3 3
•a line 2 3 4
•+|a line + a point 3 4 5
•+|+|a line + 2 points 3 5 6
•+•2 lines 3 5 7
•+•+|2 lines + a point 3 6 8
•+•+•3 lines 3 6 9
others others 3 6 10
Table 2. Different motion patterns (first column) and the ranks of the generalized structure tensors for
1, 2, and 3 motions (table rows). This table summarizes our results by showing the correspondence
between the different motion patterns and the tensor ranks that can, in turn, be used to estimate the
confidence for a particular pattern, i.e., a proper motion model. Note that the rank of JNinduces a
natural order of complexity for patterns consisting of Nadditive layers.
The approach described above for two motions can be ex-
tended to estimate the motion fields of an additive superposi-
tion f(x, t)of Ntransparent image layers g1,...,gNmoving
with constant but different velocities v1,...,vN.
It is known [18] that fand the velocities are constrained
by
M
X
j=1
cIjfIj= 0 (12)
where fIj, j = 1,...,M =1
2(N+ 1)(N+ 2) are the inde-
pendent Nth-order partial derivatives of the image sequence
f, i.e., Ij= (ij1,...,ijN)is an ordered sequence with com-
ponents in {x, y, t}and fIjis the Nth-order partial derivative
of fwith respect to the components of Ij.The mixed motion
parameters cIare the symmetric function of the coordinates
of Vn=vn+et, for n= 1,...,N,and etis the time axis.
The generalized structure tensor for Nmotions is defined
by
JN=ZΩ
f2
I1fI1fI2··· fI1fIM
fI1fI2f2
I2··· fI2fIM
.
.
..
.
..
.
.
fI1fIMfI2fIM··· f2
IM
dΩ (13)
and can be written in short notation as
JN=ω∗dN
f⊗dN
f=ω∗dN
fdN
fT,(14)
where dN
f= (fI1, fI2, . . . , fIM)T.In this case, the vector
cN= (cI1, cI2,...,cIM)Tis a null eigenvector of JNand,
in practice, estimated as the eigenvector sNassociated to the
smallest eigenvalue of JN. The velocities are recovered from
sNby the method described in [18], which is analytical for
up to four motion layers. Obviously, the mixed-motion pa-
rameters cNcan be computed only if the null eigenvalue
is non-degenerated. In what follows, we will show which
transparent moving patterns correspond to other values of the
rank ofJN.As mentioned for two motions, the zero eigen-
value is not a sufficient condition for sNto actually represent
transparent motions. Cases where this does not happen will
be ignored in the following but discussed in the Appendix B.
In analogy to single motions, we will now analyze gener-
alized aperture problems as defined by the degree of degen-
eracy of the eigenvalues of JNand reflected in the ranks of
JN, see Table 2.
The problem of motion estimation has often been studied
in the Fourier domain and it is known that additive transpar-
ent moving patterns correspond to the additive superposition
of Dirac planes through the origin. In the Fourier domain,
Eq. (8, 12) correspond to homogeneous polynomials. The
study of homogeneous equations is greatly simplified by the
use of the projective plane. Therefore, we introduce a projec-
tive transform of fbelow.
Submitted to International Journal of Computer & Information Science 4
(a) (b) (c)
(d) (e) (f)
Figure 1. If two gratings of different orienta-
tions - as shown in (a) and (b) - are moved in
the directions shown in (c), the plaid pattern
shown in (d) is seen as moving in the direc-
tion indicated in (f) which corresponds to the
only coherent velocity that is defined by the in-
tersection of the projective lines as shown in
(e).
3. Representation of Multiple Motions in the Projective
Plane
Let Fbe the Fourier transform of fand ξ= (ξx, ξy, ξt)T
the Fourier variable, we define a projective transform of fby
Pf[ξx, ξy, ξt] = 1
ρZ+∞
−∞ |F(sξx, sξy, sξt)|ds, (15)
where ρ= (ξ2
x+ξ2
y+ξ2
t)1
2.Note that the right-hand side
of the above equation does not depend on the length of ξ.
Since planes and lines of an Euclidean space correspond to
lines and points of the projective plane (see Appendix B), this
transform allows us to think of the motion layers of f(x, t)as
points and lines of the projective plane. Besides the reduction
of dimension, the projective plane establishes a natural dual-
ity between lines and points that is not present in Euclidean
geometry. This is because a (projective) line `is exactly de-
scribed by an equation of the form
ax +by +cz = 0.(16)
Thus any line `corresponds to a dual point [a, b, c]and vice-
versa.
To illustrate the usefulness of the framework, we show
how to geometrically determine the velocity of a given 2D
moving pattern: the moving pattern is mapped to a plane in
the Fourier domain, from where it is further projected to the
projective plane where it is a Dirac line. Finally, the velocity
is found by applying the duality, here denoted with D, to the
Dirac line. The process is schematically shown below:
moving 2D pattern F
plane P
line D
velocity.
In the case of a moving 1D-pattern g(x) = ˜g(a·x), e.g. a
spatial grating, the Fourier transform reduces to a line, and its
projective transform to a point. The duality operation will de-
termine the set of admissible velocities for the grating which
is a line in the projective plane:
moving 1D pattern F
line P
point D
line of admissible
velocities.
As another example, we show how to determine the co-
herent motion of superimposed gratings (plaids) [1, 19]: the
set of admissible velocities for each layer is a line, the inter-
section of these two lines is the only admissible velocity for
both layers, that is, the coherent velocity for the plaid. Further
examples will be given in Section 4.
We summarize the main points below (for further details
see Appendix B):
•The projective transform of transparent motions is the
superposition of Dirac lines in the projective plane (in
case of moving 2D patterns).
•The dual point to each Dirac line in the projective plane
is the velocity of the respective layer.
•A moving 1D pattern corresponds to a Dirac point in the
projective plane. In this case any admissible velocity for
the grating is a point on the line that is dual to the Dirac
point in the projective plane.
•Dirac lines intersect at an ideal point if and only if the
corresponding patterns move in the same direction (with
different speeds).
•The ideal line corresponds to a static pattern.
The projective transform and its properties establish a one-
to-one correspondence between different motion patterns and
subsets of the projective plane (points and lines). Further-
more, these distinct configurations in the projective plane are
in a one-to-one correspondence to the rank of JN. Table 2
summarizes these correspondences and details of how these
correspondences have been established are given in the Ap-
pendix A. Further benefits of the projective-plane representa-
tion of motion will become evident in the next section.
4. Applications to Some Perceptual Phenomena
For the case of only one motion, the aperture problem has
a high significance for the visual perception of motion. As
argued before, the motion of a 1D pattern is ambiguous from
Submitted to International Journal of Computer & Information Science 5
(a) (b) (c)
(d) (e) (f)
Figure 2. Coherent motion of three superim-
posed gratings. To the superposition of two
gratings (a) a third grating shown in (b) is
added. The physical motions of the three grat-
ings are as shown in (c) and the lines of ad-
missible velocities for each grating in (e). The
percept is that of a coherent pattern as shown
in (d) moving in the direction indicated by the
arrow in (f). The coherent percept of one mo-
tion corresponds to the intersection of the lines
in only one point.
a theoretical point of view, and so are the percepts in the sense
that they depend on the motion of the so-called terminators,
i.e. the ends of the 1D patterns.
Similar effects appear with superimposed gratings that can
induce motion percepts that are different from the directions
orthogonal to the individual gratings. For example, two grat-
ings, one moving down and to the left, the other one mov-
ing down and to the right, are perceived as a single pattern
moving downwards under most experimental conditions - see
Fig. 2. On the other hand, three moving gratings can give rise
to three mutually exclusive percepts [1].
We are now going to explain these phenomena using our
theoretical framework presented above. We will also show
that our framework predicts an illusion for the superposition
of a grating with a random dot field and then give some ex-
perimental data for this illusion.
Finally, we will also give some data for the discrimination
of multiple motions. We will show that it is, in principle,
possible to distinguish between 2, 3, and 4 overlaid motions.
It seems that the limiting factor is not the number of motions
but rather the angular separation of motion vectors, which is,
in turn, related to the rank of Jn. Preliminary results have
(a) (b) (c)
(d) (e) (f)
Figure 3. Incoherent motion of three superim-
posed gratings. The sub-figures are according
to those in Fig. 2. However, the directions of
motions are now changed such that the lines of
motion in the projective plane do not intersect
in a single point (e). This makes the motions
undefined and causes the percept to change
dramatically such that a coherent motion is not
perceived. Observers can see either of the sin-
gle motions indicated in (f) (the other two mo-
tions are seen either individually or grouped to
a plaid motion).
been presented in [10].
4.1 Two 1D Transparent Moving Gratings
In the projective plane, two moving gratings correspond to
the {line, line}case - see Table 2. According to the theory, the
perceived motion should correspond to the intersection point
Uof the two lines and indeed it does - see Fig. 1.
4.2 Three 1D Transparent Moving Gratings
In the case of three moving gratings, a percept of one co-
herent pattern only arises when all three lines intersect in the
same point. This is, for example, the case for the configura-
tion shown in Fig. 2. On the other hand, a configuration as
shown in Fig. 3 has no unique percept: human observers see
the three 1D patterns as moving individually or see combina-
tions of one 1D pattern and a 2D plaid pattern.
Submitted to International Journal of Computer & Information Science 6
(a)
(b) (c)
Figure 4. Stimulus generation for the 2D-over-
1D entrainment (a). Admissible velocities for
the grating (line) and for the 2D stimulus (point)
are perceived as single motion (c).
4.3 Entrainment Effect for 2D Patterns Over 1D Pat-
terns
A spatial field of dots superimposed on a grating (Fig. 4)
corresponds to the {line, point}case. If the point falls on the
line, the grating should seem to move in coherence with the
random dots. To test this hypothesis, we generated sinusoidal
gratings of frequency ξ= 1/8, orientation ψ=kπ/4, k =
1, .., 8, and viewing angle size 10◦×10◦. These were trans-
lated perpendicular to their orientation (φg=ψ±π/2) with
a velocity of vg= 1.6◦/s. Mean brightness of the screen
was 10 cd/m2. Then, a 2D dot pattern with same brightness
distribution was overlaid to the grating and translated with
direction φr=φg±π/4and velocity vr=vg/√2, so that
one component of the motion vector always coincided in the
grating and the moving dot pattern. 15 of these stimuli were
presented to 7 human subjects for 1.6 seconds. After pre-
sentation of each stimulus, subjects had to rotate an arrow to
indicate the direction of the grating they had perceived. The
deviation of subjects’ responses from the true direction of the
grating is given in Fig. 7(a). If the dot pattern had exerted no
influence on the percept for the grating at all, a single peak
at 0◦could be expected. Analogously, a single peak at 45◦
would indicate that subjects always perceived a single coher-
ent pattern. Note that the small peak at 135◦actually depicts
cases of 45◦deviation since it can be attributed to the induced
motion phenomenon (the same effect that makes us see the
Figure 5. Example of a 1/f noise stimulus
platform moving while sitting in a moving train).
4.4 Entrainment Effect and the Barberpole Illusion
The shape of an aperture through which a grating is seen
can strongly influence motion perception. This phenomenon
is called the barberpole illusion. For example, the straight
lines in Fig. 6 seem to change their direction along their path
behind the aperture [26]: the bar moves as indicated by the
arrows and the perceived motion is indicated by the dashed
line.
To show that the entrainment effect is able to override
the barberpole illusion, we designed the stimuli illustrated
in Fig. 6. We masked the moving grating by an aperture
perpendicular to the orientation of the grating. This should
strengthen the percept of motion in a direction orthogonal to
the grating. As an additional modification, only the termina-
tors of the grating were overlaid with a random dot field that
moved in one coherent direction. Because this led to the rise
of new terminators at the boundary of the coherent random
dot field, the remaining middle of the stimulus was overlaid
with a white-noise pattern, which had the same density and
brightness as the coherent noise pattern. Nevertheless, the
entrainment effect seen in Fig. 7(b) is still qualitatively simi-
lar to that in Fig. 7(a) which shows that the effect dominates
over the influence of the aperture.
4.5 Discrimination of Multiple Transparent Motions
The perception of multiple overlaid motions has also been
investigated by counting the number of layers that a person
can discriminate and it has been argued that it is impossible
to discriminate more than two transparent motions [20]. To
analyze the nature of this apparent bottleneck we performed
the following experiments.
Stimuli consisted of 253 ms long, 10◦×10◦sized image
sequences made of either 2, 3, or 4 translated 1/f-noise pat-
Submitted to International Journal of Computer & Information Science 7
(a) (b)
Figure 6. Barberpole illusion (a). Stimulus with aperture orientation perpendicular to that of the 1D
grating, random noise in the center, and a random dot field moving coherently in the horizontal plane
(b).
terns that were overlaid. 1/f-noise images are characterized
by an hyperbolically-shaped spectrum, e.g. a high propor-
tion of low-frequency content and few high-frequency com-
ponents. This property has been chosen to resemble statistical
properties of natural images [2]. For an example, see Fig. 5.
Note that when several of these images are overlaid, this char-
acteristic is preserved; therefore, one cannot detect the num-
ber of overlaid motions from still images alone. These pat-
terns were translated with 12◦/s. The directional separation
of the motion vectors was 15,30, ..., 180◦respectively. Sub-
jects then had to indicate, in a 5 alternatives forced choice
paradigm, whether they had perceived 1, 2, 3, 4, or 5 motions.
In addition to the experiments described above, we performed
an additional set of experiments that differed only by the fact
that we used random dot patterns instead of the 1/f patterns.
Results can be seen in Fig. 8 and 9. Note that overall
the discriminability increases with the angular separation of
the motions. Also note, however, that the difficulty of the
discrimination task increases with the number of motions.
Nevertheless, three motions can be well discriminated with
sufficient angular separation. We therefore suggest that the
angular separation is the main limiting factor, which is, of
course, in turn limited by the number of motions. As shown
in Fig. 10, this effect can be predicted qualitatively in terms of
confidence measures based on the generalized structure ten-
sor. The confidence measure is obtained as the inverse slope
of the line fitted to the distribution of the logarithm of the
M−1largest eigenvalues of the generalized structure tensor.
The inverse slope values are shown normalized to the range
[0,1].Similar results would be obtained for the 1/f patterns.
5. Discussion
We have presented a method for categorizing transparent-
motion patterns in terms of the ranks of the generalized struc-
ture tensors. Based on our results, the confidence for a par-
ticular pattern can be evaluated computationally by either de-
termining the rank JNor by using the minors of the structure
tensors [18]. For example, we can discriminate the case of
two superimposed 1D patterns (moving plaid) and a 2D pat-
tern moving in the direction of the coherent motion of the
plaid pattern.
Our results can be seen as an extension of the concept of
intrinsic dimension [28, 5]. In the current framework, the in-
trinsic dimension corresponds to the rank of J1. As shown in
Table 2, by introducing the generalized structure tensor, we
can further differentiate the signal classes of a given (integer)
intrinsic dimension. In some sense, we thereby define frac-
tional intrinsic dimensions.
Although motion estimation is a key component of many
computer-vision and image processing systems, the motion
models are often too simple and fail with realistic data. Our
results provide (i) new means for increasing the complexity of
the motion models and (ii) measures for determining the con-
fidence for a particular model. We should note that the frame-
work can be applied to make explicit the correspondence be-
tween the ranks of JN, for a value of Nlarger than 3, and the
different moving patterns.
The theory presented in this work provides a conceptual
understanding of the difficulties in the estimation of multiple
transparent motions, which are due to the generalized aper-
ture problem. We have used the projective transform to es-
tablish a correspondence between the rank of the generalized
structure tensor and different transparent moving patterns.
Note, however, that the generalized structure tensor is de-
rived by integration of the derivatives of the image sequence
in a local neighborhood and as such can be used for the es-
timation of transparent motion in that neighborhood, includ-
ing situations were the motion vectors may vary over space.
For the estimation of Ntransparent motions, Nth-order par-
tial derivatives are involved. The ubiquitous presence of noise
can be compensated by prefiltering the sequences with proper
kernels, see [18]. Such prefiltering is equivalent to the use of
more general filters instead of the derivatives and can thus
break the unfavorable relationship between the order of dif-
ferentiation and the sensitivity to noise.
Submitted to International Journal of Computer & Information Science 8
0 50 100 150 200
Angular deviation
0
2
4
6
8
10
12
14
16
18
20
Frequency
Entrainment effect
(a)
0 50 100 150 200
Angular deviation
0
2
4
6
8
10
12
14
16
18
20
Frequency
Entrainment effect
(b)
Figure 7. Data illustrating the entrainment effect of a 2D pattern over a 1D grating. No aperture (a).
Aperture orientation perpendicular to that of the 1D grating (b).
Finally, we have also shown how our results can be used to
explain some phenomena in biological vision. In particular,
the concept of the projective plane proved useful for describ-
ing and visualizing different visual percepts. Furthermore, we
demonstrated new illusionary percepts that are in accordance
with the ambiguities that one would expect from the theory.
6. Acknowledgment
Work is supported by the Deutsche Forschungsgemein-
schaft under Ba 1176/7-2.
7. References
[1] E. H. Adelson and J. A. Movshon. Phenome-
nal coherence of moving visual patterns. Nature,
300(5892):523–5, 1982.
[2] R. M. Balboa and N. M. Grzywacz. Power spectra and
distribution of contrasts of natural images from differ-
ent habitats. Vision Research, 43(24):2527–37, 2003.
[3] J. L. Barron, D. J. Fleet, and S. S. Beauchemin. Perfor-
mance of optical flow techniques. International Jour-
nal of Computer Vision, 12(1):43–77, 1994.
[4] E. Barth, M. Dorr, I. Stuke, and C. Mota. Theory and
some data for up to four transparent motions. Percep-
tion, 30 (Supplement):36, 2001.
[5] E. Barth and A. B. Watson. A geomet-
ric framework for nonlinear visual cod-
ing. Optics Express, 7:155–85, 2000.
http://www.opticsexpress.org/oearchive/source/23045.htm.
[6] M. A. Bertero, T. Poggio, and V. Torre. Ill-posed prob-
lems in early vision. Proceedings of IEEE, 76(8):869–
89, 1988.
[7] J. Big ¨un, G. H. Granlund, and J. Wiklund. Multidimen-
sional orientation estimation with application to texture
analysis and optical flow. IEEE Trans. Pattern Analysis
and Machine Intelligence, 13(8):775–90, 1991.
[8] M. J. Black and P. Anandan. The robust estimation
of multiple motions: parametric and piecewise-smooth
flow fields. Computer Vision and Image Understand-
ing, 63(1):75–104, Jan. 1996.
[9] T. Darrell and E. Simoncelli. Nulling filters and the
separation of transparent motions. In IEEE Conf. Com-
puter Vision and Pattern Recognition, pages 738–9,
New York, June 14–17, 1993. IEEE Computer Press.
[10] M. Dorr, I. Stuke, C. Mota, and E. Barth. Mathe-
matical and perceptual analysis of multiple motions.
In H. H. B¨ulthoff, K. R. Gegenfurtner, H. A. Mallot,
and R. Ulrich, editors, TWK 2001: Beitr ¨
age zur 4.
T¨
ubinger Wahrnehmungskonferenz, page 173. Knirsch
Verlag, Kirchentellinsfurt, Germany, 2001.
Submitted to International Journal of Computer & Information Science 9
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 20 40 60 80 100 120 140 160 180
Classification rate
Angular separation
2 motions
-0.2
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100 120 140 160 180
Classification rate
Angular separation
3 motions
Figure 8. Discrimination of multiple motions with 1/f patterns.
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
0 20 40 60 80 100 120 140 160 180
Classification rate
Angular separation
2 motions
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 20 40 60 80 100 120 140 160 180
Classification rate
Angular separation
3 motions
Figure 9. Discrimination of multiple motions with random-dot patterns.
[11] H. Haußecker and H. Spies. Motion. In B. J ¨ahne,
H. Haußecker, and P. Geißler, editors, Handbook of
Computer Vision and Applications, volume 2, pages
309–96. Academic Press, 1999.
[12] B. Horn and B. Schunck. Determining optical flow.
Artificial Intelligence, 17(1–3):185–203, Aug. 1981.
[13] M. Irani, B. Rousso, and S. Peleg. Computing occlud-
ing and transparent motions. International Journal of
Computer Vision, 12(1):5–16, Feb. 1994.
[14] K. Langley. Computational models of coherent and
transparent plaid motion. Vision Research, 39:87–108,
1999.
[15] B. Lucas and T. Kanade. An iterative image registra-
tion technique with an application to stereo vision. In
Proc. DARPA Image Understanding Workshop, pages
121–30, 1981.
[16] C. Mota, M. Dorr, I. Stuke, and E. Barth. Categoriza-
tion of transparent-motion patterns using the projective
plane. In W. Dosch and R. Y. Lee, editors, Proc. ACIS
4th Int. Conf. Software Engineering, Artificial Intelli-
gence, Networking and Parallel/Distributed Comput-
ing, pages 633–9, L ¨ubeck, Germany, Oct. 16–18, 2003.
[17] C. Mota, M. Dorr, I. Stuke, and E. Barth. Analysis and
synthesis of motion patterns using the projective plane.
In Human Vision and Electronic Imaging Conference.
16th Symposium on Electronic Imaging Science and
Technology, San Jose, CA, Jan. 19–22, 2004. To ap-
pear.
[18] C. Mota, I. Stuke, and E. Barth. Analytic solutions
for multiple motions. In Proc. IEEE Int. Conf. Image
Processing, volume II, pages 917–20, Thessaloniki,
Greece, Oct. 7–10, 2001. IEEE Signal Processing Soc.
[19] J. A. Movshon, E. H. Adelson, M. S. Gizzi, and
W. T. Newsome. The analysis of moving visual pat-
terns. In C. Chagas, R. Gattas, and C. Gross, editors,
Study group on pattern recognition mechanisms, vol-
ume 54, pages 117–151. Pontificiae Academiae Scien-
tiarum Scripta Varia, Vatican City, 1985.
Submitted to International Journal of Computer & Information Science 10
0 20 40 60 80 100 120 140 160 180
0
0.2
0.4
0.6
0.8
1
2 motions
Angular Separation
Confidence
0 20 40 60 80 100 120
0
0.2
0.4
0.6
0.8
1
3 motions
Angular Separation
Confidence
Figure 10. Simulation results obtained for the same patterns as the experimental results shown in
Fig. 9.
[20] J. B. Mulligan. Motion transparency is restricted to two
planes. Inv Ophth Vis Sci (suppl), 33:1049, 1992.
[21] M. Shizawa and K. Mase. Simultaneous multiple op-
tical flow estimation. In IEEE Conf. Computer Vision
and Pattern Recognition, volume I, pages 274–8, At-
lantic City, NJ, June 1990. IEEE Computer Press.
[22] M. Shizawa and K. Mase. A unified computational
theory for motion transparency and motion boundaries
based on eigenenergy analysis. In IEEE Conf. Com-
puter Vision and Pattern Recognition, pages 289–95,
Maui, HI, June 1991. IEEE Computer Press.
[23] I. Stuke, T. Aach, E. Barth, and C. Mota. Estimation
of multiple motions using block-matching and markov
random fields. In S. Panchanathan and B. Vasudev,
editors, Visual Communications and Image Processing
2004, IS&T/SPIE 16th Annual Symposium Electronic
Imaging, San Jose, California, Jan. 18–22, 2004. To
appear.
[24] I. Stuke, T. Aach, C. Mota, and E. Barth. Estimation of
multiple motions: regularization and performance eval-
uation. In B. Vasudev, T. R. Hsing, A. G. Tescher, and
T. Ebrahimi, editors, Image and Video Communications
and Processing 2003, volume 5022 of Proceedings of
SPIE, pages 75–86, May 2003.
[25] D. Vernon. Decoupling Fourier components of dy-
namic image sequences: a theory of signal separation,
image segmentation and optical flow estimation. In
H. Burkhardt and B. Neumann, editors, Computer Vi-
sion - ECCV’98, volume 1407/II of LNCS, pages 68–
85. Springer Verlag, Jan. 1998.
[26] S. Wuerger, R. Shapley, and N. Rubin. On the visu-
ally perceived direction of motion by Hans Wallach: 60
years later. Perception, 25:1317–67, 1996.
[27] W. Yu, K. Daniilidis, S. Beauchemin, and G. Som-
mer. Detection and characterization of multiple motion
points. In IEEE Conf. Computer Vision and Pattern
Recognition, volume I, pages 171–7, Fort Collins, CO,
June 23–25, 1999. IEEE Computer Press.
[28] C. Zetzsche and E. Barth. Fundamental limits of lin-
ear filters in the visual processing of two-dimensional
signals. Vision Research, 30:1111–7, 1990.
A. The Null-Eigenvector of JNand the Motion Vectors
The null-eigenvector of JNrepresents Ntransparent-
motion velocities if it is possible to solve the vectorial equa-
tion
c(v1, . . . , vN) = sN(17)
for v1,...,vN.In the case of two motions, this equation can
be written in matrix form as
S2=stt
2(ux, uy,1)T(vx, vy,1)
+ (vx, vy,1)T(ux, uy,1),(18)
where Sis the matrix with entries sij if i=jand sij /2if
i6=j. Therefore, s2represents two transparent motions if
and only if
det S2= 0
det S11
2+ det S22
2+ det S22
2<0,
stt 6= 0
(19)
Submitted to International Journal of Computer & Information Science 11
where det Sjj are the diagonal minors of S,e.g., det S33 =
sxxsyy −s2
xy/4.Eq. (18) can be easily extended to more than
two motions but we could not find any expressions for the
resulting SNanalogous to those in Eq. (19). Nevertheless,
it has been shown in [18] that the solutions of Eq. (17) can
be expressed as the roots of a complex polynomial whose co-
efficients are explicitly given in terms of sn.If vc
1,...,vc
N
are the roots of this polynomial, a necessary and sufficient
condition for sNto represent transparent-motion vectors is
therefore
sN=c(vc
1,...,vc
N).(20)
B. The Projective Plane
The projective plane is the set of all directions in the
three-dimensional Euclidean space. These directions (the
points of the projective plane) can be represented by homo-
geneous coordinates [x, y, z ].A pair of homogeneous coor-
dinates [x, y, z],[x0, y0, z0]represents the same point if and
only if (x, y, z ) = λ(x0, y0, z0)for some non-zero factor λ. A
point with coordinates [x, y , 0] is called an ideal point and the
set of ideal points is called the ideal line.
A point (x, y)in a Euclidean plane corresponds naturally
to a projective point by the identification (x, y) = [x, y, 1].
Therefore, we can think of the projective plane as the union
of the plane z= 1 and the ideal line.
Relevant Properties of the Projective Plane
Below we summarize the properties of the projective plane
that are useful for our analysis of moving patterns:
•Dimension reduction: lines and points of the projective
plane correspond to planes and lines through the origin
of the three-dimensional space respectively;
•Duality: each line `of the projective plane is associated
to a dual point Vby the corresponding orthogonality
of planes and lines in the three-dimensional Euclidean
space and vice-versa;
•No parallelism: any two lines of the projective plane do
intersect;
•Two projective lines intersect at an ideal point if and only
if their dual points and etare aligned.
C. The Rank of JN
From the discussion in Section 3, we have seen that the set
of admissible velocities of a moving layer gis the dual space
to the support of PG. This dual set is called the phase space
for the velocities of g. In what follows, we will suppose that
no pair of layers forming fmoves with collinear velocities
and none of the layers is static. This means that the lines
supporting two non-degenerated Dirac lines always intercept
at a finite (non-ideal) point.
U
V
W
(a)
U
WV
W
~
(b)
Figure 11. Admissible velocities of overlaid-
motions patterns in the projective plane: (a)
two overlaid 1D patterns, Uis the coherent
velocity, c(u,u),c(u,v),c(u,w),c(v,w)are in-
dependent null-eigenvectors of J2; (b) same
for one 1D pattern and two 2D patterns,
c(u,v,w)and c(u,v,˜
w)are independent null-
eigenvectors of J3.
The mixed-motion parameters vectors cN=c(v1, . . . ,
vN)can be interpreted as elements of the space of sym-
metric N-tensors (here denoted by SN). Therefore, if
β={U,V,W}is a basis for the three-dimensional
Euclidean space, the set {c(v1,...,vN) : Vn∈β,
for n= 1,...,N}is a basis for SN.For example,
{c(u,u),c(u,v),c(u,w),c(v,v),c(v,w),c(w,w)}is a
basis for S2. We will use this relationship between basis of
R3and SNto construct a maximal number of elements in the
kernel of J2and J3. By ‘kernel of JN’ we denote the set of
vectors that correspond to the zero eigenvalues of JN.
The Rank of J2
For two moving layers, the non-trivial possibilities for the
phase space of the velocities are a {line,line},{point, line},
{point, point}.
line, line: Choose a basis β={U,V,W}of R3such that
Uis the intersection of the two lines, and Vand Wbelong
to each of these lines, see Fig. 11(a). Now it is clear that
c(u,u),c(u,v),c(u,w)and c(v,w)are elements in the
kernel of J2. Since these vectors are linearly independent,
we can conclude that rank(J2)≤2.
line, point: Choose Uas the point and V,Win the line.
The vectors c(u,v),c(u,w)are null-eigenvectors of J2and
therefore rank(J2)≤4.
point, point: Choose U,Vas the two points and Wfreely.
The only element in the kernel of J2is c(u,v), therefore
rank(J2)≤5.
We found the above bounds to the rank(J2)given two
moving patterns. Since it is possible to reach these bounds,
Submitted to International Journal of Computer & Information Science 12
they are actually tight. Note that two moving patterns do not
produce rank 1 or 3. These ranks are actually produced by a
single moving object. The phase space for the two velocities,
in this case, is {line, plane}or {point, plane}. We analyze the
first case below, the other is similar.
line, plane: Choose U,Vas points in the line and Wout
of it. The only element that does not belong to the kernel of
J2is c(w,w)and therefore rank(J2) = 1.
The Rank of J3
For three moving patterns, the non-trivial possibilities for
the phase spaces of the velocities are a {line, line, line},
{point, line, line},{point, point, line}, and {point, point,
point}which correspond to the values 3, 6, 8, and 9 of the
rank of J3. Since the analyses of these cases are very similar,
we consider only the two last cases.
point, point, line: Choose U,Vas the points and Win
the line, see Fig. 11(b). In principle it appears that only the
element c(u,v,w)belongs to the kernel of J3. Also note
that any two lines intersect in the projective plane. Let ˜
W
be the intersection of the given line with the line determined
by Uand V. Now, if we assure that Wdoes not coincide
with ˜
W, we find the second independent symmetric tensor
in the kernel of J3, that is, c(u,v,˜
w).We conclude that
rank(J3)≤8. Since these are all the possibilities, except
maybe for degenerate cases, the bound 8is tight.
point, point, point: Choose U,V,Was these points.
Only c(u,v,w)belongs to the kernel of J3. Hence,
rank(J3) = 9 except for degenerate cases.
Similar to the case J2, three moving patterns do not fill
all the possibilities for the rank of J3. The gaps are filled by
single or two moving patterns. These correspond to ranks 1,4
and 2,5,7 respectively. Table 2 summarizes the possibilities
for the ranks of JNfor N= 1,2,3.
Cicero Mota is on leave from
University of Amazonas, Brazil
and is currently with the Insti-
tute for Neuro- and Bioinformat-
ics, University of Luebeck, Ger-
many. His research interests are in
computer vision and information
theory. Dr. Mota studied mathe-
matics at the University of Ama-
zonas and at the Institute for Pure
and Applied Mathematics – IMPA
(Ph.D. 1999) in Brazil. He has
been with the Institute for Signal Processing, University of
Luebeck (visiting scientist) and with the Institute for Applied
Physics, University of Frankfurt, both in Germany. His re-
search has been supported by the Brazilian agencies CNPq
and CAPES, and by the German agencies DAAD and DFG.
Michael Dorr is currently work-
ing towards his degree in Com-
puter Science, Bioinformatics,
and Biomathematics at the Uni-
versity of Luebeck, Germany. His
research interests are the psy-
chophysics of motion estimation
and the modeling of eye move-
ments and attention. In 2001,
he has been awarded the ”Student
Poster Prize” at the Tuebingen
Perception Conference for [10].
Ingo Stuke is currently a
scientist at the Institute for
Signal Processing, Univer-
sity of Luebeck, Germany.
His main research interests
are on the field of motion
and orientation estimation
for image processing and
computer vision applica-
tions. From 1995 to 2001 he
studied computer science at
the University of Luebeck.
Erhardt Barth is currently a sci-
entist at the Institute for Neuro-
and Bioinformatics, University of
Luebeck, Germany. His main
interests are in the areas of hu-
man and computer vision. He
studied electrical and communi-
cations engineering at the Univer-
sity of Brasov and at the Techni-
cal University of Munich (Ph.D.
1994). He has been with the Uni-
versities of Melbourne and Mu-
nich, the Institute for Advanced Study in Berlin, the NASA
Vision Science and Technology Group in California, and the
Institute for Signal Processing in Luebeck. In May 2000 he
received a Schloessmann Award.
