Wavelet Expansion and Highorder Regularization for Multiscale Fluidmotion Estimation
ABSTRACT We consider a novel optic flow estimation algorithm based on a wavelet expansion of the velocity field. In particular, we propose an efficient gradientbased estimation algorithm which naturally encompasses the estimation process into a multiresolution framework while avoiding most of the drawbacks common to this kind of hierarchical methods. We then emphasize that the proposed methodology is wellsuited to the practical implementation of highorder regularizations. The powerfulness of the proposed algorithm and regularization schemes are finally assessed by simulation results on challenging image sequence of turbulent fluids.
 Citations (9)
 Cited In (0)
 Publicacions Matemàtiques; Vol.: 35 Núm.: 1 Conf. Mathematical Analysis. José Luis Rubio.
 SourceAvailable from: Dominique Heitz
Conference Paper: Bayesian selection of scaling laws for motion modeling in images
[Show abstract] [Hide abstract]
ABSTRACT: Based on scaling laws describing the statistical structure of turbulent motion across scales, we propose a multiscale and nonparametric regularizer for opticflow estimation. Regularization is achieved by constraining motion increments to behave through scales as the most likely selfsimilar process given some image data. In a first level of inference, the hard constrained minimization problem is optimally solved by taking advantage of lagrangian duality. It results in a collection of firstorder regularizers acting at different scales. This estimation is nonparametric since the optimal regularization parameters at the different scales are obtained by solving the dual problem. In a second level of inference, the most likely selfsimilar model given the data is optimally selected by maximization of Bayesian evidence. The motion estimator accuracy is first evaluated on a synthetic image sequence of simulated bidimensional turbulence and then on a real meteorological image sequence. Results obtained with the proposed physical based approach exceeds the best state of the art results. Furthermore, selecting from images the most evident multiscale motion model enables the recovery of physical quantities, which are of major interest for turbulence characterization.Computer Vision, 2009 IEEE 12th International Conference on; 11/2009  [Show abstract] [Hide abstract]
ABSTRACT: An image registration algorithm is developed to estimate dense motion vectors between two images using the coarsetofine waveletbased motion model. This motion model is described by a linear combination of hierarchical basis functions proposed by Cai and Wang (SIAM Numer. Anal., 33(3):937–970, 1996). The coarserscale basis function has larger support while the finerscale basis function has smaller support. With these variable supports in full resolution, the basis functions serve as largetosmall windows so that the global and local information can be incorporated concurrently for image matching, especially for recovering motion vectors containing large displacements. To evaluate the accuracy of the waveletbased method, two sets of test images were experimented using both the waveletbased method and a leading pyramid splinebased method by Szeliski et al. (International Journal of Computer Vision, 22(3):199–218, 1996). One set of test images, taken from Barron et al. (International Journal of Computer Vision, 12:43–77, 1994), contains small displacements. The other set exhibits low texture or spatial aliasing after image blurring and contains large displacements. The experimental results showed that our waveletbased method produced better motion estimates with error distributions having a smaller mean and smaller standard deviation.International Journal of Computer Vision 01/2000; 38:129152. · 3.62 Impact Factor
Page 1
apport ?
?
de recherche?
ISSN 02496399
ISRN INRIA/RR????FR+ENG
Observation and Modeling for Environmental Sciences
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
Wavelet Expansion and Highorder Regularization
for Multiscale Fluidmotion Estimation
Pierre Dérian — Patrick Héas — Cédric Herzet — Étienne Mémin
N° ????
July 2010
inria00505780, version 1  26 Jul 2010
Page 2
inria00505780, version 1  26 Jul 2010
Page 3
Centre de recherche INRIA Rennes – Bretagne Atlantique
IRISA, Campus universitaire de Beaulieu, 35042 Rennes Cedex
Téléphone : +33 2 99 84 71 00 — Télécopie : +33 2 99 84 71 71
Wavelet Expansion and Highorder
Regularization
for Multiscale Fluidmotion Estimation
Pierre Dérian , Patrick Héas , Cédric Herzet , Étienne Mémin
Theme : Observation and Modeling for Environmental Sciences
ÉquipeProjet Fluminance
Rapport de recherche n° ???? — July 2010 — 19 pages
Abstract:
a wavelet expansion of the velocity field. In particular, we propose an effi
cient gradientbased estimation algorithm which naturally encompasses the es
timation process into a multiresolution framework while avoiding most of the
drawbacks common to this kind of hierarchical methods. We then emphasize
that the proposed methodology is wellsuited to the practical implementation
of highorder regularizations. The powerfulness of the proposed algorithm and
regularization schemes are finally assessed by simulation results on challenging
image sequence of turbulent fluids.
We consider a novel optic flow estimation algorithm based on
Keywords:
optic flow, wavelets, motion regularity, fluid flows, turbulence
inria00505780, version 1  26 Jul 2010
Page 4
Formulation en ondelettes et régularisation
d’ordre élevé pour l’estimation multiéchelles de
mouvements de fluides.
Résumé :
basé sur une décomposition en ondelettes du champ de vitesse. Cet algorithme
inscrit le processus d’estimation dans un cadre multirésolution, en évitant la
plupart des écueils qui accompagnent généralement ce type d’approches séquen
tielles. Cette approche permet en outre l’implémentation aisée de régularisations
d’ordre élevé. Les performances de l’algorithme d’estimation et des régularisa
tions introduits sont évaluées sur des séquences d’images de fluides turbulents.
Un algorithme original d’estimation du flux optique est introduit,
Motsclés :
turbulence
flux optique, ondelettes, estimation de mouvement, fluide,
inria00505780, version 1  26 Jul 2010
Page 5
Wavelet Expansion and Regularization for Fluidmotion Estimation
3
1Introduction
Optic flow estimation consists in recovering the apparent motion of a scene
through spatial and temporal variations of an image intensity. This estimation
process usually requires to solve complex nonlinear and underdetermined in
verse problems. Therefore, since the seminal work of Horn and Schunck [6],
numerous approaches have been proposed to address these issues.
A standard procedure to deal with the nonlinear nature of the problem is
to resort to multiresolution strategies [2]. These approaches consist in solving
the optic flow estimation problem by considering a sequence of coarsetofine
linearized subproblems. Although leading to good empirical results, this tech
nique has nevertheless a number of drawbacks. First, the estimates computed
at coarse levels can never be reevaluated at finer scales. Moreover, the sub
problems considered at each level follow from a “adhoc" filtering of the image
sequence. In particular, there is usually no obvious connections between the
data processed at each level. Finally, standard multiresolution techniques rely
on linear approximation of the initial problem. Now, the validity of such ap
proximation can be limited in case of large displacements.
The undertermined nature of the optic flow estimation problem is usually
referred to as “aperture problem". Resolving the underdetermination imposes to
add some prior information about the sought motion field. In many contribu
tions dealing with rigidmotion estimation, firstorder regularization (enforcing
the spatial gradient of the velocity to be weak) is considered with success. How
ever, when tackling more challenging problems such as motion estimation of
turbulent fluids, this simple prior turns out to be inadequate. Instead, higher
order regularizers allowing to enforce physicallysound constraints have to be
considered [4, 12, 5]. Unfortunately, current implementations of such regulariz
ers suffer from instabilities and turn out to be a difficult problem.
In this paper, we propose an optic flow estimation procedure based on a
wavelet expansion of the velocity field. This approach turns out to offer a nice
mathematical framework for multiresolution estimation algorithms. In particu
lar, we emphasize that estimating wavelet coefficients in a sequential way reduces
to a multiresolution estimation algorithm which avoids the common drawbacks
mentioned above. We propose an efficient implementation of the corresponding
optic flow estimation problem whose complexity scales linearly with the problem
dimensions. Note that another algorithm based on wavelet expansion has been
previously proposed in [11]. However, unlike the algorithm presented hereafter,
its complexity is of intractable polynomial order.
Moreover, we consider the effective implementation of highorder regulariza
tion schemes. In particular, we emphasize that enforcing highorder regulariza
tion on the velocity field is equivalent to imposing very simple constraints on the
wavelet coefficients at proper scales. Based on this observation, we elaborate
three possible implementations of highorder regularization scheme and assess
their relevance on challenging image sequence of turbulent fluid motions. Simu
lation results prove that the proposed approach outperforms the most effective
stateoftheart algorithms while having a lower complexity order.
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 6
Wavelet Expansion and Regularization for Fluidmotion Estimation
4
2 Introduction to optic flow
2.1Aperture problem
Optic flow estimation is a wellknown difficult illposed inverse problem. It
consists in estimating the apparent motion of a 3D scene through image intensity
I(x,t) variations in space x = (x1,x2) ∈ Ω ⊂ R2and time t ∈ R. The optic
flow, identified by a 2D velocity field v(x,t) : Ω×R+?→ R2is the projection on
the image plane of the 3D scene velocity. Under rigid motion and stable lighting
conditions, v = (u,v)Tsatisfies the standard Displaced Frame Difference (DFD)
equation. Let us denote by I0(x) and I1(x) two consecutive image samples of
the continuous sequence I(x,t) which has been discretized in time with a unit
interval. The DFD equation reads:
fd(I,v) = I1(x + v(x)) − I0(x) = 0.
(1)
A linearized version of model (1), socalled the motion compensate Optic Flow
Constraint (OFC) equation, is obtained by linearization of I1around x+˜ v(x)1:
˜fd(I,v) = I1(x + ˜ v(x)) − I0(x) + ∇I1(x + ˜ v(x)) · v?= 0.
where v?= v − ˜ v.
For other configurations, many other brightness evolution models have been
proposed in the literature to link the image intensity function to the sought
velocity fields [8]. In the sequel, we will sometimes use the generic notation
M(I,v) to denote the model linking images and their underlying velocity field.
However, all these evolution models remain underconstrained, as they pro
vide for each time t only one equation for two unknowns (u,v) at each spatial
location x = (x1,x2)T. To deal with this underconstrained estimation problem,
so called aperture problem, the most common setting consists in enforcing some
spatial coherence to the solution.
(2)
2.2Regularization schemes
This coherence is imposed either: i) explicitly, by constraining the motion field
to be of the form v = Φ(Θ), where Φ is function parameterized by Θ (piece
wise polynomial functions are often used); ii) globally, through a regularization
functional defined over the whole image domain. We briefly describe these two
approaches hereafter.
Explicit regularization schemes penalize discrepancies from model (1) by
minimizing an “energy" with respect to Θ, i.e.
?
where
Jobs(I,Φ(Θ))=1
2
Ω
If model M is linear, motion estimate ˆ v is simply obtained by solving a
lowdimensional system of linear equations. Alternatively to quadratic penal
ization, robust functions (socalled Mestimators) can be used to penalize model
ˆ v = Φargmin
Θ
Jobs(I,Φ(Θ))
?
,
(3)
?
[M(I,Φ(Θ))]2dx.
(4)
1The current motion estimate is usually used to define ˜ v(x).
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 7
Wavelet Expansion and Regularization for Fluidmotion Estimation
5
discrepancies. This yields to semiquadratic functionals which preserve locally
convex properties [3]. For clarity of the presentation, we will however restrict
ourselves to quadratic penalty functions in the following.
Global regularization schemes in their simplest form define the estimation
problem through the minimization of a functional composed of two terms bal
anced by a regularization coefficient γ > 0:
J(I,v,γ) = Jobs(I,v) + γJreg(v).
(5)
Thus, motion estimate ˆ v satisfies ˆ v = argminvJ(v,I,γ). The first term, Jobs
(the “data term”) is defined by (4) with v = Φ(Θ) = Θ. The second term,
Jreg (the “regularization term”), encourages the solution to follow some prior
smoothness model formalized with function fr:
?
where .2denotes the L2norm. An norder regularization writes in its simplest
form:
fr(u,x) =∂nu(x)
∂xn
A firstorder regularizer (i.e. n=1) enforcing weak spatial gradients of the two
components u and v of the velocity field v is very often used [6].
order regularizers (i.e. n > 1) have been proposed in the literature in the
case of fluid flows [4, 12]. However, since motion variables are considered on
the pixel grid, an approximation of continuous spatial derivatives by discrete
operators is required. For regular pixel grids, it is usually done using finite
difference schemes. Nevertheless, it is well known that ensuring stability of the
discretization schemes of highorder regularizer constitutes a difficult problem.
Jreg(v)=
1
2
Ω
fr(u,x)2+fr(v,x)2dx
(6)
, fr(v,x) =∂nv(x)
∂xn
.
(7)
Higher
2.3Common multiresolution strategy
A major problem with differential models such as (2) is the estimation of ve
locities for large displacements in the images. Indeed, the equations for the
inversion are only valid if the solution remains in the region of linearity of the
image intensity function. A standard approach for tackling nonlinearity is to
rely on a multiresolution strategy [2]. This approach consists in choosing some
sufficiently coarse resolution in order to make the linearity assumption valid,
and to estimate a first displacement field with a lowpass version of the original
images. Then, a socalled GaussNewton strategy is used by applying successive
linearizations around current estimate and warping a multiresolution represen
tation of the images accordingly.
More explicitly, let us introduce the following incremental decomposition of
the displacement field at scale 2j:
vj= ˜ vj+ v?
j,
(8)
where the field v?
scale 2jand ˜ vj ??
grid considered at scale 2j. In order to respect the Shannon sampling theorem,
jrepresents the unknown incremental displacement field at
i<jPj(v?
previous scales; Pj(v?
i) is a coarse motion estimate computed at the
i) denotes a projection operator which projects v?
ionto the
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 8
Wavelet Expansion and Regularization for Fluidmotion Estimation
6
the coarse scale data term is derived by a lowpass filtering with a kernel2Gjof
the original images following by subsampling at period 2j. Using (8), the coarse
scale image Ij(x) and the motioncompensated image˜Ij(x) are then defined as:
?
where ↓2j denotes an operator subsampling the filter output with a period 2j.
It yields a functional Jj
?
Finally, the sought motion estimate ˆ v = v?
by solving a system of coupled equations associated to scales s ∈ [2C,2F]:
v?
where the finest scale s = 2Fcorresponds to the pixel resolution and the coarsest
scale is noted s = 2C.
In pratice, equations in (11) are usually solved independently from the coars
est to the finest scales. This approach has the drawback of freezing (i.e. leaving
unchanged), at a given scale, all the previous coarser estimates. Moreover, the
major weakness of this strategy is the arbitrary approximation of the original
functional (4) by a set of coarse scale data terms (10), which are defined at
different scales by a modification of the original input images with (9) and by a
linearization of model (1) around the previous motion estimate.
In the next section, we will see that this multiresolution strategy has a
mathematicallysound formulation within the framework of wavelet representa
tions.
Ij(x) =↓2j ◦ (Gj? I0(x))
˜Ij(x) =↓2j ◦ (Gj? I1(x + ˜ vj(x))),
(9)
obsdefined as a linearized version of (1) around ˜ vj(x):
C)=1
2
Ωj
Jj
obs(Ij,v?,v?
j−1,...,v?
?˜Ij(x)−Ij(x) +v?(x) · ∇˜Ij(x)
F+ ˜ vF= v?
?2
dx.
(10)
F+?
i<FPF(v?
i) is given
v?
C= argmin
v?
JC
obs(IC,v?)
.
v?
j= argmin
v?
Jj
obs(Ij,v?,v?
j−1,...,v?
C)
.
v?
F= argmin
JF
obs(IF,v?,v?
F−1,...,v?
C),
(11)
3Fast multiscale motion estimation on wavelet
bases
To estimate motion structures of very different sizes minimizing the functional
(4), it is necessary to use a proper scalespace representation. The wavelet
transform provides a consistent multiresolution representation by motion de
composition on a basis of scalespace atoms.
3.1
Let φ ∈ L2(R) be a scaling function and let ψ ∈ L2(R) be its associated wavelet.
Then, the set of functions
Wavelet decomposition of optic flow
{φk(x),ψ1
j,k(x),ψ2
j,k(x),ψ3
j,k(x)},k ∈ N2,j ∈ N+
0,
(12)
2A Gaussian kernel of variance proportional to 2jis commonly used.
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 9
Wavelet Expansion and Regularization for Fluidmotion Estimation
7
where
φk(x)
ψ1
ψ2
ψ3
? φ(x1− k1)φ(x2− k2),
? ψ(x12−j− k1)φ(x22−j− k2),
? φ(x12−j− k1)ψ(x22−j− k2),
? ψ(x12−j− k1)ψ(x22−j− k2),
j,k(x)
j,k(x)
j,k(x)
(13)
forms an orthogonal basis of L2(R2) [9]. We define the following subsets of
L2(R2):
Vj? {f(x) ∈ L2(R2)  f(x) =
?
k
θ0
kφk(x) +
3
?
i=1
j
?
l=1
?
k
θi
k,lψi
l,k(x);
?
i,l,k
(θi
k,l)2< ∞}.
(14)
These sets representing approximation space at scale 2jwill prove to be useful
in the interpretation of the proposed method in section 4.
We consider the decomposition of the components of the motion vector
v(x) = [u(x)v(x)]Ton basis (12). The projection coefficients of u(x) writes
?(θu)0
where ?·,·? denotes the inner product in L2(R). Therefore, u(x) can be expressed
as
?
A similar decomposition can be consider for v(x). These decompositions then
lead to two sets of coefficients associated to the two motion components u and
v, namely
?Θu= {((θu)0
C,k=
j,k=
?u(x),φC,k(x)?, ∀k ∈ N2
?u(x),ψi
(θu)i
j,k(x)?, ∀k ∈ N2,i ∈ {1,2,3},j ∈ [F,C] ⊂ N,
(15)
u(x) =
k
(θu)0
C,kφC,k(x) +
F
?
j=C
?
i,k
(θu)i
j,kψi
j,k(x).
(16)
C,k,(θu)i
C,k,(θv)i
j,k)Ti ∈ {1,2,3}, j ∈ [F,C],k ∈ [0..2j− 1]}
j,k)Ti ∈ {1,2,3}, j ∈ [F,C],k ∈ [0..2j− 1]}.
Θv= {((θv)0
(17)
We denote by Θ = [Θu,Θv]Tthe set of all coefficients, so that (16) can be
rewritten as:
?φC,k(x), ··· , ψi
= Φ(x)Θ.
v(x) =
F,k(x)


0, ··· , 0
0, ··· , 0
φC,k(x), ··· , ψi
F,k(x)
?
Θ,
(18)
3.2Multiscale estimation of optic flow
In this section, we emphasize that a multiresolution estimation algorithm natu
rally arises when considering sequential optimizations of wavelet coefficients Θ.
Hereafter, we present two different algorithms which estimate coefficients Θ by
minimizing functional (4).
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 10
Wavelet Expansion and Regularization for Fluidmotion Estimation
8
3.2.1 Hessianbased algorithm.
An algorithm, based on the calculation of the Hessian of functional (4), has been
proposed in [11] to solve this problem with the motion compensate OFC model
(2). Using the linear decomposition (18) on v = ˜ v+v?yields ΦΘ = Φ˜Θ+ΦΘ?
and:
Jobs(Θ?) ? Jobs(I,Φ˜Θ + ΦΘ?),
=1
2
Ω
=1
2Θ?TAΘ?− Θ?Tb,
?
(ΦΘ?)TggT(ΦΘ?) + 2(I1(x + Φ˜Θ) − I0(x))(ΦΘ?)Tg dx
(19)
where:
• g(x) = ∇I1(x + Φ˜Θ), i.e. the gradient of the motion compensated image
˜I1;
?
• A =1
• b =?
Minimizing the data term in Θ?, we search an increment satisfyingˆΘ
˜Θ = argminJobs(Θ?). As the energy is quadratic, the minimum of the convex
functional linearized around˜Θ is reached at the point where gradient vanish.
Therefore, the solution reads:
2
ΩΦT(x)g(x)gT(x)Φ(x)dx;
Ω(I0(x) − I1(x + Φ˜Θ))g(x)TΦ(x)dx.
?=ˆΘ −
ˆΘ
?= A−1b
(20)
However, the algorithm complexity is very high since the Hessian calculation is
very demanding: O(KN5) for a signal of size N = 2−Cand a conjugate mirror
filter (associated to the wavelet) of size K. This limits the use of this algorithm
to coarse scales 2F? 20and to wavelets of very limited support.
3.2.2Fast gradientbased algorithm.
We propose a lowcomplexity algorithm based on the calculation of the gra
dient of functional (4) by two independent wavelet transforms. Conversely to
the previous algorithm, this one minimizes the original (nonlinearized) model
M = fddefined in (1). More explicitly, since the functional is convex, the min
imizer can be simply obtained by cancelation of the functional gradient. Using
decomposition (18), the derivative of the DFD data term with respect to vector
Θ reads:
?
=
Ω
∂Jobs(Θ)
∂Θ
T
dΘ ? lim
β→0
?
∂
∂β
Ω
1
2
?
I1(x + Φ(x)[Θ + βdΘ]) − I0(x)
?
?2
dx
?
I1(x + Φ(x)Θ) − I0(x)
∇IT
1(x + Φ(x)Θ)Φ(x)dΘdx
(21)
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 11
Wavelet Expansion and Regularization for Fluidmotion Estimation
9
and therefore using (15), one gets:
∂Jobs(Θ)
∂Θu
Ω
∂Jobs(Θ)
∂Θv
Ω
=
?
?
?
?
I1(x + Φ(x)Θ) − I0(x)
?∂I1
?∂I1
∂x1(x + Φ(x)Θ)(φC,k(x), ··· , ψi
F,k(x))Tdx
=
I1(x + Φ(x)Θ) − I0(x)
∂x2(x + Φ(x)Θ)(φC,k(x), ··· , ψi
F,k(x))Tdx.
(22)
As a consequence, components of the cost function gradient are simply the
coefficients of the decomposition of [I1(x + Φ(x)Θ) − I0(x)]∂I1
and [I1(x + Φ(x)Θ) − I0(x)]∂I1
gradient based algorithm, we obtain the minimizerˆΘ. The complexity of the
algorithm is much lower. Suppose that the conjugate mirror filters associated
to the wavelet and scaling functions have K nonzero coefficients. Then, a step
of the gradient algorithm has the complexity of two fast wavelet transforms of
complexity O(KN) [9].
∂x1(x + Φ(x)Θ)
∂x2(x + Φ(x)Θ) on the wavelet basis. Using a
4 Highorder regularization
In order to introduce waveletbased highorder multiscale regularizers we first
need to define the notion of “Lipschitz regularity” and derive accordingly the
properties of wavelet coefficients decay.
4.1 Vanishing moments, Lipschitz regularity and coeffi
cient decay
A wavelet ψ(x) ∈ L2(R) has n vanishing moments if :
?
Hence a wavelet with n vanishing moments is orthogonal to any polynomial of
degree n − 1.
A function w(x) ∈ L2(R) is said to be uniformly Lipschitz α over Ω if it
satisfies:
w(x) − pν(x) ≤ Kx − να,
where pν(x) is a polynomial of degree m = ?α? and K a constant independent of
ν. It can be shown that a function w(x) which is Lipschitz α > m is necessarily
m times continuously differentiable (i.e. w(x) ∈ Cm(R)).
The decay of the wavelet coefficients with the scale can be related to the
Lipschitz regularity of w(x) and the number of vanishing moments of ψ(x) [9].
In particular, we can distinguish between the two following cases:
• if w(x) is Lipschitz α > n, the wavelet coefficients decay as
R
x?ψ(x)dx = 0, for 0 ≤ ? < n.
(23)
∀ν ∈ Ω,∀x ∈ R,
(24)
?w(x),ψj,k(x)? ∼ 2j(n+1
2).
(25)
This result is a direct consequence of the link which exists between wavelet
coefficients and the nthderivative of a differentiable function:
?w(x),ψj,k(x)?
2j(n+1
lim
j→∞
2)
∝∂nw(x)
∂xn
.
(26)
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 12
Wavelet Expansion and Regularization for Fluidmotion Estimation
10
Note that an interesting particular case of (26) arises when w(x) is a
polynomial of degree m < n. In such a case, α = ∞ and we also have
from (23) that all the wavelet coefficients are equal to zero.
• if w(x) is Lipschitz α < n, the wavelet coefficients decays as ?w(x),ψj,k(x)? ∼
2j(α+1
2). A proof of this result can be found in [7].
These results can be easily extended to the case of twodimensional signals.
In particular, projecting a function u(x) ∈ C2n(R2) onto ψi
cally equivalent to applying an nthorder oriented derivative operators, i.e.
j,k(x) is asymptoti
lim
j→−∞
(θu)1
2j(n+1)∝∂nu(x − k2−j)
(θu)2
j,k
2j(n+1)∝∂nu(x − k2−j)
(θu)3
j,k
2j(n+1)∝∂2nu(x − k2−j)
j,k
∂xn
1
,
(27)
lim
j→−∞
∂xn
2
,
lim
j→−∞
∂xn
1xn
2
,
where (θu)i
follows that
j,k? ?u(x),ψi
j,k(x)?. Therefore, if u(x) has bounded derivatives it
(θu)i
j,k ∼ 2j(n+1)
∀i ∈ {1,2,3}.
(28)
In particular, (28) holds if u(x) is Lipschitz α > n.
Moreover, if u(x) is Lipschitz α < n it can be shown following the same
reasoning as in [9] that:
(θu)i
j,k ∼ 2j(α+1).
(29)
4.2Highorder optic flow regularization
Let ψ be a mother wavelet with a fast decay and n vanishing moments and let
α be the uniform Lipschitz isotrope regularity of motion components u and v.
a) Hard constraints on optic flow regularity.
wavelet has a sufficiently high number of vanishing moments n and assume optic
flow is a polynomial function of order m < n. Then, (27) implies that wavelet
coefficients vanish when scales tend to zero. Therefore, in this case solving the
optic flow estimation problem on a wavelet basis with a nthorder regularizer:
This is equivalent to use an explicit regularization scheme, estimating the motion
field in an approximation space VL, where C ? L > F. Thus, it results in
estimating optic flow on a truncated wavelet basis, where components at scales
We suppose the considered
ˆ v = Φ
?
argminΘJobs(I,Φ(Θ))
s.t. ∀x ∈ R2,
∂nv(x)
?
=∂2nu(x)
∂xn
=∂2nv(x)
∂xn
∂nu(x)
∂xn
1
=∂nu(x)
∂xn
=∂nv(x)
∂xn
21xn
2
= 0
= 0.
∂xn
121xn
2
(30)
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 13
Wavelet Expansion and Regularization for Fluidmotion Estimation
11
s ∈ [2F,2L] are omitted. Hence, the solution of (30) is simply obtained by
solving the lowerdimensional problem:
?
s.t.{(θu)i
As proposed in the previous section, the minimum of (31) can be simply obtained
by canceling the gradients (22). Note, that by varying the number of wavelet
vanishing moments, we are able to impose a hard constraint on the nth order
regularity.
ˆ vL= Φ
?
argminΘJobs(I,Φ(Θ))
j,k= (θv)i
?
j,k= 0,∀i ∈ {1,2,3},∀k ∈ R2,∀j < L}.
(31)
b) Multiscale interpolation.
only in the limit case of scale s ∈ [2F,2L] close to zero. Since the pixel reso
lution is the finest accessible scale, wavelet coefficients at the finest scale 2F
do not necessarily vanish in practice. Moreover, in general the solution may
deviate from a polynomial of order m, and is generally only an approximation
of a function Lipschitz α < n (see previous section) with ?α? = m. Therefore,
instead of imposing wavelet coefficients to vanish in the scale range [2F,2L], we
rather propose to prolongate the motion regularity (i.e. the wavelet coefficient
decay) in this range by multiscale interpolation [9]. Indeed, multiscale interpo
lation is the orthogonal projection of the solution estimated at scale 2Lon the
finest approximation space VF. For an approximation ˆ vLin the space VL, the
solution ˆ v interpolated at scale 2Freads:
The equivalence between (30) and (31) stands
ˆ v =
+∞
?
n1=−∞
+∞
?
n2=−∞
ˆ vL(n12F,n22F)φF(x1− n12F
2F
,x2− n22F
2F
).
(32)
The interpolation function φFis defined as the autocorrelation of the orthogonal
scaling function: φF= φ?¯φ, where ? and¯φ denote respectively the convolution
operator and the complex conjugate of φ. The solution being at small scales a
polynomial of order m ≤ n, wavelet coefficients decay in m + 1 (i.e. ?α? + 1).
The decay imposed at small scales [2F,2L] therefore constitute an approximated
prolongation of the decay in α + 1 estimated at coarser scales [2L,2C].
c) Soft constraints on optic flow regularity.
general case where the solution is not limited to polynomial function but is
Lipschitz α > n (see previous section), we propose to introduce a global nth
order regularization functional designed on the properties of wavelet coefficient
decay given in (27):
?
=1
2
i,j,k
Considering now the more
Jreg(Θ) =1
2
Ω
?∂nu
?
∂xn
1
?2
+
?∂nu
j,k)2+ (βj(θv)i
∂xn
2
?2
+
?∂2nu
j,k)2
∂xn
1xn
2
?2
+
?∂nv
∂xn
1
?2
+
?∂nv
∂xn
2
?2
+
?∂2nv
∂xn
1xn
2
?2
dx
(βj(θu)i
(33)
where βj = 2−j(n+1)denote multiplicative factors.
regularization is simply tuned by varying the number of vanishing moments of
Note that the order of
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 14
Wavelet Expansion and Regularization for Fluidmotion Estimation
12
the mother wavelet. The gradient of the regularization functional thus reads:
Assembling the gradients of the data term (21) and the gradient of the nthorder
regularizer (34), we obtain the gradient of the global functional (5). A gradient
descent algorithm is then used to obtain the minimizerˆΘ and therefore achieve
multiscale motion estimation. When α < n, the proposed regularizer does no
longer constitute a nthorder regularizer (see previous section). However, it can
constitute a good approximation as long as n is not too large. In any case,
we can make the regularizer exact since there always exist a wavelet with a
sufficiently low number of vanishing moment satisfying n < α.
∂Jreg(Θ)
∂Θ
=
...
βj(θu)i
j,k
...
...
βj(θv)i
j,k
...
(34)
5 Experiments
5.1Implementation
Daubechies wavelets of variable order have been chosen since they have a min
imum support size for a given number of vanishing moments [9]. Input images
being finite signals, it is necessary to construct wavelet bases of L2[0,1]. Thus,
the wavelet basis of L2(R) is transformed by periodizing each scaling function
and wavelet of the basis. In order to handle nicely large displacements [11],
wavelet coefficients are sequentially estimated from the coarsest to the finest
scales. More precisely, beginning from the estimation of the coarsest approxi
mation in VC, wavelet basis is refined with details in Vj−1−Vjat scale 2juntil
scale 2Lis reached. At each refinement level, minimization of the cost function
is efficiently achieved with a QuasiNewton gradient algorithm and using Wolf
conditions to fix the optimal gradient step (LBFGS algorithm [10]). The gradi
ent descent is initialized with the estimate obtained at the previous level. This
strategy enables the update of the coarser coefficients while estimating details
at finer scales.
5.2Fluid image data sets
Sequences of images depicting fluid flows have been chosen for assessing the
methods. Accurate motion estimation on such data remains very challenging.
Nevertheless, we show in the following that highorder multiscale optic flow
regularizers are well designed for the characterization of fractal motion such as
turbulence.
5.2.1 Synthetic turbulence
The first data set used for evaluation is a synthetic sequence of Particle Imagery
Velocimetry (PIV) images of 256 × 256 pixels, representing small particles (of
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
Page 15
Wavelet Expansion and Regularization for Fluidmotion Estimation
13
Figure 1: From left to right: Up, color legend [1], ground truth and sample
input PIV image. Below, estimated motion (case b of section 4.2), along with
associated end point error and Barron error maps. In the visualization of [1]
color and intensity respectively code for vector orientation and magnitude.
radius below 4 pixels) advected by a twodimensional periodic turbulent flow.
The dynamic of the fluid flow is given by numerical simulation of 2D Navier
Stokes equations. Since true velocity fields are known, comparisons and errors
computation can easily be done. An image of the sequence is displayed in Fig. 1
together with its associated motion ground truth. Estimated velocity fields are
evaluated based on the Root Mean Squared end point Error (RMSE) and Mean
Barron’s angular Error (MBE).
Horn& Schunck (1981)
firstorder [6]
0.13851
4.2656
Corpetti& al (2002)
divcurl [4]
0.13402
4.3581
Yuan& al (2007)
divcurl [12]
0.0960
3.0458
Heas& al (2009)
selfsimilar [5]
0.0914
2.8836
Proposed method
7thorder
0.0905
2.8994
Reg.
RMSE
BME
Table 1: Comparison of RMSE and BME between state of the art and proposed
estimators. Results obtained solving (32) (case b of section 4.2) have been
projected onto the nulldivergence space [5] to make them consistent with results
in [12] [5].
The two hard constraint regularization schemes (cases a and b of section 4.2)
have first been evaluated on this sequence. According to the image size of 28,
scales in the range [20,27] have been considered in the dyadic multiresolution
representation. An approximation space VL corresponding to scale 2L= 25
was chosen for solving the hardconstraint estimation problems defined in (31)
and (32). Motion field estimate obtained with a hard constraint regularization
complemented by multiscale interpolation is presented in Fig. 1 together with
the end point error and the Barron error maps. As shown in table 1, this high
RR n° 0123456789
inria00505780, version 1  26 Jul 2010
View other sources
Hide other sources
 Available from Cedric Herzet · May 28, 2014
 Available from inria.fr