Selfsimilar regularization of opticflow for turbulent motion estimation
ABSTRACT Based on selfsimilar models of turbulence, we propose in this paper a multiscale regularizer in order to provide a closure to the opticflow estimation problem. Regularization is achieved by constraining motion increments to behave as a selfsimilar process. The associate constrained minimization problem results in a collection of firstorder opticflow regularizers acting at the different scales. The problem is optimally solved by taking advantage of lagrangian duality. Furthermore, an advantage of using a dual formulation, is that we also infer the regularization parameters. Since, the selfsimilar model parameters observed in real cases can deviate from theory, we propose to add in the algorithm a bayesian learning stage. The performance of the resulting opticflow estimator is evaluated on a particle image sequence of a simulated turbulent flow. The selfsimilar regularizer is also assessed on a meteorological image sequence.

Conference Paper: Inference on Gibbs opticflow priors : Application to atmospheric turbulence characterization
[Show abstract] [Hide abstract]
ABSTRACT: In this paper, Bayesian inference is used to select the most evident Gibbs prior model for motion estimation given some image sequence. The proposed method supplements the maximum a posteriori motion estimation scheme proposed in HeÂ¿as et al. (2008). Indeed, in this recent work, the authors have introduced a family of multiscale spatial priors in order to cure the illposed inverse motion estimation problem. We propose here a second level of inference where the most likely prior model is optimally chosen given the data by maximization of Bayesian evidence. Model selection and motion estimation are assessed on Meteorological Second Generation (MSG) image sequences. Selecting from images the most evident multiscale model enables the recovery of physical quantities which are of major interest for atmospheric turbulence characterization.Geoscience and Remote Sensing Symposium,2009 IEEE International,IGARSS 2009; 08/2009
Page 1
Selfsimilar regularization of opticflow
for turbulent motion estimation
Patrick H´ eas1,2, Etienne M´ emin1, Dominique Heitz2
{Patrick.Heas, Etienne.Memin}@irisa.fr, Dominque.Heitz@cemagref.fr
1INRIA, Vista Project, Center of Rennes
2CEMAGREF, Center of Rennes
Abstract. Based on selfsimilar models of turbulence, we propose in
this paper a multiscale regularizer in order to provide a closure to the
opticflow estimation problem. Regularization is achieved by constrain
ing motion increments to behave as a selfsimilar process. The associate
constrained minimization problem results in a collection of firstorder
opticflow regularizers acting at the different scales. The problem is opti
mally solved by taking advantage of lagrangian duality. Furthermore, an
advantage of using a dual formulation, is that we also infer the regular
ization parameters. Since, the selfsimilar model parameters observed in
real cases can deviate from theory, we propose to add in the algorithm a
bayesian learning stage. The performance of the resulting opticflow esti
mator is evaluated on a particle image sequence of a simulated turbulent
flow. The selfsimilar regularizer is also assessed on a meteorological im
age sequence.
1 Introduction
The estimation of highly nonrigid image flows is an important problem in var
ious application areas of fluid flow image analysis like remote sensing, medical
imaging, and experimental fluid mechanics. Such flows, which cannot be rep
resented by a single parametric model, are typically estimated by variational
approaches. In this framework, regularization models are required to remove the
motion estimation ambiguities. However, standard regularizers acting in a lim
ited spatial neighborhood are insufficient to recover accurately the multiscale
structures of turbulent flows. Furthermore, they do not rely on any physical
prior knowledge and, moreover, raise the open question of tuning the regularizer
weight.
The objective of this contribution is to provide a multiscale regularizer based
on turbulent motion selfsimilarity. In contrast to standard approaches, this self
similar prior is physically sound and presents the valuable advantage of solving
the aperture problem while fixing regularizer weights at the different scales. The
paper is organized as follows. In the next section, we first highlight the limita
tion of standard opticflow regularizers. Besides, we introduce selfsimilar mod
els issued from theoretical works on turbulence. Then, in section 3, selfsimilar
constraints are defined and a dual approach is proposed to solve optimally con
strained opticflow estimation problems using convex optimization methods. In
order to introduce uncertainty in the parameters of the selfsimilar model given
by theory, a bayesian estimation framework is presented in section 4. Finally, a
numerical evaluation with synthetic flow and results obtained with experimental
data reveal the interest of selfsimilar regularization for fluid flows.
inria00325807, version 1  30 Sep 2008
Author manuscript, published in "Dans The 1st International Workshop on Machine Learning for Visionbased Motion Analysis 
MLVMA’08 (2008)"
Page 2
2P. H´ eas, E. M´ emin, D. Heitz
2 Related work
2.1Opticflow state of the art
Aperture problem The apparent motion v = (u,v), perceived through image
intensity I(s,t) variations, respects the standard Optical Flow Constraint (OFC)
observation model:
It+ v · ∇I = 0.
Apparent motion and the real underlying velocity field are identical when consid
ering rigid motion and stable lighting conditions. For fluids, this identity remains
valid in the case of 2D incompressible flows. Based on mass conservation, the
integrated continuity equation has been proposed in the literature for various 3D
fluid flows visualized in a projected image plane in order to link the image inten
sity function I to a vertically averaged horizontal velocity field v [1–3]. However,
observation models can not be used alone, as they provide only one equation
for two unknowns at each spatiotemporal locations (s,t). This constitute the
socalled aperture problem.
(1)
Standard regularizer limitations To deal with this problem, the most com
mon assumption consists in enforcing locally spatial coherence. Global regular
ization schemes over the entire image domain Ω are convenient to model spatial
dependencies on the complete image domain. On the contrary to disjoint esti
mation approaches [4], dense velocity fields are estimated even in the case of
noisy, low contrasted and incomplete observations. More precisely, the motion a
posteriori estimation problem is defined as the global minimization of an energy
function composed of two components:
f(I,v) = fd(I,v) + αfr(v)(2)
The first energy fd(v,I), called the data term, penalizes discrepancies from
the observation models and thus can be related to a likelihood probability. For
example, discretizing in time the OFC equation, one can build the data term:
fd(I,v)=1
2
Ω
where˜I denotes the image I(t + ∆t). A robust penalty function φ can be in
troduced in the data term for attenuating the effect of observation outliers de
viating significantly from the model. In this work, φ are Leclerc semiquadratic
Mestimator as proposed in [5]. The second component fr(v), called the reg
ularization term, acts as a spatial prior enforcing the solution to follow some
smoothness properties. In the previous expression, α > 0 denotes a regulariza
tion parameter controlling the balance between the smoothness and the global
adequacy to the observation model. In this framework, Horn and Schunck [6]
proposed a firstorder regularization of the two spatial components u and v of
velocity field v:
fr(v) =1
2
Ω
However, a first order regularization is not adapted to fluid flows as it penalizes
spatially nonhomogeneous velocity fields. Second order regularizers on motion
?
φ
?
(˜I−I+v · ∇˜I)2?
ds
(3)
?
∇u2+∇v2ds
(4)
inria00325807, version 1  30 Sep 2008
Page 3
Selfsimilar regularization of opticflow for turbulent motion estimation3
vorticity and divergence have been proposed to overcome those limitations [2,
7,8]. Nevertheless, such regularization models fail to describe the multiscale
structure of turbulent velocity fields. Efficient multiscale regularizers based on
fractal priors have already been introduced in [9]. However, such multiscale
models have not been linked to physical prior on turbulence. Furthermore, using
an energy based on two components raises the difficult problem of fixing some
regularization parameters.
Multiresolution approach
is the estimation of large displacements. Indeed, these equations are only valid
if the solution is in the region of linearity of the image intensity function. A
standard approach for tackling nonlinearity is to apply successive linearizations
around a current estimate and to warp a multiresolution representation of the
data accordingly. More precisely, a large displacement field ˜ v is first estimated
with the original data term at coarse resolution, where the linearity assumption
is valid. Then, introducing the decomposition:
v = ˜ v + v?,
motion is refined through an incremental fields v?estimated using a linearized
motioncompensated data term while going down the resolution levels of an
image pyramid [10]. Splines are in this context advantageous interpolators useful
to derive accurate motioncompensated images at the different resolutions [11].
One major problem with observation models
(5)
2.2
In the turbulence community, turbulent motion is known to be structured as a
scale invariant spatial process. In order to define scale invariance, let us introduce
the function of velocitity increments:
δv(s,?) = v(s + ?) − v(s).
where ? represents the norm of increment ?. We also introduce the longitudinal
δv?(?) and transverse δv⊥(?) functions defined as δv?(s,?) = v(s + ?t) − v(s)
and δv⊥(s,?) = v(s + ?n) − v(s), with t and n denoting the tangential and
the normal unitary vectors of any bidimensional orthogonal basis of the image
plane. No confusion should be made here with the normal and tangential optic
flow components. As a classical hypothesis in turbulence studies, we assume
homogeneity and isotropy, that is to say we consider that the statistical proper
ties of the velocity field are invariant under translation of spatial location s and
rotation of ?. In agreement with these assumptions, index to spatial locations
s can be dropped and a simple scalar velocity increment function δv(?) can be
defined in the bidimensional plane either by δv?(?) · t,
or δv⊥(?)·n. For any of these scalar quantities, Kolmogorov [12] demonstrated
that from a statistical point of view the turbulent flow is selfsimilar, i.e. there
exists a unique scaling exponent h ∈ R such that:
δv(λ?) = λhδv(?), ∀λ ∈ R+,
A corollary is that the second order moment of the probability distribution func
tion P?(δv) of velocity increments, namely the second order structure function:
?
Turbulence selfsimilar models
(6)
δv?(?) · n, δv⊥(?) · t
(7)
E[δv(?)2] =
R
δv(?)2P?(δv(?))dδv(?)(8)
inria00325807, version 1  30 Sep 2008
Page 4
4 P. H´ eas, E. M´ emin, D. Heitz
follows a power law of universal exponent ζ:
E[δv(?)2] − β?ζ= 0.
(9)
For three dimensional turbulence, the scale range I of the power law, socalled
inertial range, is defined for ? ∈ [η,?0], where η represents the dissipative scale
and where ?0is much smaller than the diameter L of the largest vortex. In this
range, Kolmogorov demonstrated that ζ = 2/3. Analogously, for bidimensional
turbulence with energy injection at scale ?0, Kraichnan showed that there exist
two different selfsimilar processes: ζ = 2 within a range I1 = [η,?0] and ζ =
2/3 within a range I2= [?0,L] [13]. For atmospheric turbulence, measurements
proved an inversion of the the two former power laws: ζ = 2/3 within a range I1=
[1,500] kilometers and ζ = 2 within a range I2= [1000,3000] kilometers [14].
Moreover, for any flow, there exits a power law of scaling exponent ζ = 2 in the
dissipative range I0= [0,η].
3
In this section we propose to close the opticflow equations by introducing
physicalbased selfsimilar constraints. Besides providing a closure for motion
estimation, such a selfsimilar regularizer yields several benefits:
– first, selfsimilar processes are multiscale models which will structure motion
fields across scales in agreement with physics,
– second, solving optimally the opticflow minimization problem under self
similar constraints will lead to a nonparametric method where the problem
of fixing regularization parameter α does no longer exist,
– last, the motion estimation problem can be solved using standard convex
optimization methods.
Selfsimilar regularization of opticflow
3.1
Let us first formalize selfsimilar constraints. The second order structure function
E[δv(?)2] is an expectation defined in Eq. 8. We approximate this expectation
by a statistical average over the image domain Ω:
1
Ω
where Ω denotes the image domain area. In order to obtain an accurate expec
tation estimator over the bidimensional plane and in order to avoid boundary
effects, we build a scalar structure function by averaging the norm of transverse
and longitudinal structure functions in the 2 directions:
?
A constraint g?(v) is then defined for each scale ? as the difference between
the second order structure function depending on the velocity field v, and the
predicted power law. Thus, for any scale ? ∈ ∪Ii, an estimated motion field
should respect the constraint :
g?(v) =1
2(E[δv(?)2] − βi?ζi) = 0, ∀? ∈ ∪Ii
for given scaling exponent ζiand factor βi.
Selfsimilar constraints
E[δv(?)2] ≈
?
Ω
δv(?)2ds
(10)
δv(?) =
1
8(δv?(?)2+ δv⊥(?)2+ δv?(−?)2+ δv⊥(−?)2)(11)
(12)
inria00325807, version 1  30 Sep 2008
Page 5
Selfsimilar regularization of opticflow for turbulent motion estimation5
3.2
Referring to section 2.1, the minimization related to the unclosed opticflow
estimation problem reads:
(ˆ v) = argmin
v
Adding the selfsimilar constraints, we obtain the following constraint minimiza
tion problem:
3.3Discrete problem formulation
Let us now express the constraint problem in its discrete form. The derivatives
∇vfd(I,v) related to any motioncompensated data term (which is quadratic
with respect to motion increments v?) can be expressed in the matricial form
A0v?−b0, when discretized on an image grid S of m points with a finite difference
scheme. The two discretized components of v?now represents a field of n = 2m
variables supported by the grid S, A0is a n×n symmetric positivedefinite,b0∈
Rnrepresents a vector of size n. The discrete data term can be rewritten as:
fd(I,v) =1
2v?TA0v?− bT
where c0 ∈ R denotes a scalar. For the selfsimilar constraints, as detailed in
appendix A, the quadratic constraint derivatives can be expressed in the vectorial
form A?v?− b?, where A?are symmetric positive semidefinite matrices and b?
are vectors of size n. Thus, using discretized variables v?yields :
g?(v) =1
2v?TA?v?− bT
where c? ∈ R are scalars. In particular, in appendix A we demonstrate that
A?v?= ∇v?g?(v?) and b?= −∇˜ vg?(˜ v) where the derivative reads:
∇vg?(v) = −
4Ω−{Γt}
where ∆?represents a discretized laplacian operators defined on a grid with
a mesh equal to ? using a centered second order finite difference scheme and
where we have exculded vertical Γtand horizontal Γnimage borders of width
?. The discretized laplacian operators can thus be interpreted as a collection
of firstorder regularizers performing at the different scales. They represent to
some extent a generalization of the Horn and Schunck firstorder regularizer
(Eq. 4) to multiscale. However, an important difference here is that motion
spatial derivatives are not penalized in our case, but constrained to follow a
power law across scales.
The constraint motion estimation problem defined in Eq. 14 can thus be rewrit
ten in its discrete form as:
where for simplification we have dropped the dependance to the image I.
Constrained motion estimation problem
fd(I,v).
(13)
minvfd(I,v)
under the constraints:
g?(v) = 0, ∀? ∈ ∪Ii
v ∈ Rn
.
(14)
0v?+ c0.
(15)
?v?+ c?= 0, ∀? ∈ ∪Ii,
(16)
1
??
Ω−{Γt+Γn}∆?u ds
Ω−{Γt+Γn}∆?v ds
?
?
+ borderterms,
(17)
(P)
minvfd(v) =1
under the constraints:
g?(v) =1
v = v?+ ˜ v ∈ Rn
2v?TA0v?− bT
0v?+ c0.
2v?TA?v?− bT
?v?+ c?= 0, ∀? ∈ ∪Ii
.
(18)
inria00325807, version 1  30 Sep 2008
Page 6
6 P. H´ eas, E. M´ emin, D. Heitz
3.4
A first idea would be to solve P by penalizing with a quadratic cost deviations
from the selfsimilar constraints, i.e. f(v) = fd(v) +?
to the fractal regularizer proposed in [9]. However, this new functional is no
longer convex because of (g?(v))2and one may face difficulties when resolving
nonlinear problems and fixing the regularization parameters {α?}. An advan
tageous alternative is to solve instead the associated constraint optimization
problem using classical lagrangian duality. To define optimality conditions, we
now introduce the lagrangian function L(v,λ) associated to (P):
?
In the lagrangian duality formalism, the optimal solutions of the socalled primal
problem P, are obtained by searching saddle points of the lagrangian function.
Saddle points denoted by (v∗,λ∗) are defined as the solutions of the socalled
dual problem:
?L(v∗,λ∗) = maxλw(λ) = maxλ{minvL(v,λ)}
where w(λ) denotes the dual function. As the functions f and g?are convex and
as the constrained group is not empty, for positive and large enough lagrangian
multipliers λ?, L is convex and the minimization problem (P) has a unique saddle
point i.e. an optimal solution v∗which is unique1. Thus in this framework, each
lagrangian multiplier λ?representing the regularization parameter at scale ? can
naturally be inferred.
Dual problem and optimal solution
∪Iiα?g?(v)2, where {α?}
are positive scalars. Such a multiscale model would be to some extent similar
L(v,λ) = f(v) +
∪Ii
λ?g?(v), λ = {λ?}.
(19)
(D)
λ?∈ R+, ∀? ∈ ∪Ii
,
3.5
The minimum ˆ v?of the convex lagrangian function at point λ can be obtained
by solving the following EulerLagrange equations:
Convex optimization
∇vL(v,λ) = ∇vfd(v) +
?
∪Ii
λ?∇vg?(v) = 0.
(20)
which reduce (using to Eq. 15 and Eq. 16) to solve the linear system :
?
The resolution of EulerLagrange large system is efficiently achieved using a
conjugate gradient squared optimization method. The dual function is then given
by:
w(λ) = ˆ v?T?
A0+
?
∪Ii
λ?A?
?
ˆ v?= b0+
?
∪Ii
λ?b?.
(21)
A0+
?
∪Ii
λ?A?
?
ˆ v?−
?
b0+
?
∪Ii
λ?b?
?T
ˆ v?+ c0+
?
∪Ii
λ?c?. (22)
1Let us note that for negative lagrangian multipliers, the convexity of the functional is
no longer insured, and for too small lagrangian multipliers, the problem may remain
illposed with no guarantee of the unicity of the solution. However, in practice the
problem admits a solution if there exist at least one scale ? with λ??= 0.
inria00325807, version 1  30 Sep 2008
Page 7
Selfsimilar regularization of opticflow for turbulent motion estimation7
The dual function is by definition concave and possesses socalled subgradients
equal to g?(ˆ v?+ ˜ v). We employ a classical gradient method to find λ∗which
maximizes the dual function and thus obtain the solution v∗. Finally, the con
straint motion estimation method results in the Uzawa algorithm presented be
low, which is used to converge towards the unique saddle point (v∗,λ∗), i.e. the
optimal motion estimate under selfsimilar constraints.
– (a) From any initial point λ0> 0 and estimate ˜ v:
– (b) At iteration k, find increment ˆ v?defining w(λk) by solving Eq. 21
– (c) Define λk+1by: ∀? ∈ ∪Ii, λk+1
– (d) If stopping criterion valid :(v∗,λ∗) = (ˆ v?+˜ v,λk), END.
Else increment k and go back to (b)
?
= λk
?+ ρkg?(ˆ v?+˜ v)
ρkdenotes the displacement step at iteration k. The latter parameter is adjust
at each iteration using a relaxation method proposed in [15].
Uzawa algorithm converging towards (v∗,λ∗).
4 Learning turbulence statistics
In this section we consider some uncertainty in the scaling exponent of the
selfsimilar model used for motion estimation. Indeed, there may exist for some
particular turbulent flows deviations from theory. The idea is thus to use an a
posteriori estimation framework to learn the power law parameters based on a
coarse motion estimate and theoritical priors.
4.1
Uncertainty on scaling exponents is introduced in the motion estimation scheme
by associating to the unknown selfsimilar model parameter (ζ,β) an a priori
Gaussian probability distribution:
p(ζ) ∼ N(ζK41,σ2
In Eq.23, the mean ζK41 denotes the exponent predicted by Kolmogorov (see
section 2.2). We also define a Gaussian likelihood probability distribution at
scale ? of the logarithm of the structure function:
Prior distribution for scaling exponents
ζ)(23)
p(logE[δv(?)2]β,ζ) ∼ N(log(β?ζ),σ2) (24)
Thus, the standard deviation σζis a parameter tuning the degree of uncertainty
on Kolmogorov parameters, while the standard deviation σ represents the al
lowed deviation of the structure function estimate E[δv(?)2] from the predicted
law.
4.2 Learning power laws
Using Eq. 23 and Eq. 24, scaling exponent ζ is estimated using the Maximum
A Posteriori (MAP) estimator and Bayes law:
inria00325807, version 1  30 Sep 2008
Page 8
8P. H´ eas, E. M´ emin, D. Heitz
ˆζMAP= argmax
ζ
p(ζ)
?
?∈I
p(logE[δv(?)2]ζ),
(25)
where we have assumed that likelihood probabilities p(logE[δv(?)2]β,ζ) are in
dependent at the different scale ?. In Eq.25, I denotes a scale interval. As there
is usually no a priori for β, it is estimated in the sense of the Minimum Mean
Square Error (MMSE). Thus, the model parameter (ˆβMMSE,ˆζMAP) are ob
tained by minimization of the functional J(β,ζ):
(ˆβMMSE,ˆζMAP) = arg min
??
The minimization is achieved by searching the solution of the linear equations
∇β,ζJ(β,ζ) = 0 based on a coarse estimation of the velocity field with any
standard regularizer. Therefore, an analytical solution is obtained by solving the
two linear equations.
β,ζJ(β,ζ)(26)
?
= arg min
β,ζ
?∈I
(logE[δv(?)2] − log(β?ζ))2+σ2
σ2
ζ
(ζK41− ζ)2
5Experiments
To evaluate the performance of the selfsimilar regularization, a synthetic par
Simulated particule image velocity ground truthvelocity estimate
Fig.1. Estimation of twodimensional turbulence. Left: particle image obtained
by DNS of 2D NavierStokes equations. Middle: true velocity field Right: estimate
ticle image sequence was generated based on a twodimensional turbulent flow
obtained by the direct numerical simulation (DNS) of NavierStokes equations,
and based on a particle image generator [16]. Fig. 1 presents one of the particle
images of 256 by 256 pixels, the true underlying velocity field and our estimation
obtained by minimizing the OFC based dataterm (Eq. 1) under selfsimilarity
constraints. Parameters of the selfsimilarity model were inferred in the dissi
pative scale range of I0= [1,10] pixels using a Horn&Schunck estimate and a
Gaussian prior power exponent centered on the theoretical value of ζK41= 2 and
with standard deviation σζ= 0.3. Using Eq. 26, we obtained a MAP estimate
equal toˆζMAP= 1.8064. The estimated power law is plotted in Fig. 2 together
with the second order structure function E[δv(?)2] obtained with Horn&Schunck
algorithm and with the proposed method. Note that constraining motion incre
ments to behave as a selfsimilar process at small scales yields an enhancement of
the structure function at fine but also at large scales. Therefore, one can conclude
inria00325807, version 1  30 Sep 2008
Page 9
Selfsimilar regularization of opticflow for turbulent motion estimation9
0.01
0.1
1
1 10
increment [pixel]
100
second order moment of increment distribution
’ground truth’ u 1:2
g(x)
’estimate with Horn&Schunck algorithm’ u 1:2
’estimate with proposed method’ u 1:2
1e06
1e05
1e04
0.001
0.01
0.1
1
10
100
0.001 0.01 0.1 1
spectral density
frequency [1/pixel]
’ground truth’ u 1:3
’Horn&Schunck (1981)’ u 1:3
’correlation technique (LaVision)’ u 1:3
’Corpetti&al. (2002)’ u 1:3
’proposed method’ u 1:3
2nd order structure functionEnergy spectrum
average RMSE(ˆ v) = 0,1699
Corpetti & al. (2002)
(divcurl regularization)
average RMSE(ˆ v) = 0,1451
Horn & Schunck (1981)
(gradient penalization)
average RMSE(ˆ v) = 0,1313
LaVision company
(correlation maximization)
average RMSE(ˆ v) = 0,1227
proposed method
(selfsimilar regularization)
Fig.2. Numerical evaluation of the selfsimilar regularizer. Upper left: power
law g(x) obtained by a posteriori estimation using a Horn&Schunck regularizer. It fits
the true and the estimated 2nd order structure function E[δv(?)2] obtained with the
proposed regularizer. Upper right: spectral comparison between a first order, a div
curl or a selfsimilar regularizer and an operational correlationbased method (PIV
technique) from LaVision. Below: spatial distribution and average RMSE of different
methods.
that the multiscale structural information has been propagated through scales.
A comparison with the state of the art is also presented in Fig. 2. One can remark
that the spatial distribution of the Root Mean Square Errors (RMSE) of velocity
field estimated using the selfsimilar model presents in average much lower values
than RMSE obtained with a divcurl [2] or a first order regularizer [6] or even
with operational correlationbased techniques. The energy spectrum comparison
displayed in the same figure proves that the proposed multiscale regularization
enhances in particular the estimation of small scales displacements. However,
large scale enhancements can be also noticed when visualizing energy spectra in
natural coordinates.
The multiscale regularizer has also been assessed on real data. A benchmark
has been constituted with a METEOSAT Second Generation meteorological
image sequence acquired at a rate of an image every 15 min. The image spatial
resolution was 3×3km2at the center of the whole Earth image disk. According
to the physicalbased methodology presented in [3], sparse image observations
related to a layer at intermediate altitude have been derived. Moreover, a robust
data term relying on a layer mass conservation model has been used to relate the
inria00325807, version 1  30 Sep 2008
Page 10
10P. H´ eas, E. M´ emin, D. Heitz
0.01
0.1
1
10
100
1000
0.01 0.1
spectral density
frequency [1/pixel]
g_I1(x)
g_I2(x)
’autosimilar regularization’ u 1:2
’first order regularization’ u 1:2
Fig.3. Estimation of atmospheric winds. An horizontal wind field of an atmo
spheric layer at midaltitude has been estimated using the physicalbased data term
proposed in [3] and the selfsimilar regularizer. Left: estimated velocity field superim
posed on a sparse input image of the sequence. Right: estimated energy spectrum fits
the power laws gI1(x) ∝ x−5
the contrary to results obtained by first order regularization
3 and gI2(x) ∝ x−3known to rule atmospheric flows, on
image intensity function to a vertically averaged horizontal wind field. Using the
predicted power exponents of ζ = 2/3 in a range I1= [1,10] pixels (contained
in the theoretical interval of [1,500] kilometers), parameters (ˆβMMSE,ˆζMAP)
were derived with Eq. 26 based on a first order regularized solution. Using this
learnt power law, the proposed Uzawa algorithm was used to converge towards
the solution of minimal cost respecting the selfsimilar constraints. In Fig. 3,
one can visualize the estimated velocity field which has been superimposed on a
sparse image. A comparison is also provided with a first order regularization in
the spectral domain. On the contrary to classical regularization, one can notice
that the energy spectrum estimated with the proposed method respects the two
power laws known to rule atmospheric flows (−ζ − 1 = −5/3 in I1 = [1,100]
pixels and −ζ − 1 = −3 for scales greater than 100 pixels [12,14]).
6 Conclusions
A closure to the aperture problem for fluid motion estimation is provided in this
paper. It relies on constraining motion increments to follow selfsimilar processes
which are well known models in the turbulence community. Solving optimally
the associate constrained minimization problem using lagrangian duality leads to
a nonparametric method where the problem of fixing regularization parameter
does no longer exist. Furthermore, standard convex optimization methods can
be used to infer the optimal motion field and its associated lagrangian multipli
ers. The resulting multiscale regularizer structures motion fields across scales in
agreement with physics. The methods also integrates a learning stage in order to
authorize deviations from theory: an a posteriori estimation framework is used
to infer the power law parameters characterizing the selfsimilar model. The su
periority of the selfsimilar model on state of the art regularizers is demonstrated
on synthetic particle images obtained by simulation of NavierStokes equations.
Experiments performed on a real meteorological image sequence proves that the
inria00325807, version 1  30 Sep 2008
Page 11
Selfsimilar regularization of opticflow for turbulent motion estimation 11
selfsimilar regularizer enhances the motion spectralconsistency in agreement
with atmospheric measurements.
A Discrete form of the selfsimilar constraints
From Eq. 10 and Eq. 11, one obtains :
?
Excluding vertical Γt and horizontal Γn image borders of width ? from the
calculation of the statistical average, one gets:
?
+
Ω−{Γn}
with γ = 8Ω−{Γt} = 8Ω−{Γn}. Manipulating the derivate with respect to
the motion horizontal component u of the constraints g?(v) which is defined in
Eq. 12, one obtains:
?
+
Ω−{Γ+
+
Ω−{Γn}
+
Ω−{Γ+
where (Γ−
included in Γt and Γn. A similar expression can be obtained for the derivate
with respect to vertical component v. Noting the presence of laplacian operators
in the previous equation, the derivate can thus be rewritten in a compact form:
∇vg?(v) = −2
γ
E[δv(?)2] ≈
1
Ω
Ω
1
8(δv?(?)2+ δv⊥(?)2+ δv?(−?)2+ δv⊥(−?)2)ds
(27)
γE[δv(?)2] ≈
Ω−{Γt}
?
(v(s) − v(s + ?t)2+ v(s) − v(s − ?t))2)ds
(28)
(v(s) − v(s + ?n)2+ v(s) − v(s − ?n)2)ds
γ∇ug?(v) =
Ω−{Γt}
(2u(s)−u(s + ?t)−u(s − ?t))ds
(29)
?
?
?
t}
(u(s)−u(s + ?t))ds+
?
Ω−{Γ−
t}
(u(s)−u(s − ?t))ds
(2u(s)−u(s + ?n)−u(s − ?n))ds
n)
(u(s)−u(s + ?n))ds+
?
Ω−{Γ−
n}
(u(s)−u(s − ?n))ds,
t,Γ+
t) and (Γ−
n,Γ+
n) denote respectively the left and right borders
??
Ω−{Γt+Γn}∆?u ds
Ω−{Γt+Γn}∆?v ds
?
?
+ borderterms,
(30)
where ∆?represents a twodimensional discretized laplacian operators defined
on a grid with a mesh equal to ? using a centered second order finite difference
scheme. Considering now the velocity field decomposition v = ˜ v + v?used in
multiresolution, as the operator is linear one obtains:
∇vg?(v) = ∇˜ vg?(˜ v) + ∇v?g?(v?)(31)
Using Eq. 29 and discretizing the velocity field, the constraints can finally be
written in their discrete form:
inria00325807, version 1  30 Sep 2008
Page 12
12P. H´ eas, E. M´ emin, D. Heitz
g?(v) =1
2v?TA?v?− bT
?v?+ c?= 0(32)
where A?v?= ∇v?g?(v?), b?= −∇˜ vg?(˜ v) and
c?= −β?ζ
22γ
Ω−{Γt}
+
2γ
Ω−{Γn}
+
1
?
˜ v(s) − ˜ v(s + ?t)2+ ˜ v(s) − ˜ v(s − ?t))2ds
1
?
˜ v(s) − ˜ v(s + ?n)2+ ˜ v(s) − ˜ v(s − ?n)2ds
References
1. Fitzpatrick, J.: The existence of geometrical densityimage transformations cor
responding to object motion. Comput. Vision, Graphics, Image Proc. 44 (1988)
155–174
2. Corpetti, T., M´ emin, E., P´ erez, P.: Dense estimation of fluid flows. IEEE Trans.
Pattern Anal. Mach. Intell. 24 (2002) 365–380
3. Heas, P., Memin, E., Papadakis, N., Szantai, A.: Layered estimation of atmospheric
mesoscale dynamics from satellite imagery. IEEE trans. on Geo. and Rem. Sensing
45 (2007) 4087–4104
4. Lucas, B., Kanade, T.: An iterative image registration technique with an applica
tion to stereovision. In: Int. Joint Conf. on Artificial Intel. (IJCAI). (1981) 674–679
5. Black, M., Anandan, P.: The robust estimation of multiple motions: Parametric
and piecewisesmooth flow fields. Computer Vision and Image Understanding 63
(1996) 75–104
6. Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17 (1981)
185–203
7. Suter, D.: Motion estimation and vector splines. In: Proc. Conf. Comp. Vision
Pattern Rec., Seattle, USA (1994) 939–942
8. Yuan, J., Schnoerr, C., Memin, E.: Discrete orthogonal decomposition and varia
tional fluid flow estimation. Journ. of Math. Imaging & Vison 28 (2007) 67–80
9. Luettgen, M., Karl, W., Willsky, A.: Efficient multiscale regularization with appli
cations to the computation of optical flow. IEEE Trans. Im. Processing 3 (1994)
41–64
10. Bergen, J., Burt, P., Hingorani, R., Peleg, S.: A 3frame algorithm for estimating
twocomponent image motion. IEEE Trans. Pattern Anal. Mach. Intell. 14 (1992)
886–895
11. Unser, M.: Splines: A perfect fit for signal and image processing. IEEE Signal
Processing Magazine 16 (1999) 22–38
12. Frisch, U.: Turbulence : the legacy of A.N. Kolmogorov. Cambridge university
press (1995)
13. Kraichnan, R.: Inertial ranges in twodimensional turbulence. Phys. Fluids 10
(1967) 1417–1423
14. Lindborg, E.: Can the atmospheric kinetic energy spectrum be explained by two
dimensional turbulence. J. Fluid Mech. 388 (1999) 259–288
15. Minoux, M.: Programmation math´ ematique : th´ eorie et algorithmes. Dunod Edi
tions (1983)
16. Carlier, J., Wieneke, B.: Report 1 on production and diffusion of fluid mechanics
images and data. Fluid project deliverable 1.2. http://www.fluid.irisa.fr (2005)
inria00325807, version 1  30 Sep 2008
View other sources
Hide other sources
 Available from Dominique Heitz · May 16, 2014
 Available from psu.edu