Page 1

Self-similar regularization of optic-flow

for turbulent motion estimation

Patrick H´ eas1,2, Etienne M´ emin1, Dominique Heitz2

{Patrick.Heas, Etienne.Memin}@irisa.fr, Dominque.Heitz@cemagref.fr

1INRIA, Vista Project, Center of Rennes

2CEMAGREF, Center of Rennes

Abstract. Based on self-similar models of turbulence, we propose in

this paper a multi-scale regularizer in order to provide a closure to the

optic-flow estimation problem. Regularization is achieved by constrain-

ing motion increments to behave as a self-similar process. The associate

constrained minimization problem results in a collection of first-order

optic-flow regularizers acting at the different scales. The problem is opti-

mally solved by taking advantage of lagrangian duality. Furthermore, an

advantage of using a dual formulation, is that we also infer the regular-

ization parameters. Since, the self-similar model parameters observed in

real cases can deviate from theory, we propose to add in the algorithm a

bayesian learning stage. The performance of the resulting optic-flow esti-

mator is evaluated on a particle image sequence of a simulated turbulent

flow. The self-similar regularizer is also assessed on a meteorological im-

age sequence.

1 Introduction

The estimation of highly non-rigid image flows is an important problem in var-

ious application areas of fluid flow image analysis like remote sensing, medical

imaging, and experimental fluid mechanics. Such flows, which cannot be rep-

resented by a single parametric model, are typically estimated by variational

approaches. In this framework, regularization models are required to remove the

motion estimation ambiguities. However, standard regularizers acting in a lim-

ited spatial neighborhood are insufficient to recover accurately the multi-scale

structures of turbulent flows. Furthermore, they do not rely on any physical

prior knowledge and, moreover, raise the open question of tuning the regularizer

weight.

The objective of this contribution is to provide a multi-scale regularizer based

on turbulent motion self-similarity. In contrast to standard approaches, this self-

similar prior is physically sound and presents the valuable advantage of solving

the aperture problem while fixing regularizer weights at the different scales. The

paper is organized as follows. In the next section, we first highlight the limita-

tion of standard optic-flow regularizers. Besides, we introduce self-similar mod-

els issued from theoretical works on turbulence. Then, in section 3, self-similar

constraints are defined and a dual approach is proposed to solve optimally con-

strained optic-flow estimation problems using convex optimization methods. In

order to introduce uncertainty in the parameters of the self-similar model given

by theory, a bayesian estimation framework is presented in section 4. Finally, a

numerical evaluation with synthetic flow and results obtained with experimental

data reveal the interest of self-similar regularization for fluid flows.

inria-00325807, version 1 - 30 Sep 2008

Author manuscript, published in "Dans The 1st International Workshop on Machine Learning for Vision-based Motion Analysis -

MLVMA’08 (2008)"

Page 2

2P. H´ eas, E. M´ emin, D. Heitz

2 Related work

2.1Optic-flow state of the art

Aperture problem The apparent motion v = (u,v), perceived through image

intensity I(s,t) variations, respects the standard Optical Flow Constraint (OFC)

observation model:

It+ v · ∇I = 0.

Apparent motion and the real underlying velocity field are identical when consid-

ering rigid motion and stable lighting conditions. For fluids, this identity remains

valid in the case of 2D incompressible flows. Based on mass conservation, the

integrated continuity equation has been proposed in the literature for various 3D

fluid flows visualized in a projected image plane in order to link the image inten-

sity function I to a vertically averaged horizontal velocity field v [1–3]. However,

observation models can not be used alone, as they provide only one equation

for two unknowns at each spatio-temporal locations (s,t). This constitute the

so-called aperture problem.

(1)

Standard regularizer limitations To deal with this problem, the most com-

mon assumption consists in enforcing locally spatial coherence. Global regular-

ization schemes over the entire image domain Ω are convenient to model spatial

dependencies on the complete image domain. On the contrary to disjoint esti-

mation approaches [4], dense velocity fields are estimated even in the case of

noisy, low contrasted and incomplete observations. More precisely, the motion a

posteriori estimation problem is defined as the global minimization of an energy

function composed of two components:

f(I,v) = fd(I,v) + αfr(v)(2)

The first energy fd(v,I), called the data term, penalizes discrepancies from

the observation models and thus can be related to a likelihood probability. For

example, discretizing in time the OFC equation, one can build the data term:

fd(I,v)=1

2

Ω

where˜I denotes the image I(t + ∆t). A robust penalty function φ can be in-

troduced in the data term for attenuating the effect of observation outliers de-

viating significantly from the model. In this work, φ are Leclerc semi-quadratic

M-estimator as proposed in [5]. The second component fr(v), called the reg-

ularization term, acts as a spatial prior enforcing the solution to follow some

smoothness properties. In the previous expression, α > 0 denotes a regulariza-

tion parameter controlling the balance between the smoothness and the global

adequacy to the observation model. In this framework, Horn and Schunck [6]

proposed a first-order regularization of the two spatial components u and v of

velocity field v:

fr(v) =1

2

Ω

However, a first order regularization is not adapted to fluid flows as it penalizes

spatially non-homogeneous velocity fields. Second order regularizers on motion

?

φ

?

(˜I−I+v · ∇˜I)2?

ds

(3)

?

||∇u||2+||∇v||2ds

(4)

inria-00325807, version 1 - 30 Sep 2008

Page 3

Self-similar regularization of optic-flow for turbulent motion estimation3

vorticity and divergence have been proposed to overcome those limitations [2,

7,8]. Nevertheless, such regularization models fail to describe the multi-scale

structure of turbulent velocity fields. Efficient multi-scale regularizers based on

fractal priors have already been introduced in [9]. However, such multi-scale

models have not been linked to physical prior on turbulence. Furthermore, using

an energy based on two components raises the difficult problem of fixing some

regularization parameters.

Multi-resolution approach

is the estimation of large displacements. Indeed, these equations are only valid

if the solution is in the region of linearity of the image intensity function. A

standard approach for tackling non-linearity is to apply successive linearizations

around a current estimate and to warp a multi-resolution representation of the

data accordingly. More precisely, a large displacement field ˜ v is first estimated

with the original data term at coarse resolution, where the linearity assumption

is valid. Then, introducing the decomposition:

v = ˜ v + v?,

motion is refined through an incremental fields v?estimated using a linearized

motion-compensated data term while going down the resolution levels of an

image pyramid [10]. Splines are in this context advantageous interpolators useful

to derive accurate motion-compensated images at the different resolutions [11].

One major problem with observation models

(5)

2.2

In the turbulence community, turbulent motion is known to be structured as a

scale invariant spatial process. In order to define scale invariance, let us introduce

the function of velocitity increments:

δv(s,?) = v(s + ?) − v(s).

where ? represents the norm of increment ?. We also introduce the longitudinal

δv?(?) and transverse δv⊥(?) functions defined as δv?(s,?) = v(s + ?t) − v(s)

and δv⊥(s,?) = v(s + ?n) − v(s), with t and n denoting the tangential and

the normal unitary vectors of any bidimensional orthogonal basis of the image

plane. No confusion should be made here with the normal and tangential optic-

flow components. As a classical hypothesis in turbulence studies, we assume

homogeneity and isotropy, that is to say we consider that the statistical proper-

ties of the velocity field are invariant under translation of spatial location s and

rotation of ?. In agreement with these assumptions, index to spatial locations

s can be dropped and a simple scalar velocity increment function δv(?) can be

defined in the bidimensional plane either by δv?(?) · t,

or δv⊥(?)·n. For any of these scalar quantities, Kolmogorov [12] demonstrated

that from a statistical point of view the turbulent flow is self-similar, i.e. there

exists a unique scaling exponent h ∈ R such that:

δv(λ?) = λhδv(?), ∀λ ∈ R+,

A corollary is that the second order moment of the probability distribution func-

tion P?(δv) of velocity increments, namely the second order structure function:

?

Turbulence self-similar models

(6)

δv?(?) · n, δv⊥(?) · t

(7)

E[δv(?)2] =

R

δv(?)2P?(δv(?))dδv(?)(8)

inria-00325807, version 1 - 30 Sep 2008

Page 4

4 P. H´ eas, E. M´ emin, D. Heitz

follows a power law of universal exponent ζ:

E[δv(?)2] − β?ζ= 0.

(9)

For three dimensional turbulence, the scale range I of the power law, so-called

inertial range, is defined for ? ∈ [η,?0], where η represents the dissipative scale

and where ?0is much smaller than the diameter L of the largest vortex. In this

range, Kolmogorov demonstrated that ζ = 2/3. Analogously, for bi-dimensional

turbulence with energy injection at scale ?0, Kraichnan showed that there exist

two different self-similar processes: ζ = 2 within a range I1 = [η,?0] and ζ =

2/3 within a range I2= [?0,L] [13]. For atmospheric turbulence, measurements

proved an inversion of the the two former power laws: ζ = 2/3 within a range I1=

[1,500] kilometers and ζ = 2 within a range I2= [1000,3000] kilometers [14].

Moreover, for any flow, there exits a power law of scaling exponent ζ = 2 in the

dissipative range I0= [0,η].

3

In this section we propose to close the optic-flow equations by introducing

physical-based self-similar constraints. Besides providing a closure for motion

estimation, such a self-similar regularizer yields several benefits:

– first, self-similar processes are multi-scale models which will structure motion

fields across scales in agreement with physics,

– second, solving optimally the optic-flow minimization problem under self-

similar constraints will lead to a non-parametric method where the problem

of fixing regularization parameter α does no longer exist,

– last, the motion estimation problem can be solved using standard convex

optimization methods.

Self-similar regularization of optic-flow

3.1

Let us first formalize self-similar constraints. The second order structure function

E[δv(?)2] is an expectation defined in Eq. 8. We approximate this expectation

by a statistical average over the image domain Ω:

1

|Ω|

where |Ω| denotes the image domain area. In order to obtain an accurate expec-

tation estimator over the bi-dimensional plane and in order to avoid boundary

effects, we build a scalar structure function by averaging the norm of transverse

and longitudinal structure functions in the 2 directions:

?

A constraint g?(v) is then defined for each scale ? as the difference between

the second order structure function depending on the velocity field v, and the

predicted power law. Thus, for any scale ? ∈ ∪Ii, an estimated motion field

should respect the constraint :

g?(v) =1

2(E[δv(?)2] − βi?ζi) = 0, ∀? ∈ ∪Ii

for given scaling exponent ζiand factor βi.

Self-similar constraints

E[δv(?)2] ≈

?

Ω

δv(?)2ds

(10)

δv(?) =

1

8(||δv?(?)||2+ ||δv⊥(?)||2+ ||δv?(−?)||2+ ||δv⊥(−?)||2)(11)

(12)

inria-00325807, version 1 - 30 Sep 2008

Page 5

Self-similar regularization of optic-flow for turbulent motion estimation5

3.2

Referring to section 2.1, the minimization related to the unclosed optic-flow

estimation problem reads:

(ˆ v) = argmin

v

Adding the self-similar constraints, we obtain the following constraint minimiza-

tion problem:

3.3Discrete problem formulation

Let us now express the constraint problem in its discrete form. The derivatives

∇vfd(I,v) related to any motion-compensated data term (which is quadratic

with respect to motion increments v?) can be expressed in the matricial form

A0v?−b0, when discretized on an image grid S of m points with a finite difference

scheme. The two discretized components of v?now represents a field of n = 2m

variables supported by the grid S, A0is a n×n symmetric positive-definite,b0∈

Rnrepresents a vector of size n. The discrete data term can be rewritten as:

fd(I,v) =1

2v?TA0v?− bT

where c0 ∈ R denotes a scalar. For the self-similar constraints, as detailed in

appendix A, the quadratic constraint derivatives can be expressed in the vectorial

form A?v?− b?, where A?are symmetric positive semi-definite matrices and b?

are vectors of size n. Thus, using discretized variables v?yields :

g?(v) =1

2v?TA?v?− bT

where c? ∈ R are scalars. In particular, in appendix A we demonstrate that

A?v?= ∇v?g?(v?) and b?= −∇˜ vg?(˜ v) where the derivative reads:

∇vg?(v) = −

4|Ω−{Γt}|

where ∆?represents a discretized laplacian operators defined on a grid with

a mesh equal to ? using a centered second order finite difference scheme and

where we have exculded vertical Γtand horizontal Γnimage borders of width

?. The discretized laplacian operators can thus be interpreted as a collection

of first-order regularizers performing at the different scales. They represent to

some extent a generalization of the Horn and Schunck first-order regularizer

(Eq. 4) to multi-scale. However, an important difference here is that motion

spatial derivatives are not penalized in our case, but constrained to follow a

power law across scales.

The constraint motion estimation problem defined in Eq. 14 can thus be rewrit-

ten in its discrete form as:

where for simplification we have dropped the dependance to the image I.

Constrained motion estimation problem

fd(I,v).

(13)

minvfd(I,v)

under the constraints:

g?(v) = 0, ∀? ∈ ∪Ii

v ∈ Rn

.

(14)

0v?+ c0.

(15)

?v?+ c?= 0, ∀? ∈ ∪Ii,

(16)

1

??

Ω−{Γt+Γn}∆?u ds

Ω−{Γt+Γn}∆?v ds

?

?

+ borderterms,

(17)

(P)

minvfd(v) =1

under the constraints:

g?(v) =1

v = v?+ ˜ v ∈ Rn

2v?TA0v?− bT

0v?+ c0.

2v?TA?v?− bT

?v?+ c?= 0, ∀? ∈ ∪Ii

.

(18)

inria-00325807, version 1 - 30 Sep 2008

Page 6

6 P. H´ eas, E. M´ emin, D. Heitz

3.4

A first idea would be to solve P by penalizing with a quadratic cost deviations

from the self-similar constraints, i.e. f(v) = fd(v) +?

to the fractal regularizer proposed in [9]. However, this new functional is no

longer convex because of (g?(v))2and one may face difficulties when resolving

non-linear problems and fixing the regularization parameters {α?}. An advan-

tageous alternative is to solve instead the associated constraint optimization

problem using classical lagrangian duality. To define optimality conditions, we

now introduce the lagrangian function L(v,λ) associated to (P):

?

In the lagrangian duality formalism, the optimal solutions of the so-called primal

problem P, are obtained by searching saddle points of the lagrangian function.

Saddle points denoted by (v∗,λ∗) are defined as the solutions of the so-called

dual problem:

?L(v∗,λ∗) = maxλw(λ) = maxλ{minvL(v,λ)}

where w(λ) denotes the dual function. As the functions f and g?are convex and

as the constrained group is not empty, for positive and large enough lagrangian

multipliers λ?, L is convex and the minimization problem (P) has a unique saddle

point i.e. an optimal solution v∗which is unique1. Thus in this framework, each

lagrangian multiplier λ?representing the regularization parameter at scale ? can

naturally be inferred.

Dual problem and optimal solution

∪Iiα?g?(v)2, where {α?}

are positive scalars. Such a multi-scale model would be to some extent similar

L(v,λ) = f(v) +

∪Ii

λ?g?(v), λ = {λ?}.

(19)

(D)

λ?∈ R+, ∀? ∈ ∪Ii

,

3.5

The minimum ˆ v?of the convex lagrangian function at point λ can be obtained

by solving the following Euler-Lagrange equations:

Convex optimization

∇vL(v,λ) = ∇vfd(v) +

?

∪Ii

λ?∇vg?(v) = 0.

(20)

which reduce (using to Eq. 15 and Eq. 16) to solve the linear system :

?

The resolution of Euler-Lagrange large system is efficiently achieved using a

conjugate gradient squared optimization method. The dual function is then given

by:

w(λ) = ˆ v?T?

A0+

?

∪Ii

λ?A?

?

ˆ v?= b0+

?

∪Ii

λ?b?.

(21)

A0+

?

∪Ii

λ?A?

?

ˆ v?−

?

b0+

?

∪Ii

λ?b?

?T

ˆ v?+ c0+

?

∪Ii

λ?c?. (22)

1Let us note that for negative lagrangian multipliers, the convexity of the functional is

no longer insured, and for too small lagrangian multipliers, the problem may remain

ill-posed with no guarantee of the unicity of the solution. However, in practice the

problem admits a solution if there exist at least one scale ? with λ??= 0.

inria-00325807, version 1 - 30 Sep 2008

Page 7

Self-similar regularization of optic-flow for turbulent motion estimation7

The dual function is by definition concave and possesses so-called sub-gradients

equal to g?(ˆ v?+ ˜ v). We employ a classical gradient method to find λ∗which

maximizes the dual function and thus obtain the solution v∗. Finally, the con-

straint motion estimation method results in the Uzawa algorithm presented be-

low, which is used to converge towards the unique saddle point (v∗,λ∗), i.e. the

optimal motion estimate under self-similar constraints.

– (a) From any initial point λ0> 0 and estimate ˜ v:

– (b) At iteration k, find increment ˆ v?defining w(λk) by solving Eq. 21

– (c) Define λk+1by: ∀? ∈ ∪Ii, λk+1

– (d) If stopping criterion valid :(v∗,λ∗) = (ˆ v?+˜ v,λk), END.

Else increment k and go back to (b)

?

= λk

?+ ρkg?(ˆ v?+˜ v)

ρkdenotes the displacement step at iteration k. The latter parameter is adjust

at each iteration using a relaxation method proposed in [15].

Uzawa algorithm converging towards (v∗,λ∗).

4 Learning turbulence statistics

In this section we consider some uncertainty in the scaling exponent of the

self-similar model used for motion estimation. Indeed, there may exist for some

particular turbulent flows deviations from theory. The idea is thus to use an a

posteriori estimation framework to learn the power law parameters based on a

coarse motion estimate and theoritical priors.

4.1

Uncertainty on scaling exponents is introduced in the motion estimation scheme

by associating to the unknown self-similar model parameter (ζ,β) an a priori

Gaussian probability distribution:

p(ζ) ∼ N(ζK41,σ2

In Eq.23, the mean ζK41 denotes the exponent predicted by Kolmogorov (see

section 2.2). We also define a Gaussian likelihood probability distribution at

scale ? of the logarithm of the structure function:

Prior distribution for scaling exponents

ζ)(23)

p(logE[δv(?)2]|β,ζ) ∼ N(log(β?ζ),σ2) (24)

Thus, the standard deviation σζis a parameter tuning the degree of uncertainty

on Kolmogorov parameters, while the standard deviation σ represents the al-

lowed deviation of the structure function estimate E[δv(?)2] from the predicted

law.

4.2 Learning power laws

Using Eq. 23 and Eq. 24, scaling exponent ζ is estimated using the Maximum

A Posteriori (MAP) estimator and Bayes law:

inria-00325807, version 1 - 30 Sep 2008

Page 8

8P. H´ eas, E. M´ emin, D. Heitz

ˆζMAP= argmax

ζ

p(ζ)

?

?∈I

p(logE[δv(?)2]|ζ),

(25)

where we have assumed that likelihood probabilities p(logE[δv(?)2]|β,ζ) are in-

dependent at the different scale ?. In Eq.25, I denotes a scale interval. As there

is usually no a priori for β, it is estimated in the sense of the Minimum Mean

Square Error (MMSE). Thus, the model parameter (ˆβMMSE,ˆζMAP) are ob-

tained by minimization of the functional J(β,ζ):

(ˆβMMSE,ˆζMAP) = arg min

??

The minimization is achieved by searching the solution of the linear equations

∇β,ζJ(β,ζ) = 0 based on a coarse estimation of the velocity field with any

standard regularizer. Therefore, an analytical solution is obtained by solving the

two linear equations.

β,ζJ(β,ζ)(26)

?

= arg min

β,ζ

?∈I

(logE[δv(?)2] − log(β?ζ))2+σ2

σ2

ζ

(ζK41− ζ)2

5Experiments

To evaluate the performance of the self-similar regularization, a synthetic par-

Simulated particule image velocity ground truthvelocity estimate

Fig.1. Estimation of two-dimensional turbulence. Left: particle image obtained

by DNS of 2D Navier-Stokes equations. Middle: true velocity field Right: estimate

ticle image sequence was generated based on a two-dimensional turbulent flow

obtained by the direct numerical simulation (DNS) of Navier-Stokes equations,

and based on a particle image generator [16]. Fig. 1 presents one of the particle

images of 256 by 256 pixels, the true underlying velocity field and our estimation

obtained by minimizing the OFC based data-term (Eq. 1) under self-similarity

constraints. Parameters of the self-similarity model were inferred in the dissi-

pative scale range of I0= [1,10] pixels using a Horn&Schunck estimate and a

Gaussian prior power exponent centered on the theoretical value of ζK41= 2 and

with standard deviation σζ= 0.3. Using Eq. 26, we obtained a MAP estimate

equal toˆζMAP= 1.8064. The estimated power law is plotted in Fig. 2 together

with the second order structure function E[δv(?)2] obtained with Horn&Schunck

algorithm and with the proposed method. Note that constraining motion incre-

ments to behave as a self-similar process at small scales yields an enhancement of

the structure function at fine but also at large scales. Therefore, one can conclude

inria-00325807, version 1 - 30 Sep 2008

Page 9

Self-similar regularization of optic-flow for turbulent motion estimation9

0.01

0.1

1

1 10

increment [pixel]

100

second order moment of increment distribution

’ground truth’ u 1:2

g(x)

’estimate with Horn&Schunck algorithm’ u 1:2

’estimate with proposed method’ u 1:2

1e-06

1e-05

1e-04

0.001

0.01

0.1

1

10

100

0.001 0.01 0.1 1

spectral density

frequency [1/pixel]

’ground truth’ u 1:3

’Horn&Schunck (1981)’ u 1:3

’correlation technique (LaVision)’ u 1:3

’Corpetti&al. (2002)’ u 1:3

’proposed method’ u 1:3

2-nd order structure functionEnergy spectrum

average RMSE(ˆ v) = 0,1699

Corpetti & al. (2002)

(div-curl regularization)

average RMSE(ˆ v) = 0,1451

Horn & Schunck (1981)

(gradient penalization)

average RMSE(ˆ v) = 0,1313

LaVision company

(correlation maximization)

average RMSE(ˆ v) = 0,1227

proposed method

(self-similar regularization)

Fig.2. Numerical evaluation of the self-similar regularizer. Upper left: power

law g(x) obtained by a posteriori estimation using a Horn&Schunck regularizer. It fits

the true and the estimated 2-nd order structure function E[δv(?)2] obtained with the

proposed regularizer. Upper right: spectral comparison between a first order, a div-

curl or a self-similar regularizer and an operational correlation-based method (PIV

technique) from LaVision. Below: spatial distribution and average RMSE of different

methods.

that the multi-scale structural information has been propagated through scales.

A comparison with the state of the art is also presented in Fig. 2. One can remark

that the spatial distribution of the Root Mean Square Errors (RMSE) of velocity

field estimated using the self-similar model presents in average much lower values

than RMSE obtained with a div-curl [2] or a first order regularizer [6] or even

with operational correlation-based techniques. The energy spectrum comparison

displayed in the same figure proves that the proposed multi-scale regularization

enhances in particular the estimation of small scales displacements. However,

large scale enhancements can be also noticed when visualizing energy spectra in

natural coordinates.

The multi-scale regularizer has also been assessed on real data. A benchmark

has been constituted with a METEOSAT Second Generation meteorological

image sequence acquired at a rate of an image every 15 min. The image spatial

resolution was 3×3km2at the center of the whole Earth image disk. According

to the physical-based methodology presented in [3], sparse image observations

related to a layer at intermediate altitude have been derived. Moreover, a robust

data term relying on a layer mass conservation model has been used to relate the

inria-00325807, version 1 - 30 Sep 2008

Page 10

10P. H´ eas, E. M´ emin, D. Heitz

0.01

0.1

1

10

100

1000

0.01 0.1

spectral density

frequency [1/pixel]

g_I1(x)

g_I2(x)

’auto-similar regularization’ u 1:2

’first order regularization’ u 1:2

Fig.3. Estimation of atmospheric winds. An horizontal wind field of an atmo-

spheric layer at mid-altitude has been estimated using the physical-based data term

proposed in [3] and the self-similar regularizer. Left: estimated velocity field superim-

posed on a sparse input image of the sequence. Right: estimated energy spectrum fits

the power laws gI1(x) ∝ x−5

the contrary to results obtained by first order regularization

3 and gI2(x) ∝ x−3known to rule atmospheric flows, on

image intensity function to a vertically averaged horizontal wind field. Using the

predicted power exponents of ζ = 2/3 in a range I1= [1,10] pixels (contained

in the theoretical interval of [1,500] kilometers), parameters (ˆβMMSE,ˆζMAP)

were derived with Eq. 26 based on a first order regularized solution. Using this

learnt power law, the proposed Uzawa algorithm was used to converge towards

the solution of minimal cost respecting the self-similar constraints. In Fig. 3,

one can visualize the estimated velocity field which has been superimposed on a

sparse image. A comparison is also provided with a first order regularization in

the spectral domain. On the contrary to classical regularization, one can notice

that the energy spectrum estimated with the proposed method respects the two

power laws known to rule atmospheric flows (−ζ − 1 = −5/3 in I1 = [1,100]

pixels and −ζ − 1 = −3 for scales greater than 100 pixels [12,14]).

6 Conclusions

A closure to the aperture problem for fluid motion estimation is provided in this

paper. It relies on constraining motion increments to follow self-similar processes

which are well known models in the turbulence community. Solving optimally

the associate constrained minimization problem using lagrangian duality leads to

a non-parametric method where the problem of fixing regularization parameter

does no longer exist. Furthermore, standard convex optimization methods can

be used to infer the optimal motion field and its associated lagrangian multipli-

ers. The resulting multi-scale regularizer structures motion fields across scales in

agreement with physics. The methods also integrates a learning stage in order to

authorize deviations from theory: an a posteriori estimation framework is used

to infer the power law parameters characterizing the self-similar model. The su-

periority of the self-similar model on state of the art regularizers is demonstrated

on synthetic particle images obtained by simulation of Navier-Stokes equations.

Experiments performed on a real meteorological image sequence proves that the

inria-00325807, version 1 - 30 Sep 2008

Page 11

Self-similar regularization of optic-flow for turbulent motion estimation 11

self-similar regularizer enhances the motion spectral-consistency in agreement

with atmospheric measurements.

A- Discrete form of the self-similar constraints

From Eq. 10 and Eq. 11, one obtains :

?

Excluding vertical Γt and horizontal Γn image borders of width ? from the

calculation of the statistical average, one gets:

?

+

Ω−{Γn}

with γ = 8|Ω−{Γt}| = 8|Ω−{Γn}|. Manipulating the derivate with respect to

the motion horizontal component u of the constraints g?(v) which is defined in

Eq. 12, one obtains:

?

+

Ω−{Γ+

+

Ω−{Γn}

+

Ω−{Γ+

where (Γ−

included in Γt and Γn. A similar expression can be obtained for the derivate

with respect to vertical component v. Noting the presence of laplacian operators

in the previous equation, the derivate can thus be rewritten in a compact form:

∇vg?(v) = −2

γ

E[δv(?)2] ≈

1

|Ω|

Ω

1

8(||δv?(?)||2+ ||δv⊥(?)||2+ ||δv?(−?)||2+ ||δv⊥(−?)||2)ds

(27)

γE[δv(?)2] ≈

Ω−{Γt}

?

(||v(s) − v(s + ?t)||2+ ||v(s) − v(s − ?t))||2)ds

(28)

(||v(s) − v(s + ?n)||2+ ||v(s) − v(s − ?n)||2)ds

γ∇ug?(v) =

Ω−{Γt}

(2u(s)−u(s + ?t)−u(s − ?t))ds

(29)

?

?

?

t}

(u(s)−u(s + ?t))ds+

?

Ω−{Γ−

t}

(u(s)−u(s − ?t))ds

(2u(s)−u(s + ?n)−u(s − ?n))ds

n)

(u(s)−u(s + ?n))ds+

?

Ω−{Γ−

n}

(u(s)−u(s − ?n))ds,

t,Γ+

t) and (Γ−

n,Γ+

n) denote respectively the left and right borders

??

Ω−{Γt+Γn}∆?u ds

Ω−{Γt+Γn}∆?v ds

?

?

+ borderterms,

(30)

where ∆?represents a two-dimensional discretized laplacian operators defined

on a grid with a mesh equal to ? using a centered second order finite difference

scheme. Considering now the velocity field decomposition v = ˜ v + v?used in

multi-resolution, as the operator is linear one obtains:

∇vg?(v) = ∇˜ vg?(˜ v) + ∇v?g?(v?)(31)

Using Eq. 29 and discretizing the velocity field, the constraints can finally be

written in their discrete form:

inria-00325807, version 1 - 30 Sep 2008

Page 12

12P. H´ eas, E. M´ emin, D. Heitz

g?(v) =1

2v?TA?v?− bT

?v?+ c?= 0(32)

where A?v?= ∇v?g?(v?), b?= −∇˜ vg?(˜ v) and

c?= −β?ζ

22γ

Ω−{Γt}

+

2γ

Ω−{Γn}

+

1

?

||˜ v(s) − ˜ v(s + ?t)||2+ ||˜ v(s) − ˜ v(s − ?t))||2ds

1

?

||˜ v(s) − ˜ v(s + ?n)||2+ ||˜ v(s) − ˜ v(s − ?n)||2ds

References

1. Fitzpatrick, J.: The existence of geometrical density-image transformations cor-

responding to object motion. Comput. Vision, Graphics, Image Proc. 44 (1988)

155–174

2. Corpetti, T., M´ emin, E., P´ erez, P.: Dense estimation of fluid flows. IEEE Trans.

Pattern Anal. Mach. Intell. 24 (2002) 365–380

3. Heas, P., Memin, E., Papadakis, N., Szantai, A.: Layered estimation of atmospheric

mesoscale dynamics from satellite imagery. IEEE trans. on Geo. and Rem. Sensing

45 (2007) 4087–4104

4. Lucas, B., Kanade, T.: An iterative image registration technique with an applica-

tion to stereovision. In: Int. Joint Conf. on Artificial Intel. (IJCAI). (1981) 674–679

5. Black, M., Anandan, P.: The robust estimation of multiple motions: Parametric

and piecewise-smooth flow fields. Computer Vision and Image Understanding 63

(1996) 75–104

6. Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17 (1981)

185–203

7. Suter, D.: Motion estimation and vector splines. In: Proc. Conf. Comp. Vision

Pattern Rec., Seattle, USA (1994) 939–942

8. Yuan, J., Schnoerr, C., Memin, E.: Discrete orthogonal decomposition and varia-

tional fluid flow estimation. Journ. of Math. Imaging & Vison 28 (2007) 67–80

9. Luettgen, M., Karl, W., Willsky, A.: Efficient multiscale regularization with appli-

cations to the computation of optical flow. IEEE Trans. Im. Processing 3 (1994)

41–64

10. Bergen, J., Burt, P., Hingorani, R., Peleg, S.: A 3-frame algorithm for estimating

two-component image motion. IEEE Trans. Pattern Anal. Mach. Intell. 14 (1992)

886–895

11. Unser, M.: Splines: A perfect fit for signal and image processing. IEEE Signal

Processing Magazine 16 (1999) 22–38

12. Frisch, U.: Turbulence : the legacy of A.N. Kolmogorov. Cambridge university

press (1995)

13. Kraichnan, R.: Inertial ranges in two-dimensional turbulence. Phys. Fluids 10

(1967) 1417–1423

14. Lindborg, E.: Can the atmospheric kinetic energy spectrum be explained by two-

dimensional turbulence. J. Fluid Mech. 388 (1999) 259–288

15. Minoux, M.: Programmation math´ ematique : th´ eorie et algorithmes. Dunod Edi-

tions (1983)

16. Carlier, J., Wieneke, B.: Report 1 on production and diffusion of fluid mechanics

images and data. Fluid project deliverable 1.2. http://www.fluid.irisa.fr (2005)

inria-00325807, version 1 - 30 Sep 2008