# Completion Energies and Scale.

**ABSTRACT** Abstract The detection of smooth curves in images and their completion over gaps are two important problems,in perceptual grouping. In this paper we examine,the notion of completion energy and introduce a fast method,to compute,the most likely completions in images. Specifically, we develop two novel analytic approximations,to the curve of least energy. In addition, we introduce a fast numerical method to compute the curve of least energy, and show that our approximations are obtained at early stages of this numerical computation. We then use our newly developed energies to find the most likely completions in images through a generalized summation,of induction fields. Since in practice edge elements are obtained by applying filters of certain widths and lengths to the image, we adjust our computation to take these parameters into account. Finally, we show that, due to the smoothness of the kernel of summation, the process of summing,induction fields can be run in time that is linear in the number of different edge elements in the image, or in log where is the number of pixels in the image, using multigrid methods.

**0**Bookmarks

**·**

**62**Views

- [Show abstract] [Hide abstract]

**ABSTRACT:**We present the first ratio-based image segmentation method that allows imposing curvature regularity of the region boundary. Our approach is a generalization of the ratio framework pioneered by Jermyn and Ishikawa so as to allow penalty functions that take into account the local curvature of the curve. The key idea is to cast the segmentation problem as one of finding cyclic paths of minimal ratio in a graph where each graph node represents a line segment. Among ratios whose discrete counterparts can be globally minimized with our approach, we focus in particular on the elastic ratio [Formula: see text] that depends, given an image I, on the oriented boundary C of the segmented region candidate. Minimizing this ratio amounts to finding a curve, neither small nor too curvy, through which the brightness flux is maximal. We prove the existence of minimizers for this criterion among continuous curves with mild regularity assumptions. We also prove that the discrete minimizers provided by our graph-based algorithm converge, as the resolution increases, to continuous minimizers. In contrast to most existing segmentation methods with computable and meaningful, i.e., nondegenerate, global optima, the proposed approach is fully unsupervised in the sense that it does not require any kind of user input such as seed nodes. Numerical experiments demonstrate that curvature regularity allows substantial improvement of the quality of segmentations. Furthermore, our results allow drawing conclusions about global optima of a parameterization-independent version of the snakes functional: the proposed algorithm allows determining parameter values where the functional has a meaningful solution and simultaneously provides the corresponding global solution.IEEE Transactions on Image Processing 02/2011; 20(9):2565-81. · 3.20 Impact Factor - SourceAvailable from: mas.ecp.fr[Show abstract] [Hide abstract]

**ABSTRACT:**In this work we introduce a hierarchical representation for object detection. We represent an object in terms of parts composed of contours corresponding to object boundaries and symmetry axes; these are in turn related to edge and ridge features that are extracted from the image. We propose a coarse-to-fine algorithm for efficient detection which exploits the hierarchical nature of the model. This provides a tractable framework to combine bottom-up and top-down computation. We learn our models from training images where only the bounding box of the object is provided. We automate the decomposition of an object category into parts and contours, and discriminatively learn the cost function that drives the matching of the object to the image using Multiple Instance Learning. Using shape-based information, we obtain state-of-the-art localization results on the UIUC and ETHZ datasets.International Journal of Computer Vision 01/2011; 93:201-225. · 3.62 Impact Factor - [Show abstract] [Hide abstract]

**ABSTRACT:**We propose a theory for cortical representation and computation of visually completed curves that are generated by the visual system to fill in missing visual information (e.g., due to occlusions). Recent computational theories and physiological evidence suggest that although such curves do not correspond to explicit image evidence along their length, their construction emerges from corresponding activation patterns of orientation-selective cells in the primary visual cortex. Previous theoretical work modeled these patterns as least energetic 3D curves in the mathematical continuous space R(2) × S(1), which abstracts the mammalian striate cortex. Here we discuss the biological plausibility of this theory and present a neural architecture that implements it with locally connected parallel networks. Part of this contribution is also a first attempt to bridge the physiological literature on curve completion with the shape problem and a shape theory. We present completion simulations of our model in natural and synthetic scenes and discuss various observations and predictions that emerge from this theory in the context of curve completion.Neural Computation 09/2012; · 1.76 Impact Factor

Page 1

Completion Energies and Scale

Eitan Sharon, Achi Brandt

Dept. of Applied Math

The Weizmann Inst. of Science

Rehovot, 76100, Israel

?, Ronen Basri

y

Abstract

The detection of smooth curves in images and their com-

pletion over gaps are two important problems in perceptual

grouping. In this paper we examine the notion of com-

pletion energy and introduce a fast method to compute the

most likely completions in images. Specifically, we develop

two novel analytic approximations to the curve of least en-

ergy. In addition, we introduce a fast numerical method to

compute the curve of least energy, and show that our ap-

proximations are obtained at early stages of this numerical

computation. We then use our newly developed energies to

find the most likely completions in images through a gener-

alized summation of induction fields. Since in practice edge

elements are obtained by applying filters of certain widths

and lengths to the image, we adjust our computationto take

these parameters into account. Finally, we show that, due

to the smoothness of the kernel of summation, the process

of summing induction fields can be run in time that is linear

in the number of different edge elements in the image, or in

O

using multigrid methods.

?N log

N? where

N is the number of pixels in the image,

1. Introduction

The smooth completion of fragmented curve segments

is a skill of the human visual system that has been demon-

stratedthroughmanycompellingexamples. Duetothisskill

people often are able to perceive the boundaries of objects

eveninthelackofsufficientcontrastorinthepresenceofoc-

clusions. Anumberofcomputationalstudieshaveaddressed

the problem of curve completion in an attempt to both pro-

vide a computational theory of the problem and as part of a

processofextractingthesmoothcurvesfromimages. These

studies commonly obtain two or more edge elements (also

referredto as edgels)andfindeitherthe most likelycomple-

tionsthatconnecttheelementsorthesmoothestcurvestrav-

?Research supported in part by Israel Ministry of Science Grant 4135-

1-93 and by the Gauss Minerva Center for Scientific Computation.

yResearch supported in part by the Unites States-Israel Binational Sci-

ence Foundation, Grant No. 94-00100.

elingthroughthem. The methodsproposedforthisproblem

generally require massive computations, and their results

strongly depend on the energy function used to evaluate the

curves in the image. It is therefore important to develop

methods which simplify the computation involved in these

methods while providing results competitive with the exist-

ing approaches. Below we present such a method that di-

rectlyrelatestoanumberofrecentstudiesofcompletionand

curvesalience [9, 18, 5, 19, 12, 7] (see also [2, 6, 8, 14, 15]).

Along with simplifying the computationsproposed in these

studies our method also takes into account the size of edge

elements, allowing for a proper computation of completion

and saliency at different scales.

A number of studies have addressed the problem of de-

termining the smoothest completion between pairs of edge

elements [17, 14, 2, 9, 18, 5]. These studies seek to define

a functional that, given two edge elements defined by their

location and orientation in the image, selects the smoothest

curve that connects the two as its minimizing curve. The

most common functional is based on the notion of elas-

tica, that is, minimizing the total squared curvature of the

curve [9]. Scale invariant variations of this functional were

introducedin [18, 5]. While the definitionof scale-invariant

elastica is intuitive, there exists no simple analytic expres-

sionto calculateits shapeoritsenergy,andexistingnumeri-

cal computationsare orders-of-magnitudetoo expensive, as

will be shown below.

In the first part of this paper we revisit the problem

of determining the smoothest completion between pairs

of edges and introduce two new analytic approximations

to the curve of least energy. The first approximation is

obtained by assuming that the deviation of the two input

edgels from the straight line connecting them is relatively

small. This assumption is valid in most of the examples

used to demonstrate perceptual completions in humans and

monkeys [10, 11]. We show that under this simplifying

assumption the Hermite spline (see, e.g., [13]) provides a

good approximation to the curve of least energy and a very

good approximation to the least energy itself. We further

develop a second expression, which directly involves the

Page 2

anglesformedby the edgelsandthe straight line connecting

them. The second expression is shown to give extremely

accurate approximations to the curve of least energy even

when the input edgels deviate significantly from the line

connecting them. We then introduce a new, fast numerical

method to compute the curve of least energy and show that

our analytic approximations are obtained at early stages of

this numerical computation.

Several recent studies view the problems of curve com-

pletion and salience as follows. Given

thespaceofallcurvesconnectingpairsofelementsisexam-

ined in an attempt to determine which of these completions

is most likely using smoothness and length considerations.

For this purpose [7, 19] define an affinity measure between

two edge elements that grows with the likelihood of these

elements being connected by a curve. By fixing one of the

elements and allowing the other element to vary over the

entire image an induction field representing the affinity val-

ues induced by the fixed element on the rest of the image

is obtained. The system finds the most likely completions

for the

summation of the induction fields for all

In the second part of this paper we use our newly devel-

oped completion energies to define an affinity measure that

encourages smoothness and penalizes for gap length. We

thenuse the inductionfieldsdefinedby thisaffinity measure

to solve the problem of finding the most likely completions

for

dimensionless, because they are usually obtained by apply-

ing filters of a certain width and length to the image, we

adjust our affinity measure to take these parameters into ac-

count. We do so by relating the scale of these filters to the

range of curvatures which can be detected by them and to

the orientational resolution needed. Finally, we show that

ouraffinitymeasureis asymptoticallysmooth,andso can be

implementedusing multigrid methods and run efficiently in

time complexity

and

The paper is divided as follows. In Section 2 we review

the notion of elastica and its scale invariant variation. In

Section 3 we introduce the two analytic approximations to

the curve of least energy. Then, in Section 4 we develop a

fast numerical method to compute the curve of least energy

and compare it to our analytic approximations. Finally,

in Section 5 we construct an affinity measure taking into

account the length and width of the edge filters applied to

the image. We then discuss a multiscale (multigrid)method

for fast summation of induction fields.

M edge elements,

M elements by applying a process that includes a

M elements.

M elements. Since in practice edge elements are never

O?nm? (where

n is the number of pixels

m is the numberof discrete orientationsat every pixel).

2. Elastica

Considertwo edgeelements

e1

?e2positionedat

P1

?P2

?

R2with directed orientations Ψ1and Ψ2respectively mea-

sured from the right-hand side of the line passing through

P1and

that Ψ1

may conveniently assume that

This is illustrated in Fig. 1(a).

P2. Below we shall confine ourselves to the case

?Ψ2

???

?

2

?

?

2

?. Denote by

r?kP2

?P1

k, we

P1

?

Let

?0 ?0 ? and

P2

??r?0 ?.

C12denote the set of

C2curves through

e1and

e2. Denote such a curve by its

orientation representation Ψ ?s?, where 0

arclength along the curve. That is,

and

the curve at

?s?L is the

x?s??

R

s

0cos

?Ψ ?ˆ

s ??dˆ

s

y?s??

R

s

0sin ?Ψ ?ˆ

s ??dˆ

s. Also denote the curvature of

s by

??s??dΨ ?s??ds.

r

(a)

Ψ

p

ΦΦ

y

L

x

21

1

f(x)

1

Ψ

2

2

p

Ψ

1

Φ

Φ

θ

Ψ

1

2

2

1θ

=Φ2

Φ1

2

−Ψ1+ =Ψ2

P

L

r

(b)

P

Figure 1. (a) The planar relation between two edge ele-

ments,

Φ1

2222

measured from the line

(b) The more general relation between Φ

?P1

?Ψ1

? and

?P2

?Ψ2

?.This relation is governed by

???

?

?

?

?, Φ2

???

?

?

?

?, and

r, where Φ1and Φ2are

P1

P2, hence Φ1

? Ψ1and Φ2

??Ψ2.

iand Ψ

i.

The most common functional used to determine the

smoothestcurvetravelingthrough

orientations Ψ1and Ψ2is the elastica functional. Namely,

the smoothest curve through

which minimizes the functional Γ

Elastica was already introduced by Euler. It was first ap-

plied to completion by Ullman [17], and its properties were

further investigated by Horn [9].

One of the problems with the classical elastica model is

that it changes its behavior with a uniform scaling of the

image. In fact, according to this model if we increase

the distance between the two input elements, the energy

of the curve connecting them proportionately decreases,

as can be easily seen by rescaling

somewhat counter-intuitive since psychophysical and neu-

robiological evidence suggests that the affinity between a

pair of straight elements drops rapidly with the distance

between them [11]. Also, the classical elastica does not

yieldcirculararcstocompletecocircularelements. Tosolve

theseproblemsWeiss[18,5]proposedtomodifytheelastica

model to make it scale invariant. His functional is defined

as Γ

0

adjustment of the completion energy to scale must take into

account not only the length of the curve (or equivalentlythe

distance between the input elements), but also the dimen-

sionsoftheinputedgeelements. Boththeelasticafunctional

anditsscaleinvariantversionassumethattheinputelements

P1and

P2withrespective

e1and

e2is the curve Ψ ?s?

el

?Ψ ?

def

?

R

L

0

?2

?s?ds?

r,

s (cf. [1]).This is

inv

?Ψ ?

def

?L

R

L

?2

?s?ds? We believe that a proper

Page 3

have no dimensions. In practice, however, edge elements

are frequentlyobtainedby convolvingthe image with filters

of some specified width and length. A proper adjustment

of the completion energy as a result of scaling the distance

between the elements should also consider whether a corre-

spondingscalingin thewidthandlengthofthe elementshas

taken place. Below we first develop useful approximations

tothe scale invariantfunctional. (Theseapproximationscan

alsoreadilybeusedwithslightmodificationstotheclassical

elasticameasure.) Later,inSection5,wedevelopanaffinity

measure between elements that also takes into account both

the distance between the elements and their dimensions.

3. Analytic simplification of Γ

inv

Althoughthedefinitionofboththeclassical andthescale

invariant elastica functionals is fairly intuitive, there is no

simple closed-form expression that specifies the energy or

the curve shape obtained with these functionals. In this

section we introduce two simple, closed-form approxima-

tions to these functionals. Our first approximation is valid

when the sum of angles

assumptionrepresentstheintuitionthatinmostpsychophys-

ical demonstrations gap completion is perceived when the

orientationsofthecurveportionstobe completedare nearly

collinear. Withthisassumptionwemayalsorestrictfornow

the range of applicable orientations to Ψ1

The second approximationwill only assume that

is small, i.e., that the curve portions to be completed are

nearly cocircular.

Since the curve of least energy is supposed to be very

smooth, it is reasonable to assume that within the chosen

range of Ψ1

Consequently,it canbedescribedasafunction

inFig.1(a). Expressingthecurvatureintermsof

obtainthat Γ

0

For

of

Γ

jΦ1

j?jΦ2

j is relatively small. This

?Ψ2

???

?

2

?

?

2

?.

jΦ1

? Φ2

j

?Ψ2the smoothest curve will not wind much.

Ψ ?C12Γ

y?f?x?, as

xand

y we

inv

?Ψ ??L

R

L

?2

?s?ds?L

R

r

0

?f

??

?2

?1??f

?

?2

?

5

2

dx?

?Φ1

?Φ2

?? 0 we get that

L?r, and that the variation

?1

??f

?

?2

? becomes unimportant for the comparison of

inv

?Ψ ? over different curves Ψ

?C12, so that Γ

inv

?Ψ ? ?

r

R

r

0

?f

??

?2

dx? Hence

E

inv

def

? min

inv

?Ψ ??r min

Ψ ?C12

Z

r

0

?f

??

?2

dx?

(1)

The minimizing curve is the appropriate cubic Hermite

spline (see [1])

f?x??x?x?r?

?1

r2

?t1

?t2

?x?

t1

r

?

?

(2)

where

t1

? tanΦ1and

t2

? tanΦ2, so that

E

inv

? 4 ?t2

1

?t2

2

?t1

t2

??

(3)

Evidently, this simple approximation to

independent. This leads us to define the scale-invariant

E

inv is scale-

spline completion energy as:

Although the spline energy provides a good

approximation to the scale invariant elastica measure for

small values of

values. An alternative approximation to

structed by noticing that for such small values tanΦ1

and tanΦ2

E

spln

?Φ1

?Φ2

?

def

? 4 ?t2

1

?

t2

2

?t1

t2

??

jΦ1

j?jΦ2

j, the measure diverges for large

E

invcan be con-

? Φ1

? Φ2. Thus, we may define:

E

ang

?Φ1

?Φ2

?

def

? 4 ?Φ2

1

? Φ2

2

? Φ1Φ2

??

(4)

We refer to this functional as the scale-invariant angular

completion energy. This measure does not diverge for large

values of

obtain

that this angular energy is obtained in an early stage of the

numericcomputationof

accurate approximations to the scale invariant least energy

functional even for relatively large values of

pecially for small

cocircular elements. Using the numeric computation we

can also derive the smoothest curve according to

jΦ1

j?jΦ2

j. In fact, when Φ1

? Φ2

???2 we

E

ang

?E

inv

??2. In Section 4 below we show

E

inv,andthatitprovidesextremely

jΦ1

?

j+jΦ2

? 2. That

j, es-

jΦ1

? Φ2

j, i.e., for the range of nearly

E

ang:

¯Ψ ?s?? 3 ?Ψ1

? Ψ2

?s2

??4Ψ1

? 2Ψ2

?s? Ψ1

?

(5)

The angular completion energy can be generalized as

follows:

E

gang

?Φ1

?Φ2

??a?Φ2

1

? Φ2

2

??b?Φ1

? Φ2

?2

?

(6)

where Eq. (4) is identical to Eq. (6) with

is, the angular completion energy is made of an equal sum

of two penalties. One is for the squared difference between

Φ1and Φ2, and the other is for the growth in each of them.

This suggests a possible generalization of

weights

energies such as

more elaborate study of these types of energies and their

properties is presented in [1].

Finally, we note that the new approximations at small

anglescan also be used to approximatethe classical elastica

energy, since

essary condition for˜Ψ ?˜

should satisfy for some

2˜Ψ

ab

E

ang to other

a? 0 and

b? 0. One can also think of using

E

circ

?Φ1

?Φ2

?

def

?aΦ2

1

?b?Φ1

? Φ2

?2. A

E

el

def

? min

Ψ ?C12Γ

el

?Ψ ??1

r

E

ang

?1

r

E

spln

?

(7)

4. Computation of

We use the scale-invariance property of Γ

reformulatethe minimizationproblemintominimizingover

all

Euler-Lagrangeequations(see, e.g., [13]) we get that a nec-

E

inv

invin order to

C12curves of length

L? 1. (See also [5]). Applying

s? to be an extremal curve is that it

?:

??

??cos˜Ψ

s?t?

(8)

Z1

0

sin ?˜Ψ ?˜

s??d˜

s? 0 ?

˜Ψ ?0 ?? Ψ1

?

˜Ψ ?1 ?? Ψ2

?

Page 4

Considering the very nature of the original minimization

problem, and also by repeatedly differentiating both sides

oftheODEequation,itcanbeshownthatitssolutionmustbe

very smooth. Hence, we can well approximate the solution

by a polynomial of the form

˜Ψ

n

?s???1

?s?Ψ1

?sΨ2

?s?1

?s?

n

X

k?0

a

k

s

k

?

(9)

where

same problem presented in [5] is far less efficient, since it

does not exploit the infinite smoothness of the solution on

the full interval (0,1). As a result the accuracy in [5] is

only second order, while here it is “

decreases exponentially in the number of discrete variables

n is small. (By comparison, the discretization of the

?-order”, i.e., the error

n+2 ( i.e.,

??a0

?????a

n from Eq. (8) and (9) ).) Fixing

n, as well as two other integers ¯

followingsystem of

n and

p, we will build the

n?2equationsforthe

n?2unknowns

a0

?a1

?????a

n

?and?

˜Ψ

??

n

?

i? 1

n? 2

???

ncos˜Ψ

n

?

i? 1

n? 2

?? 0

??0

?i?n?

collocating the ODE, and

where

merical integration.

and increase ¯

the discretization error will not be governed by the dis-

cretization error of the integration. The nonlinearsystem of

P¯

n

j?0

w

jsin˜Ψ

n

?j?¯

n?? 0,

w

j

?0

?j? ¯

n? are the weights of a

Generally, we increase

p-order nu-

n gradually

n and p as functions of

n in such a way that

n

Newton-Raphson; see, e.g. [13].) We start the Newton iter-

ationsfroma solutionpreviouslyobtainedfora system with

a lower

for each value of

this way convergence is extremely fast. At each step, in

just several dozen computer operations, the error in solving

the differential equation can be squared. In fact, due to the

smoothnessofthesolutionfortheODE,alreadyforthesim-

ple

? 2 equations is solved by Newton iterations (also called

( p

n. Actually, only one Newton iteration is needed

values of ( n,¯

by simple analytic expressions, as indeed we show in [1] by

comparingbetweenseveralsimpleapproximationsto

Fig. 2 illustrates some of the completionsobtained using

n if

n is not incremented too fast. In

?n? 0 ? ¯

n? 2 ?-system andtheSimpsonintegrationrule

? 3), a very good approximationto the accurate solution

˜Ψ ?s??

[1].

? and also to

The good approximations obtained already for small

E

inv is obtained, as can be seen in

n) suggest that

E

invcan be well approximated

E

inv.

E

It can be seen that the differences between the three curves

is barely noticeable, except in large angles where

diverges. Notice especially the close agreement between

thecurveobtainedwiththeangularenergy(Eq.(5))andthat

obtained with the scale-invariant elastica measure even in

large angles and when the angles deviate significantly from

cocircularity.

invand the two analytic approximations

E

angand

E

spln.

E

spln

Note that although the spline curve does not approxi-

mate the scale invariant elastica curve for large angles

and

elements. In fact, when the two elements deviate from co-

circularity the elastica accumulates high curvature at one of

its ends, whereas in the spline curve continues to roughly

followthetangentto thetwo elementsat bothends(see, e.g,

Fig. 2(b)) . This behavior is desirable especially when the

elements represent long curve segments (see Section 5.2).

jΦ1

j

jΦ2

j it still produces a reasonable completion for the

5. Completion field summation

Until now we have consideredthe problemof findingthe

smoothest completion between pairs of edge elements. A

natural generalization of this problem is, given an image

from which

likelycompletionsconnectingpairsofelementsintheimage

and rank them according to their likelihoods. This problem

has recently been investigated in [7, 19]. In these studies

affinity measures relating pairs of elements were defined.

Themeasuresencourageproximityandsmoothnessofcom-

pletion. Using the affinity measures the affinities induced

by an element overall other elements in the image (referred

to as the induction field of the element) are derived. The

likelihoods of all possible completions are then computed

simultaneously by a process which includes summation of

the induction fields for all

An important issue that was overlooked in previous ap-

proaches, however,is the issue of size of the edge elements.

Most studies of curve completion assume that the edge el-

ements are dimensionless. In practice, however, edge ele-

ments are usually obtained by convolving the image with

filters of certain width and length. A proper handling of

scale must take these parameters into account. Thus, for

example, one may expect that scaling the distance between

two elements would not result in a change in the affinity of

thetwoelementsiftheelementsthemselvesarescaledbythe

same proportion. Below we first present the general type of

non-scaled induction underlying previous works. We then

modify that induction to properly account for the width and

length of the edge elements.

Finally, the process of summing the induction fields may

be computationallyintensive. Nevertheless, in the third part

of this section we show that the summation kernel obtained

with our method is very smooth. Thus, the summation of

our induction fields can be speeded up considerably using a

multigrid algorithm. This result also applies to the summa-

tionkernelsin[19,16,7],andsoanefficientimplementation

of these methods can be obtained with a similar multigrid

algorithm.

M edge elements are extracted, find the most

M elements.

5.1. Non-scaled induction

In[12,19]amodelforcomputingthelikelihoodsofcurve

completions, referred to as Stochastic Completion Fields,

Page 5

00.20.40.60.81

0

0.05

0.1

(a)

0 0.2 0.40.6 0.81

0

0.2

0.4

0.6

0.8

1

(b)

Figure 2. Completion curves: elastica in solid line,¯Ψ?s? (Eq. (5)) in dotted line, and the cubic Hermite spline (Eq. (2)) in dashed line. (a)

Φ1

? 30

?

?Φ2

? 15

?, (b) Φ1

? 80

?

?Φ2

? 20

?.

was proposed. According to this model, the edge elements

in the image emit particles which follow the trajectories

of a Brownian motion. It was shown that the most likely

path that a particle may take between a source element and

a sink element is the curve of least energy according to

the Elastica energy function1. To compute the stochastic

completionfieldsaprocessofsummingtheaffinitymeasures

representing the source and sink fields was used. In [1]

we show, by further analyzing the results in [16], that the

affinity measure used for the induction in [19, 16] is of

the general type:

Fig. 1(b) ), where

parameters. These parameters need to be adjusted properly

according to the scale involved (see Sec. 5.2). Note that

for small values of (

A?e1

?e2

?

def

?e

?r ?r0

e

?E

ang

??r?0

?(see

r0and

?0are strictly positive a-priori set

jΦ1

j,jΦ2

j):

E

ang

?r?E

el. Hence,

A?e1

?e2

??e

?r ?r0

e

?E

el

??0.

Another method which uses summation of induction

fieldstocomputethesalienceofcurveswaspresentedin[7].

In their method the affinity between two edge elements

whicharecocircularhastheform:

are strictly positiveconstants,

connecting

e

??r

e

???, where

? and

?

? is the curvatureof the circle

e1and

e2, and

r is the distance between

e1and

e2. A reasonableand straightforwarddefinitionin that spirit

is˜

proximation for

an example of computing the “stochastic completion field,”

suggested by Williams and Jacobs in [19], while replacing

their affinity measure with the simple expression˜

It can be verified by comparing the fields obtained with our

affinity measure with the fields presented in [19] that the

results are very similar although a much simpler affinity

measure was employed.

A?e1

?e2

?

def

?e

??r

e

??E

splnwhere

E

splnserves as an ap-

E

inv according to Eq. (3). Fig. 3 shows

A?e1

?e2

?.

5.2. Induction and scale

Givenanimage,anedgeelementisproducedbyselecting

a filter of a certain length

filters) and convolving the filter with the image at a certain

position and orientation. The result of this convolution is

a scalar value, referred to as the response of the filter. An

edge filter may, for example, measure the contrast along

its primary axis, in which case its response represents the

“edgeness level”, or the likelihood of the relevant subarea

of the image to contain an edge of

l and width

w (e.g., rectangular

?l?w? scale. Similarly,

1Actually, the path minimizes the energy functional

for some predetermined constant

R

L

0

?2

?s?ds? ?L

?.

(a)(b)

Figure 3. Stochastic completion fields (128

orientations) with the induction

and Φ2

resemble those obtained in [19].

? 128 pixels, 36

e

?2r

e

?20E

spln. (a) Φ1

? 30

?

? 30

?, (b) Φ1

? 30

?and Φ2

??30

?. The results closely

a filter may indicate the existence of fiber-like shapes in the

image, in which case its response represents the “fiberness

level”oftherelevantsubareaoftheimage. Belowweusethe

term “straight responses” to refer to the responses obtained

by convolving the image with an edge or a fiber filter.

Consider now the edge elements obtained by convolving

the image with a filter of some fixed length

Every edge element now is positioned at a certain pixel

and is oriented in two opposite directed orientations Ψ and

Ψ

fully represent the image at this scale depends on

l and width

w.

P

??. The number of edge elements required to faith-

tion ( O

image) of all significantly different edge elements.

Given a particular scale determined by the length

width

pletion field for this scale. Note that only curves within

a relevant range of curvature radii can arouse significant

responses for our

vature radius that will arouse a still significant response by

l and

w. Thus, long and thin elements require finer resolution

in orientation than square elements. In fact, the orienta-

tional resolution required to sample significantly different

orientationsincreases linearlywith

elements of larger size require less spatial resolution than

elements of smaller size. Brandt and Dym ([4]) use these

observations in order to introduce a very efficient computa-

l?w (see [4]). Similarly,

?N log

N?, where

N is the number of pixels in the

l and

w of edgeelements, we wouldlike to computea com-

l?w elements. Denote the smallest cur-

?

cant responses also in larger

a farther-reaching and more orientation-specific continua-

tion.)

By Fig. 4(a) we see that

???l?w?. (Larger curvature radii will arouse signifi-

l ?w scales, implying there for

l?2

??sin

???? and

w?

???cos

????2

?2. Consequently, we have

?l?2 ??2

?

Page 6

?2

( l,w) we define the completion energy between the pair of

straight responses so as to depend on the scaled turning an-

? 2 w

Next, consider a pair of straight responses. Assuming

these elements are roughly cocircular, then, using the rela-

tions defined in Fig. 4(b) , the differential relation Ψ

pΦ2

??, implying that

??l2

?8 w.

?

?s??

1 ???s?canbeapproximatedby

so that Ω

Hence,forcompletionataparticularscale

sonabletodefineforeverypairofpoints

the turning angle Ω given by

?Ψ1

?Ψ2

??r? 2 ???1

??2

?,

?r ??.

?l?w?,itisrea-

P1and

P2ascalefor

r ???l?w?. That is, in the scale

gleΩ??r. SinceΩ

? Φ1

?Φ2, it isstraightforwardto show

that0 ?5Ω

for the scaled angular energy, therefore, is a monotonically

decreasing function of

?

p

E

ang

?Φ1

?Φ2

?? Ω. A reasonabledefinition

???r?

1

? Φ2

2

? Φ1Φ2.

l

w

α

(a)

ρcosα

ρ

+

2

Ω =Ψ1−

−Φ11

= Ψ

Ψ

Φ2

Φ1

2

Ψ

2

Φ

r

(b)

2= Φ1+ Φ2

Ψ

Ψ1

Ω

ρ

ρ1

2

Figure 4. (a) The relation between

l, w,and the curvature radius

?. (b) The turn Ω that a moving particle takes in its way between

two straight responses, characterized each by a planar location and

an orientation.

Obviously,inanygivenscaleofstraightresponses,

for every Φ1and Φ2, the induction of

decreasewith an increaseof

inducedbyan element

?l?w?,

P1upon

P2should

r??. Hence, we define the field

e1of length

l andwidth

w at location

P1and directed orientation Ψ1on a similar element

e2at

?P2

?Ψ2

? by

G

?l?w?

?e1;

e2

?f?u1

??

(10)

where

some appropriate function of this response, and

u1denotes the strength of response at

e1,

f?u1

? is

G

?l?w?

?e1;

e2

??F

d

?

r

?

?

F

t

?

?

r

q

Φ2

1

? Φ2

2

? Φ1Φ2

?

?

(11)

F

respectively) are smoothly decreasing dimensionless func-

tions that should be determined by further considerations

and experience. Thus, our summation kernel is a product

of the orientational and the spatial components involved in

completing a curve between

below, this definition has many computational advantages.

Let

scale

edges

dand

F

t(the distance and turning attenuation functions,

e1 and

e2. As we shall see

fu

i

g denote the set of straight responses for a given

?l?w?, where each

u

iis associated with two directed

e

i

??P

i

?Ψ

i

? and ¯

e

i

??P

i

?Ψ

i

???. The total

field induced at any element

e

j

??P

j

?Ψ

j

? by all elements

fe

i

? ¯

e

i

g is expressed by

v

j

def

?

X

i

?

G

?l?w?

?

e

i;

e

j

?

?G

?l?w?

?¯

e

i;

e

j

??

f?u

i

?? (12)

The total field induced at ¯

e

jby

fe

i

? ¯

e

i

g is given by

¯

v

j

def

?

X

i

?

G

?l?w?

?

e

i; ¯

e

j

?

?G

?l?w?

?¯

e

i; ¯

e

j

??

f?u

i

?? (13)

Since in general the responses obtained by convolving the

image with edge filters are bi-directional we may want to

combine these two fields into one. This can be done in

variousways. Thesimplestwayisto takethesum

as the completion field. Another possibility, in the spirit of

[19], is to take the product

Note that the field of a long straight response should

be very different (farther-reaching and more orientation-

specific) than the sum of the fields of shorter elements

composing it, and should strongly depend on its width (see

Fig. 5). This suggests that for a comprehensive completion

process one must practice a multiscale process, performing

a separate completion within each scale. The scaled in-

duction field (10)-(11), avoids a fundamental difficulty of

non-scaled fields like [7, 19, 16]. The latter exhibit so weak

a completion for far elements, that it would be completely

masked out by local noise and foreign local features.

fv

j

? ¯

v

j

g

fv

j¯

v

j

g as the completion field.

0

10

20

30

40

50

60

(a)

0

10

20

30

40

50

60

(b)

Figure 5. Induction fields (200

using

?200 pixels) in different scales

F

d

????e

??1

?

?1,and

F

t

????e

??2

?

?2,where

?1

? 0 ?5,

?1=0.5,

long element:

sum of induction fields of the three shorter elements composing

this long element, each consist of:

orientations.

?2

? 128, and

?2

? 1 ?5. (a) The induction field of one

l? 9,

w? 1 ?2, 25 directed orientations. (b) The

l? 3,

w? 1 ?2, 12 directed

5.3. Fast multigrid summation of induction-fields

Let

n?n?l?w? be the number of sites (

P), and

m

that are required in order to describe all the

responses that are significantly different from each other.

It can be shown (see [4]) that if

pixel units then, for any N-pixel picture,

and

?m?l?w? the number of orientations (Ψ) at each site,

l?w straight

l and

w are measured in

n?O?N?lw?

m?O?lN?w?, so the total number of

l?w elements

Page 7

is

(e.g.,

elements is

the responses at all these elements can be calculated in only

O?N ?w?. Hence, for any geometric sequence of scales

l=1,2,4,...,and

w=1,3,9,...) thetotalnumberofstraight

O?N log

N?. It has been shown (in [4]) that all

O

constructs longer-element responses from shorter ones.

At any given scale

tions (Eqs. (12) and (13)), summing over

for each value of

tal of

can be performed in parallel to each other, as in [20]).

However, using the smoothness properties of the partic-

ular kernel (11), the summation can be reorganized in a

multiscale algorithm that totals only

(and the number of unparallelizable steps grows only log-

arithmically in

usually take on the typical form

For such choices of the functions,

and practically for any other reasonable choice, the kernel

?N log

N? operations, using a multiscale algorithm that

l?w, it seems that the summa-

i? 1 ?2 ????? nm

j? 1 ?2 ????? nm, would require a to-

O?n2

m2

? operations (even though some of them

O?nm? operations

nm). Indeed, the functions in (11) would

F

d

????e

??1

?

?1, and

F

t

????e

??2

?

?2.

G

of “asymptotic smoothness.” By this we mean that any

orderderivativeof

decays fast with

higher

(even the smallest, i.e.,

function of Ψ

Due to the asymptotic smoothness, the total contribution

to

of

eachj,butcanbeinterpolated(

small an error as desired by using sufficiently high

its valuesat a few representativepoints. Forthis and similar

reasons, multiscale algorithms, which split the summations

intovariousscalesof farness(see detailsin [3])canperform

all the summations in merely

6. Conclusion

?G?e

i;

e

j

??G?x

i

?y

i

?Ψ

i;

x

j

?y

j

?Ψ

j

? has the property

q-

G withrespecttoanyofitssixarguments

r

ij

?

?

?x

i

?x

j

?2

??y

i

?y

j

?2

?1

2, and the

q is the faster the decay is. Also, for any fixed

r

ij

r

ij

?

O

O

?nm? operations.

?l?), G is a very smooth

iand of Ψ

j.

v

j(and ¯

v

j)ofallelementsfarfrom

P

jisasmoothfunction

?x

j

?y

j

?Ψ

j

?, henceitneednotbecomputedseparatelyfor

q-orderinterpolation,withas

q) from

Important problems in perceptual grouping are the de-

tection of smooth curves in images and their completion

over gaps. In this paper we have simplified the computa-

tion involved in the process of completion, exploiting the

smoothnessof the solution to the problem,and have defined

affinity measures for completion that take into a proper ac-

count the scale of edge elements. In particular, we have

introduced new, closed-form approximations for the elas-

tica energy functional and presented a fast numeric method

to compute the curve of least energy. In this method the

error decreases exponentially with the number of discrete

elements. We then have used our approximations to de-

fine an affinity measure which takes into account the width

and length of the edge elements by considering the range

of curvaturesthat can be detected with correspondingfilters

of the same scale. Finally, we have shown that solutions

to the problem of finding the most likely completions in an

image can be implemented using a multigrid algorithm in

time that is linear in the number of discrete edge elements

in the image. This last observation applies also to recent

methods for completion and salience [7, 19]. In the future

we intend to use the multigrid algorithm to simultaneously

detect completions at different scales in order to combine

these completions into a single saliency map.

References

[1] E. Sharon, A. Brandt, and R. Basri, “Completion energies and scale,”

Forthcoming Technical Report.

[2] M. Brady, W. E. L. Grimson, and D. J. Langridge, “Shape Encoding

and Subjective Contours,” Proc. First Annual Conf. Artif. Int., 1980.

[3] A. Brandt, “Multilevel computations of integral transforms and parti-

cleinteractionswithoscillatorykernels,” Comp.Phys.Comm.,65:24–

38, 1991.

[4] A. Brandt and J. Dym, “Fast computation of multiple line integrals,”

SIAM J. Sci. Comp., to appear.

[5] A. M. Bruckstein, and A. N. Netravali, “On Minimal Energy Trajec-

tories,” CVGIP, 49:283–296, 1990.

[6] S. Grossberg and E. Mingolla, “The Role of Illusory Contours in

Visual Segmentation,” The Perception of Illusory Contours, Petry, S.

and Meyer, G. (eds.), Springer-Verlag, New York:116–125, 1987.

[7] G. Guy and G. Medioni, “Inferring Global Perceptual Contours from

Local Features,” Proc.DARPAImage Understanding Workshop:881–

892, 1993.

[8] S. Heitger, and R¨ udiger von der Heydt, “A Computational Model of

Neural Contour Processing: Figure-Ground Segregation and Illusory

Contours,” Proc. of ICCV:32–40, 1993.

[9] B. K. P. Horn, “The Curve of Least Energy,” ACM Trans. on Math.

Soft., 9(4):441–460, 1983.

[10] G. Kanizsa, “Organization in Vision,” Praeger, New York, 1979.

[11] K. K. Mitesh, L. Minami, D. E. Charles, and W. Gerald, “Improve-

ment in visual sensitivity bychanges in local context: Parallel studies

in human observers and in V1 of alert monkeys,” Neuron, 15:843–

856, 1995.

[12] D. Mumford, “Elastica and Computer Vision,” Algebraic Geometry

andItsApplications, ChandrajitBajaj(ed.), SpringerVerlag, pp.491–

506, 1994.

[13] C. E. Pearson (ed.), “Handbook of applied mathematics - Second

Edition,” Van Nostrand Reinhold Company, New York, 1983.

[14] W. S. Rutkowski, “Shape Completion,” CVGIP, 9:89–101, 1979.

[15] A. Sha’ashua and S. Ullman, “Structural Saliency: The Detection

of Globally Salient Structures Using a Locally Connected Network,”

2nd ICCV:321–327, 1988,

[16] K. K. Thornber and L. R. Williams, “Analytic Solution of Stochastic

Completion Fields,” Biological Cybernetics, 75:141–151, 1996.

[17] S. Ullman, “Filling-In the Gaps: The Shape of Subjective Contours

and a Model for Their Generation,” Biological Cybernetics, 25:1–6,

1976.

[18] I. Weiss, “3D Shape Representation by Contours,” CGVIP, 41:80–

100, 1988.

[19] L. R. Williams and D. W. Jacobs, “Stochastic Completion Fields: A

Neural Model of Illusory Contour Shape and Salience,” ICCV:408–

415, 1995.

[20] L. R. Williams and D. W. Jacobs, “Local Parallel Computation of

Stochastic Completion Fields,” CVPR:161–168, 1996.

#### View other sources

#### Hide other sources

- Available from Ronen Basri · May 22, 2014
- Available from psu.edu
- Available from psu.edu