Page 1

Arc of ellipse detection for video image registration

Arnaud Le Troter, R´ emy Bulot, Jean-Marc Boi, Jean Sequeira

Laboratoire des Sciences de l’Information et des Syst` emes (LSIS - UMR 6168)

Equipe LXAO (ESIL) - case 925 - 163, avenue de Luminy

13288 Marseille cedex 9, France

Email: Jean.Sequeira@esil.univ-mrs.fr

Abstract—In this paper, we give a detailed presentation of a

robust algorithm for detecting arcs of ellipse in a binary image.

The characterization of such arcs of ellipse enables the identifica-

tion between some video image elements and the corresponding

landmarks in a 3D model of the scene to be represented. This

algorithm is based on a classical ellipse property that enables

its parameters separation. It provides interesting results even in

noisy images or when these arcs are small and partially hidden.

I. INTRODUCTION

The Simulfoot project has been developed in the LXAO

group of the LSIS laboratory with the participation of research

scientists in the fields of Computer and Cognitive Sciences.

The goal of this project is to provide a support for team sport

analysis using video sequences, with dedicated applications to

soccer [1].

A key step of this process is the 2D-3D registration that

consists in identifying a set of specific elements within the

images and associating them with the corresponding ones in

the 3D model, all these landmarks being referenced in the 2D

model of the field.

This problem can be handled in various ways depending

on video capture conditions. We suppose that we do not have

any knowledge on the capture parameters: the camera has not

been calibrated, it can move and zoom without any constraint,

and we can skip from a camera to another one within a video

sequence. Thus, we will not base the registration process on

the sequence coherence but only on the detection of landmarks

as straight and circular lines that have been drawn on the field

and that appear as straight and elliptic lines in the images.We

take advantage of homography classical properties [2] and the

representation of these transformations as linear applications

in the homogeneous space to perform the transformation from

the images to the 2D model of the field. A 3 lines and 3

columns matrix represents this transformation.

Thus, our problem is:

• To automatically detect the image area corresponding to

the field (a 2D space in the 3D one).

• To find the matrix that performs the transformation be-

tween the points of the image area and the 2D model.

Research works have been developed in the frame of this

project to automatically detect the Region of Interest (the field)

and the straight lines in this area [3]. When the number of

straight lines in this area is too small, we need to find other

relevant features such as arcs of ellipses that are the projection

of circular lines.

We find points of the primary field itself (i.e. the grass, or

the earth) through the characterization of color coherence in

the region of interest [3]. Thus, by setting to 0 all the points

selected as primary field and all the points that lay out of the

region of interest, we obtain a binary image in which points

to 1 may be associated with the players, the referee, the ball

and the lines drawn on the field [4]. Once straight lines have

been detected in this binary image, we can produce another

binary image in which the points associated with these lines

have been retrieved.

Our goal is to extract arcs of ellipses from this binary image.

II. ARC OF ELLIPSE DETECTION

Many algorithms for detecting ellipses in images have

been developed and published. Most of them use variable

ellipses that automatically match the data ([5],[6]). We have

developed an algorithm based on the Hough transform applied

to two subsets of separated variables.The most general ellipse

equation is expressed with six parameters and a proportionality

relation on them: that means that ellipses are defined by 5

effective parameters (or by using 5 points). We could think

to use a ”5 to 1” Hough transform to provide such detection

but it would not be acceptable for complexity (the parameter

space would be R5) and non-homogeneity reasons.

Fig. 1.The center O of the ellipse belongs to the line IJ

We have taken advantage of an ellipse geometrical property

to separate this research into two other ones, one in R2and one

in R3), both of them in parameter spaces that have a correct

homogeneity. This property is expressed as follows: ”Let M1

and M2be two points of the ellipse, and let T1and T2be the

tangent lines to the ellipse in these two points. Let I be the

intersection point of T1 and T2, and let J be the middle of

M1M2. Then the IJ line contains the center O of the ellipse”.

This property has already been used for detecting ellipses in

grey-level images[7].

Page 2

We use it in a different way to detect arcs of ellipse in

noisy binary images. In the following sub-sections, we give

all the details of the proposed algorithm, each sub-section

corresponding to a main step of the algorithm:

• Characterization of relevant couples (point, tangent): we

analyze the neighborhood of each point to decide if this

point supports a straight line; in this case, we consider

that this point belongs to a curve and this straight line is

its tangent.

• Ellipse center detection: we find the ellipse centers by

using the property stated before on the set of selected

relevant couples.

• Evaluation of a, b and θ parameters: the ellipse being

centered at the origin, it only depends on three parameters

that are the angle θ between its main axis and Ox, and

the a and b parameters of its normalized expression (a

and b being the half-lengths of its axes)

• Ellipse arcs detection: the ellipse being detected, we find

which points have been associated with its detection and

we analyze their distribution to extract the corresponding

arcs.

A. Relevant points and tangents characterization

Points with a value equal to 1 in the binary image are

supposed to be related to an ellipse. But they also may be noise

or they may be associated with another scene element. Our first

goal is to estimate, for each point, if it belongs to a curve and,

in this case, to evaluate the tangent to the curve at this point.

The study of point distribution in its neighborhood shows if

there is a main orientation that could be the tangent one. The

method we propose is simple and robust. We first define a

set of masks corresponding to all the possible orientations of

”thick lines” going through its center. The algorithm consists

in applying all these masks to each point (i.e. the mask is

a validation window centered on the studied point) and in

selecting the orientation associated with the mask that provides

the most important number of points, if this number is over

a given threshold S (i.e. below S, we consider the tangent is

not relevant).

A mask is defined by its ”radius” K, its orientation and the

thickness of its line. A mask is a square window of (2K+1) by

(2K +1) points. An orientation is associated with each point

at the edge of the window (in fact, we only consider the points

of the right and the top edges, the two other ones giving the

same orientations). E is the number of parallel lines over and

below the main line when the orientation is given by a point

that belongs to the right edge (and similar for the number of

parallel lines on the right and on the left when the orientation

is given by a point that belongs to the top edge).

The examples below use K = 15 and E = 2.

Fig. 2. Masks for tangent detection

The threshold value S has been chosen equal to K in order

to validate couples (point, tangent) in areas with only a few

points. This algorithm produces a set of N couples (Mi,Ti)

that will be used at the next step to detect the ellipse center.

B. Ellipse center detection

Let us consider now all the couples of couples: ((Mi,Ti),

(Mj,Tj)). We compute the corresponding line Dij= AijBij

where Aij is the middle of segment MiMj and Bij the

intersection point of Tiand Tj.

Because of approximation errors, we do not keep the lines

Dij associated with points Mi and Mj that are too close:

in this case the points Aij and Bij are too close to provide

a reliable line Dij. We could think there may be another

case of error due to the approximation when Ti and Tj are

parallel (Bij being rejected at the infinite): it is not the case

if we compute all the points in the homogenous space (in the

projective space, a point at the infinite has the same status

and is computed in the same way than the others ones). All

these lines focus on the same point if the couples (Mi,Ti)

and (Mj,Tj) belong to a same ellipse. We compute all the

intersections of the Dijand Dkland we study the distribution

of these points in the plane: concentrations areas correspond

to the center of the ellipses to be detected. The method for

finding these concentration areas is classical (Hough space).

C. Ellipse parameterization

Once we have found its center P, we can express the ellipse

equation in a referential of which the origin is P. The most

general expression of this equation is:

Ax2+ Bxy + Cy2= 1

We select those points Mi of the binary image that have

been effectively used to obtain the center P of the ellipse

(points Miused for the construction of Dijwhich contains P

- or which is close to P). Let us call SP this set of points.

Let us consider all the subsets of three points (Mi,Mj,Mk)

from SP. We can compute the coefficient A, B and C of the

corresponding ellipse (that contains Mi,Mjand Mk) and thus:

Mi,Mj,Mk∈ Sp: (Mi,Mj,Mk) → (A,B,C)ijk

We could think to use a ”3 to 1” Hough transform on the

(A,B,C) coefficients but we do not have any control on the

homogeneity of the corresponding space. Thus, we chose to

work in the (a,b,θ) space where θ is angle (Ox,∆) - ∆ being

the ellipse main axis - and where a and b are the half-lengths

of its axes. The expression of (a,b,θ) as a function of (A,B,C)

is classical:

a2=

2

A + C −?B2+ (A − C)2

2

A + C +?B2+ (A − C)2

θ =1

2arctan

b2=

B

A − C+kΠ

2

Page 3

As we did before to find the ellipse center, we use a classical

procedure to find the concentrations (there will be only one

in the frame of the problem we are interested in) and thus the

final coefficients (a,b,θ)P.

D. Arc of ellipse detection

For each ellipse detected in the image, we have its parame-

ters (center, orientation and axes lengths) and the set of points

that are in its neighborhood. In order to characterize the arcs

of ellipse associated with this set of points, we apply to these

points the homography that transforms the ellipse into the

circle centered at the origin and which radius is 1 (unit circle).

This transformation is expressed in the homogeneous space

as a 3 lines and 3 columns matrix. It takes into account

consecutively:

• A move from P (ellipse center) to O (origin)

• A rotation of −θ

• A scaling of 1/a in the Ox direction

• A scaling of 1/b in the Oy direction

After this transformation, all the selected points are close

to the unit circle. First, we eliminate the points that are not

very close to this circle: we only keep the point that are at a

distance to the origin belonging to [1-?,1+?] (? being chosen

as 0.1). Then, we sort all the remaining points by increasing

values of θ. Finally, we segment this set θ values by merging

all of the points (or θ values) of which value difference is less

that δθ (we have chosen δθ = 10obut any other choice should

also be valid as a representation with a different granularity).

The images below illustrate all this stpdf: the ellipse detec-

tion, the point selection and distribution analysis, and finally

the arc detection.

Fig. 3. Ellipse detection, homography, arc detection

III. RESULTS

In this section, we present some results we obtained in the

frame of the Simulfoot project and how we exploit them to

provide the registration.

Fig. 4.Arc of ellipse detection close to the goal area

Figure 4 shows us the detection of an arc of ellipse.

There are two arcs of ellipse and a full ellipse on a soccer

field. Knowing that we have an arc of ellipse and having its

orientation gives us strong information on which part of the

field is visualized. But there is to take care: if the detected

arc(s) is (are) close to the image boundary it may have been

clipped and then, the rules for characterizing the part of the

field are different.

Fig. 5.Ellipse detection in the center of the field

Figure 5 shows us the detection of a full ellipse that

corresponds to the circle drawn at the center of the field. In

fact, it is not a full ellipse but a set of arcs that all belong to

same ellipse due to the players who hide parts of it.Studying

the logic links of the resulting arcs and having knowledge on

the scene enable us to determine which part of the field is

visualized.

IV. CONCLUSION

The detection of arcs of ellipse is a key process for

image to model registration when analyzing team sports video

sequences. This is due to the presence of circles (or arcs of

circle) on the field in most cases (basket-ball, handball,).

We have designed a new algorithm that enables such detec-

tion in a noisy binary image. This algorithm is robust and fast

enough for the Simulfoot project applications.

We will now focus our research works on the automatic

understanding of the scene in relation with the different

elements extracted from the images (ellipses, lines, ) and the

way they are connected.

REFERENCES

[1] G. Poplu, J. Baratgin, S. Mavromatis, H. Ripoll, ”What kinds of processes

underlie decision making in soccer simulation? An implicit-memory

investigation”, in International Journal of Sport and Exercise Psychology,

vol. 1, n 4, pp. 390-405, 2003.

[2] P. Samuel, ”Projective Geometry”, in UTM; Springer Verlag , Heidelberg,

1988.

[3] S. Mavromatis, J. Baratgin, J. Sequeira, ”Reconstruction and simulation of

soccer sequences”, in MIRAGE 2003 INRIA Rocquencourt, Paris, March

2003.

[4] A. Le Troter, S. Mavromatis, J. Sequeira, ”Soccer field detection in

video images using color and spatial coherence”, in ICIAR - International

Conference on Image Analysis and Recognition vol. ICIAR 2004 Vol 2,

pp. 265-272, Springer Verlag, September 2004.

[5] K. Kanatani, N. Otha, ”Automatic Detection of Circular Objects by

Ellipse Growing”, in Memoirs of the Faculty of Engineering, Okayama

University, Vol. 36, No. 1, pp. 107116, December 2001.

[6] B. Matei, P. Mirr, ”Reduction of Bias in Maximum Likelihood Ellipse

Fitting”, in International Conference on Pattern Recognition (ICPR 00),

Barcelone, September 2000.

[7] H.K. Yuen, J. Illingworth, J. Kittler, ”Detecting partially occluded ellipses

using the Hough transform”, in Image and Vision Computing Vol. 7, issue

1, pp.31-37, February 1989.