ArticlePDF Available

The Gradient Product Transform: An Image Filter for Symmetry Detection



The Gradient Product Transform (GPT) is an image filter that converts a grayscale image into a float image, such that points representing a point reflection symmetry center obtain a high score. Beside the symmetry score, it also yields an estimator for the size of the symmetry region around each point. Apart from describing the GPT, the article also explains its application for two use cases: detection of objects with a point reflection or C 2m rotational symmetry, and the extraction of blood vessel skeletons from medical images. For the detection of symmetric objects, a score normalization procedure is suggested that allows to choose a fixed threshold for score values representing actual symmetries. Source Code The reviewed C++ source code for this algorithm is available from the web page of this article 1. Compilation and usage instructions are included in the README.txt file of the archive.
Published in Image Processing On Line on 2019–12–09.
Submitted on 2019–07–09, accepted on 2019–11–22.
ISSN 2105–1232 c
2019 IPOL & the authors CC–BY–NC–SA
This article is available online with supplementary materials,
software, datasets and online demo at
2015/06/16 v0.5.1 IPOL article class
The Gradient Product Transform: An Image Filter for
Symmetry Detection
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
Institute for Pattern Recognition (iPattern), Niederrhein University of Applied Sciences, Krefeld, Germany
The Gradient Product Transform (GPT) is an image filter that converts a grayscale image into
a float image, such that points representing a point reflection symmetry center obtain a high
score. Beside the symmetry score, it also yields an estimator for the size of the symmetry region
around each point. Apart from describing the GPT, the article also explains its application for
two use cases: detection of objects with a point reflection or C2mrotational symmetry, and
the extraction of blood vessel skeletons from medical images. For the detection of symmetric
objects, a score normalization procedure is suggested that allows to choose a fixed threshold for
score values representing actual symmetries.
Source Code
The reviewed C++ source code for this algorithm is available from the web page of this article1.
Compilation and usage instructions are included in the README.txt file of the archive.
Keywords: symmetry detection; blood vessel extraction; image filtering
1 Introduction
Symmetry is an important feature in human perception [18]. Consequently, computer algorithms for
automatic symmetry detection have been an area of research for some time, and there are conference
workshops entirely devoted to this problem [9,5]. Formally, a symmetry is a geometric transformation
under which an image region is invariant. The class of the transformation defines the symmetry type:
reflection,rotation, or translation. These symmetry types are not mutually exclusive, but often occur
in combination. The object in Figure 1(b), e.g., is invariant under reflections at the six red axes,
and also under rotations with angles α=k·2π/6 (kZ) around the intersection point of the axes.
Rotations by multiples of 2π/m form the symmetry group Cm, with the special case Cdenoting
rotation invariance under arbitrary angles.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch,The Gradient Product Transform: An Image Filter for Symmetry Detection, Image
Processing On Line, 9 (2019), pp. 413–431.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
(a) pure rotational C3symmetry (b) rotational C6and axial symmetries
Figure 1: Symmetric object with a pure C3rotation symmetry (left), and with a combination of reflection symmetries along
each of the red axes and a C6rotation symmetry (right).
Algorithms for symmetry detection can either use an entire image as input, or a list of charac-
teristic points that have been picked from the image by some pre-processing algorithm like an edge
detector or SIFT [11,12]. Surveys on symmetry detection algorithms can be found in [10,13]. The
present article deals with an algorithm that takes an entire grayscale image as input and assigns
each image point a “symmetry score” that measures how suitable the point is as a symmetry center
of a C2m(point reflection) symmetry. As the score value is based on scalar products of gray value
gradients at corresponding positions, the method is called Gradient Product Transform (GPT) [4].
The image of symmetry score values thus represents a “symmetry transform”, and symmetry points
correspond to maxima in the symmetry transform image.
The algorithm was originally defined in [3] for square shaped symmetry regions and a simple
criterion for discriminating between rotational and axial symmetries was suggested. In [4], this was
extended to rectangular regions, a more robust method for determining the size of the symmetry
region was introduced, a quadratic discriminant analysis with more features was suggested to dis-
criminate between axial and rotational symmetries, and an optional modification was proposed to
make the GPT applicable to blood vessel extraction. Although the GPT was originally devised for
2D symmetry detection, is has also been applied to 3D symmetry detection in voxel data [15].
The present article provides a comprehensive description of the GPT computation and its ap-
plication for object recognition and vessel detection. For object recognition, we propose a new
threshold on a normalized score for identifying actual symmetry points. This problem was circum-
vented in [3,4] by simply looking for the highest symmetry score in an image, which always yielded
exactly one symmetry point, irrespective of the actual number of symmetric objects. We now give
a threshold recommendation for detecting more than one (or, which might also happen: none) sym-
metry. Moreover, we provide a reference implementation of both the GPT and its applications in
standard C++. Apart from libpng2for reading and writing images, the implementation does not
depend on any third party libraries.
This article is organized as follows: Section 2describes the computation of the GPT, Section 3
describes the two applications of detecting C2m-symmetric objects (Section 3.1) and of blood vessel
extraction (Section 3.2), Section 4discusses some run time considerations, Section 5presents some
examples and a typical case of failure, and Section 6describes the online demo provided on the web
page of this article3.
The Gradient Product Transform: An Image Filter for Symmetry Detection
2 Gradient Product Transform (GPT)
The gradient product transform computes three float images from a grayscale image: an image
Srepresenting the symmetry score at each point ~x, and two images Rx, Ryrepresenting half the
diagonal of the rectangular symmetry region corresponding to the symmetry score value in S. The
computation of these three images requires three steps for each image point ~x:
1. For a range of radii ~r = (rx, ry) with 0 < rx, ryrmax, a raw symmetry score s(~x, ~r) is
computed as described in Equation (3) below.
2. From these values, an “optimal” radius ~
R(~x) = (Rx, Ry) is determined as
R(~x) = arg~r max s(~x, ~r)
max(rx, ry)rmax(1)
3. The symmetry score is then set to
S(~x) = s(~x, ~
(Rx(~x) + Ry(~x))α(2)
The parameter α[0,1] is typically set to α= 0.5. The following subsections describe the details
of the computation of s(~x, ~r) and the reasoning for the choice of the parameter α.
2.1 Computation of the Symmetry Score
The raw symmetry score s(~x, ~r ) is based on the two gradient images ~
G= (Gx, Gy), which are
computed from the grayscale image via convolution with Sobel-kernels [6]. Starting with the gradient
has the advantage that plain homogeneous regions do not obtain a high symmetry score, even though
they are formally perfectly symmetric. The GPT utilizes the two observations that, under point
reflection symmetry, the mirrored gradient ~
G0is anti-parallel to the gradient ~
G(see Figure 2), and
that the scalar product h~
G, ~
G0iis minimal for ~
G. The raw symmetry score for the rectangle
with circumradius ~r = (rx, ry) around the center point ~x = (x, y), i.e., the rectangle with upper left
~x ~r and lower right ~x +~r, is therefore defined as
s(~x, ~r) =
dx=1 D~
G(dx, dy),~
where ~
G(dx, dy) denotes the gradient at position (x+dx, y +dy). At first sight, it seems unintuitive
to use a rectangular symmetry region for a rotational symmetry instead of a circular region or, for
raster images, a square region. In real world images, however, an actually circular object usually
appears ellipsoidal due to perspective distortion (see Figure 3) and the Cmsymmetry only holds
approximately. Testing for regions with rectangular bounding boxes thus makes the symmetry score
more robust with respect to skew, which was experimentally confirmed in [4].
As the size of the symmetry region is not known in advance, the score (3) must be computed for
all radii 1 rx, ryrmax. This is less computationally intensive than it might seem at first sight,
because Equation (3) only consists of sums. This allows to compute all scores s(~x, ~r) in a single run
over the symmetry region with the following recursion formula for rx, ry>1
s~x, (rx, ry)=s~x, (rx1, ry)+s~x, (rx, ry1)s~x, (rx1, ry1)
G(rx, ry),~
G(rx, ry)i.(4)
The computation of the GPT is summarized in Algorithm 1.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
x + d
= −G
x − d
Figure 2: A point reflection at ~x maps the gradient ~
Gat ~x +~
donto the gradient ~
Gat ~x ~
(a) front view (b) skewed view
Figure 3: Due to perspective skew, the bounding box of a rotationally symmetric object can become rectangular.
Algorithm 1: Gradient Product Transform
input : grayscale image F,rmax,α
output: float images S(symmetry score) and ~
R= (Rx, Ry) (symmetry radius)
1Gxx-component of gradient of Fwith Sobel-Operator
2Gyy-component of gradient of Fwith Sobel-Operator
G(Gx, Gy)
4foreach pixel position ~x = (x, y)do
5ssum ← −D~
G(x, y + 1),~
G(x, y 1)E
6foreach x-component of radius rxwith 0< rxrmax do
7ssum ssum D~
G(x+rx, y + 1),~
G(xrx, y 1)E
G(x+rx, y),~
G(xrx, y)ED~
G(x+rx, y 1),~
G(xrx, y + 1)E
9~r (rx,1)
10 s(~r)ssum // Score for candidate region with radius ~r.
11 foreach radius ~r = (rx, ry)with 0< rx, ryrmax do
12 s(~r)s(rx1, ry) + s(rx, ry1) s(rx1, ry1) // Recursion formula (4).
13 D~
G(x+rx, y +ry),~
G(xrx, y ry)ED~
G(x+rx, y ry),~
G(xrx, y +ry)E
14 ~r arg~r max {s(~r)·(rx+ry)α|max(rx, ry)rmax}// Symmetry radius after Equation (1).
15 ~
16 S(~x)s(~r)·(rx+ry)α// Symmetry score after Equation (2).
2.2 Symmetry Size Normalization Parameter α
The parameter αhas been introduced in [4] in Equations (1) and (2) to compensate for the observation
that the raw symmetry score s(~x, ~r ) tends to increase with the size of ~r due to background noise.
Without this scale normalization, the radius with the highest score tends to be quite close to rmax,
The Gradient Product Transform: An Image Filter for Symmetry Detection
(r/r )
0s(r) / s(r )
r / r0
0 0.5 1 1.5 2
α = 0.0
α = 0.5
α = 1.0
(a) average over all images
α = 0.0
α = 0.5
α = 1.0
(b) an example image
Figure 4: Impact of αon the symmetry radius determination for all images from the data set [3] (left) and a different
sample image (right). r0is for each image the actual ground truth radius of the symmetry region.
even when the actual symmetric object is much smaller.
To understand this phenomenon, we have computed the symmetry score as a function of the
radius rwith square shaped regions (rx=ry=r) for all ground truth symmetry points in the data
set [3]. To make the measurements comparable, we have normalized all radii with the ground truth
radius r0, and all scores with the score s(r0). As can be seen in Figure 4(a) (curve for α= 0), the
score increases steeply for rr0, as expected, but still increases as rbecomes larger than r0, albeit
not as steep. This is simply due to the fact that the number of summands in (3) grows with rso
that random fluctuations can increase the sum.
To remedy this problem, the symmetry score needs to be normalized by a factor that depends on
~r. Even though the number of summands in (3) is proportional to the area of the symmetry region,
which is rx·ry, only scalar products with large gradients will contribute significantly to the sum.
As the gradient is large for edges, and the number of edge points is proportional to the surface of
the symmetry region, it seems natural to normalize with rx+ry. As can be seen from the curve for
α= 1.0 in Figure 4(a), this indeed leads on average to a maximum score at the actual symmetry
size, but it also has the side effect that the score values of small symmetry regions are increased.
To control the trade off between these two effects, we normalize with (rx+ry)α, where α= 0.5 is
a compromise between both effects. The effect of the choice of αon the symmetry radius detection
for rectangular regions on a sample image can be seen in Figure 4(b), where we have set rmax to the
size of the image and for each value of αthe highest symmetry is shown.
3 Symmetry Detection
The GPT is merely an image filter that assigns a symmetry score to each pixel. Locating actual
symmetry centers requires further analysis of the transformed image. The following subsections
describe two different post-processing algorithms of the GPT image for the detection of rotationally
symmetric objects, and for extracting symmetry axes in blood vessel images.
3.1 Finding Rotationally Symmetric Objects
In principle, the center point of a symmetric object is the location ~x of a local maximum in the GPT
score image Swith a high score value S(~x), and the borders of the object are given by the GPT
radius images Rxand Ry. In practice, there are however two questions that need to be addressed:
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
1. Not only rotational symmetries, but also pure axial symmetries can obtain high symmetry
scores (see Figure 5(a)). How can we discriminate between a merely reflectional and an actual
rotational C2msymmetry?
2. Local maxima are greater than their neighborhood, but are not guaranteed to be high in an
absolute sense. What is a reasonable threshold for a “high” symmetry score?
(a) schematic example (b) example image (from [3]) (c) GPT scores of example image
Figure 5: In some cases, C2symmetries are reflection symmetries, too. The GPT score image has been converted to
grayscale and dilated in order to make high scores better visible.
An example for the first problem can be seen in Figure 5(c): for the axial symmetries the entire
symmetry axis tends to obtain high symmetry scores, while, for the rotational symmetry with respect
to the center of the cross sign, the symmetry score decays with the distance to the symmetry center
independent from the direction. Inspired by this observation, four features for the discrimination
between rotational and axial symmetries were defined in [4]. As these features measure local proper-
ties in a symmetric neighborhood of the symmetry score maximum, they are computed over square
shaped windows. Table 1lists sample values of these features for two different exemplary points, one
representing a rotational symmetry center, and one an axial symmetry center:
Edge directedness. This feature is based on the angle histogram of the gradients of the symmetry
transform in a 7×7 window centered at ~x. We compute the gradients by applying a Sobel filter
on the symmetry transform and build a weighted histogram with 16 bins of the gradient angles.
Each angle is weighted in the histogram with the corresponding absolute value of the gradient.
The edge directedness is then defined as the highest occurring frequency in this histogram,
which should be higher for axial symmetries. In Table 1, this effect can be seen for two sample
Skeleton size. Starting from ~x, we follow the skeleton until the score falls below S(~x)/2. Here,
“skeleton” does not refer to the skeleton extracted with the algorithm described in section 3.2.
Instead it means the point sequence obtained by following in opposite directions the ridge of
weakest descent from the local maximum. The feature is then the ratio between the skeleton
length and the size pR2
yof the symmetry region, where ~
R= (Rx, Ry) is the circumradius
(half diagonal) of the symmetry region. It should be higher for axial symmetries, as indeed is
the case for the example in Table 1.
Antiparallel directions. This feature is based on the gradient of the original grayscale image. We
compute the direction histogram with eight bins of all gradients in a window with the radius
min{Rx, Ry}. Only those gradients are taken into account for which the mirrored gradient
is “antiparallel”, i.e. the cosine of the angles between the gradients is less than -0.975. The
The Gradient Product Transform: An Image Filter for Symmetry Detection
point Apoint B
edge directedness 0.1203 0.5303
skeleton size 0.0236 1.0000
antiparallelity 0.2095 0.4935
covariance eigenratio 0.8196 0.4389
Table 1: Values of the four features for two typical symmetry center candidate points for the image from Figure 5. Point
Arepresents a rotational symmetry and point Ban axial symmetry. The image on the left shows the (dilated) symmetry
feature is the highest relative frequency in the direction histogram. The value for “antiparallel
directions” should be lower for rotational symmetries. It should be higher for axial symmetries,
as indeed is the case for the example in Table 1.
Covariance eigenratio. For the points in a 7×7 window around (x, y) in the symmetry transform
image, we compute the covariance matrix Kas4
S(x+dx, y +dy)×dx dx dx dy
dy dx dy dy ,(5)
where S(x, y) is the symmetry transform value at (x, y), and the normalization factor Nis
the sum over all symmetry values in the window. The eigenvalues of Kindicate how strongly
the values spread in the direction of the corresponding eigenvector. Consequently, the ratio
between the smaller and the larger eigenvalue should be higher for rotational symmetry, as can
be seen in the example in Table 1.
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
0.1 0.4 0.7
0.0 0.6
0.2 0.4
0.1 0.3 0.5 0.7
0.0 0.6
0.2 0.3 0.4 0.5
Figure 6: Scatterplots of the four features for the manually labeled rotational (blue) and axial (red) symmetry centers from
the dataset [3].
Whilst Table 1demonstrates the discriminating suitability of the four features for a particular
example, the scatterplots in Figure 6confirm these tendencies for a greater number of symmetry
4Beware that dx and dy are not derivatives, but displacements and the matrix Kis thus unrelated to and must
not be confused with the Hessian matrix.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
center points. These points were taken from the data set published with [3], which includes a list of
ground truth points that are labeled as either axial or rotational symmetry centers. In each individual
scatterplot of Figure 6, there is, however, considerable overlap between the classes. It is thus better
to use a combination of the features for discrimination. In [4], a quadratic discriminant analysis
was therefore applied, which decides for the class (axial or rotational) with the largest discriminant
ξ) = wi− k(~
where the index i∈ {a, r}denotes the class (axial or rotational), and ~
ξis the row vector of the four
features in the same order as in the list above. The parameters ~µi,wiand Wihave been determined
with the labeled ground truth data from [3] used as training data via the function qda as implemented
in the MASS package or the R language for statistical computing5. The values reported in [4] are:
wa=15.03338 wr=21.49041 (7)
~µa= (0.4334,0.4975,0.4417,0.4352) ~µr= (0.1859,0.0654,0.2302,0.7136)
9.3921 3.1451 7.8727 0.8677
02.4364 0.3522 0.1780
0 0 18.5275 3.1907
0 0 0 4.3365
14.3116 6.1912 15.5453 6.7814
0 20.3183 0.3754 0.9648
0 0 20.5954 3.8730
0 0 0 7.7489
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.5 1.0 1.5 2.0
Figure 7: Kernel density plots of the distribution of snorm for true and false high symmetry scores in the data set [3].
The second aforementioned question, i.e., how to define a threshold for the symmetry score
representing an actual symmetry, has not yet been discussed in the literature. Here we propose such
a threshold on a normalized score value that is guaranteed to fall in the range [1,1]. The highest
symmetry score s(~x, ~r) according to Equation (3) is obtained for a perfect C2msymmetry. In this
case, all mirrored gradients are exactly anti-parallel, such that h~
G, ~
G0k. The greatest
possible score value is thus
smax(~x, ~
R(~x)) =
dx=1 k~
G(dx, dy)k·k~
where ~
R(~x) = (Rx, Ry) is the value of the GPT radius images at pixel ~x. A normalized score in the
range [1,1] is thus
snorm(~x) = s(~x, ~
smax(~x, ~
R(~x)) =S(~x)·(Rx(~x) + Ry(~x))α
smax(~x, ~
R(~x)) [1,1].(9)
The Gradient Product Transform: An Image Filter for Symmetry Detection
To choose a reasonable threshold for snorm , we took all images of the dataset [3] and extracted the five
greatest local maxima in the symmetry score image S(~x) that were classified as rotational symmetry
centers by the discriminant analysis (6) and (7). We have manually labeled these points as correctly
or wrong identified and estimated the distribution of snorm for the different classes with a kernel
density estimator [16], as shown in Figure 7. From these data, we conclude that the threshold on
snorm should be at least 0.5 for points reported as a symmetry center. It should be noted that this
threshold is not meant to discriminate between axial and rotational symmetries, but for suppressing
responses from region not representing symmetries at all. In our reference implementation (see
Section 6), the threshold defaults to 0.6 and can be changed by a command line option.
It should be noted that, for the purpose of finding symmetry center candidate points as local
maxima in the score image, the symmetry score S(~x) cannot be replaced by snorm(~x), because the
normalization (9) has the side effect of increasing the score values for regions with low contrast and
henceforth with small gradients. We therefore proceed as follows for detecting symmetry center
1. Find all local maxima ~x within a k×kwindow with k= 56in the GPT score image S(~x).
These points are subsequently processed in the descending order of their score values.
2. Test whether a point is a rotational symmetry with the quadratic discriminant (6).
3. Test whether snorm(~x)t, where 0 < t < 1 is some threshold. If this is not the case, no further
points are examined.
In order to avoid the detection of small symmetry regions inside a larger region (for example screw
heads inside a wheel), we optionally ignore points lying inside already detected symmetry regions.
The resulting symmetric object detection is formally summarized in Algorithm 2, and Figure 8shows
its result on a sample image.
Algorithm 2: Detection of rotationally symmetric objects
input : float images S(GPT score) and ~
R= (Rx, Ry) (GPT radius) and ~
G= (Gx, Gy)
(gradient image), window size k, flag skipinside, threshold t
output: symmetry regions A={(~x1, ~r1), . . .}
1A← {},P← {}
2foreach pixel position ~x = (x, y)do
3if S(~x)is highest score in k×kwindow around ~x then PP∪ {~x}
4sort Pin descending order of S(~x)
5foreach ~x Pdo
6if skipinside and ~x contained in a region from Athen continue
7if ga(~x)> gr(~x)then continue // see Equation (6), requires S,~
Rand ~
8if snorm(~x)< t then break // see Equation (8), requires S,~
Rand ~
9AA∪ {(~x, ~
3.2 Extracting Vessel Skeletons
The problem of blood vessel extraction occurs for many types of medical images, e.g. for angiographs,
fundus photographs, or MRT-TOF datasets. As blood vessels are locally C2symmetric (see Fig-
ure 5(a)), the GPT assigns high scores to the medial axis of the vessels. When the GPT symmetry
6The value k= 5 is somewhat arbitrary and can be optionally changed on the command line (see section 6). We
have not noticed much differences on our test images, though, whether it is set to 5, 7, or 11.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
(a) image with detected symmetries (b) dilated GPT scores with local maxima regions
Figure 8: An example image with the GPT score image and the symmetric objects (cyan) detected with Algorithm 2. Red
rectangles have been sorted out by the quadratic discriminant analysis. For this 600 ×400 image, we used rmax = 80 and
set the symmetry threshold to t= 0.6.
(a) dark blood vessels (from [17]) (b) original GPT score (c) GPT score with Θ(+)
Figure 9: Effect of Heaviside’s step function on the GPT score image (rmax = 3). The GPT score images have been
converted to grayscale and their histograms were equalized in order to make the high scores better visible.
score image is interpreted as a height map, the problem of vessel skeleton extraction is thus trans-
formed into the problem of ridge extraction. In practice, there are however two questions that need
to be addressed:
1. Parallel vessels lead to a high score in the middle of the background between them, too. How
can we ensure that only symmetries in the foreground are detected?
2. How do we extract the dominant ridges?
An example for the first problem can be seen in Figure 9(b): many high GPT scores occur in
the background, too. We can, however, easily circumvent this problem by suppressing contributions
to the symmetry score for which the gradient points into the wrong direction with respect to the
symmetry center. We do this by introducing Heaviside’s step function Θ (zero for negative arguments,
and one for positive arguments) as a factor in each summand of the symmetry score in order to
suppress contributions from gradients pointing to (or away from) the symmetry center
s(~x, ~r) = X
G(~x +~
G(~x ~
d, ~
G(~x +~
d, ~
G(~x ~
The Gradient Product Transform: An Image Filter for Symmetry Detection
(a) fundus image (from [17]) (b) extracted GPT score ridges
Figure 10: Symmetry score ridges (yellow) extracted with Algorithm 3from a fundus image. Blue pixels match criteria
(11) and (12), but have been filtered out by threshold (14). The GPT has been applied with rmax = 3 and with Θ(+).
The sign of the argument in Θ determines which objects are detected. For a dark foreground, the
plus sign should be used because it has the effect that only gradients pointing outwards contribute
to the sum. The result can be seen in Figure 9(c).
The second problem, i.e., ridge detection, naturally occurs in the context of geological terrain
maps, and different algorithms have been proposed based on simulating dropping water [19,8], on
building and pruning neighborhood graphs [2,1], or on estimating local curvature features [17,7].
For demonstrating the application of the GPT to vessel medial axis extraction, we have implemented
the curvature based method by Staal et al., which defines the following condition for a ridge point
~x [17]
1 = 1
sign h~
GS(~x +~v1), ~v1isign h~
GS(~x ~v1), ~v1i
where λ1is the eigenvalue with the largest absolute value of the Hessian matrix at position ~x of
the GPT score image, ~v1is the corresponding eigenvector (normalized to k~v1k= 1), and ~
GSis the
gradient of the GPT score image. The gradients and the Hessian matrix are computed with the
Sobel operator. Note that, unlike Staal et al., we do not compute these derivatives on the original
grayscale image, but on the GPT score image.
Application of criterion (11) does not result in a one pixel wide skeleton, but in a dilated skeleton
at curved positions. We therefore added as an additional criterion for an actual ridge point the
criterion of the first step of the profile-recognition by Chang and Sinha [2]. This imposes the condition
that the score value at the center is greater than the values of opposite neighbors for at least one
combination of neighbors (N-S, NE-SW, E-W, SE-NW), i.e.
S(~x +~
d)< S(~x) and S(~x ~
d)< S(~x),(12)
for at least one of the step vectors ~
d∈ {(0,1),(1,1),(1,0),(1,1)}. These two conditions still lead
to many ridges in regions with low GPT score values. To suppress these responses, we additionally
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
Algorithm 3: Medial axis extraction of vessels
input : float images S(GPT score) and ~
R= (Rx, Ry) (GPT radius) and m(mean gradient
absolute value of original image)
output: medial axis points P={~x1, . . .}
1P← {}
2GxSobel filter of Sin x-direction // gradient ~
Gof GPT score
3GySobel filter of Sin y-direction
4Hxx Sobel filter of Gxin x-direction // Hessian Hof GPT score
5Hyy Sobel filter of Gyin y-direction
6Hxy Hyx Sobel filter of Gxin y-direction
7foreach pixel position ~x = (x, y)do
8rxRx(~x) and ryRy(~x)
9if S(~x)·(rx+ry)α<2·max(rx, ry)·m2then continue // threshold (14)
10 compute eigenvalues and eigenvectors H~vi=λi~viwhere |λ1|>|λ2|and k~vik= 1
11 if 16=1
sign h~
GS(~x +~v1), ~v1isign h~
GS(~x ~v1), ~v1i
12 continue // ridge criterion (11) matches
13 ridgefound False
14 foreach ~
d∈ {(0,1),(1,1),(1,0),(1,1)}do
15 if S(~x +~
d)< S(~x)and S(~x ~
d)< S(~x)then
16 ridgefound True // ridge criterion (12) matches
17 if not ridgefound then continue
18 compute snorm(~x) after Equation (9)
19 if snorm(~x)0.4then PP∪ {~x}// threshold (13)
apply two thresholds on the GPT score value
S(~x)·(Rx(~x) + Rx(~x))α2·max(Rx(~x), Ry(~x)) ·mean(k~
where the mean absolute value is computed over the entire gradient image of the input image. The
first threshold ensures that the returned ridge points are actual symmetries. The second threshold
is only added to suppress responses from homogeneous regions, as can be seen in Figure 10(b). The
reasoning behind the second threshold is that a symmetry axis should have significant contributions
from at least the two longer border sides, which correspond to 2·max(Rx, Ry) in Equation (10). The
resulting algorithm for ridge extraction is formally summarized in Algorithm 3, and Figure 10 shows
its result on a sample image. Note that the ridge criteria are applied in Algorithm 3in such an order
that the most computationally expensive test, which includes the computation of snorm, is done last,
and thus least frequently.
4 Runtime Considerations
There are a number of O(n) operations in the GPT algorithms, where nis the number of image
pixels, e.g. the Sobel operators in Algorithm 1, the location of local maxima in Algorithm 2, or
the application of ridge criteria in Algorithm 3. All these are dwarfed, however, by the GPT score
computation because the computation of all scores up to a radius rmax at an image point (x, y)
The Gradient Product Transform: An Image Filter for Symmetry Detection
(a) angiography (b) GPT ridges (-which all) (c) GPT ridges (-which dark)
Figure 11: Examples showing the results of the GPT in combination with Algorithm 3. The option -which dark activates
Heaviside’s step function in Equation (10). For this 400 ×600 image, we have used rmax = 80.
according to Equations (3) and (4) requires O(r2
max) additions. The total runtime of the GPT is thus
It should be noted that, when the number of image pixels is increased by a factor a, the maximum
symmetry radius rmax needs to be increased by a factor a, too, in order to detect objects of the
same maximum real world size. This means that the runtime is even O(n2) when large objects are
to be detected. In order to speed up the runtime, we therefore use parallelization and (optional)
As the GPT score is defined independently for each point, the computation of the GPT score and
radius images can be performed independently in parallel. We have used OpenMP7and let the user
specify the number of parallel threads at compilation time in the CMakefile. With the choice of four
threads on an Intel Core i7-4770 CPU @ 3.40GHz, the runtime for detecting the symmetric objects
in the 600 ×400 image in Figure 8(a) with rmax = 80 and t= 0.75 was about 1.95s.
When both the image size and rmax are large, it is reasonable to assume that the user is not
interested in responses of very small region size. In this case, the runtime can be reduced consider-
ably by downscaling the image before applying the GPT, and then remap the detected symmetry
points and regions on the original input image. In our reference implementation, we do this when
pncols ×nrows ×r2
max >105. In this case the image is scaled down such that this value is 105. This
behavior can be switched off by the command line option -noscale.
5 Examples and Cases of Failure
Figure 11 is an example for the vessel medial axis extraction: as the vessel foreground is dark, the
GPT variant (10) with the plus sign should be used (command line option -which dark). Except
from minor noise, Algorithm 3extracts the medial axis even for vessels of different sizes. When
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
specifically smaller vessels are to be detected, rmax should be chosen smaller. Although rotational
object detection is of no interest in this case, it is interesting to note that, in this case, no rotational
symmetry is found by Algorithm 2with t= 0.6, as it should be.
An example, for which Algorithm 2works very well for detecting a symmetric object, can be seen
in Figure 12(a): the global maximum in the GPT score image is the center of the dartboard, and
its normalized score is snorm 0.69. With t= 0.5, no further rotationally symmetric objects are
detected. Figure 12(c) shows the effect of the option skipinside in Algorithm 2: with this option, only
the outermost symmetry including both bird’s eyes is detected, while otherwise additional symmetries
inside are detected.
Figure 12(f) shows a typical example, for which the algorithm fails: both the background pattern
on the church wall and the window rosette in the center show rotational symmetries, but the contrast
in the background is much greater, which leads to higher gradients and thus to higher GPT scores.
Although the background pattern actually shows more than one symmetry (translational, rotational,
and axial), there are clear local maxima at the brick centers which make the quadratic discriminant
analysis decide for “rotational symmetry”. Other problems can occur due to perspective skew, as
shown in Figure 13. Although the outline of the clock is transformed to an ellipse in Figure 13(c),
the interior is no longer C2symmetric and thus does not contribute sufficiently to the symmetry
6 Online Demo and Command Line Program
The source code of a command line program gptsymmetry that implements this algorithm is provided
on the web page of this article8. On the same website, there is also an online demo built on top of this
command line program. Users can choose among several pre-uploaded images or upload their own
image in the PNG file format. Color images are automatically converted to grayscale by averaging
over the three color channels. Here is a list of the options for the online demo with the corresponding
command line parameters of the reference implementation given in parentheses:
Maximum radius (-r). The maximum radius rmax of the symmetry region in Algorithm 1.
Threshold (-t). The threshold on the normalized score snorm in Algorithm 2. When negative (de-
fault), it is set to 0.6 if which=all, and to 0.4 otherwise. When t1, only the first rotational
symmetry is reported.
Object foreground (-which). Whether all symmetries are to be detected or only those of light
(or dark) objects. Defaults to all.
Exponent α(-alpha). Exponent for normalizing the symmetry score in Algorithm 2with (rx+
ry)α. Defaults to 0.5.
Skipinside (-skipinside). When set, symmetries with lower symmetry score inside larger symme-
try region are not reported in Algorithm 2.
The online demo creates and displays four images: the GPT score image converted to grayscale,
the ridges extracted with Algorithm 3, the symmetry regions detected by Algorithm 2classified as
axial (red) or rotational (cyan) overlaid on the GPT score image, and the rotational symmetries
detected by Algorithm 2overlaid on the input image. The detected symmetries are printed in CSV
format separated by semicolons as
The Gradient Product Transform: An Image Filter for Symmetry Detection
(a) dartboard (b) GPT scores with local maxima regions
(c) metal bird (d) all local maxima (e) with skipinside
(f) church window (g) GPT scores with local maxima regions
Figure 12: Examples showing the results of the GPT in combination with Algorithm 2. The GPT score images have been
dilated. Cyan rectangles have been detected as rotational symmetries, and red rectangles have been sorted out by the
quadratic discriminant analysis. All images had height 400, and we used rmax = 80 with a symmetry threshold of t= 0.6.
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
(a) front view (b) clock detected (c) skewed view (d) nothing detected
Figure 13: Example showing the effect of strong perspective skew on symmetry detection with Algorithm 2. Cyan
rectangles have been detected as rotational symmetries. Both images had height 400, and we used rmax = 80 with a
symmetry threshold of t= 0.6.
where rot indicates whether this was classified as a rotational (1) or axial symmetry (0), (x, y) and
(rx, ry) denote the symmetry center and radius of the symmetry region, Sis the GPT score, and
snorm is the score normalized between zero and one.
The command line program gptsymmetry has the following additional options not supported by
the online demo:
-k k.Radius of the window for local maxima search. Defaults to 5.
-bw borderwidth .Width of the borders of symmetry regions drawn in result image. Defaults to 1.
-o outpng .Draw detected symmetry as rectangle in file outpng. not write images of interim steps with prefix trace-. This will speed the program up.
-noscale.Do not scale down large images with large radius.
7 Conclusion
The Gradient Product Transform is an algorithm that has already been reported in the literature
as a useful tool for symmetry detection or vessel skeleton extraction [3,4]. The score normalization
method presented in this article allows for absolute thresholds, which are helpful in both use cases.
The Gradient Product Transform: An Image Filter for Symmetry Detection
A drawback of the algorithm is that it only uses the grayscale information of the image, but does
not utilize color information. Another drawback is that all radii up to a user provided value rmax
are tested, which becomes slow for the combination of a large image size with a large rmax. We
solve this problem by scaling down the image in this case. A multiresolution approach based on a
Gaussian image pyramid [6] would be more flexible and would save the user from specifying rmax .
This would, however, introduce the problem of how to order the local maxima from different scales
in Algorithm 2and how to select the most salient symmetries. This would require extensive further
experiments on new additional ground truth data, and it is unclear whether the results would justify
the effort.
In its current form, the GPT already is a useful tool for symmetry detection. As the reference
implementation in standard C++ coming with this article does not rely on third party libraries
(except for libpng), it provides an effortless way to try out the algorithm for particular applications.
The authors are grateful to Tobias Bolten for creating the symmetry dataset [3], to Bram van Gin-
neken for making the DRIVE database [17] available to us, and to Christian Neumann for providing
to us an angiography image from the study [14]. Moreover, we thank the anonymous reviewers for
their valuable comments.
Image Credits
Tobias Bolten, partly from the dataset described in [3]
Joes Staal et al., from the dataset DRIVE (“Digital Retinal Images for Vessel Extraction”) [17]
Christoph Dalitz
Christian Neumann, from the study [14]
[1] S. Bangay, D. de Bruyn, and K. Glass,Minimum spanning trees for valley and ridge
characterization in digital elevation maps, in Proceedings of the 7th International Conference
on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa, AFRIGRAPH,
ACM, 2010, pp. 73–82.
[2] Y-C. Chang and G. Sinha,A visual basic program for ridge axis picking on DEM data using
the profile-recognition and polygon-breaking algorithm, Computers & Geosciences, 33 (2007),
pp. 229–237.
[3] C. Dalitz, R. Pohle-Fr¨
ohlich, and T. Bolten,Detection of symmetry points in images,
in International Conference on Computer Vision Theory and Applications (VISAPP), 2013,
pp. 577–585.
[4] C. Dalitz, R. Pohle-Fr¨
ohlich, F. Schmitt, and M. Jeltsch,The gradient product
transform for symmetry detection and blood vessel extraction, in International Conference on
Christoph Dalitz, Jens Wilberg, Manuel Jeltsch
Computer Vision Theory and Applications (VISAPP), 2015, pp. 177–184. http://dx.doi.
[5] C. Funk, S. Lee, M.R. Oswald, S. Tsogkas, W. Shen, A. Cohen, S. Dickinson,
and Y. Liu,2017 ICCV challenge: Detecting symmetry in the wild, in Proceedings of the
IEEE International Conference on Computer Vision, 2017, pp. 1692–1701.
[6] R.C. Gonzalez and R.E. Woods,Digital Image Processing, Prentice-Hall, New Jersey,
2nd ed., 2002.
[7] S.N. Kalitzin, J. Staal, B.M. ter Haar Romeny, and M.A. Viergever,A com-
putational method for segmenting topological point-sets and application to image analysis,
IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (2001), pp. 447–459.
[8] S. Koka, K. Nomaki, K. Sugita, K. Tsuchida, and T. Yaku,Ridge detection with a
drop of water principle, in ACM SIGGRAPH ASIA, 2010, p. 34.
[9] J. Liu, G. Slota, G. Zheng, Z. Wu, M. Park, S. Lee, I. Rauschert, and Y. Liu,
Symmetry detection from real world images competition 2013: Summary and results, in IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 200–205. https:
[10] Y. Liu, H. Hel-Or, C.S. Kaplan, and L. Van Gool,Computational symmetry in computer
vision and computer graphics, Foundations and Trends in Computer Graphics and Vision, 5
(2010), pp. 1–195.
[11] D.G. Lowe,Distinctive image features from scale-invariant keypoints, International Journal
of Computer Vision, 10 (2004), pp. 91–110.
[12] G. Loy and J.O. Eklundh,Detecting symmetry and symmetric constellations of features, in
European Conference on Computer Vision (ECCV), 2006, pp. 508–521.
[13] N.J. Mitra, M. Pauly, M. Wand, and D. Ceylan,Symmetry in 3D geometry: Extraction
and applications, in Computer Graphics Forum, vol. 32, 2013, pp. 1–23.
[14] C. Neumann, K. T¨
onnies, and R. Pohle-Fr¨
ohlich,AngioUnet - a convolutional neural
network for vessel segmentation in cerebral DSA series, in International Conference on Computer
Vision Theory and Applications (VISAPP), 2018, pp. 331–338.
[15] R. Pohle-Fr¨
ohlich and D. Stalder,3D-Symmetrietransformation zur Gef¨aßsegmentierung
in MRT-TOF-Daten, in Bildverarbeitung f¨ur die Medizin, Springer, 2014, pp. 144–149.
[16] S.J. Sheather and M.C. Jones,A reliable data-based bandwidth selection method for kernel
density estimation, Journal of the Royal Society series B, 53 (1991), pp. 683–690.
[17] J. Staal, M.D. Abramoff, M. Niemeijer, M.A. Viergever, and B. van Ginneken,
Ridge-based vessel segmentation in color images of the retina, IEEE Transactions on Medical
Imaging, 23 (2004), pp. 501–509.
The Gradient Product Transform: An Image Filter for Symmetry Detection
[18] P.A. Van der Helm,Symmetry perception, in Oxford handbook of perceptual organization,
Johan Wagemans, ed., Oxford University Press, Oxford, UK, 2015, pp. 108–128. https://doi.
[19] R. Yokoyama, A. Kureha, T. Motohashi, H. Ogasawara, T. Yaku, and D. Yoshino,
Geographical concept recognition with the octgrid method for learning geography and geology, in
Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007),
2007, pp. 470–471.
In both natural and man-made structures, symmetry provides a range of desirable properties such as uniform distributions of internal forces, concise transmission paths of forces, as well as rhythm and beauty. Most research on symmetry focus on natural objects to promote the developments in computer vision. However, countless engineering structures also contain symmetry elements since ancient times. In fact, many scholars have investigated symmetry in engineering structures, but most of them are based on analytical methods which require tedious calculations. Inspired by the application of deep learning in image identification, in this paper, we use two Convolutional Neural Networks (CNNs) to respectively identify the symmetry group and symmetry order of planar engineering structures. To this end, two different datasets with labels for symmetric structures are created. Then, the datasets are used to train and test the constructed network models. For symmetry classification, it achieves 86.69% accuracy, which takes about 0.006 s to predict one picture. On the other hand, for symmetry order recognition, it reaches 92% accuracy, which expends about 0.005 s to identify an image. This method provides an efficient approach to the exploration of structural symmetry, which can be expanded and developed further toward the identification of symmetry in three-dimensional structures.
Conference Paper
Full-text available
Motivated by various new applications of computational symmetry in computer vision and in an effort to advance machine perception of symmetry in the wild, we organize the third international symmetry detection challenge at ICCV 2017, after the CVPR 2011/2013 symmetry detection competitions. Our goal is to gauge the progress in computational symmetry with continuous benchmarking of both new algorithms and datasets, as well as more polished validation methodology. Different from previous years, this time we expand our training/testing data sets to include 3D data, and establish the most comprehensive and largest annotated datasets for symmetry detection to date; we also expand the types of symmetries to include densely-distributed and medial-axis-like symmetries; furthermore, we establish a challenge-and-paper dual track mechanism where both algorithms and articles on symmetry-related research are solicited. In this report, we provide a detailed summary of our evaluation methodology for each type of symmetry detection algorithm validated. We demonstrate and analyze quantified detection results in terms of precision-recall curves and F-measures for all algorithms evaluated. We also offer a short survey of the paper-track submissions accepted for our 2017 symmetry challenge.
Full-text available
Detection of (mirror) symmetry-which is abundantly present in the world-is often assumed to be an integral part of the perceptual organization process that is applied to every visual stimulus. On the one hand, as reviewed in this article, the exact role of symmetry in perceptual organization is actually rather elusive due to its interaction with other grouping factors. On the other hand, the detectability of single and multiple symmetry is indeed extraordinary in comparison to that of repetition and Glass patterns. Empirical and theoretical findings pertaining to this detectability are discussed, focusing on converging evidence rather than on details of individual studies, and putting various findings in evolutionary perspectives. Reflecting a seeming opposition between process models and representation models, a specific question is whether symmetry detection relies on crude or precise correspondences between symmetry halves. Both stances find psychophysical support, and this article ends with a unification proposal.
Conference Paper
Full-text available
This article proposes a new method for detecting symmetry points in images. Like other symmetry detection algorithms, it assigns a " symmetry score " to each image point. Our symmetry measure is only based on scalar products between gradients and is therefore both easy to implement and of low runtime complexity. Moreover, our approach also yields the size of the symmetry region without additional computational effort. As both axial symmetries as well as some rotational symmetries can result in a point symmetry, we propose and evaluate different methods for identifying the rotational symmetries. We evaluate our method on two different test sets of real world images and compare it to several other rotational symmetry detection methods.
Conference Paper
Full-text available
The " gradient product transform " is a recently proposed image filter for assigning each image point a symmetry score based on scalar products of gradients. In this article, we show that the originally suggested method for finding the radius of the symmetry region is unreliable, and a more robust method is presented. Moreover, we extend the symmetry transform to rectangular symmetry regions so that it is more robust with respect to skew, and the transform is generalised to also work with three dimensional image data. We apply the transform to two different problems: detection of objects with rotational symmetry, and blood vessel extraction from medical images. In an experimental comparison with other solutions for these problems, the gradient product transform performs comparable to the best known algorithm for rotational symmetry detection, and better than the vesselness filter for blood vessel extraction.
Full-text available
Symmetry is a pervasive phenomenon presenting itself in all forms and scales in natural and manmade environments. Its detection plays an essential role at all levels of human as well as machine perception. The recent resurging interest in computational symmetry for computer vision and computer graphics applications has motivated us to conduct a US NSF funded symmetry detection algorithm competition as a workshop affiliated with the Computer Vision and Pattern Recognition (CVPR) Conference 2011 and 2013. This 2013 competition sets a more comprehensive benchmark for com-puter vision symmetry detection algorithms. In this report we explain the evaluation metric and the automatic evalua-tion workflow. We also present and analyze the algorithms submitted, and show their results on three test sets of real world images depicting reflection, rotation and translation symmetries respectively. This competition establishes a per-formance baseline for future work on symmetry detection.
We present a new method for data‐based selection of the bandwidth in kernel density estimation which has excellent properties. It improves on a recent procedure of Park and Marron (which itself is a good method) in various ways. First, the new method has superior theoretical performance; second, it also has a computational advantage; third, the new method has reliably good performance for smooth densities in simulations, performance that is second to none in the existing literature. These methods are based on choosing the bandwidth to (approximately) minimize good quality estimates of the mean integrated squared error. The key to the success of the current procedure is the reintroduction of a non‐stochastic term which was previously omitted together with use of the bandwidth to reduce bias in estimation without inflating variance.
The concept of symmetry has received significant attention in computer graphics and computer vision research in recent years. Numerous methods have been proposed to find, extract, encode and exploit geometric symmetries and high-level structural information for a wide variety of geometry processing tasks. This report surveys and classifies recent developments in symmetry detection. We focus on elucidating the key similarities and differences between existing methods to gain a better understanding of a fundamental problem in digital geometry processing and shape understanding in general. We discuss a variety of applications in computer graphics and geometry processing that benefit from symmetry information for more effective processing. An analysis of the strengths and limitations of existing algorithms highlights the plenitude of opportunities for future research both in terms of theory and applications.
It is important to display effectively 3D terrain maps with features such as ridges [Lopez et al., 1999]. This paper deals with displaying problems of broad and accurate 3D terrain maps with the feature information. As feature extraction methods a discrete Laplace transform method, and the steepest descent line method [Yokoyama et al., 2007] are known. We consider that those methods are insufficient, since the obtained maps are not accurate. In this paper, we modify the steepest descent line method and propose the steepest ascent method (cf. [Cousty et al., 2009]). Furthermore, we apply it and generate the ridge lined entire Japan 3D terrain map with around 4.5 x 107 cells.
Conference Paper
A novel and efficient method is presented for grouping feature points on the basis of their underlying symmetry and characterising the symmetries present in an image. We show how symmetric pairs of features can be efficiently detected, how the symmetry bonding each pair is extracted and evaluated, and how these can be grouped into symmetric constellations that specify the domi- nant symmetries present in the image. Symmetries over all orientations and radii are considered simultaneously, and the method is able to detect local or global symmetries, locate symmetric figures in complex backgrounds, detect bilateral or rotational symmetry, and detect multiple incidences of symmetry.