Content uploaded by Deng Fuqin
Author content
All content in this area was uploaded by Deng Fuqin on Apr 27, 2021
Content may be subject to copyright.
An edge detection algorithm based on rectangular Gaussian
kernels for machine vision applications
Fuqin Denga, Kenneth S.M. Fungb, Jiangwen Dengb, and Edmund Y. Lama
aImaging Systems Laboratory, Department of Electrical and Electronic Engineering,
University of Hong Kong, Pokfulam Road, Hong Kong
bASM Assembly Automation Ltd, Kwai Chung, Hong Kong
ABSTRACT
In this paper, we develop rectangular Gaussian kernels, i.e. all the rotated versions of the first order partial
derivatives of the 2D nonsymmetrical Gaussian functions, which are used to convolve with the test images for
edge extraction. By using rectangular kernels, one can have greater flexibility to smooth high frequency noise
while keeping the high frequency edge details. When using larger kernels for edge detection, one can smooth more
high frequency noise at the expense of edge details. Rectangular kernels allow us to smooth more noise along one
direction and detect better edge details along the other direction, which improve the overall edge detection results
especially when detecting line pattern edges. Here we propose two new approaches in using rectangular Gaussian
kernels, namely the pattern-matching method and the quadratic method. The magnitude and directional edge
from these two methods are computed based on the convolution results of the small neighborhood of the edge
point with the rectangular Gaussian kernels along different directions.
Keywords: Directional convolution, edge detection, rectangular kernel, Gaussian kernel, machine vision appli-
cations
1. INTRODUCTION
Edge detection is commonly taken for granted in high-level image analysis and computer vision applications.
However, the errors in the detected edge position and direction are often not negligible, especially when one
intends to compute differential properties of image curves. Attempts have been made in edge detection at a sub-
pixel accuracy. For examples, Huertas and Medioni use a refinement of the zero-crossing of Laplacian to arrive
at sub-pixel edge detection,1while Devernay proposes a method with sub-pixel (which is less than one tenth of
a pixel in this case) accuracy of edge feature position based on simple bilinear or quadratic interpolation and
non-maxima suppression method.2Comparatively, more researchers are concerned with improving the accuracy
of the edge feature position, while fewer researchers are concerned with improving the accuracy of the direction
of the edge features. However, high-level image analysis and computer vision applications depend significantly
on the accuracy of the edge features, such as detecting lines and circle and general arbitrary shapes through
Hough transform,3, 4 robust corresponding points matching in stereo vision5and image registration,6and motion
estimation through optical flow.7
The directional information of the edge feature is very valuable in providing robust information for high level
processing. However, the magnitude of directional error is often not well described in the current literature. In
this paper, the rectangular kernels are used for edge detection, which will improve the accuracy of edge direction.
An experimental method is given to measure the error of directional information, and two new methods, called
the pattern-matching method and the quadratic method, are proposed. The paper is organized as follows. In
Section 2, the square kernels and rectangular kernels are compared in details. In Section 3, two new approaches
to edge detection are proposed, and experiments are given to show that our methods are more accurate than
Canny’s method. In Section 4, real images of semiconductor dies are tested to show the efficacy of our method.
Some other applications in machine vision with our method and concluding remarks are given in Section 5 and
Section 6 respectively.
Further author information: (Send correspondence to Edmund Y. Lam)
Edmund Y. Lam: E-mail: elam@eee.hku.hk, Telephone: 852 2241 5942
Image Processing: Machine Vision Applications II, edited by Kurt S. Niel, David Fofi, Proc. of SPIE-IS&T Electronic Imaging,
SPIE Vol. 7251, 72510N · © 2009 SPIE-IS&T · CCC code: 0277-786X/09/$18 · doi: 10.1117/12.805241
SPIE-IS&T/ Vol. 7251 72510N-1
(a) (b) (c) (d)
Figure 1. (a) The symmetrical Gaussian function, where the variances along xand ydirections are equal; (b) The 45◦
rotated version of nonsymmetrical Gaussian function with different variances along xand y; (c) and (d) The 45◦rotated
versions of the first order partial derivative of symmetrical Gaussian and nonsymmetrical Gaussian, respectively.The sum
of variances along xand ydirections are the same, so they have equal ability to smooth noise while detecting the edge
components.
2. SQUARE KERNELS AND RECTANGULAR KERNELS
As edge is essentially high frequency content, gradient measurements are frequently used in edge detection. In
the earliest methods, differential operators such as Prewitt kernels, Roberts kernels, or Sobel kernels3, 8 are used
to calculate the local variation of the gray levels in the image. However, it is also known that noise will be
magnified by using these kernels. In order to smooth noise, square kernels developed according to the criteria
for edge detection from Canny9are better choices. The square kernels, which are so called because the kernel
shape is the same in the xand ydirections, also have many good properties when they are the first order partial
derivatives of Gaussian function:
1. The Gaussian function is separable, so high dimensional problem can be formulated into several 1D prob-
lems.
2. The Gaussian function is sufficiently smooth and it is a good choice for reducing high frequency noise.
3. The symmetrical Gaussian kernel is isotropic since the variances along xand ydirections are the same.
The invariance to rotational transformation can be used to extract some feature invariants.10,11 Furthermore,
the square kernel is steerable under the rotation Lie transformation group action. Thus, only finite partial
derivatives have to be performed in order to obtain the derivatives in any direction.12–15
The resulting steerable pyramid with steerable property retains some advantages of orthogonal wavelet trans-
forms (e.g., the bases are localized in space and frequency domain), while improves on some of their disadvantages
(e.g., those representations are heavily aliased),16 and the steerable property has wide applications in local ori-
entation analysis,17 motion estimation,18 texture analysis,19,20 efficient filtering,21, 22 and so on.
In spite of so many good properties for square kernels, they are only suitable for detecting non-oriented
structures. Because of the direction insensitive property of square kernels and the discretization and quantization
effects, using the square kernels is not accurate for detecting line pattern structures, especially for detecting the
direction of the lines as shown in later experiments. For practical applications such as vision-based high precision
measurement, alignment, and defect inspection, we need to detect more oriented structures such as line patterns.
Based on the above requirements, all the rotated versions of the rectangular kernels, which are the cases where
the variances along xand ydirections are not the same, are used for edge detection in this paper. Fig. 1 shows
the images of square and rectangular kernels. Fig. 2 gives the comparison results of convolutions with square
kernel and rectangular kernel shown in Fig. 1(c) and (d). In order to make the experiment comparable, we choose
the square kernel and rectangular kernel that have the same total ability to smooth noise along all the directions.
In the case of square kernel, some high frequency edge details are blurred when smoothing noise. The scale of
the square kernel affects the result: If the scale is large, it smooths more noise and loses more edge details, while
SPIE-IS&T/ Vol. 7251 72510N-2
(a) (b) (c) (d)
Figure 2. (a) An image of two lines close to each other; (b) The same image with some random noise; (c) and (d) are
the convolution results with 45◦rotated version of square kernel and rectangular kernel shown in Fig. 1(c) and (d). In
this comparison experiment, the sum of variances along xand ydirections are the same, so the square kernel and the
rectangular kernel have the same ability to reduce noise.
if the scale is small, it detects more high frequency edge details, but at the same time noise is also magnified
and gives false edge details. Indeed, the scale of the kernel determines the tradeoff between smoothing noise and
keeping details, and choosing the appropriate scale is still on-going research.23–27 In the rectangular kernel case,
thestrategyismoreflexible: Wecansmoothmorenoisealongtheedgedirectionandsmoothlessalongthe
direction perpendicular to the edge direction to calculate the high frequency details, since the variances of the
rectangular kernels are different along different directions. If we set the same threshold in Fig. 2 (c) and (d), we
can find that the two nearby lines become one line in the case of square kernel, whereas the two lines can still
be distinguished from each other after smoothing most of the noise in the rectangular kernel case.
3. THE IMPROVED ALGORITHMS FOR EDGE DETECTION
The traditional Canny’s edge detection method for detecting the direction of the edge is based on the square
kernels. They are finite basis kernels that can span all the rotated versions of the partial derivative of the
symmetrical Gaussian function. In fact, these partial derivatives along the xdirection G1
0◦and its 90◦rotated
version G1
90◦are enough to span all its rotated versions G1
θby the following equation
G1
θ= cos(θ)G1
0◦+sin(θ)G1
90◦,(1)
for all θ∈Rwhere
G1
0◦=∂
∂xe−(x2+y2)and G1
90◦=−∂
∂ye−(x2+y2).(2)
One only needs to use these two basis kernels G1
0◦and G1
90◦to perform convolutions with the image Ito get
the edges along the xand ydirections by E1
0◦=G1
0◦∗Iand E1
90◦=G1
90◦∗I. Then the magnitude and the
directional information at each point in the image can be obtained by
Em=(E1
0◦)2+(E1
90◦)2(3)
Ed=arctan(E1
0◦,E1
90◦)(4)
The gradient ridges are then thinned using non-maximum suppression. Then, the thinned gradient magnitudes
are thresholded to produce the edge map.
The problem with Canny’s method is that it is sensitive to weak edges and susceptible to false edges.28 The
directional information detected by this method is not accurate because of the discretization, quantization, and
noise from the digital images and digital kernels as we will demonstrate later in this paper. In order to improve
the accuracy of the edge detection result, in the following methods proposed by us, all the rotated versions of
the rectangular kernel will be used as shown in Fig. 3.
SPIE-IS&T/ Vol. 7251 72510N-3
riuiiiiiuII1Ij
riruriri;;iiiiii
urI1I1
riru;iiiiu;iiiiuriI1
i;iiUI]eu;;iiuI1
Figure 3. All the rectangular kernels used for edge detection.
3.1 Pattern-matching Method
From a geometrical point of view, the convolution operation is essentially a series of inner products, and therefore
choosing rectangular kernels for detecting line patterns is more feasible since they appear more similar. Our
pattern-matching method for edge detection is based on this geometrical view. We use all the rotated versions
of the first-order partial derivative of 2D nonsymmetrical Gaussian kernels at every degree as shown in Fig. 3.
For each pixel in the image, all the kernels are used to convolve with the image and the maximum convolution
result is found. This means that the kernel matches best with the neighborhood of that pixel. Hence, the direction
of the kernel can be considered as the direction of the edge at that pixel, and the difference along the gradient
direction, which is the direction corresponding to the maximum convolution value, can be considered as the
magnitude of the edge. Fig. 4 gives the comparison results for a noisy line image. A line image with noise is
constructed first, then Canny’s method and the pattern-matching method are used to detect the direction of the
line. The errors at the points near the line are calculated by comparing the detected results with ground truth
(e.g., it is a 38◦line in the figure).
Fig. 5 gives more exhaustive results to validate the efficacy of the pattern-matching method. All the lines
from 0◦to 90◦are tested in a similar way. From these experiments, one can see that the pattern-matching
method is more accurate when detecting the direction of the edge. The accuracy depends on the number of
rotated kernels used. If higher accuracy is required, more kernels are needed, such as if we use all the rotated
versions of the first order partial derivative of rectangular Gaussian kernel at every half degree. We therefore can
suggest a mixed approach: Although the Canny’s method is not so accurate, it can be used as a rough estimation
of the direction. Based on this estimation, one only has to convolve the image with the rectangular kernels in
the neighboring directions of this estimation in order to get an accurate directional information. This technique
greatly increases the speed. The number of neighboring directions chosen for comparison is dependent on the
error bounds of the Canny’s method, which can be observed from Fig. 5.
3.2 Quadratic Method
We can improve upon the pattern-matching method above somewhat. Based on the initial estimation of the
direction θ(possibly obtained from Canny’s method), we choose another two directions, θ0−δand θ0+δ,where
SPIE-IS&T/ Vol. 7251 72510N-4
UU
(a) Line (b) Noisy line
0
50
100
150
0
50
100
150
−8
−6
−4
−2
0
2
4
6
8
10
X
y
Direction errors
Errors along the line
Original line
(c) Direction errors along the line for pattern-matching
method
0
50
100
150
0
50
100
150
−8
−6
−4
−2
0
2
4
6
8
10
X
Y
Direction errors
Errors along the line
Original line
(d) Direction errors along the line for Canny’s method
0 10 20 30 40 50 60 70 80 90 100
−10
−5
0
5
10
15
Number of edge points
Direction errors
Pattern−matching method
Max absolute=10.1681
Mean=2.226
Std=4.061
(e) Direction errors at all the edge points for pattern-
matching method
0 10 20 30 40 50 60 70 80 90 100
−10
−5
0
5
10
15
Number of edge points
Direction errors
Canny’s method
Max absolute=3
Mean=0.41121
Std=1.1155
(f) Direction errors at all the edge points for Canny’s method
Figure 4. (a) The original 38◦line; (b) The line with some Gaussian noise; (c) and (e) show the directional errors with
Canny’s method, i.e. using inverse tangent to calculate the directional information; (d) and (f) show the directional errors
with the pattern-matching method. In (c) and (d), x axis and y axis denote the x and y coordinates of the edge points
respectively. The errors for Canny’s method and pattern matching method at each edge point are plotted in (e) and (f)
respectively.
SPIE-IS&T/ Vol. 7251 72510N-5
/r
0
1
0
2
0 30
4
0 50 60
7
0 80 90
0
5
10
15
Canny’s Method
Pattern−matching Method
(a) Maximum absolute error along all the directions
0
1
0
2
0 30
4
0 50 60
7
0 80 90
−5
0
5
Canny’s Method
Pattern−matching Method
(b) Mean error along all the directions
0
1
0
2
0 30
4
0 50 60
7
0 80 90
0
1
2
3
4
Canny’s Method
Pattern−matching Method
(c) Standard deviation of error along all the directions
Figure 5. Comparison of errors between Canny’s method and the pattern-matching method with rectangular kernels when
detecting lines along all the directions. In this experiment, for every half degree, a line is constructed, and the two
methods are used to detect the direction of the line. In the plots, x-axis denotes the direction of lines, and y-axis denotes
the statistics of the error.
(a) (b) (c)
Figure 6. (a) A typical semiconductor die image; (b) The edge detection result with Canny’s method; (c) The edge
detection result based on our quadratic method with rectangular kernels.
δis a small number, we can construct a quadratic curve based on these three directions and their corresponding
convolution results. The maximum value of this quadratic curve can be considered as the magnitude of the edge,
and the symmetrical axis at which the maximum is achieved can be considered as the direction of the edge. The
value o f δfor detecting the edge along certain direction is determined by the error bound of the estimation along
that direction (e.g., it can be observed from Fig. 5 for the initial direction estimation using Canny’s method).
Actually, this process can be iterative. The direction obtained at this step can be used as the estimation for the
next step, and the value of δdecreases step by step. Other curves (e.g., spline polynomial) can also be options
for the interpolation.
4. OTHER COMPARISON EXPERIMENTS
Fig. 6 gives the edge detection results of Canny’s method and our quadratic method. In these experiments, the
thresholds are the same and we perform the same non-maximum suppression strategy with Canny’s method.
As shown in the marked region, our quadratic method not only smooths more noise in the neighborhood of the
edges of the semiconductor die image, but also keeps more edge details to distinguish the nearby edges, whereas
using Canny’s method, some adjacent lines are missed because the high frequency information is lost during the
smoothing process.
SPIE-IS&T/ Vol. 7251 72510N-6
5. APPLICATIONS IN MACHINE VISION
The above algorithms have wide application in machine vision such as semiconductor inspection and biomedical
image processing. They are especially helpful in line or circle detection through Hough transform. The directional
information can also be used for robust points matching in stereo vision, and for characteristic descriptor or
feature invariants11 in pattern recognition applications. We can also use them for optical flow computations
and motion estimation. For low-level image processing, the directional convolution can also be used for edge
detection and texture analysis.
6. CONCLUSION
Two edge detection methods are proposed in this paper. Experiments show that our methods based on the
rectangular kernels are more accurate than Canny’s method, especially for detecting the directional information
of the line patterns. In our methods, more kernels are used to gain high accuracy. We intend to discuss further
speed-up strategies in future publications.
ACKNOWLEDGMENTS
The authors would like to thank ASM Assembly Automation Ltd. for the support of this research work. The
first author would also like to thank Prof. ZeXiang Li of the Electronic and Computer Engineering Department
of the Hong Kong University of Science and Technology, Prof. Yu Wang of the Mechanical and Automation
Engineering Department of the Chinese University of Hong Kong, and Prof. XiaoJun Wu of the Control and
Mechatronics Engineering Department of the Harbin Institute of Technology ShenZhen Graduate School, for
kindhelpandsuggestions.
REFERENCES
[1] A. Huertas and G. Medioni, “Detection of intensity changes with subpixel accuracy using Laplacian-Gaussian
masks,” International Journal of Pattern Recognition and Image Analysis 8(5), pp. 651–664, 1986.
[2] F. Devernay, “A non-maxima suppression method for edge detection with sub-pixel accuracy,” Tech. Rep.
RR-2724, 1999.
[3] R.C.GonzalezandR.C.Woods,Digital Image Processing, Prentice Hall, 2nd ed., 2002.
[4] D. Ballar, “Generalizing the Hough transform to detect arbitrary shapes,” Pattern Recognition 13(2),
pp. 111–122, 1981.
[5] O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint, MIT press, 1993.
[6] B. Zitova and J. Flusser, “Image registration methods: A survey,” Image and Vision Computing 21(11),
pp. 977–1000, 2003.
[7] B. Horn and B. Schunck, “Determining optical flow,” Artificial Intelligence 17, pp. 185–203, 1981.
[8] A. Rosenfeld and A. Kak, Digital Picture Processing, New York: Academic, 2nd ed., 1982.
[9] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Analysis and Machine
Intelligence 8(6), pp. 679–698, 1986.
[10] T. Lindeberg, “Scale-space theory: A basic tool for analysing structures at different scales,” Journal of
Applied Statistics 21, pp. 225–270, 1994.
[11] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer
Vision 60, pp. 91–110, 2004.
[12] W. T. Freeman and E. H. Adelson, “The design and use of steerable filters,” IEEE Trans. Pattern Analysis
and Machine Intelligence 13(9), pp. 891–906, 1991.
[13] Y. Hel-Or and P. Teo, “A common framework for steerability, motion estimation and invariant feature
detection,” 5, pp. 337–340, 1996.
[14] Y. Hel-Or and P. Teo, “Canonical decomposition of steerable functions,” Proceedings of the IEEE Interna-
tional Conference on Computer Vision and Pattern Recognition, pp. 809–816, 1996.
[15] Y. Hel-Or and P. Teo, “Lie generators for computing steerable functions,” Pattern Recognition Letters 19(7),
pp. 7–17, 1998.
SPIE-IS&T/ Vol. 7251 72510N-7
[16] E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: A flexible architecture for multi-scale deriv-
ative computation,” Proceedings of the IEEE International Conference on Image Processing, pp. 444–447,
1995.
[17] E. P. Simoncelli and H. Farid, “Steerable wedge filters for local orientation analysis,” IEEE Trans. Image
Processing 5(9), pp. 1377–1382, 1996.
[18] K. Krajsek, “Steerable filters in motion estimation,” Tech. Rep. TR-L-0401, Frankfurt University, 2006.
[19] D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Pearson Prentice Hall, 1st ed., 2004.
[20] J. Portilla and E. P. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet
coefficients,” International Journal of Computer Vision 40, pp. 49–71, 2000.
[21] P. Perona, “Deformable kernels for early vision,” IEEE Trans. Pattern Analysis and Machine Intelli-
gence 17(5), pp. 488–499, 1995.
[22] R. Manduchi, P. Perona, and D. Shy, “Efficient deformable filter banks,” IEEE Trans. Signal Process-
ing 46(4), pp. 1168–1173, 1998.
[23] T. Lindeberg, “Edge detection and ridge detection with automatic scale selection,” International Journal
of Computer Vision 30, pp. 465–470, 1996.
[24] J. H. Elder and S. W. Zucker, “Local scale control for edge detection and blur estimation,” IEEE Trans.
Pattern Analysis and Machine Intel ligence 20(7), pp. 699–716, 1998.
[25] M. Peter and G. Bogdan, “Edge detection with embedded confidence,” IEEE Trans. Pattern Analysis and
Machine Intelligence 23(12), pp. 1351–1365, 2001.
[26] A. Almansa and T. Lindeberg, “Fingerprint enhancement by shape adaptation of scale-space operators with
automatic scale selection,” IEEE Trans. Image Processing 9(12), pp. 2027–2042, 2000.
[27] X. Zhuang, “Multi-feature edge detection with scale-rate as the measurement of local image complexity,”
Proceedings of the IEEE International Conference on Networking, Sensing and Control , 2005.
[28] M. Basu, “Gaussian-based edge-detection methods: A survey,” IEEE Trans. System, Man, and Cybernet-
ics 32(3), pp. 252–260, 2002.
SPIE-IS&T/ Vol. 7251 72510N-8
