Content uploaded by Deng Fuqin

Author content

All content in this area was uploaded by Deng Fuqin on Apr 27, 2021

Content may be subject to copyright.

An edge detection algorithm based on rectangular Gaussian

kernels for machine vision applications

Fuqin Denga, Kenneth S.M. Fungb, Jiangwen Dengb, and Edmund Y. Lama

aImaging Systems Laboratory, Department of Electrical and Electronic Engineering,

University of Hong Kong, Pokfulam Road, Hong Kong

bASM Assembly Automation Ltd, Kwai Chung, Hong Kong

ABSTRACT

In this paper, we develop rectangular Gaussian kernels, i.e. all the rotated versions of the ﬁrst order partial

derivatives of the 2D nonsymmetrical Gaussian functions, which are used to convolve with the test images for

edge extraction. By using rectangular kernels, one can have greater ﬂexibility to smooth high frequency noise

while keeping the high frequency edge details. When using larger kernels for edge detection, one can smooth more

high frequency noise at the expense of edge details. Rectangular kernels allow us to smooth more noise along one

direction and detect better edge details along the other direction, which improve the overall edge detection results

especially when detecting line pattern edges. Here we propose two new approaches in using rectangular Gaussian

kernels, namely the pattern-matching method and the quadratic method. The magnitude and directional edge

from these two methods are computed based on the convolution results of the small neighborhood of the edge

point with the rectangular Gaussian kernels along diﬀerent directions.

Keywords: Directional convolution, edge detection, rectangular kernel, Gaussian kernel, machine vision appli-

cations

1. INTRODUCTION

Edge detection is commonly taken for granted in high-level image analysis and computer vision applications.

However, the errors in the detected edge position and direction are often not negligible, especially when one

intends to compute diﬀerential properties of image curves. Attempts have been made in edge detection at a sub-

pixel accuracy. For examples, Huertas and Medioni use a reﬁnement of the zero-crossing of Laplacian to arrive

at sub-pixel edge detection,1while Devernay proposes a method with sub-pixel (which is less than one tenth of

a pixel in this case) accuracy of edge feature position based on simple bilinear or quadratic interpolation and

non-maxima suppression method.2Comparatively, more researchers are concerned with improving the accuracy

of the edge feature position, while fewer researchers are concerned with improving the accuracy of the direction

of the edge features. However, high-level image analysis and computer vision applications depend signiﬁcantly

on the accuracy of the edge features, such as detecting lines and circle and general arbitrary shapes through

Hough transform,3, 4 robust corresponding points matching in stereo vision5and image registration,6and motion

estimation through optical ﬂow.7

The directional information of the edge feature is very valuable in providing robust information for high level

processing. However, the magnitude of directional error is often not well described in the current literature. In

this paper, the rectangular kernels are used for edge detection, which will improve the accuracy of edge direction.

An experimental method is given to measure the error of directional information, and two new methods, called

the pattern-matching method and the quadratic method, are proposed. The paper is organized as follows. In

Section 2, the square kernels and rectangular kernels are compared in details. In Section 3, two new approaches

to edge detection are proposed, and experiments are given to show that our methods are more accurate than

Canny’s method. In Section 4, real images of semiconductor dies are tested to show the eﬃcacy of our method.

Some other applications in machine vision with our method and concluding remarks are given in Section 5 and

Section 6 respectively.

Further author information: (Send correspondence to Edmund Y. Lam)

Edmund Y. Lam: E-mail: elam@eee.hku.hk, Telephone: 852 2241 5942

Image Processing: Machine Vision Applications II, edited by Kurt S. Niel, David Fofi, Proc. of SPIE-IS&T Electronic Imaging,

SPIE Vol. 7251, 72510N · © 2009 SPIE-IS&T · CCC code: 0277-786X/09/$18 · doi: 10.1117/12.805241

SPIE-IS&T/ Vol. 7251 72510N-1

(a) (b) (c) (d)

Figure 1. (a) The symmetrical Gaussian function, where the variances along xand ydirections are equal; (b) The 45◦

rotated version of nonsymmetrical Gaussian function with diﬀerent variances along xand y; (c) and (d) The 45◦rotated

versions of the ﬁrst order partial derivative of symmetrical Gaussian and nonsymmetrical Gaussian, respectively.The sum

of variances along xand ydirections are the same, so they have equal ability to smooth noise while detecting the edge

components.

2. SQUARE KERNELS AND RECTANGULAR KERNELS

As edge is essentially high frequency content, gradient measurements are frequently used in edge detection. In

the earliest methods, diﬀerential operators such as Prewitt kernels, Roberts kernels, or Sobel kernels3, 8 are used

to calculate the local variation of the gray levels in the image. However, it is also known that noise will be

magniﬁed by using these kernels. In order to smooth noise, square kernels developed according to the criteria

for edge detection from Canny9are better choices. The square kernels, which are so called because the kernel

shape is the same in the xand ydirections, also have many good properties when they are the ﬁrst order partial

derivatives of Gaussian function:

1. The Gaussian function is separable, so high dimensional problem can be formulated into several 1D prob-

lems.

2. The Gaussian function is suﬃciently smooth and it is a good choice for reducing high frequency noise.

3. The symmetrical Gaussian kernel is isotropic since the variances along xand ydirections are the same.

The invariance to rotational transformation can be used to extract some feature invariants.10,11 Furthermore,

the square kernel is steerable under the rotation Lie transformation group action. Thus, only ﬁnite partial

derivatives have to be performed in order to obtain the derivatives in any direction.12–15

The resulting steerable pyramid with steerable property retains some advantages of orthogonal wavelet trans-

forms (e.g., the bases are localized in space and frequency domain), while improves on some of their disadvantages

(e.g., those representations are heavily aliased),16 and the steerable property has wide applications in local ori-

entation analysis,17 motion estimation,18 texture analysis,19,20 eﬃcient ﬁltering,21, 22 and so on.

In spite of so many good properties for square kernels, they are only suitable for detecting non-oriented

structures. Because of the direction insensitive property of square kernels and the discretization and quantization

eﬀects, using the square kernels is not accurate for detecting line pattern structures, especially for detecting the

direction of the lines as shown in later experiments. For practical applications such as vision-based high precision

measurement, alignment, and defect inspection, we need to detect more oriented structures such as line patterns.

Based on the above requirements, all the rotated versions of the rectangular kernels, which are the cases where

the variances along xand ydirections are not the same, are used for edge detection in this paper. Fig. 1 shows

the images of square and rectangular kernels. Fig. 2 gives the comparison results of convolutions with square

kernel and rectangular kernel shown in Fig. 1(c) and (d). In order to make the experiment comparable, we choose

the square kernel and rectangular kernel that have the same total ability to smooth noise along all the directions.

In the case of square kernel, some high frequency edge details are blurred when smoothing noise. The scale of

the square kernel aﬀects the result: If the scale is large, it smooths more noise and loses more edge details, while

SPIE-IS&T/ Vol. 7251 72510N-2

(a) (b) (c) (d)

Figure 2. (a) An image of two lines close to each other; (b) The same image with some random noise; (c) and (d) are

the convolution results with 45◦rotated version of square kernel and rectangular kernel shown in Fig. 1(c) and (d). In

this comparison experiment, the sum of variances along xand ydirections are the same, so the square kernel and the

rectangular kernel have the same ability to reduce noise.

if the scale is small, it detects more high frequency edge details, but at the same time noise is also magniﬁed

and gives false edge details. Indeed, the scale of the kernel determines the tradeoﬀ between smoothing noise and

keeping details, and choosing the appropriate scale is still on-going research.23–27 In the rectangular kernel case,

thestrategyismoreﬂexible: Wecansmoothmorenoisealongtheedgedirectionandsmoothlessalongthe

direction perpendicular to the edge direction to calculate the high frequency details, since the variances of the

rectangular kernels are diﬀerent along diﬀerent directions. If we set the same threshold in Fig. 2 (c) and (d), we

can ﬁnd that the two nearby lines become one line in the case of square kernel, whereas the two lines can still

be distinguished from each other after smoothing most of the noise in the rectangular kernel case.

3. THE IMPROVED ALGORITHMS FOR EDGE DETECTION

The traditional Canny’s edge detection method for detecting the direction of the edge is based on the square

kernels. They are ﬁnite basis kernels that can span all the rotated versions of the partial derivative of the

symmetrical Gaussian function. In fact, these partial derivatives along the xdirection G1

0◦and its 90◦rotated

version G1

90◦are enough to span all its rotated versions G1

θby the following equation

G1

θ= cos(θ)G1

0◦+sin(θ)G1

90◦,(1)

for all θ∈Rwhere

G1

0◦=∂

∂xe−(x2+y2)and G1

90◦=−∂

∂ye−(x2+y2).(2)

One only needs to use these two basis kernels G1

0◦and G1

90◦to perform convolutions with the image Ito get

the edges along the xand ydirections by E1

0◦=G1

0◦∗Iand E1

90◦=G1

90◦∗I. Then the magnitude and the

directional information at each point in the image can be obtained by

Em=(E1

0◦)2+(E1

90◦)2(3)

Ed=arctan(E1

0◦,E1

90◦)(4)

The gradient ridges are then thinned using non-maximum suppression. Then, the thinned gradient magnitudes

are thresholded to produce the edge map.

The problem with Canny’s method is that it is sensitive to weak edges and susceptible to false edges.28 The

directional information detected by this method is not accurate because of the discretization, quantization, and

noise from the digital images and digital kernels as we will demonstrate later in this paper. In order to improve

the accuracy of the edge detection result, in the following methods proposed by us, all the rotated versions of

the rectangular kernel will be used as shown in Fig. 3.

SPIE-IS&T/ Vol. 7251 72510N-3

riuiiiiiuII1Ij

riruriri;;iiiiii

urI1I1

riru;iiiiu;iiiiuriI1

i;iiUI]eu;;iiuI1

Figure 3. All the rectangular kernels used for edge detection.

3.1 Pattern-matching Method

From a geometrical point of view, the convolution operation is essentially a series of inner products, and therefore

choosing rectangular kernels for detecting line patterns is more feasible since they appear more similar. Our

pattern-matching method for edge detection is based on this geometrical view. We use all the rotated versions

of the ﬁrst-order partial derivative of 2D nonsymmetrical Gaussian kernels at every degree as shown in Fig. 3.

For each pixel in the image, all the kernels are used to convolve with the image and the maximum convolution

result is found. This means that the kernel matches best with the neighborhood of that pixel. Hence, the direction

of the kernel can be considered as the direction of the edge at that pixel, and the diﬀerence along the gradient

direction, which is the direction corresponding to the maximum convolution value, can be considered as the

magnitude of the edge. Fig. 4 gives the comparison results for a noisy line image. A line image with noise is

constructed ﬁrst, then Canny’s method and the pattern-matching method are used to detect the direction of the

line. The errors at the points near the line are calculated by comparing the detected results with ground truth

(e.g., it is a 38◦line in the ﬁgure).

Fig. 5 gives more exhaustive results to validate the eﬃcacy of the pattern-matching method. All the lines

from 0◦to 90◦are tested in a similar way. From these experiments, one can see that the pattern-matching

method is more accurate when detecting the direction of the edge. The accuracy depends on the number of

rotated kernels used. If higher accuracy is required, more kernels are needed, such as if we use all the rotated

versions of the ﬁrst order partial derivative of rectangular Gaussian kernel at every half degree. We therefore can

suggest a mixed approach: Although the Canny’s method is not so accurate, it can be used as a rough estimation

of the direction. Based on this estimation, one only has to convolve the image with the rectangular kernels in

the neighboring directions of this estimation in order to get an accurate directional information. This technique

greatly increases the speed. The number of neighboring directions chosen for comparison is dependent on the

error bounds of the Canny’s method, which can be observed from Fig. 5.

3.2 Quadratic Method

We can improve upon the pattern-matching method above somewhat. Based on the initial estimation of the

direction θ(possibly obtained from Canny’s method), we choose another two directions, θ0−δand θ0+δ,where

SPIE-IS&T/ Vol. 7251 72510N-4

UU

(a) Line (b) Noisy line

0

50

100

150

0

50

100

150

−8

−6

−4

−2

0

2

4

6

8

10

X

y

Direction errors

Errors along the line

Original line

(c) Direction errors along the line for pattern-matching

method

0

50

100

150

0

50

100

150

−8

−6

−4

−2

0

2

4

6

8

10

X

Y

Direction errors

Errors along the line

Original line

(d) Direction errors along the line for Canny’s method

0 10 20 30 40 50 60 70 80 90 100

−10

−5

0

5

10

15

Number of edge points

Direction errors

Pattern−matching method

Max absolute=10.1681

Mean=2.226

Std=4.061

(e) Direction errors at all the edge points for pattern-

matching method

0 10 20 30 40 50 60 70 80 90 100

−10

−5

0

5

10

15

Number of edge points

Direction errors

Canny’s method

Max absolute=3

Mean=0.41121

Std=1.1155

(f) Direction errors at all the edge points for Canny’s method

Figure 4. (a) The original 38◦line; (b) The line with some Gaussian noise; (c) and (e) show the directional errors with

Canny’s method, i.e. using inverse tangent to calculate the directional information; (d) and (f) show the directional errors

with the pattern-matching method. In (c) and (d), x axis and y axis denote the x and y coordinates of the edge points

respectively. The errors for Canny’s method and pattern matching method at each edge point are plotted in (e) and (f)

respectively.

SPIE-IS&T/ Vol. 7251 72510N-5

/r

0

1

0

2

0 30

4

0 50 60

7

0 80 90

0

5

10

15

Canny’s Method

Pattern−matching Method

(a) Maximum absolute error along all the directions

0

1

0

2

0 30

4

0 50 60

7

0 80 90

−5

0

5

Canny’s Method

Pattern−matching Method

(b) Mean error along all the directions

0

1

0

2

0 30

4

0 50 60

7

0 80 90

0

1

2

3

4

Canny’s Method

Pattern−matching Method

(c) Standard deviation of error along all the directions

Figure 5. Comparison of errors between Canny’s method and the pattern-matching method with rectangular kernels when

detecting lines along all the directions. In this experiment, for every half degree, a line is constructed, and the two

methods are used to detect the direction of the line. In the plots, x-axis denotes the direction of lines, and y-axis denotes

the statistics of the error.

(a) (b) (c)

Figure 6. (a) A typical semiconductor die image; (b) The edge detection result with Canny’s method; (c) The edge

detection result based on our quadratic method with rectangular kernels.

δis a small number, we can construct a quadratic curve based on these three directions and their corresponding

convolution results. The maximum value of this quadratic curve can be considered as the magnitude of the edge,

and the symmetrical axis at which the maximum is achieved can be considered as the direction of the edge. The

value o f δfor detecting the edge along certain direction is determined by the error bound of the estimation along

that direction (e.g., it can be observed from Fig. 5 for the initial direction estimation using Canny’s method).

Actually, this process can be iterative. The direction obtained at this step can be used as the estimation for the

next step, and the value of δdecreases step by step. Other curves (e.g., spline polynomial) can also be options

for the interpolation.

4. OTHER COMPARISON EXPERIMENTS

Fig. 6 gives the edge detection results of Canny’s method and our quadratic method. In these experiments, the

thresholds are the same and we perform the same non-maximum suppression strategy with Canny’s method.

As shown in the marked region, our quadratic method not only smooths more noise in the neighborhood of the

edges of the semiconductor die image, but also keeps more edge details to distinguish the nearby edges, whereas

using Canny’s method, some adjacent lines are missed because the high frequency information is lost during the

smoothing process.

SPIE-IS&T/ Vol. 7251 72510N-6

5. APPLICATIONS IN MACHINE VISION

The above algorithms have wide application in machine vision such as semiconductor inspection and biomedical

image processing. They are especially helpful in line or circle detection through Hough transform. The directional

information can also be used for robust points matching in stereo vision, and for characteristic descriptor or

feature invariants11 in pattern recognition applications. We can also use them for optical ﬂow computations

and motion estimation. For low-level image processing, the directional convolution can also be used for edge

detection and texture analysis.

6. CONCLUSION

Two edge detection methods are proposed in this paper. Experiments show that our methods based on the

rectangular kernels are more accurate than Canny’s method, especially for detecting the directional information

of the line patterns. In our methods, more kernels are used to gain high accuracy. We intend to discuss further

speed-up strategies in future publications.

ACKNOWLEDGMENTS

The authors would like to thank ASM Assembly Automation Ltd. for the support of this research work. The

ﬁrst author would also like to thank Prof. ZeXiang Li of the Electronic and Computer Engineering Department

of the Hong Kong University of Science and Technology, Prof. Yu Wang of the Mechanical and Automation

Engineering Department of the Chinese University of Hong Kong, and Prof. XiaoJun Wu of the Control and

Mechatronics Engineering Department of the Harbin Institute of Technology ShenZhen Graduate School, for

kindhelpandsuggestions.

REFERENCES

[1] A. Huertas and G. Medioni, “Detection of intensity changes with subpixel accuracy using Laplacian-Gaussian

masks,” International Journal of Pattern Recognition and Image Analysis 8(5), pp. 651–664, 1986.

[2] F. Devernay, “A non-maxima suppression method for edge detection with sub-pixel accuracy,” Tech. Rep.

RR-2724, 1999.

[3] R.C.GonzalezandR.C.Woods,Digital Image Processing, Prentice Hall, 2nd ed., 2002.

[4] D. Ballar, “Generalizing the Hough transform to detect arbitrary shapes,” Pattern Recognition 13(2),

pp. 111–122, 1981.

[5] O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint, MIT press, 1993.

[6] B. Zitova and J. Flusser, “Image registration methods: A survey,” Image and Vision Computing 21(11),

pp. 977–1000, 2003.

[7] B. Horn and B. Schunck, “Determining optical ﬂow,” Artiﬁcial Intelligence 17, pp. 185–203, 1981.

[8] A. Rosenfeld and A. Kak, Digital Picture Processing, New York: Academic, 2nd ed., 1982.

[9] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Analysis and Machine

Intelligence 8(6), pp. 679–698, 1986.

[10] T. Lindeberg, “Scale-space theory: A basic tool for analysing structures at diﬀerent scales,” Journal of

Applied Statistics 21, pp. 225–270, 1994.

[11] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer

Vision 60, pp. 91–110, 2004.

[12] W. T. Freeman and E. H. Adelson, “The design and use of steerable ﬁlters,” IEEE Trans. Pattern Analysis

and Machine Intelligence 13(9), pp. 891–906, 1991.

[13] Y. Hel-Or and P. Teo, “A common framework for steerability, motion estimation and invariant feature

detection,” 5, pp. 337–340, 1996.

[14] Y. Hel-Or and P. Teo, “Canonical decomposition of steerable functions,” Proceedings of the IEEE Interna-

tional Conference on Computer Vision and Pattern Recognition, pp. 809–816, 1996.

[15] Y. Hel-Or and P. Teo, “Lie generators for computing steerable functions,” Pattern Recognition Letters 19(7),

pp. 7–17, 1998.

SPIE-IS&T/ Vol. 7251 72510N-7

[16] E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: A ﬂexible architecture for multi-scale deriv-

ative computation,” Proceedings of the IEEE International Conference on Image Processing, pp. 444–447,

1995.

[17] E. P. Simoncelli and H. Farid, “Steerable wedge ﬁlters for local orientation analysis,” IEEE Trans. Image

Processing 5(9), pp. 1377–1382, 1996.

[18] K. Krajsek, “Steerable ﬁlters in motion estimation,” Tech. Rep. TR-L-0401, Frankfurt University, 2006.

[19] D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Pearson Prentice Hall, 1st ed., 2004.

[20] J. Portilla and E. P. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet

coeﬃcients,” International Journal of Computer Vision 40, pp. 49–71, 2000.

[21] P. Perona, “Deformable kernels for early vision,” IEEE Trans. Pattern Analysis and Machine Intelli-

gence 17(5), pp. 488–499, 1995.

[22] R. Manduchi, P. Perona, and D. Shy, “Eﬃcient deformable ﬁlter banks,” IEEE Trans. Signal Process-

ing 46(4), pp. 1168–1173, 1998.

[23] T. Lindeberg, “Edge detection and ridge detection with automatic scale selection,” International Journal

of Computer Vision 30, pp. 465–470, 1996.

[24] J. H. Elder and S. W. Zucker, “Local scale control for edge detection and blur estimation,” IEEE Trans.

Pattern Analysis and Machine Intel ligence 20(7), pp. 699–716, 1998.

[25] M. Peter and G. Bogdan, “Edge detection with embedded conﬁdence,” IEEE Trans. Pattern Analysis and

Machine Intelligence 23(12), pp. 1351–1365, 2001.

[26] A. Almansa and T. Lindeberg, “Fingerprint enhancement by shape adaptation of scale-space operators with

automatic scale selection,” IEEE Trans. Image Processing 9(12), pp. 2027–2042, 2000.

[27] X. Zhuang, “Multi-feature edge detection with scale-rate as the measurement of local image complexity,”

Proceedings of the IEEE International Conference on Networking, Sensing and Control , 2005.

[28] M. Basu, “Gaussian-based edge-detection methods: A survey,” IEEE Trans. System, Man, and Cybernet-

ics 32(3), pp. 252–260, 2002.

SPIE-IS&T/ Vol. 7251 72510N-8