Automated visual inspection system for wood defect classification using computational intelligence techniques

Abstract

This article presents improvements in the segmentation module, feature extraction module, and the classification module of a low-cost automated visual inspection (AVI) system for wood defect classification. One of the major drawbacks in the low-cost AVI system was the erroneous segmentation of clear wood regions as defects, which then introduces confusion in the classification module. To reduce this problem, we use the fuzzy min–max neural network for image segmentation (FMMIS). The FMMIS method grows boxes from a set of seed pixels, yielding ideally the minimum bounded rectangle for each defect present in the wood board image. Additional features with texture information are considered for the feature extraction module, and multi-class support vector machines are compared with multilayer perceptron neural networks in the classification module. Results using the FMMIS, additional features, and a pairwise classification support vector machine on a 550 test wood image set containing 11 defect categories show 91% of correct classification, which is significantly better than the original 75% of the low-cost AVI system. The use of computational intelligence techniques improved significantly the overall performance of the proposed automated visual inspection system for wood boards.

Full text

International Journal of Systems Science
Vol. 40, No. 2, February 2009, 163–172
Automated visual inspection system for wood defect classification using computational
intelligence techniques
Gonzalo A. Ruz
a
*, Pablo A. Este
´vez
b
and Pablo A. Ramı
´rez
b
a
Manufacturing Engineering Centre, Cardiff University, Cardiff, UK;
b
Department of Electrical Engineering,
Universidad de Chile, Casilla 412-3, Santiago, Chile
(Received 2 May 2006)
This article presents improvements in the segmentation module, feature extraction module, and the classification
module of a low-cost automated visual inspection (AVI) system for wood defect classification. One of the major
drawbacks in the low-cost AVI system was the erroneous segmentation of clear wood regions as defects, which
then introduces confusion in the classification module. To reduce this problem, we use the fuzzy min–max neural
network for image segmentation (FMMIS). The FMMIS method grows boxes from a set of seed pixels, yielding
ideally the minimum bounded rectangle for each defect present in the wood board image. Additional features
with texture information are considered for the feature extraction module, and multi-class support vector
machines are compared with multilayer perceptron neural networks in the classification module. Results using
the FMMIS, additional features, and a pairwise classification support vector machine on a 550 test wood image
set containing 11 defect categories show 91% of correct classification, which is significantly better than the
original 75% of the low-cost AVI system. The use of computational intelligence techniques improved
significantly the overall performance of the proposed automated visual inspection system for wood boards.
Keywords: AVI systems; image segmentation; wood defect classification; multi-class support vector machines
1. Introduction
Automated visual inspection (AVI) systems are an
automated form of quality control normally achieved
using a camera connected to a computer. The AVI
framework includes the following stages (Pham and
Alcock 2003):
(1) Image acquisition: to obtain an image of the
object to be inspected.
(2) Image enhancement: to improve the quality
of the acquired image, which facilitates later
processing.
(3) Image segmentation: to divide the image into
areas of interest and background. The result of
this stage is called the segmented image, where
objects represent the areas of interest.
(4) Feature extraction: to calculate the values of
parameters that describe each object.
(5) Classification: to determine what is represented
by each object.
AVI systems have been used with success in several
industries, achieving considerable improvements com-
pared to human inspection. Some examples are: pulp
inspection under critical lighting (Campoy, Canaval
and Pen
˜a 2005), contaminant removal from wool
(Zhang et al. 2005), inspection of colour tablets in
pharmaceutical blisters (Derganc, Likar, Bernard,
Tomazˇ evic
ˇand Pernus
ˇ2003), misplaced component
inspection for printed circuit boards (Enke and Dagli
1997), and defect detection in textile fabrics
(Abouelela, Abbas, Eldeeb, Wahdan and Nassar
2005). More industrial applications as well as impor-
tant issues and directions for designing and developing
industrial vision systems can be found in Malamas,
Petrakis, Zervakis, Petit and Legat (2003).
In the wood industry, there are human operators
that identify and locate board areas containing defects.
Errors in determining a defect and its location, known
as operator error, increases product cost. Buehlmann
and Thomas (2002) carried out a study to measure the
human operators performance in the wood defect
detection problem. Three types of operator error were
investigated: (a) when the operator marked a defect
when there was none, (b) when the operator missed
a defect, and (c) when the operator marked a defect
inside the defective area of the strip (incorrect marking
of a defect within its boundaries). Results showed that
78.2% of all defects were marked incorrectly.
Specifically, 2% of all marks made were type (a)
errors, 43.4% were type (b) errors, and 32.8% were
type (c) errors. This high error rate translated in a high
rate of rejected parts after the processing (22.0%).
*Corresponding author. Email: ruzg2@cardiff.ac.uk
ISSN 0020–7721 print/ISSN 1464–5319 online
!2009 Taylor & Francis
DOI: 10.1080/00207720802630685
http://www.informaworld.com
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

They concluded that AVI systems could solve many of
the operator errors observed in this study. Due to the
high rate of human errors, such systems do not need
to be flawless. If their defect detection ability is 50%
better than the one observed in the human case
presented in their work, payback periods for incorpor-
ating an AVI system could be as low as 1 year.
A review of AVI research applied to the inspection
of wood boards showed that segmentation is often the
most time consuming part of the process, and one that
usually does not locate all defects properly (Pham
and Alcock 1998). Colour image segmentation algo-
rithms can be classified into one or more of the
following techniques (Cheng, Jiang, Sun and Wang
2001): histogram thresholding, feature space
clustering, region-based approaches, edge detection,
fuzzy approaches, neural networks, physics-based
approaches and hybrid techniques. The selection of
a colour space is application dependent. Brunner,
Maristany, Butler, VanLeeuwen and Funck (1992)
found that for wood images there is no advantage in
transforming the red, green and blue (RGB) colour
space into other colour spaces.
In Este
´vez, Perez and Goles (2003) a low-cost AVI
system for wood defect detection was introduced. The
system is composed by a standard colour video camera
connected to a PC Pentium IV; the lighting was a
mixture of two frontal halogen lights and ceiling
fluorescent lamps. The image segmentation module
operated using a histogram-based multiple threshold-
ing technique. The defect detection rate achieved was
94%. This high rate of defect detection was achieved
at the expense of increasing the rate of false-positives
(32%), i.e. dark grain lines segmented as defects. The
feature extraction module extracted 182 features from
the segmented defects and was used as inputs to a
neural classifier, which classified the defects into one of
the 11 defect categories. To improve the performance
achieved with individual classifiers, the combination of
three or five neural classifiers was carried out by simple
averaging of their outputs. All networks had one
hidden layer with an N
i
–N
h
–N
o
architecture, where N
i
is the number of inputs, N
h
is the number of hidden
units and N
o
is the number of outputs or categories.
Networks were trained by a second-order Back-
Propagation Quasi-Newton learning algorithm (BPQ)
(Saito and Nakano 1997), which uses an adaptive step-
length. The objective function minimised was the sum
of square errors plus a penalty term consisting of the
sum over all squared weights. The penalty term was
weighted by a regularisation factor. For each simula-
tion, the weights were randomly initialised. Each
variable was normalised in the range [0,1], and the
database was randomly split into training, validation
and test sets. Experimental results showed a
performance of 75% of correct classification in the
11 class wood defect problem.
Amongst the computational intelligence techni-
ques, the flexibility of fuzzy sets and the computational
efficiency of neural networks have caused a great
amount of interest in the combination of both
techniques. In the so-called neurofuzzy approaches,
Simpson (1993) introduced the fuzzy min–max (FMM)
clustering neural network, where clusters are repre-
sented as hyperboxes in the n-dimensional pattern
space. In Gabris and Bargiela (2000), the FMM was
extended and improved.
In this work, we present improvements to a low-
cost AVI system by using an image segmentation
method based on the FMM neural networks. The
method is called fuzzy min–max neural network for
image segmentation (FMMIS).
1
The FMMIS method
has been successfully applied to wood defect detection
in Ruz, Este
´vez and Perez (2005) achieving a defect
detection rate of 95% with a false positive rate of 6%
on 300 test wood board images with an average time
processing of 0.11 s per image, other applications
include face detection (Este
´vez, Flores and Perez
2005). Also, additional features are used in the
feature extraction module and the comparison of
Multi-Class Support Vectors Machines with MLP
neural networks in the classification module is
performed.
2. Image segmentation method
The image segmentation method for wood
defect detection consists of four stages, shown in
Figure 1. Each stage is described in the following
subsections.
2.1. Seed selection process
To speed up the image segmentation process, the
FMMIS does not use all the pixels from the image
analysed. Instead, it only uses a few input pixels, called
seeds, to grow the hyperboxes. The seeds are auto-
matically determined (located) by an ad hoc procedure,
which is described in what follows. Considering the
great variability of colour of the wood boards, the seed
selection is based on adaptive thresholding. For each
board image, the mean colour intensity per channel,
!
t
, and the minimum colour intensity per channel, "
t
,
where t¼R,G,B, are measured. A cumulative histo-
gram per channel, H
t
, is defined as:
HtðnÞ¼X
n
i¼"t
htðiÞð1Þ
164 G.A. Ruz et al.
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

where nis the colour intensity level (n$255) and h
t
is
the histogram of the wood board image for channel t.
In addition, a colour intensity level is selected,
per channel, based on the cumulative histogram as
follows:
#t¼$H!t
ðÞ ð2Þ
where $is a user-defined parameter, typically $$0:01.
To detect defects that are brighter than those detected
by using #t, an additional colour intensity is consid-
ered, and calculated as:
%t¼#tþ!t
2ð3Þ
For each wood board image, the seeds are the pixels
belonging to the following intensity range:
It¼"t#t
½'^%tif #t<&
t
"t#t
½' if #t(&t
(ð4Þ
where &tis a user-defined threshold for each channel,
that allows avoiding false positives when all the
defects on the wood board image are not too dark.
In Figure 1(a) the seeds are represented as white
circles.
2.2. Input patterns
The input patterns are the spatial coordinates of the
seeds, determined in Section 2.1 with each dimension
normalised in the range [0, 1]. Let Xbe a S)2 input
matrix (Figure 1(b)), where Sis the number of
seeds selected. The position of the h-th seed in
the image is represented by the vector Xh¼
xh1,xh2
ðÞ2I2, where the first coordinate indicates
the column and the second coordinate the row of the
image.
2.3. Fuzzy min–max neural network for image
segmentation
The FMMIS is built using hyperbox fuzzy sets. A
hyperbox defines a region in the n-dimensional
geometric space by pairs of min–max points for each
spatial coordinate of the image (rectangular boxes in
the case of two-dimensional images). Each hyperbox
fuzzy set has associated a membership function, which
describes the degree of membership (spatial proximity)
of a given pixel to a hyperbox in the [0,1] interval.
Let each hyperbox fuzzy set, B
j
, be defined by the
ordered set
Bj¼Xh,Vj,Wj,bjXh,Vj,Wj
!"#$
ð5Þ
where V
j
¼('
j1
,'
j2
) is the min-point and W
j
¼(!
j1
,!
j2
)
is the max-point. The membership function for the
j-th hyperbox is 0 $bjðXh,Vj,WjÞ$1. Seeds contained
within a hyperbox have full membership value equal
to 1. The more distant the seeds are from the min–max
bounds of the hyperbox, the lower are their member-
ship values. The membership function defined in
Gabris and Bargiela (2000) is used here:
bjXh,Vj,Wj
!"
¼min
i¼1,2 %min %1*fðxhi *!ji,(
&'
,
1*fð'ji *xhi,(
&'
(( ð6Þ
where fis the ramp function defined as,
fg,(ðÞ¼
1g(>1
g(0$g($1
0g(<0
8
<
:ð7Þ
and (is the sensitivity parameter, that controls how
fast the membership value decreases when the seed is
farther from the min–max bounds of the hyperbox.
Figure 2 shows the two-dimensional membership
function of (6), with min-point V¼[0.4 0.4] and
Figure 1. Different stages of the image segmentation method: (a) seed selection process, (b) input patterns, (c) FMMIS and
(d) minimum bounding rectangles enclosing the defects, which correspond to the FMMIS output.
International Journal of Systems Science 165
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

max-point W¼[0.7 0.7], and (¼1. The membership
values ranging from 0 to 1 are represented by the gray-
scale ranging from black to white, respectively.
2.3.1. FMMIS learning algorithm
(1) Initialisation: V
j
and W
j
points are initially set
to 0. When a hyperbox is adjusted for the first
time using the input pattern Xh¼ðxh1,xh2Þ, the
min and the max points of the hyperbox are
made identically to this pattern,
Vj¼Wj¼Xh:ð8Þ
(2) Hyperbox expansion: When an input pattern
(new seed pixel) is presented, the hyperbox with
the highest degree of membership is found and
expanded to enclose the input pattern. The
hyperbox expansion is accepted only if the
region contained by the expanded hyperbox is
similar in colour to the region enclosed by the
hyperbox before the expansion. A fuzzy colour
homogeneity criterion based on the standard Z
function of the Euclidean distance between the
mean colour intensities of the two hyperboxes
is defined. In this way, the colour similarity in
the RGB space between the hyperboxes before
and after the expansion is compared. A user-
defined parameter )2[0,1] is introduced to
control the required degree of colour homo-
geneity for expanding hyperboxes.
Formally, the following constraint must be
satisfied to expand a hyperbox,
Zðd,a,b,cÞ()ð9Þ
where Zis a fuzzy membership function
(Cheng, Jiang and Wang 2002) defined as
Zðx,a,b,cÞ¼
10$x$a
1*2x*a
c*a
%(
2
a$x$b
2x*c
c*a
%(
2
b$x$c
0c$x$L
8
>
>
>
>
>
>
>
<
>
>
>
>
>
>
>
:
ð10Þ
and xis the Euclidean distance between the
mean colour intensities in the image region
covered by the two hyperboxes (before and after
expansion), measured in the RGB space. The
parameters of (10) are set to: a¼0, b¼L=2,
c¼Land L¼255 ffiffiffi
3
p. If the expansion criterion
for including the current seed is not satisfied, a
new hyperbox is created starting with that seed
as done in (8).
(3) Hyperbox overlap test: Let us assume that
the hyperbox B
j
was expanded in the previous
step. To test for overlapping, a dimension-
by-dimension comparison is performed
between B
j
and all the rest B
k
with k6¼j.
Overlap exists between B
j
and B
k
, if one of
the following four cases are met for dimen-
sions i¼1, 2:
Case 1: 'ji <'
ki <!
ji <!
ki
Case 2: 'ki <'
ji <!
ki <!
ji
Case 3: 'ji <'
ki $!ki <!
ji
Case 4: 'ki <'
ji $!ji <!
ki:
(4) Hyperbox contraction: If overlap exists
between B
j
and B
k
, both hyperboxes begin to
contract until the overlap is eliminated. The
hyperbox contraction rules, which depend on
the four cases previously described, are as
follows:
Case 1: 'ji <'
ki <!
ji <!
ki,
'new
ki ¼!new
ji ¼'old
ki þ!old
ji
2ð11Þ
Case 2: 'ki <'
ji <!
ki <!
ji,
'new
ji ¼!new
ki ¼'old
ji þ!old
ki
2ð12Þ
Case 3: 'ji <'
ki $!ki <!
ji,
If !ki *'ji <!
ji *'ki then
'new
ji ¼!old
ki ð13Þ
otherwise,
!new
ji ¼'old
ki ð14Þ
00.2 0.4 0.6 0.8 1
0
0.5
1
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Figure 2. Example of the two-dimensional membership
function associated to each hyperbox used by the FMMIS.
166 G.A. Ruz et al.
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

Case 4: 'ki <'
ji $!ji <!
ki,
by symmetry the same assignments as in
Case 3.
(5) Fine-tuning hyperbox expansion: After a
single pass through all the seeds, there is a
fine-tuning hyperbox expansion process, which
allows the hyperbox to grow, if necessary,
until the defect is completely enclosed. For
two-dimensional images, the hyperboxes are
rectangles defined by four line segments. A
horizontal line segment, hmin, and a vertical
line segment, vmin, pass through the min-
point of the hyperbox. Likewise, a horizontal
line segment hmax, and a vertical line
segment, vmax, pass through the max-point
of the hyperbox. For the hyperbox B
j
, and the
colour channel t, the following notation is
introduced: hmax
jt
(r), vmax
jt
(r), hmin
jt
(r) and
vmin
jt
(r) are the colour intensities of the r-th
pixel belonging to the line segments hmax,
vmax, hmin and vmin, respectively. Let u
t
be
a colour intensity threshold defined for
channels t¼R,G,B. A line segment (any of
the four) would be expanded if it contains
pixels darker than u
t
. The following conditions
for the while cycles should be satisfied for each
t¼R,G,B.
Case 1: while ðmaxrðhmaxjtðrÞÞ $ utÞ
!new
j2¼!old
j2þ1, update hmaxjt with !new
j2
no
ð15Þ
Case 2: while ðmaxrðvmaxjtðrÞÞ $ utÞ
!new
j1¼!old
j1þ1, update vmaxjt with !new
j1
no
ð16Þ
Case 3: while ðmaxrðhminjtðrÞÞ $ utÞ
'new
j2¼'old
j2*1, update hminjt with 'new
j2
no
ð17Þ
Case 4: while ðmaxrðvminjtðrÞÞ $ utÞ
'new
j1¼'old
j1*1, update vminjt with 'new
j1
no
ð18Þ
(6) Hyperbox merging: After finishing the
fine-tuning hyperbox expansion process
(if necessary), a final step is added in order to
merge hyperboxes belonging to the same defect.
The number of hyperboxes constructed per
defect depends on the value of )in (9). Let c
be the number of hyperboxes after the
fine-tuning hyperbox expansion process.
The centroid, CB
j
(cb
j1
,cb
j2
), of the hyperbox
B
j
is computed as:
cbji ¼'ji þ!ji
2ð19Þ
for i¼1, 2 and j¼1, ...,c. Let I
j
be the image
region contained within the limits of the
hyperbox B
j
. Let d
E
(I
j
,I
k
) be the Euclidean
distance in the colour space between the mean
intensities of the image regions I
j
and I
k
. The
fuzzy membership function !
c
, which mea-
sures the proximity in colour and space
between two hyperboxes, is defined as:
!c¼min Zd
EIj,Ik
!"
,a,b,c
!"
,bjCBj
!"!"
ð20Þ
where Zis the Z-function defined in (10), and
b
j
is the membership function defined in (6).
The merging criterion is to merge hyperboxes
whose proximity defined by (20) is greater than
a given threshold.
The FMMIS representation as a neural network
(Figure 1(c)) makes it possible to explore the paralle-
lism of the algorithm. Although the learning algorithm
is not necessarily neural (there is no biological principle
underlying the expansion–contraction process), the
execution of the network, once trained, fits in a
neural scheme. For this case, a three-layered neural
network was chosen to implement the FMMIS, as
shown in Figure 1(c). The input layer, F
X
, consists of
two processing elements (PE), one for each dimension
of the input pattern (seed) Xh¼ðxh1,xh2Þ. The second
layer, F
B
, consists of mPE (ideally one per each defect
in the wood board image). Finally, the third layer, F
C
,
does the merging of the nodes of the second layer
belonging to the same defect. There are two connec-
tions between each node of F
X
and each node of F
B
(Figure 1(c)); this is because the input can be crisp or
fuzzy (min–max value). Each node of F
B
in the three-
layered neural network represents a hyperbox fuzzy
set, where the connections from F
X
to F
B
are the min-
and max-points. The transference function of F
B
is the
membership function defined in (6). The connections
are adjusted using the learning algorithm described in
this subsection. The connections between the nodes
of F
B
and F
C
have binary values and they are stored
in the Umatrix, with value 1 if b
j
follows the
merging criterion giving origin to c
k
, and with value
0 otherwise.
2.4. Output of the FMMIS
The last stage is to draw the rectangle (ideally a
minimum bounding rectangle) on each defect using
International Journal of Systems Science 167
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

the min- and max-points of each hyperbox
formed by the FMMIS algorithm, as shown in
Figure 1(d).
3. Feature extraction
The feature extraction module of the previously
developed AVI system (Este
´vez et al. 2003) extracted
features from objects and windows of 64 by 64 pixels
centred in the object geometrical centre. The features
used in this work include: 7 object geometrical features
measured on the binarised gray image (e.g. area,
perimeter, average radius, aspect ratio, etc.); 96
object colour features (24 features measured in each
of the four channels, R, G, B and gray); and
46 window colour features (e.g. mean and variance of
window histograms, mean and variance at the edge of
windows).
3.1. Additional features
In addition to the 149 features mentioned above, new
features were added. These features are computed
using the co-occurrence matrix of the defects contained
in the rectangles found by the FMMIS; they receive the
name of co-occurrence features. We considered these
features because they are useful for separating classes
like clear wood and stain where the texture of the
image is important.
To compute the co-occurrence features, the gray
levels of the image contained in a square window are
quantised into Qlevels. Then, a two-dimensional
matrix p(i,j) is constructed (0 5i,j5Q). Point (i,j)
in the matrix represents the number of times that a
pixel with level iis followed, at a distance d([0 1],
[*1 1], [*1 0] and [*1*1]) and angle #(0+, 45+, 90+
and 135+), by a pixel with level j. A number of these
matrices can be computed for angles at intervals of 45+
and for each value of d. A more detailed description
on how to calculate the co-occurrence matrix can be
found in Gonzalez and Wood (2002), and Pham and
Alcock (2003).
A large number of features can be derived from
these matrices. In our work, we computed the
following features:
Contrast ¼X
Q
i¼1X
Q
j¼1ði*jÞ2pði,jÞð21Þ
Correlation ¼PQ
i¼1PQ
j¼1i*!x
ðÞj*!y
!"
pði,jÞ
*x*yð22Þ
where,
!x¼PQ
i¼1iPQ
j¼1pði,jÞ
Qð23Þ
!y¼PQ
j¼1jPQ
i¼1pði,jÞ
Qð24Þ
*2
x¼PQ
i¼1i*!x
ðÞ
2PQ
j¼1pði,jÞ
Qð25Þ
*2
y¼PQ
j¼1j*!y
!"
2PQ
i¼1pði,jÞ
Qð26Þ
Energy ¼X
Q
i¼1X
Q
j¼1
pði,jÞ2ð27Þ
Homogeneity ¼X
Q
i¼1X
Q
j¼1
pði,jÞ
1þði*jÞ2ð28Þ
In the extraction of the co-occurrence features we used
Q¼64. Also, for each of the four angles in the
co-occurrence matrix, the four measures were com-
puted. So at the end, 16 new features were added.
4. Classification
With the new segmentation module and the 16
additional features, the use of other classification
techniques was explored to see if any extra improve-
ments could be obtained. In addition to testing the
original MLP neural networks, the multi-class support
vector machines (SVM) were explored as well.
The multi-class SVM employed in this work were the
pairwise classification, one versus the rest, and the
binary decision tree.
The SVM pairwise classification was introduced
in Friedman (1996) and has become one of the most
popular Multi-Class SVM due to its good performance
in several classification tasks (Osuna, Freund and
Girosi 1997; Mayoraz and Alpaydin 1999). The basic
idea behind this classifier is to create a binary SVM for
each possible pair of classes, that is, for a Kclass
problem there are K(K*1)/2 binary classifiers. Then
when a new example is presented for classification,
each trained SVM is evaluated with this example and
a vote is assigned to the winning class. This method is
called the voting scheme, and the class label for
the example is the class that obtained the majority of
the votes.
The one versus the rest (Vapnik 1998)
classification scheme is built using Kbinary SVMs,
168 G.A. Ruz et al.
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

used to find a decision boundary that will separate a
certain class from the rest of the classes. This method
is faster than pairwise classification since only a few
SVMs are involved.
The SVM binary decision tree (Bennett 1999; Platt,
Cristianini and Shawe-Taylor 2000) has a binary tree
architecture, where the nodes are binary classifiers.
The nodes can contain multiple classes which are split
into two equally sizes subsets while moving along in
depth of the tree. This classifier is faster than the
previous two, since it uses less binary classifiers.
5. Methods
To test the overall performance of the AVI system
for wood defect detection, colour wood board
images (320 )240 pixels) drawn from the
University of Chile database (Este
´vez et al. 2003)
was processed and used for the FMMIS. All the
objects segmented by the FMMIS were manually
labelled (there can be more than one object per
wood board image) into one of the following 10
defect categories (see Ruz, Este
´vez and Perez 2005
for details): birdseye (be), pockets (po), wane (wa),
split (sp), stain (st), blue stain (bs), pith (pi), dead
knot (dk), live knot (lk) and hole (ho). Also, the
clear wood (cl) category was added, which corre-
sponds to non-defective areas that the FMMIS
segmented as a defective one. This error is then
hopefully detected in the classification stage. The
number of wood board images used was enough to
obtain 200 examples per each defect category. Then,
the samples were split into 1100 for training, 550 for
validation and 550 for testing.
Using the results obtained in Ruz et al. (2005)
and in Ruz and Este
´vez (2005), the parameters of
FMMIS were set to, (¼1 (sensitivity parameter),
)¼0.99 (degree of colour homogeneity used in hyper-
box expansion), u
R
¼195 (fine-tuning hyperbox
expansion parameter) and D¼0.95 (hyperbox merging
parameter).
For the multi-class SVMs, a lineal kernel, a
polynomial kernel, and a Gaussian kernel were
explored using the training set. Out of the three, the
best classification performance on the validation set
was obtained by a third-order polynomial kernel.
So this kernel was used for each multi-class SVM
when testing the overall performance.
The MLP neural network was trained by a second-
order quasi-Newton learning algorithm called BPQ
(Saito and Nakano 1997). A weight decay penalty term
was added to the cost function with a regularisation
factor ", in order to avoid overfitting and improve the
generalisation performance.
The test set was used to evaluate the correct
classification percentage on the three multi-class
SVMs and the MLP neural network. Results without
and with the added texture features were taken into
account as well.
Also, in order to identify which are the most
informative and relevant features for solving the
classification problem, a feature selection method
based on mutual information called AMIFS (Tesmer
and Este
´vez 2004) was used to rank the 165 features
from the most informative to the least informative.
The AMIFS adaptively controls the trade-off between
eliminating irrelevance or redundancy, avoiding the
need of a user defined parameter, and can handle data
of mixed nature (discrete and continuous features).
6. Experimental results
For the SVMs classification methods, the correct
classification percentage was computed taking the
average value of five different test runs.
For the pairwise classification, results on the test
set are shown in Table 1. The average correct
classification performance was 91.39%. Notice that
for this type of SVM, 11(11 *1)/2 ¼55 binary SVMs
were needed to be trained. The confusion matrix of the
best solution (test run 4) is shown in Table 2. The
numbers in the confusion matrix are the classification
percentages where each row adds up to 100. We can
appreciate that there are eight classes with a classifica-
tion rate greater than 90%, amongst them, the clear
wood class increased from 77% (previous low-cost
AVI system) to 92%. This result shows that the central
problem of the previous system, which was the
confusion with the clear wood class, is overcome.
Currently, the principal confusions are stain with blue
stain and blue stain with clear wood. Nevertheless, the
classification rate of stain increased from 40% (Este
´vez
et al. 2003) to 66%.
Table 1. Pairwise classification performance without and
with co-occurrence features.
Test run
number
Correct
classification %
without co-occurrence
features
Correct
classification %
with co-occurrence
features
1 87.45 91.45
2 86.91 90.91
3 88.91 91.64
4 88.36 91.69
5 87.09 91.27
Average 87.74 ,0.86 91.39 ,0.32
International Journal of Systems Science 169
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

We can appreciate that the classifier still presents
some confusion between stain, blue stain and clear
wood.
Table 3 shows the results obtained using the one
versus the rest, reaching an average performance of
86.21%. Here, only 11 SVM were needed to be trained.
For the SVM binary decision tree, the optimal division
of all the classes into two groups was searched first.
This was done by testing different group divisions,
searching for the one that obtained best classification
performance. In total, 462 binary classifiers were tested
finding two optimal divisions that obtained the same
results. The two possible divisions are shown in
Table 4. Using any of the two configurations, the
average performance is 82.79% shown in Table 5.
Table 6 shows the results obtained by the MLP
neural networks, for different values of the regularisa-
tion parameter "using two different architectures, one
with inputs N
i
¼165, hidden units N
h
¼15 and outputs
N
o
¼11 and the other N
i
¼165, N
h
¼25 and N
o
¼11.
Using 2000 training epochs the best performance was
found with "¼0.001 achieving 83.88%.
In general, the performance increases with the
incorporation of the co-occurrence features.
Table 1 shows a 4% increase in the average perfor-
mance when these features are considered.
The ranking of the features using AMIFS is shown
in Figure 3. The 165 features are listed left to right
starting from the most informative feature ‘116’ to
the less informative one ‘6’. From this result, it is
important to point out that all the new features
(16 co-occurrence features) appear in the top part of
Table 2. Confusion matrix when using the pairwise classification.
be po wa sp bs st pi dk lk ho cl
be 1 0 0 0 0 0 0 0 0 0 0
po 0 0.88 0.02 0.02 0 0 0.06 0 0.02 0 0
wa 0 0 1 0 0 0 0 0 0 0 0
sp 0.02 0 0 0.98 0 0 0 0 0 0 0
bs 0 0 0 0 0.84 0.02 0 0 0 0 0.14
st 0 0 0 0 0.18 0.66 0 0 0.08 0 0.08
pi 0 0.04 0 0 0 0 0.96 0 0 0 0
dk 0 0.06 0 0 0 0 0 0.92 0 0.02 0
lk 0 0.02 0 0 0 0.02 0 0.04 0.92 0 0
ho 0 0 0 0 0 0 0 0 0 1 0
cl 0 0 0 0 0.08 0 0 0 0 0 0.92
Table 4. Division of the 11 defect categories into two
optimal subsets.
Configuration Subset 1 Subset 2
1 Birdseye Pockets
Wane Pith
Split Dead knot
Blue stain Live knot
Stain Hole
Clear wood
2 Birdseye Pockets
Split Wane
Blue stain Pith
Stain Dead knot
Clear wood Live knot
Hole
Table 3. One vs. the rest performance without and with
co-occurrence features.
Test run
number
Correct
classification %
without co-occurrence
features
Correct
classification %
with co-occurrence
features
1 83.36 86.32
2 84.57 86.56
3 83.89 86.91
4 84.05 85.48
5 84.11 85.76
Average 83.99 ,0.44 86.21 ,0.58
Table 5. SVM binary decision tree performance without and
with co-occurrence features.
Test run
number
Correct
classification %
without co-occurrence
features
Correct
classification %
with co-occurrence
features
1 80.95 82.35
2 81.15 82.95
3 81.37 83.89
4 80.88 82.56
5 80.99 82.21
Average 81.07 ,0.19 82.79 ,0.67
170 G.A. Ruz et al.
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

the ranking (‘150–165’ with shaded colour) meaning
that they are very informative, thus, useful for this
classification task. Other relevant features include:
‘116’ brightest part of a histogram from a centred
window on the object (blue channel), ‘96’ brightest part
of a histogram from the object (blue channel), ‘98’
darkest part of a histogram from a centred window on
the object (red channel), ‘78’ darkest part of a
histogram from the object (red channel), ‘12’ colour
variance (red channel), ‘15’ colour variance (gray
channel), ‘13’ colour variance (green channel), ‘42’
variance in the minimum bounding box of the object
(red channel) and ‘4’ the aspect ratio (height/width).
7. Conclusions
In an AVI system for wood defect classification, each
stage plays a key role in the overall performance.
Nevertheless, the segmentation stage is critical since
the features used as inputs to the classifiers are
computed from the segmented defects. Advanced
classification techniques will not yield good results
if features are not obtained from proper segmented
defects. A high false positive detection rate (clear
wood) would produce erroneous classification of
non-defective areas as well.
The segmentation method called FMMIS is based
on the original FMM, but with a new learning
algorithm specially adapted for image segmentation
tasks. The FMMIS method combines clustering with
region-based techniques to obtain a substantially
different method than the original Simpson’s FMM.
The feature extraction module improved the
classification when the co-occurrence features were
added. The importance of these features, for this
particular classification problem, was also assessed by
the results of the ranking of the features using AMIFS.
The pairwise classification outperformed the MLP
neural network as well as the other SVM’s tested,
reaching 91.39% of correct classification. This is a
substantial improvement compared to the old system,
which reached 75% of correct classification.
Although FMMIS performs a coarse segmenta-
tion, enclosing the defects by rectangles (ideally the
MBR), results show that this level of segmentation is
enough to achieve good classification performances
and also provides correct cutting boundaries of
defective areas for the rough mill, since these are
done vertically or horizontally. Because a detailed
segmentation is not needed, one of the advantages of
FMMIS is that it performs the segmentation very
fast: in average each wood board image is segmented
in 0.11 s.
Future work should address how to reduce the
confusion between the stain, blue stain and clear wood
categories. One option could be to construct special
features for this purpose, another could be to train an
additional classifier destined to solve this specific
problem.
Acknowledgement
This work was supported in part by Conicyt-Chile, under
grant Fondecyt 1050751.
Note
1. This article is an extended version of Ruz and Este
´vez
(2005).
Figure 3. Ranking of the features obtained by AMIFS.
The most informative to the less informative features are
presented left to right, the shaded features are the
co-occurrence features.
Table 6. MLP neural network performance without and
with co-occurrence features for two different network
architectures and three values of the weight-decay regular-
isation parameter ".
N
i
–N
h
–N
o
"
Correct
classification
% without
co-occurrence
features
Correct
classification
% with
co-occurrence
features
165–15–11 0.1 78.24 79.86
165–15–11 0.01 80.65 83.09
165–15–11 0.001 81.92 83.88
165–25–11 0.1 78.55 79.09
165–25–11 0.01 80.07 81.75
165–25–11 0.001 80.93 82.77
International Journal of Systems Science 171
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009

References
Abouelela, A., Abbas, H.M., Eldeeb, H., Wahdan, A.A., and
Nassar, S.M. (2005), ‘Automated Vision System for
Localizing Structural Defects in Textile Fabrics’, Pattern
Recognition Letters, 26, 1435–1443.
Bennett, K.P. (1999), ‘Combining Support Vector and
Mathematical Programming Methods for Classification’,
in Advances in Kernel Methods: Support Vector Learning,
eds. B. SchYo
¨lkopf, C.J.C. Burges, and A.J. Smola,
Cambridge, MA: MIT Press, pp. 307–326.
Brunner, C.C., Maristany, A.G., Butler, D.A., VanLeeuwen,
D., and Funck, J.W. (1992), ‘An Evaluation of Color
Spaces for Detecting Defects in Douglas-fir Veneer’,
Industrial Metrology, 2, 169–184.
Buehlmann, U., and Thomas, R.E. (2002), ‘Impact of Human
Error on Lumber Yield in Rough Mills’, Robotics and
Computer Integrated Manufacturing, 18, 197–203.
Campoy, P., Canaval, J., and Pen
˜a, D. (2005), ‘InsPulp-I!:
An On-line Visual Inspection System for the Pulp
Industry’, Computers in Industry, 56, 935–942.
Cheng, H.D., Jiang, X.H., Sun, Y., and Wang, J. (2001),
‘Color Image Segmentation: Advances and Prospects’,
Pattern Recognition, 34, 2259–2281.
Cheng, H.D., Jiang, X.H., and Wang, J. (2002), ‘Color Image
Segmentation Based on Homogram Thresholding and
Region Merging’, Pattern Recognition, 35, 373–393.
Derganc, J., Likar, B., Bernard, R., Tomazˇ evic
ˇ, D., and
Pernus
ˇ, F. (2003), ‘Real-time Automated Visual Inspection
of Color Tablets in Pharmaceutical Blisters’, Real-time
Imaging, 9, 113–124.
Enke, D., and Dagli, C. (1997), ‘Automated Misplaced
Component Inspection for Printed Circuit Boards’,
Computers and Industrial Engineering, 33, 373–376.
Este
´vez, P.A., Perez, C.A., and Goles, E. (2003), ‘Genetic
Input Selection to a Neural Classifier for Defect
Classification of Radiata Pine Boards’, Forest Products
Journal, 53, 87–94.
Este
´vez, P.A., Flores, R.J., and Perez, C.A. (2005), ‘Color
Image Segmentation Using Fuzzy Min-max Neural
Networks’, in Proceedings of the International Joint
Conference on Neural Networks, Montreal, Canada,
pp. 3052–3057.
Friedman, J. (1996), ‘Another Approach to Polychotomous
Classification’. Stanford University Technical Report.
Available online at: http://www-stat.stanford.edu/-jhf/
ftp/poly.pdf.
Gabris, B., and Bargiela, A. (2000), ‘General Fuzzy Min-max
Neural Networks for Clustering and Classification’,
IEEE Transactions on Neural Networks, 11, 769–783.
Gonzalez, R.C., and Woods, R.E. (2002), Digital Image
Processing (2nd ed.), New Jersey: Prentice Hall.
Malamas, E.N., Petrakis, E.G.M., Zervakis, M., Petit, L.,
and Legat, J. (2003), ‘A Survey on Industrial Vision
Systems, Applications and Tools’, Image and Vision
Computing, 21, 171–188.
Mayoraz, E., and Alpaydin, E. (1999), ‘Support Vector
Machines for Multiclass Classification’, in International
Workshop on Artificial Neural Networks (IWANN99), eds.
J. Mira, and J.V. Sa
´nchez-Andre
´s, pp. 833–842.
Osuna, E., Freund, R., and Girosi, F. (1997), ‘Training
support vector machines: An application to face detection’,
in IEEE Conference on Computer Vision and Pattern
Recognition, pp. 130–136.
Pham, D.T., and Alcock, R.J. (2003), Smart Inspection
Systems, London: Academic Press.
Pham, D.T., and Alcock, R.J. (1998), ‘Automated Grading
and Defect Detection: A Review’, Forest Products Journal,
48, 34–42.
Platt, J., Cristianini, N., and Shawe-Taylor, J. (2000), ‘Large
Margin DAGs for Multiclass Classification’, in Advances
in Neural Information Processing Systems, eds. S.A. Solla,
T.K. Leen, and K.R. Myuller, Cambridge, MA: MIT
Press, pp. 547–553.
Ruz, G.A., and Este
´vez, P.A. (2005), ‘Image Segmentation
Using Fuzzy Min-max Neural Networks for Wood Defect
Detection’, in Intelligent Production Machines and Systems-
First I*PROMS Virtual Conference, eds. D.T Pham,
E.E. Eldukhri, and A.J. Soroka, pp. 183–188.
Ruz, G.A., Este
´vez, P.A., and Perez, C.A. (2005), ‘A
Neurofuzzy Color Image Segmentation Method for
Wood Surface Defect Detection’, Forest Products
Journal, 55, 52–58.
Saito, K., and Nakano, R. (1997), ‘Partial BFGS Update and
Efficient Step-length Calculation for Three-layer Neural
Networks’, Neural Computation, 9, 123–141.
Simpson, P.K. (1993), ‘Fuzzy Min-max Neural Networks.
Part 2. Clustering’, IEEE Transactions on Fuzzy Sets, 1,
32–45.
Tesmer, M., and Este
´vez, P.A. (2004), ‘AMIFS: Adaptive
Feature Selection by using Mutual Information’, in
Proceedings of IEEE International Joint Conference on
Neural Networks (IJCNN’04), Budapest, Hungary,
pp. 303–308.
Vapnik, V. (1998), Statistical Learning Theory, New York:
Wiley.
Zhang, L., Dehghanib, A., Sua, Z., Kingb, T., Greenwooda,
B., and Levesleyb, M. (2005), ‘Real-time Automated Visual
Inspection System for Contaminant Removal from Wool’,
Real-time Imaging, 11, 257–269.
172 G.A. Ruz et al.
Downloaded By: [Ruz, Gonzalo A.] At: 15:49 16 February 2009