A comparison of methodologies for fuzzy expert system creation--application to arrhythmic beat classification.

Abstract

In this work, three different methodologies for fuzzy expert systems creation are compared: a well-known neuro-fuzzy approach, a knowledge-based approach and a novel methodology, based on rule-extraction. The adaptive neuro-fuzzy information system (ANFIS) is used to automatically generate a fuzzy expert system. In the knowledge-based approach and the rule-extraction methodology, the idea is to start with a model described by crisp rules, provided by medical experts in the first case or extracted using data mining techniques in the second, and then to transform them into a set of fuzzy rules, creating a fuzzy model. In either case, the adjustment of the model's parameters is performed via a stochastic global optimization procedure. All three approaches are applied to a medical domain problem, the cardiac arrhythmic beat classification. The ability to interpret the decisions made from the created fuzzy expert systems is a major advantage compared to other "black box" approaches.

Full text

Page 1

Abstract—In this work, three different methodologies for fuzzy
expert systems creation are compared: a well-known neuro-
fuzzy approach, a knowledge-based approach and a novel
methodology, based on rule-extraction. The adaptive neuro-
fuzzy information system (ANFIS) is used to automatically
generate a fuzzy expert system. In the knowledge-based
approach and the rule-extraction methodology, the idea is to
start with a model described by crisp rules, provided by
medical experts in the first case or extracted using data mining
techniques in the second, and then to transform them into a set
of fuzzy rules, creating a fuzzy model. In either case, the
adjustment of the model’s parameters is performed via a
stochastic global optimization procedure. All three approaches
are applied to a medical domain problem, the cardiac
arrhythmic beat classification. The ability to interpret the
decisions made from the created fuzzy expert systems is a
major advantage compared to other “black box” approaches.
I. INTRODUCTION
edical expert systems are a challenging field, requiring
the synergy of different scientific areas. The
representation of medical knowledge and expertise, the
decision making in the presence of uncertainty and
imprecision, the choice and adaptation of a suitable model,
are some issues which a medical expert system should take
under consideration. Uncertainty is traditionally treated in a
probabilistic manner; recently, however, methods based on
fuzzy logic have gained ground [1]. The model’s parameter
adaptation amounts to optimizing a properly constructed
“error” function. Expert systems are a branch of artificial
intelligence, which make extensive use of specialized
knowledge to solve problems at the level of a human expert.
This knowledge is represented by a set of rules [2]. An
expert system’s review of applications can be found in [3].
An expert system can be created by defining a crisp or fuzzy
model (set of rules) and then optimizing its parameters to fit
a given dataset. Several approaches have been proposed in
the literature for the development of fuzzy or crisp models;
in most of them the model is trained using a known
optimization technique, i.e. fuzzy rules with genetic
Manuscript received April 3, 2006. This research is partially funded by
the program "Heraklitos" of the Hellenic Ministry of Education. Also it is
supported by the European Commission as part of the project NOESIS
(IST-2002-507960).
M. G. Tsipouras, T. P. Exarchos and D. I. Fotiadis are with the Unit of
Medical Technology and Intelligent Information Systems, Dept. of
Computer Science, Univ. of Ioannina, Ioannina, Greece, GR 45110 (+30-
26510-98803; markos@cs.uoi.gr, me01238@cc.uoi.gr, fotiadis@cs.uoi.gr).
algorithms [4], fuzzy rules with simulated annealing [5],
multicriteria decision analysis with genetic algorithms [6].
Neuro-fuzzy algorithms have also been proposed; the fuzzy
rules are modelled using an artificial neural network (ANN)
structure and popular training techniques are applied [7].
In this work, three different methodologies for fuzzy
expert system creation are compared: a knowledge-based
approach, a rule-extraction methodology and the well-known
ANFIS approach [8]. All three are applied to a medical
domain problem, the cardiac arrhythmic beat classification
into four types of cardiac rhythms: normal sinus rhythm and
three types of arrhythmia: ventricular flutter/fibrillation,
ventricular tachycardia and 2o heart block. The classification
is performed using only the RR interval signal and features
extracted from it. For the knowledge-based approach and the
rule-extraction methodology the idea is similar: having an
initial crisp model (i.e. a model comprised from crisp rules),
transform it into a fuzzy model and then tune the parameters
of this model using global optimization. The initial rules can
be either defined by experts (knowledge-based approach) or
automatically extracted using data mining techniques (rule-
extraction methodology). In either case, the rules are initially
represented using the monotonic crisp membership function
in a disjunctive normal form (DNF). Then, the rules are
transformed from crisp to fuzzy ones, using a fuzzy
membership function and T and S norms. Using different
fuzzy membership functions and alternative definitions for
the T and S norms, several fuzzy models can be defined.
Finally, the adjustment of the model’s parameters is
performed via stochastic global optimization. The ANFIS is
also used to automatically generate a fuzzy expert system.
The MIT-BIH arrhythmia database [9] is used for optimizing
the fuzzy model’s parameters and evaluating the fuzzy
expert systems. Results, for all three approaches, along with
qualitative and quantitative comparison, are presented.
II.MATERIALS AND METHODS
A. Medical Background
Arrhythmia can be defined as any type of rhythm that
deviates from the normal sinus rhythm (NSR). Automatic
arrhythmic beat detection and classification, using the ECG
and/or features extracted from it, is a critical task in clinical
cardiology, especially when performed in real time.
A Comparison of Methodologies for Fuzzy Expert System Creation -
Application to Arrhythmic Beat Classification
Markos G. Tsipouras, Themis P. Exarchos and Dimitrios I. Fotiadis, Member, IEEE
M
Proceedings of the 28th IEEE
EMBS Annual International Conference
New York City, USA, Aug 30-Sept 3, 2006
FrA09.3
1-4244-0033-3/06/$20.00 ©2006 IEEE.2316

Page 2

Therefore several studies have been proposed in the
literature, based on neural networks [10,12], autoregresive
modelling [11], support vector machines [13], ECG
morphology [14], rule-based systems [15]. Some of the
studies are based on the analysis of the ECG signal,
extracting features from it and using them for the detection
and/or classification of cardiac beats. However, this is not
always feasible due to: (a) the presence of noise making
feature extraction difficult and in some cases impossible, and
(b) the inability to perform in real time. An alternative would
be to use only the RR-interval signal (tachogram) but, in this
case, it is expected that only certain types of arrhythmias can
be detected and classified. Also, most of the studies are
“black box” approaches and, therefore, lack the ability of
interpreting their decisions.
B. Dataset
The beat classification is performed using only the
tachogram. Therefore, a QRS detection method, proposed by
Tompkins [16] is used to detect the R waves and then the
tachogram is formed measuring the time intervals between
consecutive R waves. Then, a three RR-interval sliding
window is used to create the dataset:
[
123
RR ,RR ,RR
the
l three RR-interval window and
{}
ll
D d ,c
=
with
]
l
ld
=
th
lc the class of the middle RR interval (
2
RR ). The class
lc
is represented as
{}
4
0 1
,
lc
∈
, where, if
l
d belongs to class
i , then
arrhythmia database are used for the creation of the dataset
D . Both rhythm and beat annotations from the database are
used to specify the class, following the scheme: If
annotated as ventricular
[
1 0 0 0 , , ,
, else if
2
RR is annotated as PVC then
[
01 0 0, , ,
, else if
2
RR belongs to 2o heart block episode,
then
[
0 0 01, , ,
, else
0 010
l
i
ce
=
. All beats from all records of the MIT-BIH
2
then
RR is
flutter/fibrillation,
]
]
lc
=
=
lc
]
lc
=
[]
lc, , ,
=
.
C. The knowledge-based approach
Initial set of rules: Three rules, provided by medical
experts, comprise the initial set of rules:
Rule 1: Ventricular flutter/fibrillation (VF). If one of the
following conditions is true then the middle RR interval
(
2
RR ) of the window is classified into the VF category:
(
111 1121 2
,,,
E : RRRR
?
,,
E: RRRRRR
++<
.
Rule 2: Premature ventricular contractions (PVC). If one of
the following conditions is true then the middle RR interval
of the window is classified as PVC:
(
2 1122 13
,,
E : RR / RR RR / RR
(
2 2312 31
,,
E: RR / RRRR / RR
) () ()
313
,
???
RR
<∧<∧<
,
1 21231 4
) (
) (
)
)
22 2
,
??
>∧>
,
22 4
,
??
>∧>
,
() ()
3
2 3
,
122 5
,
2 2 6
,
2 7
,
12
2
???
RR
+
E: RRRRRR
RR RR
?
?
?
?
?
?
?
?
?
?
?
?
−<∧<∧<
() ()
1
2 4,23 2 8,2 2 9,2 10,
23
2
???
RR
+
E : RRRRRR
RR RR
−<∧<∧<
Rule 3: 2o heart block (BII). If one of the following
conditions is true then the middle RR interval of the window
is classified into BII category:
(
??
(
??
The crisp rules
(
1
R d ,
,
which comprise the crisp model, in DNF representation are:
(
11111 2,,
R d ,EE
,
2
R d ,
and
(
33 31 3 2,,
R d ,EE
, with
{
22
110,..,
and
3
are false, then the beat is classified as normal. If more than
one rules are true, then the beat is considered as unclassified.
Fuzzyfication of the set of rules: To transform the crisp
rules into fuzzy, the sigmoid function, defined as:
()
1g x,a,b e
=+
is used as the fuzzy membership
function, the
norm
T is
(
1212norm
Ta ,a a a
=
, and the
norm
S
(
121212norm
Sa ,aaaa a
=+−
. The fuzzy rules
(
2
Rd ,
and
3
Rd ,
classification problem, are defined as:
(
(
(
(
12
g RR RR
?
(
(
(
(
(
31
2
gRR / RR
+
?
(
(
12
2g RR / RR
+
?
?
?
(
(
(
(
2
g RR
?
?
) (
) (
)
31,231,3 2,12 3 3,
???
E: RR,RRRR
∈∧−<
??
,
)
3 2,231 ,3 2,
(
233 3,
)
???
E: RR ,RR
)
RR
(
2
R d ,
∈∧−<
??
.
1 ?
l
2 ?
l
and
())
33 ?
l
R d ,
,
)
?
l
=∨
)
()
22 1,
?
2 2,
?
2 3,
=
2 4,
?
l
EE
{
, ,
EE
=∨∨
}
. If all rules
∨
?
l
=∨
11
14
,j, j ,..,
=
,
}
??
, j, j
==
{}
3
??
1 2 3
, j, j
==
()
()
1
a b x
−
−
the
is the probabilistic OR :
algebraic product:
)
)
()
(
11 ?
fl
Rd ,
,
)
2 ?
fl
())
3 ?
fl
for the arrhythmia
()
)
)
)
)
()
111
,
11
,
21 2
,
1 2
,
11
31 3
,
1 3
,
31 4
,
1 4
,
??
??
?
??
??
,
ab
ab
norm
T
fl
norm
ab
ab
g RR ,,,
g RR ,,, ,
?
?
?
?
R d ,S
g RR ,,
RR ,
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
=
++
,
)
)
)
(
(
)
)
(
(
?
?
?
)
)
)
?
,
)
?
)
?
(
,
(
)
)
()
22
122 1
,
21
,
32 2 2
,
2 2
,
312 3
,
2 3
,
122 4
,
2 4
,
12 2 5
,
2 5
,
22 6
,
2 6
,
22 7
,
2 7
,
?
????
??
,
??
????

?
fl
abab
norm
T
abab
norm
T
abab
norm
norm
T
ab
norm
T
Rd ,
g RR / RR ,,,g RR / RR ,,,
g RR / RR ,,g RR / RR ,,,
g RRRR ,,,g RR ,,,
S
,
RR
g
=
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
−
)
)
()
232 8
,
2 8
,
22 9
,
2 9
,
3210
,
2 10
,
????
?
abab
ab
RRRR ,,,g RR ,,,
RR ,,
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
−
,
()
(
?
,
)
)
)
?
(
,
)
)
231
,
31 23 2
,
3 2
,
123 3
,
3 3
,
33
231
,
31 23 2
,
3 2
,
33 3
,
3 3
,
????
?
????
?
abab
,
norm
T
ab
fl
norm
abab
,
norm
T
ab
g RR ,, ,g RR ,,,
,
g RR RR ,
Rd , S
g RR ,,,g RR ,,,
RR ,
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
−
=
−
,
2317

Page 3

with
{
and
}
=
111
???
14
ab
,j,j
,, j
{
,..,
==
,
{
1
}
222
???
1 10
ab
,j,j
, , j,...,
==
17j,...,
=
The final decision is made as follows: if the maximum
value of the results
(
112
mmax Rd ,,R d ,
}
333
???
ab
, j ,j
,,
3
j ,...,
=
.
of the three
)
rules
)()()
()
(
equal to
233
???
flflfl
,R d ,
=
is less or
4 ? then
ld is classified as normal sinus rhythm. If
m is more than
of the rule that has the maximum result:
[0,0,0,1]
?
?
?
?
=?
?
?
? ?
where
{
123
?? ? ? ?
,,,
=
Optimization: The fuzzy model
with respect to ? using a training dataset (
randomly selected subset of D , containing 250 patterns
from each class, and the HTMLSL [17] optimization
method. The mean square error cost function is used:
()
1
l
train
D
=
4 ? then the
ld is classified in the category
()
(
(
)
)
()
114
224
4
334

??
[0,0,1,0]
??
?
[0,1,0,0]
?
[1,0,0,0]
??
l
l
l
l
if mR d ,
if mR d ,
M d ,
if m
if m
}
.
R d ,
=>
=
≤
=
>
>
,
4
()
?
D
l
M d ,
is optimized
train
), which is a
()
1000
?
2
1
??
ll
train
f D,M d ,c
=−
.
D. The rule-extraction methodology
In the rule-extraction methodology, an initial set of rules
is automatically extracted from a decision tree, induced from
the
train
D
. Functions of the three RR intervals (used in the
knowledge-based model) are included in the feature vector
(
12
RR RR ,
31
RR RR ,
3
RR RR ,
(
312
2RR / RRRR
+
,
1
2RR / RR
with a total of 11 features. The construction of the decision
tree is implemented using the C4.5 inductive algorithm [18].
Each internal node of the tree corresponds to a feature, while
each outgoing branch corresponds to the range of values of
that component. The leaf nodes represent the class to be
assigned to a sample. The C4.5 algorithm applies to a set of
data and generates a decision-tree, which minimizes the
expected value of the number of tests for the classification of
the data. The most important factor in the C4.5 algorithm is
its ability to automatically select the feature which is
appropriate at each node. The feature of each node is
selected in order to divide input samples effectively. The
information gain is used as a measure of effectiveness.
After the induction of the decision tree, a pruning method
is applied to reduce the tree’s size and complexity. There are
two common methods for pruning: prepruning and post-
pruning. Post-pruning tends to give better results than
prepruning since it makes pruning decisions based on a fully
grown tree, unlike prepruning, which can suffer from early
212
RR
+
RR
−
)
3
,
23
RR RR
−
+
,
)(
2
RR
,
)
123
RR RRRR
+
termination of the tree growing process [18]. Post-pruning is
performed by replacing a subtree with a new leaf node
whose class label is determined from the majority class of
records associated with the subtree (subtree replacement).
The subtree replacement was performed by calculating the
pessimistic error. The confidence value was set to 25% and
the minimum number of records per leaf was set to 2.
The crisp set of rules is created from the final decision tree
as follows: (a) One condition is created for every leaf of the
tree, by parsing the tree from the root node to a leaf. The tests
encountered along the path form the conjuncts of the
condition while the class label at the leaf node is assigned to
the rule consequent:
E
(
{
is a vector containing all thresholds,
features characterizing a record,
()
()
,,
iroot
c
rootroot
Aa
θ
Θ =∧
)
()
, ...,
jjjkkk
nnnnnn
caca
θθ∧∧
, where
i
Cond is a condition,
{
fn
is the number of
ja and ?j are the feature
}
12
fn
Aa ,a ,...,a
=
is the feature vector,
}
12
?? ??
fn
,,...,
=
and the threshold used in the conjunct j and
the respective conjunct. (b) A general rule is created for each
class, using all the conditions
consequent this class:
(
2n
jj
CondA,... CondA,
These general rules comprise the crisp set of rules, which is
exhaustive and mutually exclusive. Therefore, for each
feature vector A , one of the general rules is true, thus
defining its class. After the creation of the initial crisp set of
rules, the fuzzyfication and optimization steps are followed.
()
?
jjj
c a ,
is
( ?)
)
=
i
Cond A,
(
?
A,
, where y is the class.
having as
(
1
j
Cond
)
?
y
R
)
A,
∨
)(
??
∨∨
E. ANFIS
The neuro-fuzzy techniques have emerged from the
fusion of artificial neural networks and fuzzy inference
systems and form a popular framework for solving real
world problems. ANFIS is based on a Sugeno type fuzzy
system. The feature vector, described above, is used as input
and the learning algorithm is a hybrid method, proposed in
[19].
III. RESULTS
The dataset includes 484 beats annotated as VF, 6,183
beats annotated as PVC, 102,793 beats annotated as N and
420 beats annotated as BII, totally. For each methodology,
the normalized confusion matrix and the corresponding
accuracy are shown in Table I. The accuracy of the three
produced fuzzy expert systems is: 90.24% for the
knowledge-based approach, 94.13% for the ANFIS and
95.98% for the rule- extraction methodology.
IV. DISCUSSION
Three methodologies are compared, to create fuzzy expert
systems for the arrhythmic beat classification problem: a
knowledge-based approach, the ANFIS and a novel one
2318

Page 4

TABLE I
NORMALIZED CONFUSION MATRIX AND ACCURACY
FOR EACH METHODOLOGY(%).

Methodology VF
VF 89.87
PVC 0.36
N 0.5
BII 0
approach
Uncl 9.27
VF 98.9
PVC 1.1
N 0
methodology
BII 0
VF 98.2
PVC 1.8
N 0
BII 0

Database
PVC
1.08
81.54
17.08 92.57
0
0.3
1.44
92.4
6.16
0
1.82
90.7
7.48
0
















N BII
0
0.24
2.78
96.98
0
0
0.48
0.51
99.01
0
1.67
0
98.33
Accuracy
0.43
6.79
0.16
0.05
0.65
5.32
93.61
0.42
0.7
9.83
89.31
0.16
Knowledge-
based
90.24
Rule-extraction
95.98
ANFIS 94.13
based on rule-extraction. The later, initially extracts several
rules from a dataset, creating thus, a set of rules. The set of
rules is then fuzzyfied and, finally, the parameters entering
the model (?) are tuned using global optimization.
Compared to the ANFIS methodology, the proposed method
exhibits several advantages: (a) optimization is used in both
the data mining stage and in the fuzzy model, therefore, the
final expert system is better tuned and thus having better
results; (b) the fuzzy model is generated fuzzyfing an initial
crisp set of rules, thus it is simpler than the respective model
generated from ANFIS, and its optimization is faster. On the
other hand, the initial data mining stage, not existing in the
ANFIS, increases the computational cost. Compared to the
knowledge-based approach, both the ANFIS and the
proposed methodology, are data-driven and, thus, they do
not require a-priori knowledge but the quality of the
produced expert system depends on the quality of the initial
dataset. Also the results presented in Table I indicate that the
proposed method has the best results, with respect to the
arrhythmic beat classification problem.
Table II presents several methods proposed in the along
with the reported accuracy. All methods have accuracy in
the
TABLE II
SUMMARY OF METHODS FOR ARRHYTHMIC BEAT CLASSIFICATION
Authors Method
Dataset
(beats)
Accuracy
(%)
Osowski & Linh [10]
cumulants of the second,
third and fourth order and
fuzzy hybrid neural network
7,185 96.06
Ge et al. [11] autoregressive modelling 856 96.84
Osowski et al. [13] support vector machines 12,785 95.91
Chazal et al. [14]
ECG morphology and linear
discriminates
100,00097.50
Tsipouras et al. [15] knowledge-based system 109,880 94.26
Knowledge-based approach 90.24
Rule-extraction methodology 95.98 this work
ANFIS
109,880
94.13
range of 90% to 98%. The methods of [10,13] are based on
“black box” approaches, such as neural networks and
support vector machines. Therefore, there is no
interpretation for their decisions. Some of the proposed
methods have been tested on small subsets of the MIT-BIH
arrhythmia database [10,11,13]. In [14] ECG morphology is
utilized, which is an approach not feasible in cases of high
noise. In the proposed methodology, only QRS detection
was performed, on the ECG signal. The results were
obtained using the MIT-BIH arrhythmia database for
evaluation, while for each decision made, interpretation is
available.
REFERENCES
[1] L.H. Tsoukalas and R.E. Uhrig, Fuzzy and neural approaches in
engineering, John Wiley & Sons, 1997.
[2] J. Giartano and G. Riley. Expert systems, Principles and
programming, 3rd eds., PWS Publishing Company, 1998.
[3] S.H. Liao, “Expert system methodologies and applications-a decade
review from 1995 to 2004”, Expert Systems with Applications, vol. 28,
pp. 93-103, 2005.
[4] W. Pedrycz and J.V. de Oliveira, “An algorithmic framework for
development and optimization of fuzzy models”, Fuzzy sets and
systems, vol. 80, pp. 37-55, 1996.
[5] J.M. Garibalti and E.I. Ifeachor, “Application of simulated annealing
fuzzy model tuning to umbilical cord acid-base interpretation”, IEEE
Trans. Fuzzy Systems, vol. 7, pp. 72-84, 1999.
[6] Y. Goletsis, C. Papaloucas, D.I. Fotiadis, A. Likas and L.K. Michalis,
“Automated ischemic beat classification using genetic algorithms and
multicriteria decision analysis”, IEEE Trans. Biom. Eng., vol. 51, pp.
1717-1725, 2004.
[7] R. Babuska and H. Verbruggen, “Neuro-fuzzy methods for nonlinear
system identification”, Ann. Rev. Control, vol. 27, pp.73-85 2003.
[8] J. S. Jang, C. T. Sun and E. Mizutani, Neuro fuzzy and soft computing,
Matlab curriculum series, Prentice Hall, Upper Saddle River, USA,
1997.
[9] Harvard-MIT Division of Health Sciences and Technology, MIT-BIH
Arrhythmia Database CD-ROM, 3rd eds., 1997.
[10] S. Osowski and T.H. Linh, “ECG beat recognition using fuzzy hybrid
neural network”, IEEE Trans Biom Eng, vol. 48 pp.1265-71, 2001.
[11] D. Ge, N. Srinivasan and S.M. Krishnan, “Cardiac arrhythmia
classification using autoregressive modering”, Biomed. Eng. OnLine,
1:5, 2002.
[12] U.R. Acharya, P.S. Bhat, S.S. Iyengar, A. Rao and S. Dua,
“Classification of heart rate data using artificial neural network and
fuzzy equivalence relation”, Pat. Recognition, vol. 36, pp. 61-8 2003.
[13] S. Osowski T.H. Linh and T. Markiewicz, “Support vector machine-
based expert system for reliable heartbeat recognition”, IEEE Trans.
Biom. Eng., vol. 51, pp. 582-589, 2004.
[14] P. de Chazal, M. O’Dwyer and R. B. Reilly, “Automatic classification
of heartbeats using ECG morphology and heartbeat interval features”,
IEEE Trans. Biom. Eng., vol. 51, pp. 1196-1206, 2004.
[15] M.G. Tsipouras, D.I. Fotiadis and D. Sideris, “An arrhythmia
classification system based on the RR interval signal”, Artificial
Intelligence in Medicine, vol. 33, pp. 237-250, 2005.
[16] P.S. Hamilton and W.J. Tompkins, “Quantitative investigation of QRS
detection rules using the MIT/BIH arrhythmia database”, IEEE Trans.
Biom. Eng., vol. 33, pp. 1157-1165, 1986.
[17] F.V. Theos, I.E. Lagaris, D.G. Papageorgiou, “PANMIN: sequential
and parallel global optimization procedures with a variety of options
for the local search strategy”, CPC, vol. 159, pp. 63–69, 2004.
[18] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan
Kauffman, USA, 1993.
[19] I. Guler, E.D. Ubeyli, “Adaptive neuro-fuzzy inference system for
classification of EEG signals using wavelets coefficients”, J
Neuroscience Methods, vol. 148, pp.113-121, 2005.
2319