Page 1

Decision making with imprecise parameters

Asli Celikyilmaza,*, I. Burhan Turksenb,c

aDepartment of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA

bDepartment of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ont., Canada

cDepartment of Industrial Engineering, TOBB-Economy and Technology University, Sogutozu, Ankara, Turkey

a r t i c l ei n f o

Article history:

Available online 3 July 2010

Keywords:

Interval-valued membership functions and

imprecise functions

Cased-based type reduction

a b s t r a c t

We analyze the impact of imprecise parameters on performance of an uncertainty-model-

ing tool presented in this paper. In particular, we present a reliable and efficient uncer-

tainty-modeling tool, which enables dynamic capturing of interval-valued clusters

representations sets and functions using well-known pattern recognition and machine

learning algorithms. We mainly deal with imprecise learning parameters in identifying

uncertainty intervals of membership value distributions and imprecise functions. In the

experiments, we use the proposed system as a decision support tool for a production line

process. Simulation results indicate that in comparison to benchmark methods such as

well-known type-1 and type-2 system modeling tools, and statistical machine-learning

algorithms, proposed interval-valued imprecise system modeling tool is more robust with

less error.

? 2010 Elsevier Inc. All rights reserved.

1. Introduction

Fuzzy systems are useful tools that can deal with complex, ill defined and uncertain environment parameters, where con-

ventional mathematical models fail to give satisfactory results. In many research papers and books such as in [1–3] it has

been shown that type-1 fuzzy models can have limitations in identifying uncertainties, since membership functions to char-

acterize type-1 fuzzy sets are crisp, rather than fuzzy values. An extension of type-1 fuzzy sets, also known as higher order

fuzzy sets, e.g., type-2 fuzzy sets, can be characterized with type-2 membership functions that are themselves fuzzy. Type-2

fuzzy sets are introduced by Zadeh [4] as an extension of type-1 fuzzy sets. It has been demonstrated in many different re-

search papers [3,5,6] that with the implementation of higher order fuzzy sets to build type-2 fuzzy systems or interval-val-

ued fuzzy systems, the performance of predicted model can be improved to a certain degree compared to type-1 fuzzy

systems, e.g., [1,7–9]. Hence, in this paper we introduce and investigate a new interval-valued fuzzy system modeling strat-

egy, in short interval-valued fuzzy functions (IVFF) approach, as an extension of our earlier type-1 fuzzy functions (T1FF)

methodology [7,8,10].

In modeling real systems, implementing type-2 fuzzy sets instead of type-1 fuzzy sets can be useful when it is difficult to

determine the exact and precise membership values of data points. Nevertheless, computations with general type-2 fuzzy

sets are rather complex compared to type-1 fuzzy sets. For such reasons in the literature, interval-valued fuzzy inference

systems, e.g., [1,6,11], which implement interval-valued type-2 fuzzy sets, have commonly been used instead of type-2 fuzzy

sets to reduce the computation complexity to a certain degree. In such systems, footprint of uncertainty (FOU) is the general

term used for interval-valued membership functions, to define the uncertainty interval of membership values. The FOU

shows the uncertainty region of type-1 membership functions [14] by forming an interval that is bounded with upper

0888-613X/$ - see front matter ? 2010 Elsevier Inc. All rights reserved.

doi:10.1016/j.ijar.2010.06.002

* Corresponding author. Tel.: +1 510 229 8269.

E-mail addresses: asli@berkeley.edu, asli@eecs.berkeley.edu (A. Celikyilmaz).

International Journal of Approximate Reasoning 51 (2010) 869–882

Contents lists available at ScienceDirect

International Journal of Approximate Reasoning

journal homepage: www.elsevier.com/locate/ijar

Page 2

and lower membership functions, which are usually identified by experts. Generally, each of type-1 membership function,

which are embedded within the boundaries of FOU of interval type-2 fuzzy sets, is used to construct embedded type-1 fuzzy

models. For such systems, a type-reduction method is implemented which follows a defuzzification algorithm to obtain crisp

outputs where necessary.

In type-2 fuzzy inference systems (FISs), more usual than not, the shape or parameters of type-2 membership functions,

boundaries of FOU, or structure of the rules, are identified by domain experts [1,14,23,24]. Manually designing and tuning

the parameters of membership functions of type-2 fuzzy systems may result in false assumptions and probably affect pre-

dicted model’s performance. Usually, pre-defined shapes e.g., Gaussian or triangular fuzzy sets, are used to define fuzzy sets.

Such type-2 FISs implement mainly Takagi–Sugeno or Mamdani type fuzzy systems.

Interval-valued fuzzy inference system (IVFS), to be presented at a later point of this paper, uses a fuzzy clustering algo-

rithm to identify overlapping regions that may exist in the dataset and assigns membership values for each data point in the

dataset to each cluster. For different values of initial parameters of fuzzy clustering methods one may obtain different mem-

bership values. Hence, we construct embedded (discrete) interval membership values by iterating learning parameters of

fuzzy clustering method. For each iteration we estimate a set of fuzzy functions, one for each cluster to capture the local

input–output dependencies. Thus proposed IVFS dynamically identifies embedded fuzzy functions for each cluster using

interval-valued fuzzy sets obtained from a fuzzy clustering algorithm of imprecise parameters, particularly using a hybrid

clustering method improved fuzzy clustering (IFC) [8]. Different from the hybrid clustering methods [13] of literature, IFC

is designed to shape membership values so that they can help to shape local functions to define input–output dependencies

of each cluster. The aim is to find the optimum functions to minimize the error. The fuzzy modeling approach being pre-

sented in this paper is feasible in the sense that it implements a non-complex inference tool with a practical type-reduction

method based on case-base reasoning.

The rest of the paper is organized as follows: In Section 2, we briefly review type-1 fuzzy functions in comparison to well-

known type-1 fuzzy inference systems. In Section 3, we introduce the new interval-valued fuzzy function representation and

inference modules. In Section 4, we present the results of experiments conducted on application of proposed and benchmark

methods on a desulphurization process of steel production. Finally, we draw conclusions in Section 5.

2. Fuzzy inference system based on type-1 fuzzy functions

Before we explain the interval-valued fuzzy function methodology of this paper, in what follows, we briefly review foun-

dations of type-1 fuzzy functions approach in comparison to well-known fuzzy inference systems. A detailed introduction

and explanation of underlying theories of type-1 fuzzy functions can be found in [8,10].

2.1. General fuzzy inference systems

Traditional fuzzy representation and inference systems, i.e., either type-1 or type-2 [1,9,12,14] have various challenges

that are in need for review for this paper. Among some of these challenges are;

? Identification of types of antecedent and consequent membership functions, and their varying parameters.

? Identification of the most suitable combination operators (t-norm, t-conorm, etc.), conjunction operators while aggregat-

ing antecedents, and consequents of each rule.

? Identification of the type of implication operator to capture uncertainty associated with linguistic ‘‘AND”, ‘‘OR”, ‘‘IMP” for

representation of rules, and reasoning with them.

? Identification of the type of the defuzzification method.

These issues have been investigated in many research papers to optimize the fuzzy operations. Many different methods

are proposed to optimize the parameters of such fuzzy systems and reduce expert intervention in building hybrid fuzzy sys-

tems by using other soft-computing methods such as genetic algorithms or neural networks, e.g., [15,16,18,25–28] as a

parameter or structure optimization tool. Some researchers have approached latter issues differently and tried to reduce

computation complexity and prevent information loss in a different way as introduced in [17], which was later utilized in

several papers such as [3,7,8,10]. Such fuzzy systems are constructed under the assumption that antecedent fuzzy sets

are dependent on each other, i.e., they are interactive; hence they characterize multi-dimensional membership functions

to represent entire antecedent part of any rule. An extension of such method using Takagi–Sugeno [9] systems can be defined

as follows:

Ri: IF x 2 X is AiTHEN yi¼ aixTþ bi

In (1) fuzzy set Aiis characterized by a type-1 membership function li(x) ? [0,1], which represents entire antecedent part of

rule i and x 2 X is a multi-input vector. Since only one antecedent fuzzy set is defined for each rule, fuzzification would be

simpler, aggregation of antecedent step is eliminated, and there is no independence assumption of input variables, which

may affect the prediction performance of the overall fuzzy system model. In addition, we would eliminate the a possible per-

formance decrease due to information loss that may be encountered when mapping the multi-dimensional membership

ð1Þ

870

A. Celikyilmaz, I. Burhan Turksen/International Journal of Approximate Reasoning 51 (2010) 869–882

Page 3

functions onto each individual input dimension. Fuzzy system models based on fuzzy functions [7,8,10] also adopt interac-

tivity of input fuzzy sets as explained in the next subsection.

2.2. Type-1 fuzzy functions

Type-1 fuzzy functions (T1FF) [7,8,10] approach is an alternate representation and reasoning tool to standard fuzzy infer-

ence systems. It can be considered as an extension of Takagi–Sugeno and Sugeno–Yasukawa fuzzy inference systems, except

that such systems implement multi-dimensional antecedent fuzzy sets. Structure identification of T1FF is based on a fuzzy

clustering algorithm, e.g., fuzzy c-means (FCM) [19] algorithm or improved fuzzy clustering method [8] to find possible hid-

den structures of a given dataset and characterize input membership distributions to represent multi-dimensional member-

ship functions of input domain. These methods do not require most of the aforementioned fuzzy operations of traditional

fuzzy inference systems as mentioned in the previous subsection. In somewhat simplified view, type-1 fuzzy function archi-

tecture is demonstrated in Fig. 1.

As shown in Fig. 1, given dataset is fuzzy partitioned into overlapping clusters using a fuzzy clustering algorithm. In se-

quence, for each cluster identified, i = 1,...,c, we predict a fuzzy function~fiðx;hiÞ 2 R using a non-generative algorithm, e.g.,

regression, etc. To predict an output value of a new testing data point, xtest2 Rnv, we first use fuzzy clustering parameters to

estimate membership values to each cluster and then using estimated fuzzy function parameters,^hi¼1;...;c2 Rnvwe obtain a

scalar output value, ^ yi¼^fðxtest;^hiÞ 2 R. The arithmetic mean of the estimated output values for each cluster, which are

weighted with their cluster membership values is calculated to obtain a crisp output value for the given test data point.

We can further simplify the learning algorithm demonstrated in Fig. 1 in two steps:

(i) Fuzzy clustering: The domain X2 Rn?nvof nv dimensional input space is partitioned into c overlapping clusters using

fuzzy clustering algorithm based on a chosen degree of fuzziness constant, m, and each cluster is represented with

cluster centers, Vi2 Rnvi ¼ 1;...;c; and membership value matrix, U 2 ½0; 1?n?c; Ui;j¼ li;j2 R where i = 1,...c, j =

1,...,n.

(ii) Function approximation: To each of these regions a local fuzzy model~fiðx;hiÞ 2 R is calculated by using membership

values, li2 Rnas additional predictors to given input variables, X ¼ fxk¼1;...;ng; xk2 Rnv. Scalar valued fuzzy functions

are calculated by using any regression method, e.g., linear least squares, kernel regression, etc.

It should be pointed out that the consequent functions, which are called the fuzzy functions in this paper, are special func-

tions which are formed using not only the original input variables but also the membership values of the particular cluster

(rule) and its user defined transformations to improve the prediction performance. Thus we implement a special hybrid clus-

tering method – IFC [8] – which can shape membership values so they can improve the representation of local dependencies

by minimizing the local loss function, i.e., minP

output value, lik= lik(xk) 2 [0,1] represent its membership value to cluster i = 1,...,c, c be the total number of clusters, m > 1

be the level of fuzziness parameter. The learning algorithm of type-1 fuzzy functions approach [8] is explained below.

ilðhiÞjlðhiÞ ¼ ðy ?^fiðx;li;hiÞÞ2.

Let (xk, yk) denote each training data point, where xk2 Rnvis any kth input vector of nv dimensions, yk2 R is its observed

2.2.1. Improved fuzzy clustering (IFC)

Improved Fuzzy Clustering (IFC) is a hybrid clustering method, which combines the point-wise clustering and function

estimation methods in one objective function as follows:

minJIFC

m¼

Xc

i¼1

Xn

k¼1lm

ikd2

ik

zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{

clustering-error

þ

Xc

i¼1

Xn

k¼1lm

iklikðyk;gikÞ

zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{

function-approximation loss

ð2Þ

0 In (2), dik= kxk?vik, represents the distance from each xkto each cluster center, vi, and the first term measures the

clustering error. We minimize this loss to stabilize the inter-cluster and inner-cluster similarities [19]. The loss function

li(s) = (yk? gi(sik))2of the second term the squared deviation between approximated fuzzy models, namely the interim fuzzy

Fuzzy Clustering

Fuzzy Functions

INFERENCE

Fuzzy Output Weighing

Fuzzy Input Sets of

Type-1

Fuzzy Output Set

of Type-1

Crisp

Inputs

Crisp

Output

Type-0

Fig. 1. Type-1 fuzzy functions systems.

A. Celikyilmaz, I. Burhan Turksen/International Journal of Approximate Reasoning 51 (2010) 869–882

871

Page 4

functions gi(si) of cluster i and the actual output. We would like to minimize this loss by estimating better interim fuzzy

functions. The interim fuzzy functions can be estimated with any regression method, e.g., least squares estimation-LSE, mul-

tivariate non-linear regression, etc. At each step of the clustering algorithm, a different membership value is used as input

variable. The aim is to capture the best membership’s values as inputs that can explain the local input–output relationships

and at the same time identify regression functions that define different dependencies between inputs and output. The first

term in the objective function in (2) helps to identify the local regions and the second term helps to shape the membership

values so the interim fuzzy function loss is minimized. The corresponding membership values and their possible transfor-

mations are the only predictors of the interim fuzzy functions gi(si) excluding original variables. An example of an interim

fuzzy function can be formed using:

giðsiÞ ¼ ^ w0iþ ^ w1iliþ ^ w2ið1 þ expð?lm

Additional examples of fuzzy functions can be found in [7,8]. In (3), ^ wirepresents the regression coefficients. The second

term of the objective function can be minimized if the optimum functions can be found. Thus, the algorithm searches for

the best interim fuzzy functions, gi(si). From the Lagrange transformation of the objective function in (2) the membership

values are calculated as follows:

i¼1

i = 1,...,c, k = 1,...,n. The cluster update equation is not affected and is same as the FCM [19] method. Punishing the objective

function with an additional error forces the algorithm to capture membership values that would help to reduce the loss, but

at the same time identify the hidden partitions. Membership function in (4) yields ‘‘improved” membership values,

l?

When estimating interim fuzzy functions, we eliminated the original input variables because we wanted to capture the

membership values that can explain the output variable. Alienating membership values helps to shape them independent

from the original input variables and prevent them from the effects of dominant input variables, which are highly correlated

with the output. We demonstrate in Fig. 2 that the membership values obtained from IFC are better inputs to build a model

to explain the output compared to the membership values obtained from the well-known FCM [19] method.

We constructed a very simple toy dataset of 20 data points with a single input and single output and executed FCM and

IFC, respectively setting c = 2 and m = 1.5 for each clustering method. The top scatter diagrams in Fig. 2 are linear regression

iÞÞð3Þ

lik¼

X

c

j¼1

d2

ikþ lik

??

= d2

jkþ ljk

??hi1=ðm?1Þ

!?1

;

X

c

lik¼ 1

ð4Þ

ik2 ½0; 1?, such that the membership values improves the prediction of the local dependencies.

0.10.2 0.3 0.40.5 0.60.7 0.80.9

-10

-5

0

5

10

Membership value (cluster 1)

Residual

residuals

Linear: norm of residuals = 19.1566

0 0.10.20.3

Membership values (cluster 1)

0.40.50.60.70.80.91

0

50

100

150

Output

Membership values from cluster 1 vs. Output

y = 110*u + 1.1

00.10.2 0.3

Membership values (cluster 1)

0.4 0.50.6 0.7 0.80.9

-10

-5

0

5

10

Residual

residuals

Linear: norm of residuals = 27.9979

0 0.1 0.20.3 0.4 0.50.60.70.80.91

0

20

40

60

80

100

120

Membership values (cluster1)

Output

Membership values (cluster 1) vs. Output

y = 120*u + 1.8

Fig. 2. Performance of membership values from FCM (left) and IFC (right). The top diagrams are functions of membership values from FCM and IFC to

explain the output variable of a sample dataset. The lower diagrams are residual errors of corresponding linear functions on the top.

872

A. Celikyilmaz, I. Burhan Turksen/International Journal of Approximate Reasoning 51 (2010) 869–882

Page 5

functions estimated using the membership values of a cluster as input variable. The top left figure demonstrates the mem-

bership values from FCM and the right from IFC. It is to be noted that the membership values of IFC (on the top-right of Fig. 2)

can explain the output better and the residual error of a single regression function is less than that of the regression function

estimated with the FCM results. The scatter diagram on the top-right shows that the membership values are shaped (aligned)

close to the output variable. This is due the result of IFC, which can shape the membership values into explaining output

variable. Different transformations of membership values, e.g., exponential or logistic, can help to explain the output

variable.

2.2.2. Identification of local dependencies

One fuzzy function is approximated for each cluster to identify the input–output relations within each cluster i. The

dataset of each cluster is comprised of original input variables, x, improved membership values, l?

obtained from IFC, and their user defined transformations, e.g., ððl?

sional input space, Rnv, of each individual cluster i onto a higher dimensional feature space Rnvþnm, i.e., x ! Uiðx;l?

nm is the total number of membership value transformations used to structure a system of principle fuzzy functions,~fðUiÞ, to

determine the local relations of each cluster in (nv + nm) space. A sample fuzzy function structure Uiusing two different

membership value transformations and the original inputs is as follows:

?

...

l?

in

x1;n

???

~fðUi;biÞ 2 Rjbi;0l?

The interim fuzzy functions, gi(si) are different from principle fuzzy functions~fðUi;biÞ, since gi(si) is used to shape the

membership values during IFC and only use membership values and their transformations as input variables. Whereas prin-

ciple fuzzy functions can minimize the local loss, minP

proved membership values, without the effect of the input variables, and then use them to calculate the fuzzy functions.

ik, of particular cluster i

ikÞpðp > 1Þ; el?, etc. This is same as mapping nvdimen-

iÞ. Here

Ui¼

l?

i1

?

?

el?

i1

?

?

?

?

x1;1

...

???

xnv;1

...

xnv;n

...

?

el?

in

2

664

3

775

n?ðnmþnvÞ

ð5Þ

iþ bi;1expðl?

iÞ þ bi;2x1þ ??? þ bi;ðnvþnmÞxnv

ilðhiÞjlðhiÞ ¼ ðy ?^fiðx;li;hiÞÞ2, i.e., the error between the actual output

i. In sequence, we first obtain im-

and the model output from each cluster induced by the improved membership values, l?

3. Interval-valued fuzzy functions (IVFF) approach

One of the challenges of type-1 fuzzy functions strategies summarized above is that the learning parameters of the clus-

tering, e.g. the number of clusters, c, degree of fuzziness, m, and the parameters of the fuzzy functions, e.g., the definitions of

fuzzy functions identified with different types of transformations, siand Ui, are uncertain. One needs to define these param-

eters manually prior to model construction. Even if optimization methods such as genetic algorithms [15,18] or neural net-

works [16,20] are used to build hybrid fuzzy models, assigning a single (optimum) value to any of these imprecise

parameters and generalizing it for every object of a given dataset may not reveal an optimum solution. An alternative

way would be assigning interval-valued parameter values instead of crisp values, and precisiating such intervals during

inference. Such a rich representation of a fuzzy model would improve performance, i.e., the optimum model would present

a full solution to choose from among alternative best solutions. In this sense, the uncertainty modeling of this paper is based

on identification of interval values for imprecise learning parameters.

Using interval-valued parameters to form interval-valued type-2 fuzzy systems has been studied by different researchers.

In [10], Turksen proposed characterization of uncertainty of fuzzy systems by identifying upper and lower boundaries of the

fuzziness parameter of the fuzzy c-Means (FCM) [19] clustering algorithm. The degree of fuzziness, m, viz., a constant to rep-

resent overlapping degree of identified clusters, has been investigated by many researchers, e.g., [3,11,21], since it is shown

that changing the fuzziness parameter results in different membership values, which is a natural consequence of identifica-

tion of interval-valued membership values. In addition, for different membership value transformations different fuzzy func-

tions can be obtained, resulting interval-valued outputs for each cluster. Hence, in this paper, the new IVFF uses the fuzziness

parameter of improved fuzzy clustering (IFC) and different structures of fuzzy functions to identify embedded membership

values and embedded local scalar fuzzy functions. In particular, embedded interval membership are formed by using differ-

ent values of fuzziness parameter (m) of IFC clustering [8]. Thus, we define the footprint of uncertainty (FOU) of interval

membership values based on a fuzzy clustering parameter and function definitions. Next, we will present the architecture

of the proposed method in two steps: structure identification and inference engines.

3.1. Structure identification of the proposed IVFF

To build ICFF, we use interval-valued membership values as shown in Fig. 3, thus a definition of an interval fuzzy set is

required. Interval fuzzy sets,eA, map the domain of a variable onto membership values in the interval of [0,1] as follows:

leAðxÞ : x 2 lL

eAðxÞ;lU

eAðxÞ

hi

;

leAðxÞ 2 ½0; 1?ð6Þ

A. Celikyilmaz, I. Burhan Turksen/International Journal of Approximate Reasoning 51 (2010) 869–882

873