Page 1

Journal of Intelligent & Fuzzy Systems 12 (2002) 235–242

IOS Press

235

A greedy randomized adaptive search

procedure applied to the clustering problem

as an initialization process using K-Means as

a local search procedure

J.R. Canoa, O. Cord´ onb, F. Herreraband L. S´ anchezc

aDepartment of Software Engineering, University of Huelva, 21071 La Rabida (Huelva), Spain

E-mail: Jose.cano@diesia.uhu.es

bDepartment of Computer Science and A.I., University of Granada, 18071-Granada,Spain

E-mail: ocordon,herrera@decsai.ugr.es

cDepartment of Computer Science, University of Oviedo, Oviedo, Spain

E-mail: luciano@lsi.uniovi.es

Abstract. We present a new approach for Cluster Analysis based on a Greedy Randomized Adaptive Search Procedure (GRASP),

with the objective of overcoming the convergence to a local solution. It uses a probabilistic greedy Kaufman initialization to get

initial solutions and K-Means as a local search algorithm. The approach is a new initialization one for K-Means. Hence, we

compare it with some typical initialization methods: Random, Forgy, Macqueen and Kaufman. Our empirical results suggest

that the hybrid GRASP – K-Means with probabilistic greedy Kaufman initialization performs better than the other methods with

improved results. The new approach obtains high quality solutions for eight benchmark problems.

Keywords: Clustering, greedy randomized adaptive search procedure, K-Means

1. Introduction

Clustering is a basic process to human understand-

ing. The groupingof related objects can be found such

diverse fields as statistics, economics, physics, psy-

chology,biology,pattern recognition,engineering,and

marketing [6,14].

TheClusteringprobleminvolvespartitioningasetof

entities into a given number of subsets and finding the

location of a centre for each subset in such a way that

a dissimilarity measure between entities and centres is

minimized.

K-Means is a popular clustering algorithms [2]. Al-

thoughit is known for its robustness, it can fall in local

optimal solutions under certain conditions [13]. It is

also widely reported that the K-Means algorithm suf-

fers from its dependence on initialization conditions

(initial clustering and instance order) as shown in [11].

Since heuristic approaches are good for avoiding

convergence to a locally optimal solution, they could

be used to find a globally optimal solution. Heuristic

approaches used in clustering include those based on

simulatedannealing[4,7,12],tabusearch[1],evolution

strategies [3] and genetic algorithms [8–10,15].

In this paper, we focus on the problem suffered

by the K-Means due to the initial conditions, propos-

ing a heuristic procedure to this problem. We pro-

poseaGreedyRandomizedAdaptiveSearchProcedure

(GRASP) [5] applied to the Clustering problem as an

initialization process, using K-Means as a local search

procedure.

Ourapproachtriestoeliminatetheclassical problem

of the K-Means algorithm, by permitting a higher ex-

1064-1246/02/$8.00 2002 – IOS Press. All rights reserved

Page 2

236

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

ploration and exploitation of the search space, with a

low computational cost.

A GRASP is an iterative process with each iteration

consisting of two phases, a construction phase and a

local search phase:

– The construction phase is based on a probabilistic

greedy Kaufman initialization to get initial solu-

tions. It encourages search space exploration.

– The local search phase uses the K-Means algo-

rithmtoexploitthesearchspace. WeuseK-Means

as a local search technique because it offers bet-

ter solutions than otherclassical algorithmswith a

low computational cost.

The construction phase builds a feasible solution,

whoseneighborhoodis exploredbyalocalsearch. The

best solution over all GRASP iterations is returned as

the result.

In order to introduce this new approach, this paper

is organized as follows. Section 2 introduces a back-

ground on clustering, K-means and initialization ap-

proaches. Section 3 presents the GRASP approach to

clustering. Section4showsexperimentsandtheiranal-

ysis, and finally, Section 5 points out some concluding

remarks.

2. Background

Acommonprobleminclusteranalysisispartitioning

objects into a fixed number of groups to optimize an

objective function-based clustering. These objects are

measuredaccordingtoseveralfeaturesthatcharacterize

them. Patterns can be viewed as vectors in a high

dimensional space, where each dimension corresponds

to one feature.

Inthissectionweintroducetheformalizationofclus-

tering, the K-Means algorithms, and four initialization

approaches.

2.1. Clustering problem

The clustering problem can be formalized as fol-

lows [2]: Considering N entities ei, each with an as-

sociated weight wi (i = 1,...,N), we search for k

centres cj(j = 1,...,k) minimizing:

f(c1,...,ck) =

N

?

i=1

min

j

(wid(ei,cj))

where d(ei,cj) measures the dissimilarity between ei

and cj. In our case, where the entities are described

Table 1

Iris results, k = 3

K = 3

Arithmetic mean

97.339

97.326

97.326

97.326

97.326

Standard deviation

0.010

0

0

0

0

Best solution

97.326

97.326

97.326

97.326

97.326

Random

Forgy

Macq.

Kaufm.

GRASP

Table 2

Iris results, k = 4

K = 4

Arithmetic mean

83.769

83.749

83.750

83.786

83.729

Standard deviation

0.026

0.027

0.031

0

0

Best solution

83.729

83.729

83.729

83.786

83.729

Random

Forgy

Macq.

Kaufm.

GRASP

by their co-ordinates in ?m, d(ei,cj) is the Euclidean

distance.

Basically, clustering can be considered as a combi-

natorial optimization problem, formulated as follow:

Let Q be set containing all objects to be clustered,

C be the set of all feasible clustering of Q,

J: C → ? be the internal clustering criterion,

then

Minimize J(c) subject to c ∈ C.

The complexity of the clustering problem is due to

different factors:

1. Clustering is an NP-HARD problem when it is

considered as the optimization of a certain mea-

sure. Therefore, an exhaustive approach is not

practicable due to the exponential number of po-

tentialpartitionsoftheinputdata. Thenumberof

possible partitions of N elements into k clusters

is given by

?

(k,N) =1

k!

k

?

j=1

(−1)k−j

?k

j

?

(j)N.

2. Clustering complexity grows if the number of

groups is unknown. In such a case he number of

solutions becomes:

?

(k,N) =

k

?

i=1

1

i!

i

?

j=1

(−1)i−j

?i

j

?

(j)N.

Due to these reasons, trying to get a global optimal

solution by means of an efficient and robust method is

difficult. Thus, there is a considerable interest in the

design of new heuristics to solve large-sized practical

clustering problems.

Page 3

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

237

Table 3

Ruspini results, k = 4

K = 4

Arithmetic mean

720.142

720.142

720.142

720.142

720.142

Standard deviation

0

0

0

0

0

Best solution

720.142

720.142

720.142

720.142

720.142

Random

Forgy

Macq.

Kaufm.

GRASP

Table 4

Ruspini results, k = 5

K = 5

Arithmetic mean

654.918

654.074

655.437

647.672

647.672

Standard deviation

12.193

7.320

6.692

0

0

Best solution

647.672

647.672

647.672

647.672

647.672

Random

Forgy

Macq.

Kaufm.

GRASP

2.2. K-Means algorithm

K-means evaluates a set of k selected objects, which

are considered representatives for the k clusters to be

foundwithin the source set of N objects. Given the set

of representative objects, the remaining objects are as-

signedto the nearestrepresentativeone,usinga chosen

distance measure.

The philosophy is that a better set of clusters is ob-

tained when the k representative objects are more cen-

trally located in the cluster they define. For this rea-

son, a suitable objective function to be minimized is

the sum of the distances from the respective centers to

all the other objects of the same cluster. The function

value depends on the current partition of the database

{C1,...,Ck}:

J :

?

withπk(Ω)beingthesetofallpartitionsofthedatabase

Ω = {e1,...,eN} in k non-empty clusters. Each in-

stance eiof the N instances in the database Ω is an

m-dimensional vector.

The K-Means algorithm finds locally optimal so-

lutions using the Euclidean distance in the clustering

criterion. This criterion is sometimes referred to as

square-error criterion. Therefore, it follows that:

k(Ω) → ?

J({C1,...,Ck}) =

k

?

i=1

ki

?

j=1

||eij− ci||

where k is the number of clusters, kithe number of

objects of the cluster i, eijis the j-th object of the i-th

cluster and ciis the centroid of the i-th cluster defined

Table 5

Glass results, k = 2

K = 2

Arithmetic mean

323.323

323.323

323.323

323.323

323.323

Standard deviation

0

0

0

0

0

Best solution

323.323

323.323

323.323

323.323

323.323

Random

Forgy

Macq.

Kaufm.

GRASP

as:

ci=

1

ki

k

?

j=1

eij,i = 1,...,k

The pseudo-code for this algorithm is:

1. Select an initial partition of the database in k

clusters {C1,...,Ck}

2. Calculate cluster centroids, using the expression

of its definition.

3. For every ei in the database and following the

instance order DO

– Reassign instance eito its closest cluster cen-

troid.Hence, ei ∈ Cs is moved to Ct if

||ei−et|| ? ||ei−cs|| forall t = 1,...,k,t ?=

s.

– Recalculate centroids for those clusters.

4. If cluster membership is stabilized then stop else

go to step 3.

The K-Means algorithm has the following draw-

backs:

– It assumes that the number of clusters k in the

database is known, which is not necessarily true.

– Itisespeciallysensitivetoinitialconditions(initial

clusters and instance order).

– It converges finitely to a local minimum, defined

by a deterministic mapping from the initial condi-

tions to the final solution.

2.3. Initialization approaches

Thesecondproblem,sensitivity toinitial conditions,

may be mitigated using different values of k, some

instance orders and different initialization methods.

In this section we describe four initialization ap-

proachesanalyzedin [11]. Each one generatesk initial

clusters following some kind of heuristic and produces

a different K-Means response.

These four classical methods are:

Page 4

238

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

Table 6

Glass results, k = 7

K = 7

Arithmetic mean

206.632

206.493

206.635

211.658

205.239

Standard deviation

0.704

0.723

0.676

0

0.486

Best solution

205.307

205.889

205.889

211.158

204.992

Random

Forgy

Macq.

Kaufm.

GRASP

Table 7

Glass results, k = 10

K = 10

Standard deviation

2.670

1.205

1.389

0

0.089

Arithmetic mean

179.042

176.710

177.144

188.691

176.058

Best solution

175.945

176.044

176.044

188.691

175.945

Random

Forgy

Macq.

Kaufm.

GRASP

– Random: Divides the database into a partition of

K randomly selected clusters.

– Forgy: k instances of the database (seeds) are

chosen at random and the rest of the instances are

assigned to the cluster represented by the nearest

seed.

– Macqueen: k instancesof the database(seeds) are

chosen at random. Following the instance order,

the rest of the instances are assigned to the cluster

with the nearest centroid. After each assignment,

a recalculation of the centroid has to be carried

out.

– Kaufman: The initial clustering is obtained by the

successive selection of k representative instances.

The first representative is chosen to be the most

centrallylocatedinstanceinthedatabase. Therest

of representative instances are selected according

to the heuristic rule of choosing the instances that

promise to have around them a higher number of

the rest of instances and that are located as far as

possible from the previously fixed ones.

Differences between these initialization methods

are:

– Random and Forgy generate an initial partition

independently of the instance order.

– Macqueen generates an initial partition that de-

pends on the instance order.

– Kaufman is the only deterministic initialization,

based on a greedy approach.

Table 8

Titanic results, k = 2

K = 2

Arithmetic mean

1617.563

1617.564

1617.564

1617.564

1617.562

Standard deviation

0

0

0

0.00066

0

Best solution

1617.563

1617.564

1617.564

1617.563

1617.562

Random

Forgy

Macq.

Kaufm.

GRASP

3. GRASP approach to the clustering problem

In this section we introduce the GRASP approach

and present its application to clustering.

3.1. Greedy randomized adaptive search procedure

(GRASP)

A generic GRASP pseudo-code is shown as fol-

lows [5]:

Procedure grasp( )

InputInstance();

For GRASP stopping criterion not satisfied

ConstructGreedyRandomizedSolution(Solution);

LocalSearch(Solution);

UpdateSolution(Solution,BestSolutionFound);

Rof

Return(BestSolutionFound);

End grasp;

In the construction phase, a feasible solution is it-

eratively constructed, choosing one element at a time.

At each construction algorithm iteration, the choice of

the next element to be added is determinedby ordering

all elements in a candidate list with respect to a greedy

selection function. This function measures the benefit

of selecting each element.

The probabilistic component of a GRASP is char-

acterized by randomly choosing one of the best candi-

dates in the list, but not necessarily the top one. The

list of the best candidates is called the restricted can-

didate list (RCL) and has dimension l. This choice

technique allows different solutions to be obtained at

each GRASP iteration, but does not necessarily com-

promise the power of the adaptive greedy component

of the method.

The GRASP construction phase pseudo-code is:

Procedure ConstructGreedyRandomizedSolution

(Solution)

Solution= {};

For Solution construction not done

MakeRCL(RCL);

s = SelectElementAtRandom(RCL);

Page 5

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

239

Table 9

Titanic results, k = 4

K = 4

Arithmetic mean

1110.034

1437.635

1442.327

1170.012

1071.458

Standard deviation

19.662

297.603

301.718

0

1.368

Best solution

1080.426

1282.327

1282.327

1170.012

1070.661

Random

Forgy

Macq.

Kaufm.

GRASP

Table 10

Titanic results, k = 10

K = 10

Standard deviation

71.877

82.042

84.564

0

4.196

Arithmetic mean

880.530

992.681

991.914

340.696

329.234

Best solution

670.166

930.914

930.914

340.696

327.234

Random

Forgy

Macq.

Kaufm.

GRASP

Solution = Solution ∪ s;

AdaptGreedyFunction(s);

Rof

End ConstructGreedyRandomizedSolution;

The solutions generated by a GRASP construction

algorithmare not guaranteedto be locally optimalwith

respect to simple neighborhood solutions. Hence, it is

almost always useful to apply a local search to attempt

to improve each constructed solution.

A local search algorithm works in an iterative fash-

ion by successively replacing the current solution by a

better solution in the neighborhood of the current one.

Thekeytosuccessforalocalsearchalgorithminvolves

a suitablechoicefora neighborhoodstructure,efficient

neighborhoodsearch techniques, and the starting solu-

tion.

Finally, the GRASP local search phase pseudo-code

is:

Procedure Local (P,N(P),s)

For s not locally optimal

Find a better solution t ∈ N(s);

Let s = t;

Rof

Return (s as local optimal for P)

End local;

with P being a problem (clustering in our case), s and

t being a solution for P, and N(s) being a mechanism

to obtain neighbors for s.

3.2. Using GRASP for the clustering problem

Following the generic GRASP structure, it is easy

to adapt the algorithm to the clustering problem using

Table 11

Image segmentation results, k = 6

K = 6

Arithmetic mean

158581.932

158261.497

158334.78

159233.335

158246.504

Standard deviation

470.346

626.579

321.169

0.0178

312.505

Best solution

158141.047

157923.297

157923.297

159233.297

157917.547

Random

Forgy

Macq.

Kaufm.

GRASP

Table 12

Image segmentation results, k = 12

K = 12

Standard deviation

3263.117

7.570

8.724

0

1.066

Arithmetic mean

117819.845

114244.441

115645.357

114248.172

114221.817

Best solution

114229.797

114237.953

114237.842

114248.172

114220.75

Random

Forgy

Macq.

Kaufm.

GRASP

the K-means algorithm and greedyKaufmaninitializa-

tion. The stopping criterion is defined in terms of the

maximum number of iterations, while the K-Means is

introduced as a local search algorithm. To construct

the greedy randomized solution, the Kaufman initial-

ization [6] is taken as a base, because it is a greedy

deterministic initialization algorithm. Using the Kauf-

man criterion, the RCL list is generated by the most

promising objects for each center of the solution and

one of those candidates is randomly selected.

A generic pseudo-code of the GRASP construction

phase for clustering is:

Step 1. Select the most centrally located instance as

the first seed.

Step 2. FOR every non selected instance eiDO

Step 2.1. FOR every non selected instance

ejDO

Calc Cji = max(Dj − dji,0)

where dji= ||ei− ej||

and Dj= minsdsj being s one of

the selected seeds

Step 2.2. Calculate the gain of selecting ei

by?

Step 3. MakeRCL(RCL)byselectingthelinstances

ei which maximizes?

Step 4. SelectElementAtRandom(RCL)

Step 5. If there are k selected seeds THEN stop

ELSE go to Step 2.

Step 6. For having a clustering assign each non-

selected instance to the cluster represented

by the nearest seed.

jCji

jCji

Page 6

240

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

Table 13

Texture results, k = 6

K = 6

Arithmetic mean

406.635

406.635

406.635

406.640

406.635

Standard deviation

0

0

0

0

0

Best solution

406.635

406.635

406.635

406.640

406.635

Random

Forgy

Macq.

Kaufm.

GRASP

Table 14

Texture results, k = 12

K = 12

Standard deviation

0.782

1.032

0.964

0

0.135

Arithmetic mean

361.731

361.361

361.491

360.354

359.989

Best solution

360.172

360.468

360.468

360.354

359.803

Random

Forgy

Macq.

Kaufm.

GRASP

When the k objects havebeentaken, the local search

(K-Means in this case) is applied taking the clustering

obtained as initialization. Then, we compare the local

solution cost with the best solution cost found so far,

and we take the best. This process continues until all

the iterations have been done.

Theuse ofK-Meansoffersan efficientandlowcom-

putational cost method to obtain relatively good solu-

tions, but converges to a local minimum. The GRASP

construction phase corrects this problem, performinga

wide spaceexplorationsearch. Notethat ouralgorithm

constitutes a new initialization method for K-Means,

which better explores the search space.

4. Experimental results

Next, we present the sample process, the results ob-

tained in our experiments, and their analysis.

4.1. Sampling process

The performance of our algorithm is studied with

various instance sets, trying to get conclusions inde-

pendentofthe problem. Eightreal-worlddatabases are

considered:

– Iris, which has 150 instances, 4 attributes and 3

classes.

– Ruspini , which has 75 instances, 2 attributes and

4 classes.

– Glass, which has 214 instances, 9 attributes and 7

clusters that can be grouped in 2 bigger classes.

Table 15

Vehicle results, k = 4

K = 4

Arithmetic mean

427.314

427.351

427.635

427.314

427.314

Standard deviation

0

0.124

0.116

0

0

Best solution

427.314

427.314

427.314

427.314

427.314

Random

Forgy

Macq.

Kaufm.

GRASP

– Titanic, which has 2200instances, 4 attributes and

2 classes.

– Image Segmentation, which has 2310 instances,

19 attributes and 7 classes.

– Textures, which has 1024 instances, 32 attributes

and 6 classes.

– Vehicle,whichhas846instances,18attributesand

4 classes.

– Pima, which has 768 instances, 8 attributes and 2

classes.

Since the K-Means algorithm strongly depends on

initial conditions, this problem is mitigated using dif-

ferent values of k and some instance orders. In this

case, the followinginitial numberof clusters havebeen

considered:

– Iris: k = 3, 4

– Ruspini: k = 4, 5

– Glass: k = 2, 7, 10

– Titanic: k = 2, 4, 10

– Textures: K = 6, 12

– Image Segmentation: k = 6, 12

– Vehicle: K = 4, 8, 12

– Pima: k = 6, 10

First, we applied three K-Means variants, each one

with its own initialization: Random, Forgy and Mac-

queen.

The sampling process followed is based on the com-

bination of four initial partitions and four instance or-

ders (see [11]),runningthe K-Means overeveryone of

them, and taking the best result of these sixteen runs.

This sampling process is repeated ten times and the

following three values are taken:

– the arithmetic mean of the objective function,

square-error criterion (J(·)), of the best solution

obtainedineachoneoftherunsofthetensampling

processes, denoted by “Arithmetic mean”.

– the standard deviation, and

– the best solution (objective function value) ob-

tained from the run of the ten sampling processes.

Page 7

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

241

Table 16

Vehicle results, k = 8

K = 8

Arithmetic mean

370.256

370.344

369.945

371.934

369.7

Standard deviation

0.79

0.754

0.464

0

0.001

Best solution

369.768

369.703

369.703

371.934

369.699

Random

Forgy

Macq.

Kaufm.

GRASP

Table 17

Vehicle results, k = 12

K = 12

Standard deviation

1.585

0.94

1.163

0

0.675

Arithmetic mean

335.661

334.343

334.671

336.131

333.313

Best solution

333.045

333.267

333.301

336.131

332.845

Random

Forgy

Macq.

Kaufm.

GRASP

On the other hand, the evaluation of K-Means with

Kaufman initialization has been done using 10 random

instance orders of each problem. Although Kaufman

initialization seems to be a completely deterministic

initializationmethod,thisisnotcompletelytrue. When

one instance has the same Euclidean distance to two

cluster centers, this instance will be associated to the

firstofthem. Ifwehavesomeinstancesinthissituation,

their order will modify the evolution of the k centers.

TheproposedGRASPalgorithmisstudiedusingsix-

teen iterations in each execution (running sixteen K-

means). Ten executions of the GRASP algorithm are

made, getting the arithmetic mean, the standard devia-

tion and the best solution. The RCL size used is fifteen

(l = 15), which is flexible enough to obtain optimal

solutions due the database sizes. Therefore, in order

to compare GRASP evaluation with the remaining K-

Means initialization methods, we use the same number

of K-Means runs as GRASP iterations for every exe-

cution of Random, Forgy and Macqueen initialization

methods.

4.2. Results

Experimental results are presented in Tables 1–20.

4.3. Analysis

Comparing results, we notice that Forgy and Mac-

queen usually give very similar results. They seem to

have the same performance, keeping the same or sim-

ilar local minima. Their results improve those given

by Random initialization, with any number of clusters,

Table 18

Pima results, k = 2

K = 2

Arithmetic mean

52072.211

52072.211

52072.211

52072.233

52072.233

Standard deviation

0.005

0.005

0.005

0.010

0.009

Best solution

52072.203

52072.203

52072.203

52072.219

52072.219

Random

Forgy

Macq.

Kaufm.

GRASP

Table 19

Pima results, k = 6

K = 6

Arithmetic mean

29415.066

29426.369

29428.447

30729.2

29403.763

Standard deviation

15.681

14.519

15.117

0.013

0.001

Best solution

29403.762

29403.762

29403.762

30729.195

29403.762

Random

Forgy

Macq.

Kaufm.

GRASP

except in the Titanic database. Thus, both Forgy and

Macqueen initializations respond better than Random,

with this difference more significant when k grows.

It is clear that Kaufman initialization by itself does

not obtain the best results compared to the classical

initialization methods. We conclude that the heuristic

used in Kaufman initialization needs a more flexible

centroid selection to reach the global optimum.

Finally, we compare the results obtained from

GRASP with the remaining K-Means initialization

methods. As said, this comparison is feasible because

we are comparing ten GRASP runs versus ten sam-

pling processes runs (16 K-Means runs with different

initialization approaches). This comparison is based

on effectiveness and robustness:

– If we study each individual table, we notice that

GRASP gets the best results in every problem for

the majority of executions.

Table 21 shows the percentage of times that

GRASP executions are the best, equal or worse

than the remaining ones (names in the fist row)

with respect to the “Arithmeticmean column”and

“Best Solution column” in the 20 tables of results,

denoting them as AM and BS respectively.

– On the other hand, GRASP presents similar ro-

bustness (small values for the standard deviation)

to Kaufman K-Means in most of the cases, and

better robustness than the remaining approaches,

Random, Forgy and Macqueen.

Although GRASP is computationally a little more

demandingthan the remainingK-Means initializations

with the same number of runs/iterations (due to the

Page 8

242

J.R. Cano et al. / A greedy randomized adaptive search procedure applied to the clustering problem

Table 20

Pima results, k = 10

K = 10

Standard deviation

134.991

26.718

23.687

0

7.929

Arithmetic mean

24287.956

24136.125

24137.881

24340.811

24127.770

Best solution

24120.801

24117.748

24117.748

24340.811

24117.748

Random

Forgy

Macq.

Kaufm.

GRASP

Table 21

Percentage of the performance improvement of GRASP versus the

remaining initializations

Random

AM

75% 50% 75% 50% 75% 50% 70% 70%

GRASP equal 20% 45% 20% 45% 20% 45% 25% 25%

GRASP worst5% 5%5%

Forgy

AM

Macq.

AM

Kaufm.

AM BSBS BS BS

GRASP best

5%5% 5%5%5%

computationaltime needed by the construction phase),

GRASP induces the K-Means algorithm to present a

better performance and a more robust behaviour.

5. Concluding remarks

The K-Means algorithmsuffersfrom its dependence

on initial conditions. We present a GRASP algorithm

totheclusteringproblembasedonthegreedyKaufman

initializationtoavoidthisdrawback. TheGRASPalgo-

rithm has been empirically compared with four classi-

cal initialization methods (Random, Forgy, Macqueen,

and Kaufman), and it proved to be the most effective

algorithm.

References

[1] K.S. Alsultan, A Tabu Search Approach to the Clustering

Problem, Pattern Recognition 28 (1995), 1443–1451.

[2] M.R. Anderberg, Cluster Analysis and Applications, Aca

demic Press, 1973.

G.P. Babu and M.N. Murty, Clustering with Evolution Strate-

gies, Pattern Recognition 27 (1994), 321–329.

D.E.BrownandC.L.Hutley, APractical Application ofSimu-

lated Annealing to Clustering, Pattern Recognition 25 (1992),

401–412.

T.A.Feo and M.G.C. Resende, Greedy Randomized Adaptive

Search Procedure, Journal of Global Optimization 2 (1995),

1–27.

L. Kaufman and P.J. Rousseeuw, Finding Groups in Data, An

Introduction to Cluster Analysis, Wiley, 1990.

R.W. Klein and R.C. Dubes, Experiments in Projection and

Clustering by Simulated Annealing, Pattern Recognition 22

(1989), 213–220.

K. Krishna and M.N. Murty, Genetic K-Means Algorithm.

IEEE Trans. on Systems, Man and Cybernetics – Part B:

Cybernetics 29 (1999), 433–439.

U. Maulik and S. Bandyopadhyay, Genetic Algorithm-Based

Clustering Technique, Pattern Recognition 33 (2000), 1455–

1465.

J.A. Lozano and P. Larra˜ naga, Applying Genetic Algorithms

to Search for the Hierarchical Clustering of Data sets, Pattern

Recognition Letters 20 (1999), 911–918.

J.M. Pe˜ na, J.A. Lozano and P. Larra˜ naga, An Empirical Com-

parison of Four Initialization Methods for the K-Means Algo-

rithm, Pattern Recognition Letters 20 (1999), 1027–1040.

S.Z.SelimandK.Alsultan, ASimulated Annealing Algorithm

for the Clustering Problem, Pattern Recognition 24 (1991),

1003–1008.

S.Z. Selim and M.A. Ismail, K-Means-Type Algorithm: Gen-

eralized Convergence Theorem and Characterization of Local

Optimality, IEEE Trans. Pattern Anal. Machine Intelligence 6

(1984), 81–87.

S. Theodoridis and K. Koutroumbas, Pattern Recognition,

Academic Press, 1999.

L.Y.TsengandS.B.Yang,AGenetic Clustering Algorithmfor

Data with Non-Spherical-Shape Clusters, Pattern Recognition

33 (2000), 1251–1259.

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]