Page 1

Journal of Computer Engineering

Research of Blind Signals Separation with Genetic Algorithm and Particle

Swarm Optimization Based on Mutual Information

Samira Mavaddaty

Department of Electrical and Computer Engineering

Babol Noshirvani University of Technology,

Babol, Iran

Email: s.mavaddaty@stu.nit.ac.ir

Abstract

Blind source separation technique separates mixed signals blindly without any

information on the mixing system. In this paper, we have used two evolutionary

algorithms, namely, genetic algorithm and particle swarm optimization for blind

source separation. In these techniques a novel fitness function that is based on the

mutual information and high order statistics is proposed. In order to evaluate and

compare the performance of these methods, we have focused on separation of noisy

and noiseless sources. Simulations results demonstrate that proposed method for

employing fitness function have rapid convergence, simplicity and a more favorable

signal to noise ratio for separation tasks based on particle swarm optimization and

continuous genetic algorithm than binary genetic algorithm. Also, particle swarm

optimization enjoys shorter computation time than the other two algorithms for

solving these optimization problems for multiple sources.

Keywords: Blind source separation, mutual information, high order statistics, Continuous

and Binary genetic algorithm, Particle swarm optimization.

1. Introduction

Blind source separation (BSS) has important applications in many area of signal

processing such as medical data processing, speech recognition and radar signal

communication [1-4]. In BSS, the source signals and the parameter of mixing model are

unknown. The unknown original source signals can be separated and estimated using

only the observed signals which are given through unobservable mixture [5]. In the

literature, the theory of BSS has been approached in several ways and as a result,

various algorithms have been proposed. For example independent component analysis

(ICA), principle component analysis (PCA), high order statistical cumulants and others

[6-8]. The most important and simplest of them is ICA as a statistical method that its

purpose is to find components of signal which have the most statistical independence.

ICA is based on random and natural gradient [9]. This algorithm is susceptible to the

local minima problem during the learning process and is limited in many practical

applications such as BSS that requires a global optimal solution. Also, the neural

networks have been proposed which their operation depends on an update formula and

activation function that are updated for maximizing the independence between

estimated signals [10]. These algorithms depend on the distribution of source signals.

Since this separation is executed blindly and there is no information about source

1 (2009) 63-76

77

Ataollah Ebrahimzadeh

Department of Electrical and Computer Engineering

Babol Noshirvani University of Technology,

Babol, Iran

Email: e_zadeh@nit.ac.ir

Page 2

Research of Blind Signals Separation…

signals, the distribution function of source signals should be estimated early.

Consequently, it leads to reduce the accuracy of problem solving. Thus, developing new

BSS algorithms on the basis of global optimization independent of gradient techniques

is an important issue [11-15]. The BSS problem is identified as a popular search among

researchers because it can work based on evolutionary algorithms such as continuous

genetic algorithm (CGA) and binary genetic algorithm (BGA), PSO and so on [16, 17].

It is obvious that GA and PSO are successful evolutionary algorithms that provide

heuristic solutions for combinatorial optimization problems.

In this paper, the BSS approach for linear mixed signals is studied to get the

coefficients of separating matrix by using PSO and both forms of GA. The operation of

these algorithms principally depends on the fitness function which in this paper uses

mutual information (MI) as a main criterion in information theory and high order

statistics (HOS) of kurtosis [18, 19, 20]. MI is a main quantity that measures the mutual

dependence of the two variables. Also the kurtosis is a simple and necessary criterion

for estimating dependency among signals [21]. This paper proposes the fusion of these

important criteria as a suitable fitness function for separation of different sources in

linear BSS model. Using this fitness function in evolutionary algorithms, it does not

need to have activation functions like what is required in neural network [22]. The

simulation results demonstrate the BSS scheme based on PSO and CGA is robust to

achieve global optimal solutions from any initial values of the separation system. These

results show high accuracy, suitable SNR and fast convergence of these evolutionary

algorithms than BGA. The BGA has bad convergence and lower values of accuracy and

SNR in source separation for more than three sources. The analyses show the

convergence of BSS using PSO is essentially faster than CGA and BGA for any number

of original signals. Thus, PSO is totally effective for this kind of optimization problems

especially in applications that needs high speed or low cost for time computations.

S.Mavaddaty & A.Ebrahimzadeh.

78

2. Linear Mixing Model and Separating Process

Assume that there exist n unknown signal

independent as possible. It is supposed that the source signals in linear model of BSS

are linearly mixed together With a matrix

A

is ,i 1,...,n

=

which are as mutually

n n

× that is unknown:

x As

=

(1)

Where

and n is the number of sources. The goal in solve of BSS problem is to discover the

source signals from x without knowing the nature of mixing matrixA. For doing this

task, separating matrixW should be found that it is

1

n

s [s ,...,s ]

=

and

1

n

x [x ,...,x ]

=

are n− dimensional source and mixed signals

1

WA

−

=

in ideal situation:

yWx

=

(2)

So that

model of BSS problem with sparse representation, which illustrated as Figure 1 includes

three procedures: an unknown mixing model, a recognition of mixing matrix and a

source signal retrieval process.

1n

y[y ,...,y ]

=

includes n− dimensional estimation of source signals. A general

Page 3

Journal of Computer Engineering

3. Preprocessing of BSS

1 (2009) 77-88

79

A. Centering

One of the most basic and necessary part of preprocessing is to center mixing signals

x so as to subtract its mean vector

{ }

m E x

=

that means convert x to a zero-mean signal

x [8]. This step should be executed because kurtosis basically obtains as follows:

{ }{ }

{ } { }

{ } { }{ }

(

{ }

E x

( )

()

()

{ }

()

{ }

2

()

)

()

2

2

42

2

2

22

4

2

2

E x3 E x

12 E x

E x

Kurt x

3 E x E x

4E x E x36 E x

−

=+

+

−

Figure 1. The general BSS flowchart consisting of unknown mixing model,

recognition of mixing model and source retrieval operation

The assumption of mixed data centering, makes easy the calculation of kurtosis. So,

the kurtosis can be computed by simple following formula:

{}

{}

E x

After estimating the mixing matrix W with centered data, the estimation by adding

the mean vector of x back to the centered estimates of s is completed. The mean vector

of s is given by

W m

, where m is the mean that was subtracted in the preprocessing.

()

4

2

2

E x

Kurt(x)

3

=− (4)

1

−

B. Whitening

Another useful preprocessing is to whiten the observed signals [23]. This means that

before considering the application of the ICA algorithm (and after centering), the

observed signal x is linearly transformed so that a new signal x obtains which is white,

i.e. its components are uncorrelated and their variances are equaled in unity. In other

words, the covariance matrix of x equals the identity matrix:

{}

E x.xI

=

T

(5)

The whitening transformation is always feasible. One popular method for whitening

is to use the eigenvalue decomposition (EVD) of the covariance matrix {

where E is the orthogonal matrix of eigenvectors of {

matrix of its eigenvalues,

1n

D diag(d ,...,d )

=

. Whitening can now be calculated by:

}

TT

E x.x EDE

=

}

T

E x.x

and D is the diagonal

Mixing

Model

1 s

2 s

3 s

A

Mixing

Model

Recognition

Source

signal

Retrieval

2

x

x

1

x

3

1

y

y

y

W

2

3

Page 4

Research of Blind Signals Separation…

x=ED E x=ED

S.Mavaddaty & A.Ebrahimzadeh.

80

-1 2T -1 2T

A

E As=As

(6)

The benefit of whitening is that it works based on the fact that the new mixing matrix

A is orthogonal. This characteristic of separating matrix reduces the number of

parameters needs to be estimated. Instead of estimating the

elements of the separating matrix, the new orthogonal separating matrix is estimated

that n(n 1) 2

−

contains degrees of freedom.

2

n parameters which are the

4. GA and PSO Algorithms for BSS

The algorithms that work based on evolutionary mechanism can be the best solution

for solving BSS problem through finding optimum and accurate coefficients of

separating matrix. According to these algorithms, Primary population can be converted

into a new population that independence among its components is maximized using a

suitable fitness function. Since GA and PSO intrinsically use evolutionary technique,

we take advantages of us as a successful and fast algorithm to jump out of the potential

local minimum.

A. Fitness function

There are two types of contrast function of BSS which are based on information

theory and high order statistics. The former methods include one of the mentioned

types. The fitness function proposed in this paper takes the fusion of two criteria,

kurtosis and mutual information. The kurtosis is a very simple and essential measure

that can be defined as:

•

Kurtosis < 3 for sub Gaussian signal

•

Kurtosis = 3 for Gaussian signal

•

Kurtosis > 3 for super Gaussian signal

According to central limit theorem that is totally practical in ICA, the distribution of

a sum of independent random variables tends toward a Gaussian distribution. Thus, a

sum of two independent random variables usually has a distribution that is closer to

Gaussian than any of the two original random variables. In BSS, if the kurtosis of

estimated signals is maximized and distanced from the kurtosis of Gaussian signal then

the reverse of the theorem is confirmed and independence among estimated signals is

guaranteed. So the fitness function can be defined based on the sum of the absolute

values of kurtosis in estimated signals. Another natural measure of dependence between

signals is inspired by information theory that is minimization of mutual information.

The mutual information I between n random variables

differential entropy is defined as follows:

()() ( )

12ni

i 1

=

That H is the entropy of mixed signals and

non-negative, and zero if and only if the variables are statistically independent that it

takes the form:

( )()()

ii

i 1

=

i y ,i1,...,n

=

using the concept of

n

I y ,y ,...,yH y H y

=−

∑

(7)

12n

Y[y ,y ,...,y ]

=

. The entropy is always

n

H Y P ylogP y

= −−

∑

(8)

Page 5

Journal of Computer Engineering

Thus, mutual information takes into account the whole dependence structure of the

variables, and not only the covariance, like PCA and related methods. So, the fitness

function can be defined as:

{ }

i 1

iii

J(y) [E y 3Ey H y H y ]

=

=−−+−

∑

Where

1n

y ,...,y are estimate of source speech signals. The dependence among the

estimated signals is minimized when Fitness is maximized. In this method, it is not

necessary to assume that the sources have the same sign of kurtosis, because the

absolute of fitness function is directly maximized. So the super Gaussian signals and

sub Gaussian signals can be separated from each other successfully.

1 (2009) 77-88

81

{ }

() ( )

422

n

(9)

B. Orthogonalization

The Orthogonalization plays a main and practical rule in BSS that the algorithm

would be completely defective without it. The estimate of coefficients using

maximization of fitness function to retrieve independent components is not enough.

Doing these steps until now, outputs of BSS algorithm are n similar speech signals that

are the estimate of source signal that its kurtosis is maximum. It should be mentioned

that algorithm does its task correctly because only when fitness function is maximized

that all n estimated signals have the analogy and maximum kurtosis. So the

orthogonalization is applied in order to avoid this problem. The orthogonal separating

matrix can be obtained by orthogonalization and satisfies (5). The orthogonalization is

applied to GA and PSO before fitting each population. When fitness function is

maximized, estimated signal is mutually independent as possible. Two main methods

for orthogonallization exist: Deflationary and Symmetric orthogonalization. Usually

Symmetric orthogonalization is used in ICA because of higher applicability and obtains

through the following formula:

T 1 2

W W.Real(inv(W.W ))

=

Doing Symmetric orthogonalization as the last necessary step for BSS, independence

among separated signals is guaranteed. The structure of the BSS based evolutionary

algorithm is shown in Figure 2.

−

(10)

5. Evaluating Criteria

In order to check the effectiveness of the proposed algorithm, the Euclidean distance

of the two vectors: the kurtosis of the estimated and source signals as error of proposed

method is investigated. The results of the separating process are better whatever this

criterion be less. Also, we utilize the SNR (signal-to-noise ratio) to confirm the

accuracy of Euclidean distance as evaluating criteria. We define SNR as:

2

i

i

2

SNR10log

Eytst

−

( )( )

()

ii

E (s (t))

=

(11)